If you just want to try the exercises then skip the explanation and scroll down to Exercise 1.
One of the quickest ways to send your AI agent off track is to give it conflicting information. Conflicting information in files, memory, or context almost guarantees unreliable results. One run it’ll do X, the next run it’ll go off and do Y.
Now this might seem, on the surface, like quite an easy thing to avoid. However, when you’re getting AI agents to generate information in the first place, you can (most likely will) end up with a lot of data and files. And let’s face it — we don’t always read and review all of the content that’s generated.
As you create all these documents you’re not just creating documents. You’re building an information system that shapes your AI’s behavior. That behaviour becomes more unpredicatable the more conflict it’s exposed to. Conversly the more consistency you provide the more predictability you’ll get.
The content in your system MUST be consistent if you want to build a good system.
Most people think about documentation as storage. However, if you start thinking about this from a higher-level you start to understand that your documentation is the behavioral programming for AI system.
Lower-Level Thinking: “I need to organize my project documents better”
Higher-Level Thinking: “I need choherent and consistent documentation so that conflicting signals are minimised”
This isn’t about being tidy. It’s about understanding that every document is a behavior modifier for your AI system. Conflicts create behavioral instability. Consistency creates predictable and powerful AI behavior.
After months of working with AI agents, and debugging when things are going wrong, I discovered this technique. It’s deceptively simple, but it’s the difference between AI that works and AI that works reliably.
First I’ve found it useful to structure my data/documents into distinct areas:
Domain A: The WHAT Documents (Templates/Examples/Specifications)
Domain B: The HOW Documents (Process/Instructions/Methods)
Domain C: The WHO/WHEN/WHERE Context (Persistent Memory)
In many ways this approach helps ME as much as it helps the AI engine.
That’s because I can break down what I’m trying to do into easier to manage and understand components. When you separate WHAT from HOW, you’re creating clear lanes where different aspects of your information can be created with clarity and consistency.
Think of it like this:
The big reason I do this is because it makes it easier to pick up on conflicts and build consistency.
I’ve designed two exercises so that you can try this immediately, regardless of your setup. You just need a browser and a claude.ai / chatgpt account.
Exercise 1 – simple personal fitness project
Exercise 2 – test case development system
Each exercise focuses on creating the WHAT, HOW and WHO/WHEN/WHERE documents. Then using a consistency check prompt like this…
Please read these three documents.
Then analyse these documents against each other for inconsistencies, discrepancies and/or conflicts
Please rate the level of consistency between these documents on a scale of 1 to 100.
Please provide me with suggestions on how to get to 100.
You can create variations on this prompt but you get the idea. If you want you could supplement this prompt with something like..
List conflicts by severity:
- BLOCKING: AI cannot proceed without resolution
- DIVERGENT: AI will make inconsistent choices
- PREFERENTIAL: AI might vary but within acceptable bounds"
Let’s start with exercise 1 then.
This exercise gets you to build a personal fitness program. It’s based on creating 3 documents:
This is a simple exercise with blindingly obvious discrepancies. The idea is to force home the concept and technique … not actually help you run a 10k race!
You can do this entire exercise in a single browser session (with 4 tabs open). It’s designed to give you the “aha moment” in under 10 minutes.
Complete these actions:
Open claude.ai or ChatGPT in a browser (Tab 1) and use this exact prompt:
"Please create a WHAT document for my fitness goals. I want to:
- Lose 15 pounds over 6 months
- Complete a 10K race
- Improve overall strength and muscle tone
Define what success looks like for each goal, what the finished state looks like, and what standards I need to meet. Format this as a clear and concise document I can reference."
Save this response as a document called what.md
Open a second browser tab with claude.ai or ChatGPT (Tab 2) and use this exact prompt:
"Please create a HOW document for my fitness training approach. My methods are:
- Running 4 times per week building up distance gradually
- Strength training 3 times per week focusing on compound movements
- Calorie deficit of 500 calories per day through portion control
Describe how I'll approach training, how I'll structure my week, and how I'll execute this plan. Format this as a clear and concise document."
Save this response as a document called how.md
Open a third browser tab with claude.ai or ChatGPT (Tab 3) and use this exact prompt:
“Please create a WHO/WHEN/WHERE context document for my fitness situation. My reality:
Describe my constraints, my current situation, and the resources and limitations I’m working with. Format this as a clear and concise document.”
Save this response as a document called who-when-where.md
Open a fourth browser tab with claude.ai/chatGPT (Tab 4) :
Upload the following documents
what.md how.md who-when-where.md
Then enter the following prompt:
"I have three documents about my fitness plan. Analyze them for conflicts, contradictions, and inconsistencies."
Identify:
1. Goal conflicts (goals that work against each other)
2. Method mismatches (approaches that don't support the goals)
3. Reality gaps (constraints that make goals/methods impossible)
4. Resource conflicts (time/access issues)
Be specific about which parts of which documents conflict with each other."
You could even experiment with an additional prompt like this…
Please rate the level of consistency between these documents on a scale of 1 to 100.
Then provide me with suggestions on how to get to 100.
From here you can work with your AI agent to create 3 updated documents that are consistent and work together.
This conflict analysis isn’t just debugging documentation for AI systems. In many ways it’s debugging your own decision-making process. The clearer your thought process is the clearer you can make it for an AI system.
Maybe I could even go as far as saying humans and AI agents share very similar fundamental requirements. Consistent information produces consistent behavior. Contradictory information — whether in documents, plans, or your own thinking, produces conflict.
Build consistency into your system, and you build reliability into your results.
In this exercise we create the document set that would form the basis of a system that gets an AI agent to create test cases. We’ll create the following documents:
Complete these actions:
Open claude.ai or ChatGPT in a browser (Tab 1) and use this exact prompt:
Create test-case-format.md with the following prompt:
Create a test case template that shows the standard structure and format for writing software test cases. Include:
- All essential fields (ID, title, preconditions, steps, expected results, etc.)
- 2-3 concrete examples showing the template in use
- Clear formatting that makes it easy to replicate
Keep it practical and scannable.
Open claude.ai or ChatGPT in a browser (Tab 2) and use this exact prompt:
Create test-principles.md with the following prompt:
Define 5-7 core principles for writing effective test cases. For each principle:
- State the principle clearly (e.g., "Test one thing at a time")
- Explain why it matters in 1-2 sentences
- Give a brief example or counter-example
Focus on principles that lead to maintainable, clear, and valuable test cases.
I for this one you can just create the markdown document with the following content.
Create project-context.md document using this prompt:
# Project Context
## Team
- 4 developers with varying TDD experience
- QA lead reviews all test PRs
- Product owner reads test names for validation
## Environment
- JavaScript/Node.js project
- Jest test framework
- CI/CD runs tests on every commit
- Test coverage target: 80%
## History
- Previous tests were inconsistent
- Team agreed to define structure in last retro
Open claude.ai or ChatGPT in a browser (Tab 2) and use this exact prompt:
Now give this prompt to Claude Code:
Analyze these three test documentation files for conflicts or inconsistencies:
1. test-case-format.md (WHAT we build)
2. test-principles.md (HOW we build)
3. project-context.md (WHO/WHEN/WHERE constraints)
Specifically identify:
- BLOCKING conflicts: Where HOW contradicts WHAT
- TENSION points: Where following one might compromise another
- GAPS: What's undefined that could cause inconsistency
Then create a test case following all three documents and show me where you had to make judgment calls.
You’ll probably find significant conflicts here – and that’s precisely the point!
You’ve just experienced what your AI experiences every time it reads your project documents.
Those contradictions between template structure and guiding principles? That tension between comprehensive documentation and incomplete examples? Your AI navigates these conflicts constantly, making different choices each run.
Build consistency into your process, not after it. Create documents sequentially, each informing the next.
You’re not organizing information – you’re architecting behavior.
This isn’t really about documents. It’s about understanding that you’re designing the operating system for AI behavior.
Every document is a behavioral instruction. Every conflict is a defect in that operating system. Every consistency check is behavioral debugging.
When you think this way, you stop being a prompt writer and become a system architect.
For the next week, try this:
Instead of asking “How do I organize my documents?”, ask “What behavioral do I want from my AI system?“
Instead of fixing document conflicts, ask “What AI behavior am I debugging?“
This is the shift from lower-level to higher-level thinking.
You’re not managing documents. You’re shaping behavior.
Ultimately you’re building the system that builds the system.
