Month: October 2025

Can you treat test cases like source code that compiles into automation? I wondered if this might work… so I tried it. The Question How far can I take building a fully “prompt driven” test automation system? In previous experiments, I’ve been exploring different aspects of AI-driven test automation: Experiment #1: Creating test cases from screen recordings with FFMPEG Experiment #2: Running test cases directly with Chrome DevTools MCP Experiment #3: The “run twice” pattern – agentic discovery then deterministic YAML This experiment brings it all together into a systematic approach I’m calling The Test Automation Compiler. The core idea: Treat markdown test cases as source code that gets compiled into executable automation through a defined process – just like...

Can you transform unreliable agentic tests into deterministic, repeatable tests using a “run twice” pattern? I wondered if this works… so I tried it. The Question Why did Claude Code suddenly start building a test automation framework I didn’t ask for? Towards the end of Experiment #2, things went sideways. I asked Claude Code to create a slash command for test automation. Instead, it started generating commands for converting tests to YAML, creating action libraries, building test runners… way beyond what I’d asked for. Then I realized: I’d forgotten about a markdown document in my project folder. A document about “deterministic test automation” that I’d been exploring with Claude Code in a previous session. When I said “read all the...

Can you actually go straight from test case doc to automated execution? Thought I’d give this a go and see what kind of results I came up with. The Question “Can I just give Claude Code my test case document, connect the Chrome DevTools MCP, and have it run the tests? Like, actually run them in a real browser, without writing any test code or setting up a framework?” Potential scenarios for this include… when you have documented test cases but no automation yet – could this give you instant automated execution? Maybe you could use this to figure out how to interact with difficult applications before building proper test frameworks. Anyway, no point speculating about how to use this...

Can you actually record yourself testing and get test case documentation back automatically? I wondered if this works… so I tried it. The Question I hate writing test case documentation, especially after I’ve already done the testing manually. So I wondered… what if I just record a video of myself testing and let Claude Code write the test cases? Could this actually work? I’m kind of thinking of two potential scenarios if this works. First, when you have something to test but nobody has bothered to write a decent spec – you could record yourself exploring the application and generate test cases from that. Second, maybe you have a wireframe of the application (before it’s built) and you can record...