The more I work with AI for coding and testing the more I keep falling back to tried and tested principles. And yet some principles seem to take on even more weight when working with AI.
Take building out a regression test suite as an example.
Specs go stale. Tests don’t (or shouldn’t).
Design docs and requirement specs go stale because nothing alerts you when they are out of date. And why should anything alert you? Once implemented they are rarely used ever again.
The code moves on, the document doesn’t, and nobody notices.
Tests are different. At least they are different if you’re maintaining your regression test suite.
The moment a test stops matching the code, it fails. Someone has to look. Either the code is wrong or the test is, and one of them gets fixed.
So a well-kept pack of test cases and scripts ends up being the most accurate spec of what your code should do.
None of this is new. What’s new is its importance!
Your agents read the spec too
Every agent session starts with no memory. It works from whatever it can read. If your agent has access to a requirements doc that’s six months out of date it’ll be influenced by something that’s out of date.
If your agent has access to a pack of regression tests that still hold true, then your agent will be influenced by something that’s in date.
When the code drifts, a test script fails. If that failure is picking up a regression issue it’s guiding your agent and pulling your agent back on track.
That’s not something an out of date requirements spec is ever going to do.
Testing isn’t the afterthought anymore
As agents write more of the code, the test cases become the critical component that guides them. Testing moves from the end of the line and starts threading its way through end-to-end.
You might say testing has always been threading its way through end-to-end. But has it? Has it really? The overhead of test maintenance has always meant we settled for whatever we could keep up with.
With agents able to maintain test case specifications and keep automated test scripts in sync with the code, maintaining regression packs is no longer the challenge it’s always been.
My prediction is that the ability to manage the test process with AI is going to become THE most important component to generating code with AI.
No longer the second-class citizen! The most important citizen!