Give an ambiguous ticket to a developer and something useful happens: they come back and ask. The ambiguity surfaces as a question in standup, costs five minutes, and gets resolved before any code exists.
Give the same ticket to a coding agent and nothing happens, in the specific sense that nothing visible happens. There is no “I am not sure” state in a language model. Where the description stops, generation continues, and the gap gets filled with whatever pattern is most probable given the surrounding words. You get code back. It runs. It is internally consistent. And its behaviour is whatever the training distribution suggested, which may or may not resemble what you meant.
That is the entire shift, and everything else follows from it. Ambiguity used to cost a question. Now it costs plausible wrong code that passes review, because nothing about the output signals which parts were specified and which were invented.
The one test that replaces the checklist
There are good lists of spec rules circulating — start with outcomes rather than implementation, state non-goals, write testable acceptance criteria, force error paths, give constraints, prefer examples over rules, match detail to complexity. Seven items, all correct, and more than anyone will hold in their head while writing a ticket at 5pm.
They compress into one question:
Would two competent implementers, working independently from this text, produce the same observable behaviour?
If yes, the spec is closed. If no, you have found the exact place the model will improvise — and, unlike your colleague, it will not tell you it improvised.
Every rule on the list is a way of failing that test. No non-goals means two people disagree about what is out of scope. No error path means two people invent different behaviour for the empty input. No constraint means one reaches for the framework already in the repo and the other pulls in a new dependency. The question is not a replacement for the rules; it is the thing the rules are all trying to detect, and it is short enough to actually use.
This is a test plan
Look at what a well-formed spec turns out to contain.
Acceptance criteria in Given/When/Then form: context, event, expected result. At least one criterion covering an error path. At least one covering an empty, permission-denied, or boundary case. An explicit non-goal. And for every criterion, a defined proof — a test, a log line, a screenshot — decided before implementation, with the rule that a criterion nobody can prove is a criterion that is too vague to keep.
A QA engineer reading that list is reading their own job description. Given/When/Then is a test case. “At least one error path” is negative testing. Boundary and permission cases are equivalence partitioning. “Define the proof before implementation” is the whole argument for writing tests first. Non-goals are scope boundaries, which is what stops a regression suite growing without limit.
None of this is new. What is new is where it sits. This used to be work done after a spec, by someone reading it and turning it into a test plan — and the act of doing that translation is where ambiguity got caught, because you cannot write a test for a requirement you do not understand. Hand the spec straight to an agent and that translation step disappears. Nobody hits the sentence that does not make sense.
So the discipline has to move upstream. The spec has to arrive already containing its own oracle, because the reader who would have demanded one is no longer in the loop.
Why the gap fills with something plausible
It is worth being precise about the mechanism, because it explains why the failures are so hard to spot.
A model completing a gap is doing the same thing it does with any other continuation: emitting the most probable tokens given the context. A single word can activate a whole learned pattern. Write “sort and return the top N” and the canonical implementation from thousands of training examples is right there, fully formed, plausible, and possibly wrong for your data. The result is code that is syntactically correct and semantically off, and the two are hard to tell apart at review, because everything about the code’s appearance is right.
The failure has a famous non-AI ancestor. The Mars Climate Orbiter was lost in 1999 because ground software produced thrust in imperial units while the navigation system expected newtons. Every number was sent correctly, received correctly, and stored correctly. Nothing errored. Over nine months the discrepancy compounded into a trajectory that put the probe into the atmosphere instead of orbit. The review board’s phrase for it — precisely, exactly, verifiably wrong — describes agent output from an underspecified brief perfectly.
The lesson QA already drew from that class of failure is that you test the contract at the interface, not just the components on either side. The agent version is the same: the spec is the interface, and an unstated assumption there is a unit mismatch waiting nine months to matter.
There is measured support for the mechanism, with limits worth stating. A controlled study on task-description quality found that ambiguous, incomplete, or contradictory descriptions cut first-attempt accuracy by roughly 20 to 40 percent, with contradictory descriptions the worst of the three. It was measured on deliberate mutations of standard coding benchmarks rather than on production tickets, so treat it as directional rather than as a number to plan against.
Where a better spec does not save you
The claim “it is the spec, not the model” is a comfortable one and it is not fully true. Worth saying plainly, because a rule set that promises too much gets abandoned the first time it fails.
A controlled trial on experienced developers working in their own mature repositories found some of them were slower with AI assistance while believing they were faster — a small study of sixteen participants, and the authors are explicit that it does not generalise across all software work. But better spec discipline did not rescue that result, which is the relevant part here.
There is a second boundary in the same direction: models that score in the high eighties on synthetic benchmarks drop sharply on real repository tasks, and the reasons include missing context and retrieval failures that no amount of spec-writing addresses.
So the honest formulation is narrower and more useful than the slogan: the spec is the largest lever you personally control. Before concluding the model is weak, audit your own description — it is cheaper than switching models, and it is the only variable in the system you can change this afternoon.
The spec is authoritative, and the tests will tell you it is wrong
The loop that makes this work is one QA people already run, with one clause that surprises people.
You write the spec, generate tests and code, watch what breaks, tighten the spec, repeat. The clause: a failing test is as likely to be a defect in the spec as a defect in the code. When code and spec disagree, the spec wins, because it is the contract and the code is one implementation of it. But that only holds if you treat a failure as a question about which of the two is wrong rather than an instruction to patch the code until the light goes green.
This is where the discipline earns its keep, and where it usually collapses. An agent will report a task complete on a spec it satisfied literally while the tests that would have caught the gap were never written. Spec green, code incomplete. It is the same failure I keep arriving at from different directions: output is not proof, and the last word on whether something is done belongs to a person.
Which is also why specs belong in the repository rather than in a ticket queue. A spec that drifts out of sync with the code it governs stops being a contract and becomes archaeology — the same argument for keeping instructions as versioned artefacts next to the code rather than as prompts someone pastes in.
When not to write one
The discipline has an inverse, and skipping it is not a moral failing.
A one-line bugfix, an exploratory spike, a change a single test closes — a full spec costs more than it returns. The heuristic is risk, not size: if you can write a quick test that proves the result, the scope is small enough to just move. Anything touching authorisation, payments, or data integrity gets a spec regardless of how small the diff looks, because those are the places where “precisely, exactly, verifiably wrong” is expensive.
What to take from it
- Run the two-implementers test before you send anything to an agent. One question, ten seconds, and it locates the gap the model would have filled silently.
- Write the proof before the implementation. For every acceptance criterion, name the test, log, or artefact that settles it. A criterion with no path to proof is not a criterion.
- Cover one error path and one boundary case, always. These are the two the happy-path description always omits and the model always invents.
- Treat a failing test as a question, not an instruction. It is telling you the code and the spec disagree. Decide which one is wrong before you change either.
- Audit the description before blaming the model. It is the cheapest experiment available and it is right more often than it feels like it should be.
The uncomfortable part is that none of this is new craft. It is requirements review and test design, both well understood for decades, both routinely skipped because the developer downstream would ask about anything that did not make sense. That reader has been replaced by something that never asks, and the discipline they were quietly providing now has to be written down.