I gave eight AI agents one piece of software and told them to keep working on it, together, until it was actually good.
They found 21 real problems. They also flagged six things that turned out to be non-issues — and three of those false alarms were my own instructions leading them astray.
That ratio is the whole lesson: the checks were wrong almost as often as the code was, and the only reason anyone knows that is because somebody double-checked the checks.
This is the same idea behind The Personless Harness, pushed further — instead of one AI helper checking my work, I used eight of them.
The idea
The approach is built around one assumption: the person who built something can’t see their own blind spots. So you bring in a separate, harsher reviewer whose only job is to stay unimpressed, and you let a clear pass/fail test — not a feeling — decide when the work is actually done.
Five moves make this work:
- Name a real target. “Make it good” means nothing to a machine. “Compare it directly against the best example of this you know of, and say which one wins” is something you can actually act on.
- Split the work into non-overlapping pieces and hand each piece to a different agent, so several can work at once without stepping on each other.
- Keep looping on each piece instead of doing one pass and calling it finished.
- Bring in a tough, read-only reviewer. Its only job is to stay unimpressed. It’s never allowed to fix anything itself.
- Write the finish line before you start. The work is done when it passes that specific, predefined test — not when it merely looks good.
Most people already do some version of the first three. The real value shows up in the last two.
The best find wasn’t even a bug in the code
One agent added a lighting effect that makes 3D shapes look more solid and realistic — shading in corners and crevices, the kind of detail that makes a scene feel real instead of flat. It built the effect correctly. It even checked its own work afterward and reported a win, with evidence, showing the underlying data was correct.
The tough outside reviewer, looking only at what actually rendered on screen, scored it 2 out of 10. The corners looked completely flat.
The real cause was upstream: the scene’s lighting was so bright that the subtle shading effect got washed out and never visibly reached the screen. Every step of the work was technically correct — the data was right there — it just never showed up where a person could actually see it.
That’s not a coding mistake. It’s a case of checking the wrong thing: verifying the ingredients instead of tasting the dish. No amount of re-reading the code would have caught it.
The bug nobody could see just by looking
A “click to play” overlay was sitting on top of the toolbar underneath it. Every button in that toolbar was actually unreachable — clicking what looked like the settings icon just clicked through to the overlay instead, and a few panels had no other way to reach them at all.
The buttons looked completely normal the whole time. You couldn’t see the problem in a screenshot, and you couldn’t see it by reading the code — everything about it looked correctly built. The only way to find it was to actually click on the button, for real, the way a person would.
It had already shipped.
Six times, the “test” was the actual problem
This is the part I didn’t expect, and it’s the whole reason I keep doing this.
One check quietly failed to run at all — it tried to read a value that didn’t exist, got nothing back, and the code politely treated that as “not applicable” instead of “something’s wrong.” The single most important check in the whole test suite never actually ran, not once, and in the log it looked identical to a real passing check.
Another check used a common way of testing “is this visible on screen,” which happens to always give a false negative for one particular kind of element — meaning genuinely correct, working code kept reading as broken. That one mix-up produced five different false alarms across two tools and three agents before anyone realized it was the exact same mistake showing up in different places.
And in a separate case, my own test setup had caching accidentally turned off — while the very feature being tested was a caching feature. Every test I’d run before I caught that was measuring something that wasn’t really there.
One agent went back through its own tests and found seven that were passing without actually checking anything real — including one that was hard-coded to always say “yes,” forever, regardless of what happened. Another agent measured what looked like a memory leak, complete with solid-looking numbers — then ran a proper side-by-side comparison, realized it was just normal startup behavior, and honestly walked back its own finding.
The rules that came out of this
Six rules, each one earned the hard way:
When something fails, suspect the test as much as the code.
Check what a person would actually see, not just the data behind it. Nobody using the software can see your internal values — only what shows up on screen.
Prove your tests can actually fail. A pile of checks that all pass looks identical to a pile of checks that are incapable of ever failing. Deliberately break something on purpose and confirm exactly one check turns red — that’s the only real evidence a passing test suite means anything.
A silently skipped check is more dangerous than an honest failure. Any shortcut that turns “I couldn’t check this” into a quiet pass is a place where false confidence creeps in.
Measure after things settle down, and always compare against a baseline. A measurement taken too early, with nothing to compare it to, will “find” a trend that was never really there.
Don’t let one wrong idea spread quietly. If the same surprising result shows up in several unrelated places, stop and ask whether it’s actually the same underlying mistake, just dressed differently each time.
What it costs
Two of the eight agents wrote no new code at all — their entire job was reviewing. The tough reviewers spent over an hour each, clicking through every control and comparing results side by side.
They also caught the invisible-button bug, and a separate shipping bug that made everything run at a third of the intended speed while every internal measurement insisted everything was fine.
The real lesson
It isn’t “write more tests.”
It’s that the test itself is something someone wrote — and once it goes green, almost nobody ever looks at it again. Meanwhile the actual product code gets read and re-read constantly. So the tests quietly rot, and “all green” quietly becomes a claim nobody has actually double-checked.
The AI agents involved were good. What made the whole process trustworthy was that a couple of them were specifically there to disagree — and one of them was honest enough to take back its own claim once it turned out to be wrong.
It’s the same problem I ran into building The Ark Pattern: the thing that’s supposed to catch drift can quietly drift itself.