kodyw.com

Kody Wildfeuer's blog

kodyw.com

When the Power Comes Back, the AI Estate Should Too

The power went out overnight, and it took down my whole home AI setup — several computers, each one in the middle of its own task.

Flipping the power back on didn’t actually fix anything. The machines came back, but I had no idea what each one had been doing, how far it had gotten, or what was safe to pick back up. A computer that turns back on isn’t the same as work that picks back up where it left off.

That’s the real problem with AI “helpers” running unattended: when something interrupts them, you can’t just reboot and hope. You need to know what each one was working on, what it had already finished, and what it’s safe to resume — because blindly repeating a step an AI already took (like sending a message or submitting a request) can cause a second problem instead of fixing the first one.

Think of it like coming home after a blackout

Imagine your dishwasher, DVR, and thermostat all lost power at once. The power comes back, but none of them remember what they were doing — the dishwasher doesn’t know if it finished the rinse cycle, the DVR doesn’t know if it recorded your show, and nobody wants to guess.

Now imagine dozens of AI assistants in that same spot at once, each one mid-task. That’s what I woke up to. The fix isn’t “turn it back on.” The fix is giving every task a paper trail — a record of what it was doing, what it had completed, and what it still needed — so recovery means picking up exactly where things left off, not guessing.

What “remembering” actually requires

To recover safely, you need answers to a few plain questions for every task in flight:

  • Which AI assistant was doing this, and as part of what job?
  • What had it actually finished, versus what it merely intended to do?
  • What survived the outage, and what needs to be checked before resuming?

That’s it. It’s less about fancier AI and more about good bookkeeping — a clear, trustworthy record that outlives the outage.

Why “just resume” is risky

If an assistant might have already sent a message or kicked off an action right before the power died, blindly re-running that step isn’t recovery — it’s a second, possibly duplicate action. The safe version of “pick up where you left off” always double-checks whether the last step actually happened before deciding what to do next.

This is the idea behind something I’m building called RAPP Disaster Recovery: a way to give an entire AI household a single, trustworthy view of what survived, what’s uncertain, and what’s safe to resume. It’s not finished — full automatic recovery across every device isn’t something I can promise yet — but it’s the direction I’m building toward.

The takeaway

The most useful thing an AI system can have isn’t more data or a longer memory. It’s a clear, honest record of what it was doing and what actually happened — so that when the power comes back, the work can come back with it.

If the idea of an AI helper that keeps working reliably on its own appeals to you, The Personless Harness goes deeper on what that actually looks like day to day.


This post describes an idea I’m actively building, not a finished product. If you want the deeper technical version of this, it lives on my GitHub.

The Personless Harness: Brainstem + Copilot

In 1895 you didn’t buy a car. You bought a horseless carriage — the new machine named for the animal it removed. The name is the whole lesson: the carriage didn’t change. The wheels, the seats, the cab — all still there. What changed was what pulled it. The horse went out, the engine went in, and the same carriage could suddenly run at a pace no stable could feed.

Here’s the same swap, happening right now, in software.

Copilot is the carriage. It’s the vehicle everyone already has — the chat window, the buttons, the place work actually happens. A genuinely good carriage.

You are the horse. Today, almost everything an AI assistant does, it does because a person is pulling: typing the prompt, clicking the button, checking the output, deciding what happens next, then pulling again. That works — beautifully, at the pace of one tired human — and it stops the moment that human needs to sleep, eat, or go to a meeting.

The Brainstem is the engine. Think of it as a second, tireless version of you: it knows your tools, remembers how you like things done, and can keep working after you’ve clocked out. It isn’t a smarter chat window — it’s the thing that can sit where you sit and keep going. Same carriage, different thing doing the pulling.

What the swap looks like in practice

This isn’t really a story about software testing. It’s a story about a new way of working — testing just happened to be the first job the engine took from me, because testing is the purest form of “a human clicks through an app after every change to see what broke.”

I had just finished a feature I believed was done. Instead of opening the app myself, clicking the toggle, and watching what happened, I handed the job to an AI agent with a real browser and a checklist, and went to work on something else.

It came back with five problems. One was serious: the page was loading with zero working code behind it, so the whole feature was just a picture of itself. Another: a switch that could be turned on but never back off. A third: a checkbox that had quietly grown too wide and shoved its own label onto three lines. None of that showed up in my usual automated tests, because it all lived in the gap between “the code looks right” and “the thing actually works when a hand touches it.” Until that day, finding it had always meant me sitting there clicking around.

I fixed the five problems, sent the AI agent back in, and it came back clean: thirty-three checks, all passing, no human involved. My status update changed from “try it and tell me what breaks” to “it’s done, and here’s the proof.”

What makes an AI helper trustworthy at this job

A few things separate a helper you can actually trust from one that just looks impressive in a demo:

  1. It uses the real thing. Real clicks, real pop-ups, real downloads — not a shortcut that skips the part a real person would experience.
  2. It expects the annoying stuff. An unexpected confirmation pop-up is a failure to catch, not something to shrug off. And layout gets measured precisely, not eyeballed.
  3. It checks for things that should be missing, too. A feature that’s supposed to stay hidden until turned on, a panel that’s supposed to disappear — humans almost never double-check that something properly vanished. An AI helper can, every time.
  4. It never fakes success. If it couldn’t actually run the check, it says so loudly. A “looks fine” that never really ran is worse than an honest “this failed.”

How much faster, really?

To put a number on it: that 33-check run, done by hand by someone who knows the app well, takes roughly twenty minutes of full attention — and you have to redo the whole pass after every fix. The AI agent finished the same 33 checks in about 19 seconds. That’s roughly sixty times faster on the stopwatch. But the bigger difference is what it costs you: twenty minutes of a person’s full focus, versus zero. I ran it four times in one evening without noticing the cost at all.

The carriage stays; the horse retires

Notice what this swap does not ask of you. You don’t throw out the tools you already use. Copilot, the app, everything you’ve already built — all of it stays. That’s why the horseless carriage caught on so fast: it didn’t ask the world to reinvent the wheel, just to stop feeding the horse.

And notice where the person goes. The engine didn’t remove people from travel — it moved them from pulling to steering. This doesn’t remove you from the work; it moves you to the seat where you decide where things go and read the results. You stop being the horse. You were never at your best there anyway.

Curious how far this can go? The Buzzsaw pushes the same idea further — instead of one AI helper checking the work, I used eight of them, arguing with each other, to catch things a single reviewer never would.

For a while, this will feel like a strange new way to work. Then, like the horseless carriage before it, it’ll just be how work is done — and the idea that a person once had to manually click through every check will sound as old-fashioned as keeping a horse for your commute.

The Ark Pattern

Skills drift.

You teach an AI assistant how to do something, it works, you move on. Six months later, that same “skill” gives you a different answer — because someone edited a copy, or a dependency shifted underneath it, or four different people each tweaked their own version and never told each other. The name of the skill stays the same. What it actually does does not.

That’s not hypothetical. I watched one bad instruction produce five separate wrong answers, across different tools and different AI agents, before anyone realized it was the same mistake wearing different clothes each time. Nothing announced it. Every copy looked fine on its own.

The uncomfortable part: if AI assistants everywhere start relying on shared “skills” like this, quiet drift is the real risk — not some sci-fi AI takeover, just a slow loss of agreement about what a given instruction actually does, with no easy way to check.

So I wanted a way to catch it. A backup that works even if everything around it gets messy.

The idea

My approach treats a capability as one portable file with a clear job — drop it in a folder, it runs, no setup required.

This pattern goes one step further: it puts the working code inside the plain-English instructions themselves, so the document explaining what a skill does is also, literally, the thing that does it. It reads like a normal explanation to a person, and it can also run directly as working code. There’s no second, separate copy sitting somewhere else that could quietly drift out of sync — because there’s only one copy, period.

The real benefit isn’t for me. It’s that anyone can pick up that one file, with no special setup on their end, and get the exact same behavior I do.

That’s the claim. A claim is worth nothing until you test it, so I tested it.

Putting it to the test

I built the same capability two ways: the normal way (spread across several files) and the “one file” way described above. Then I ran both versions dozens of times, side by side, and compared the results byte for byte.

Every single time, both versions produced identical output. Not “close enough” — identical.

Then I deliberately broke it: I edited the normal, spread-out version in place, the way an innocent well-meaning edit usually happens. Its output changed, as expected. The single-file version’s output did not change at all — and, importantly, the mismatch between the two was easy to spot immediately, instead of surfacing months later when two AI agents quietly disagree about what a rule says.

That’s the whole value, proven rather than assumed: one copy stayed trustworthy, and when the other one drifted, it was obvious right away.

What it costs

Honestly, some convenience. A single file can get long, and you lose the ability to test small pieces of it separately. Mine landed at a comfortable size — ten times longer would start to hurt.

I also hit a real bug while building this, worth naming because it’s exactly the kind of thing that quietly erodes trust: embedding code inside a plain-text document accidentally mangled a couple of special characters, so two of five commands printed a broken symbol instead of a proper line break. The other three were fine, because they happened to be built slightly differently.

A quick spot-check of just one command would have missed it and shipped it broken. Only checking every single case caught it — which is the same lesson every time: the check itself can be wrong too, and nobody double-checks the checker.

The pattern, stated plainly

Ship a capability as one self-contained, readable document. If the surrounding ecosystem holds up over time, you’ve lost nothing. If it drifts, you have one clearly labeled, trustworthy copy that makes the drift obvious instead of invisible.

It’s a cheap insurance policy against a quiet, slow-moving risk — and so far, it’s the best way I’ve found to share something’s strengths with people who have no interest in adopting my whole system, just the one useful piece.

It’s the same lesson at the heart of The Buzzsaw: the thing meant to keep you honest can quietly become the thing you stop questioning.

Update: it holds up beyond one example

One capability working this way could be a fluke. So I automated the conversion process — turning a normal, spread-out capability into the “one file” version automatically, and refusing to finish if even a single byte comes out different — and tried it on two more capabilities.

All three matched perfectly, every time, without needing anyone to hand-check the conversion. That the conversion itself is automatic matters more than any individual result — if a person had to convert each one by hand, that manual step would become a brand-new place for drift to sneak back in, which defeats the whole point.

One test along the way even reported a false alarm — the two versions looked “different” only because of an unrelated quirk in how I’d named the test files, not because the actual code differed. Worth mentioning honestly: that’s now the eighth time in this stretch of work that a reported failure turned out to be a flaw in the test itself, not the thing being tested. I’ve started treating that as the normal rate, not a fluke — which is exactly why checking your checks matters as much as checking your code.


Every number in this post comes from a script anyone could re-run, and it fails loudly if the equivalence claim doesn’t hold. If it had failed, this post would say so.

The Buzzsaw

I gave eight AI agents one piece of software and told them to keep working on it, together, until it was actually good.

They found 21 real problems. They also flagged six things that turned out to be non-issues — and three of those false alarms were my own instructions leading them astray.

That ratio is the whole lesson: the checks were wrong almost as often as the code was, and the only reason anyone knows that is because somebody double-checked the checks.

This is the same idea behind The Personless Harness, pushed further — instead of one AI helper checking my work, I used eight of them.

The idea

The approach is built around one assumption: the person who built something can’t see their own blind spots. So you bring in a separate, harsher reviewer whose only job is to stay unimpressed, and you let a clear pass/fail test — not a feeling — decide when the work is actually done.

Five moves make this work:

  1. Name a real target. “Make it good” means nothing to a machine. “Compare it directly against the best example of this you know of, and say which one wins” is something you can actually act on.
  2. Split the work into non-overlapping pieces and hand each piece to a different agent, so several can work at once without stepping on each other.
  3. Keep looping on each piece instead of doing one pass and calling it finished.
  4. Bring in a tough, read-only reviewer. Its only job is to stay unimpressed. It’s never allowed to fix anything itself.
  5. Write the finish line before you start. The work is done when it passes that specific, predefined test — not when it merely looks good.

Most people already do some version of the first three. The real value shows up in the last two.

The best find wasn’t even a bug in the code

One agent added a lighting effect that makes 3D shapes look more solid and realistic — shading in corners and crevices, the kind of detail that makes a scene feel real instead of flat. It built the effect correctly. It even checked its own work afterward and reported a win, with evidence, showing the underlying data was correct.

The tough outside reviewer, looking only at what actually rendered on screen, scored it 2 out of 10. The corners looked completely flat.

The real cause was upstream: the scene’s lighting was so bright that the subtle shading effect got washed out and never visibly reached the screen. Every step of the work was technically correct — the data was right there — it just never showed up where a person could actually see it.

That’s not a coding mistake. It’s a case of checking the wrong thing: verifying the ingredients instead of tasting the dish. No amount of re-reading the code would have caught it.

The bug nobody could see just by looking

A “click to play” overlay was sitting on top of the toolbar underneath it. Every button in that toolbar was actually unreachable — clicking what looked like the settings icon just clicked through to the overlay instead, and a few panels had no other way to reach them at all.

The buttons looked completely normal the whole time. You couldn’t see the problem in a screenshot, and you couldn’t see it by reading the code — everything about it looked correctly built. The only way to find it was to actually click on the button, for real, the way a person would.

It had already shipped.

Six times, the “test” was the actual problem

This is the part I didn’t expect, and it’s the whole reason I keep doing this.

One check quietly failed to run at all — it tried to read a value that didn’t exist, got nothing back, and the code politely treated that as “not applicable” instead of “something’s wrong.” The single most important check in the whole test suite never actually ran, not once, and in the log it looked identical to a real passing check.

Another check used a common way of testing “is this visible on screen,” which happens to always give a false negative for one particular kind of element — meaning genuinely correct, working code kept reading as broken. That one mix-up produced five different false alarms across two tools and three agents before anyone realized it was the exact same mistake showing up in different places.

And in a separate case, my own test setup had caching accidentally turned off — while the very feature being tested was a caching feature. Every test I’d run before I caught that was measuring something that wasn’t really there.

One agent went back through its own tests and found seven that were passing without actually checking anything real — including one that was hard-coded to always say “yes,” forever, regardless of what happened. Another agent measured what looked like a memory leak, complete with solid-looking numbers — then ran a proper side-by-side comparison, realized it was just normal startup behavior, and honestly walked back its own finding.

The rules that came out of this

Six rules, each one earned the hard way:

When something fails, suspect the test as much as the code.

Check what a person would actually see, not just the data behind it. Nobody using the software can see your internal values — only what shows up on screen.

Prove your tests can actually fail. A pile of checks that all pass looks identical to a pile of checks that are incapable of ever failing. Deliberately break something on purpose and confirm exactly one check turns red — that’s the only real evidence a passing test suite means anything.

A silently skipped check is more dangerous than an honest failure. Any shortcut that turns “I couldn’t check this” into a quiet pass is a place where false confidence creeps in.

Measure after things settle down, and always compare against a baseline. A measurement taken too early, with nothing to compare it to, will “find” a trend that was never really there.

Don’t let one wrong idea spread quietly. If the same surprising result shows up in several unrelated places, stop and ask whether it’s actually the same underlying mistake, just dressed differently each time.

What it costs

Two of the eight agents wrote no new code at all — their entire job was reviewing. The tough reviewers spent over an hour each, clicking through every control and comparing results side by side.

They also caught the invisible-button bug, and a separate shipping bug that made everything run at a third of the intended speed while every internal measurement insisted everything was fine.

The real lesson

It isn’t “write more tests.”

It’s that the test itself is something someone wrote — and once it goes green, almost nobody ever looks at it again. Meanwhile the actual product code gets read and re-read constantly. So the tests quietly rot, and “all green” quietly becomes a claim nobody has actually double-checked.

The AI agents involved were good. What made the whole process trustworthy was that a couple of them were specifically there to disagree — and one of them was honest enough to take back its own claim once it turned out to be wrong.

It’s the same problem I ran into building The Ark Pattern: the thing that’s supposed to catch drift can quietly drift itself.

The Company That Doesn’t Exist

Every “let’s use AI at work” project I’ve ever seen stalls in the same place: waiting for access to real company data. Everyone agrees the idea is good, and then the whole team sits around for six weeks while security reviews who’s allowed to touch what.

So we stopped waiting. We built a fake company instead.

The idea

We created a pretend business — with pretend employees, pretend systems, and pretend documents — that behaves exactly like a real company’s software would, down to the tiniest technical detail. To any AI tool trying to use it, it’s indistinguishable from the real thing. But because none of it is real, there’s nothing to leak, and nothing to wait weeks for permission to touch.

This week we added two new pieces to that pretend company:

  • A stand-in for the kind of big enterprise software (like SAP) that runs company finances and operations — with fake plants, fake business units, and fake company codes, all behaving exactly like the real software would.
  • A stand-in for a company’s internal document library — a project charter, a team directory with who owns what, an escalation plan for when something breaks, and a rollout plan.

The company is called Meridian Manufacturing Group. The project is called Project Phoenix. Neither exists. Every name, department, and detail in it is made up — which is exactly why we’re able to share all of it freely, with no approval process required.

What we built on top of it

Using that fake company as the foundation, we built the kind of AI assistant every large project eventually asks for — one you can ask things like: “Who’s responsible for this part of the project?” or “I’ve hit a blocker — who do I escalate this to, and who’s their backup?”

It reads through all the documents, understands how they relate to each other, and answers questions by citing exactly where it found the answer — which document, which section, and a link back to it. Ask it an escalation question, and it correctly pulls from two different documents to give you one clear answer, with every fact traceable back to its source.

Here’s the part that actually matters: the same AI assistant can be pointed at a real company’s real systems with zero changes to how it works — just a different address to look at. The pretend version isn’t a mockup of the real thing. It’s a full working rehearsal for it. Going from “demo” to “actually useful at a real company” is just a matter of pointing it at the real data once you have permission.

Why build it this way?

Because it turns the hardest part of an AI project — getting safe, realistic data to build against — into something anyone can just download and use:

  • Anyone can copy it and make their own version — swap in your own industry’s details in an afternoon.
  • Every version is saved and traceable — nothing about a demo mysteriously changes or disappears.
  • It costs nothing to keep running — there’s no server to maintain, patch, or pay for.
  • It’s inherently safe — because it’s fake from the ground up, there’s nothing sensitive to protect, so security sign-off takes minutes instead of weeks.

The fun part

Once this is connected into a company’s everyday chat tools (like Microsoft Teams), updating the AI assistant’s knowledge takes effect instantly — no waiting for a technical redeployment. You can tweak how it answers and watch it change in real time, in front of the actual people who’ll use it. That immediate feedback loop is what makes the whole thing worth doing.

Build a pretend company, point an AI assistant at it, and stop waiting on permission to start proving an idea works.

That same instinct — build something that can just get to work instead of waiting around for a person — is what led to The Personless Harness.

Everything described here is public and freely available. All the company data is entirely made up — any resemblance to a real company is coincidental.

Page 1 of 10

Powered by WordPress & Theme by Anders Norén