How AI Builders Should Verify Their Agent Actually Worked

For today's meta, we have a really cool PDF that co-founder Greg C. from Spec Story created, and him and 6 engineers built their product Stoa across over 1,300 captured agent sessions and 4,670 commits over 8 months from September 2025 to May 2026. Almost none of the code was written by hand, and they wrote a book on their 25 patterns in agentic Engineering: The Field Guide. This free 65-page book has 25 rules for building with AI agents, and every pattern is one you'll want to know as an AI builder. So let's get into it. I really like this. It was really well done. Uh, this is from Spec Story. We'll shout out the original LinkedIn post that I got in my feed where I read this. And they have really great visualizations like this, like you in a traditional copilot versus vibe coding versus agentic engineering. Just to visualize the differences in methodology. Really nice table of contents. And when you get into patterns here, there's like this part, verification is the job. When the agent writes the code, your irreducible job is to prove the system does what you meant against real runtime evidence. These four patterns are how you hold the line. So verification is the job is the very first one. And the four patterns are calibrated distrust, treat every agent claim verified, accurate tests pass as an unverified assertion. Until you have seen the check, turning your doubt to fire on confidence, not on content. Pattern 2: Keep a source of truth outside the agent's reach. Hold at least one ground truth source the agent did not author and cannot write, and check every claim against that, never against another agent output. Pattern 3: The premise auditor. Name your prime suspect and make the agent try to clear it. The disconfirmation, not the fix, is the deliverable. And for this one, this is stating your prime suspect explicitly, ask the agent to clear or convict it with the mechanism, not a yes. And when choosing between options, ask, should we do X or Y? And reward neither, refusing to pick an option you do not yet understand. And pattern 4 of job verification, the read-only turn. This is a specific move. Commissioning a diagnosis whose entire deliverable is a forensic report under a hard do not edit files fence. So you audit report, you can reopen instead of a diff, you have to trust. Jonathan, what do you think from your CTO perspective about the— maybe just start like holistically here on verification is the job, this statement here, and then the 4 patterns we just named.

Yeah. Uh, actually this page is what grabbed me when I looked at the PDF. I think this is key number one principle I've learned as an engineer, you know, my career, and like, how do you know something actually works? And now that you have agenting tools that can do so much, it really is just the job to like make sure that it actually did what it did. And now you're just steering the agent. I think they're coming from like really, really similar kind of like engineering philosophy. You know, verification is everything. I think figuring out how you're going to verify, and for instance, when in our PR lifecycle, we have this process where we like enumerate requirements up front and then we write tests that are gonna validate those requirements. And I see it as like, it's my job by the time I'm reviewing the PR, verify that the tests that we wrote actually fulfill the requirements that we set out. Like it's my job, I'm the gate, I gotta make sure that's the thing. And I, I don't have an LLM judge that can verify on my behalf. And so that's become where it like, where is my attention going? It's, it's like largely going towards verification of what the agent outputs. And so I, yeah, really love this and how they broke down the different ways that they do this in their own systems.

Yeah. Like in particular for this one, calibrated distrust where agentic development, when an agent says verified or tests pass type, how did we verify this? Very simple tactic is just don't trust that it said everything passed.

Yeah.

Here is one specific example. In February 2026, an agent says cost Helicones Reported total cost is accurate. Me, how did we verify this? Agent ThinkBlock, the user is rightly questioning my claim that Helicon's cost is accurate. I didn't actually verify it. I just assumed it. So they will lie to you. That's, that's the current meta is you got to verify. And actually one of the core principles we have in our company is trust but verify. So for sure, part of the job more than ever, I think because we could generate so much more than ever. Across many surface areas. And especially for the people that are, don't come from traditional engineering backgrounds where this hasn't been really inculcated into your day-to-day execution is to verify and make sure that you know the system's working as you want it. Really, really focus on verification. Very, very tactically, I guess I'll say is if you're using Cloud Code or Codex or anything like that, one of these agents with a harness to do any sort of code generation and build apps or platforms, very easy thing is go create a test strategy document that you read as the exit criteria that the agent has to uphold before it exits its autonomous goal loop. Like you feed it a spec, for example, like spec-driven development, but you make sure you read the test strategy and the test matrix of cases that you know to be the exit criteria. You confirm that, and that tells you the system is doing what you want it to do. But you have to like read that up front and make sure the agent is, um, honoring that. Just real quick pro tip.