How To Build Agents That Learn From Your Team

Robert Ta presents Clarity, an open-source harness that lets coding agents learn from your team's annotations. Agents ingest work artifacts like interview transcripts, extract story tickets, and improve when you add your perspective on who they're for and what matters. Multiple team members annotating runs builds a shared context layer that makes future agent executions smarter.
Chapters
If you're a product manager and you're thinking about how do I make sure as I use my agents, the next time I use the agent, it gets better and better. And if you're doing multiplayer, you're working with many other PMs, how do you make sure that every run that one of you executes with an agent, you annotate it and that compounds the multiplayer execution as well. So everybody's runs get better. What you're seeing here is the Clarity open source harness that we're going to be releasing soon. Soon for AI builders, AI product managers, AI engineers to use in your regular flow of work. You'll just have a skill/Clarity init harness, and as you use Cloud Code or Codex or Copilot, whatever, your agents register into this Clarity harness. So we have a concept of an agent registry, and your Cloud Code or whatever coding agent is going to spin up sessions. You're going to write stories. You're gonna upload transcripts and stuff like that. So in this case, I uploaded some transcripts from a particular interview with a customer and then it pulled out a sample artifact of a story. I could actually see the flow of work that this agent took. Had a tool call ingesting the source packet, which was a transcript. It extracted story tickets for learning and then a model call to update a story or to make a new story and then proposing a diff. And now it's waiting on me. And what I could do is I could be like, hey, the title of this, maybe I might say wrong strength. In context, I want to also focus more on hiring managers at medium enterprise. So I want it to be a bit more specific. And then that's gone ahead and marked there. And I have my annotation there. That annotation would capture from my profile. And that way it has my subjective perspective taking of what would make a better ticket. And that's captured my open coding. What I could then do is replay it. So it'll replay. And I could see it rerunning through and focusing now more on Medium Enterprise. That's one way we're thinking about how can we make it easy for an expert in the flow of work of managing agents and subagents, reviewing their runs, annotating them. So we connect the human to the agent run, and then that annotation goes into the curated context layer that then feeds better agent runs in the future and compounds and multiplayer context compounding. So you can imagine hundreds of product managers or hundreds of engineers annotating in the flow of work with whatever agent runs in their day-to-day work, and then a context layer on top aggregating the annotations from their expert perspectives to feed in-mass multiplayer compounded context and better runs.










