AI Isn't Killing PMs, It's Rebuilding Them

Is the traditional product management role dead? And if it is, what is it becoming? Welcome back to Self-Aligned. I'm Robert Ta, CEO and co-founder of Clarity by Epistemic.me, joined by my co-founder and CTO Jonathan McCoy. In this podcast, we give you a weekly AI clarity— this is the series on just signal from noise roundup from the past week. In this episode, we're gonna specifically focus on the topic of the traditional software development lifecycle, the crucial role that product management has as a function has played in the past, and how that is evolving with thought leadership from none other than Boris Cherny from the Cloud Code Team for Anthropic. So let's get right into it. As engineering, product, design, DS, etc. melt into a new kind of role, I was reflecting on what roles might look like in the future. Boris says, for example, when I look at the Cloud Code team, I see what I think is 5 archetypes. And if you're a head of product or head of engineering or staff IC, developer, PM, even on the design side, you are thinking about how does the traditional SDLC evolve? And I think this is such a great way to put it. 5 archetypes from the creator of Cloud Code. Number 1, the prototyper. This is a person that comes up with brand new ideas, churns out many ideas, most of which don't ship. I think that's a key thing. Most of which don't ship. This is just innovation idea factory. Number 2, builder quickly turns a prototype or idea into production-grade product or infra. So that's important delineation there. Number 3, sweeper. This is a person that cleans up the UI, the front end, simplifies the code and the system, unships, optimizes performance. So cutting down and paring down to refine the experience on both sides. Number 4, the grower. This person takes a product that has been built and iterates on it to improve product-market fit. You could make a product, but you also need to grow it and monetize it and get users and retain them and get them engaged and all those things. And so that, that's this person's responsibility. Number 5 archetype, the maintainer. This person owns a mature system to make it secure, reliable, fast, and efficient as it scales. So Boris says that many people span across 2 roles and sometimes 3 roles. He says that he notices that these roles are not really tied to job function. And at Anthropic, one of the leading labs out there pioneering AI-native workflows. At Anthropic, some designers match category 1, some 2, some 3. Same for engineers, PMs, and DS. A healthy team needs a mix of these depending on the product. And some considerations: product that is new or pre-PMF needs people that are strong at categories 1, 2, and 3, so prototyper, builder, sweeper. A product that is growing and has some PMF needs 2, 3, and 4, so builder, sweeper, grower. A product that has strong PMF needs 3, 4, and 5 and some 2. So that's sweeper, grower, maintainer, and some builder. And he's saying maybe product roles of the future will look more like this and less like the domain-specific roles of today. So we have enterprise clients, they're trying to figure this out. I don't think anybody's figured it out. This is a great piece of thought leadership. What are your thoughts from a CTO perspective on the evolution of the product management role and software development lifecycle, Jonathan?

Yeah, it's a really interesting week because in addition to this post from Boris, there was Lenny's podcast with guy, what's his name? Andrew Ambrosino. And he described a similar type of role dissolution and similar kind of like mechanics of these product management teams getting rebuilt, the product teams getting rebuilt. So, you know, like, this is the frontier labs. They're the ones probably using AI the most and the soonest. And this is like an interesting signal about how the reshaping is happening. We're trying to figure this out on our own team, like how, how exactly are the AI tools augmenting and changing our, our work and our roles and how do we work together on that. We have customers who we're talking to about how their product teams are evolving. And I was seeing kind of a nice lens for, like, since making this information. And I think it also overlaps with this narrative, probably a mistaken one, that AI is an automation tool that will replace jobs. And, you know, like, I think that the product team is like on the front edge of this and it shows what the technology meeting the roles actually looks like. And so I did a little bit of an analysis about how this is evolving over time.

Which is super key that there has been a narrative in industry that, hey, maybe fire all your product managers, fire your designers, right? Like fire your engineers, like you could do way more with less people. And actually I think the evidence we're seeing is that product managers no longer have to do mundane tasks like be backlog junkies. That was part of the— that was just an assumed— nobody gives you credit for it. You just had to do it to get the job done of getting good product out the door. Because if nobody's doing it, not the rigorous program management, project management, dependency graph, identifying the acceptance criteria, figuring out the right things to build, like if people aren't doing that stuff, it's like you haven't tied your shoes to play basketball, right? That's like what Coach Wooden from UCLA would say. You haven't done the basics. But now AI helps you do the fundamentals easier. So guess what? Your percentage of thinking capacity frees up enormously to do these things that Boris talks about, to be a prototyper, to be a builder, to be a sweeper, to be a grower, to be a maintainer. And you go to Claude Goode and just ask it how to do these things. And you know, like that, that is the empowerment now. So I always think that the— and this is, this is what the best entrepreneurs and builders say is accelerating your learning and taking that to drive faster, better decisions. That is the input of making great products. Work every stage. So, and ultimately the PM function traditionally has been the de facto decision maker. If there are many stakeholders, they're the ones that has to decide, we're gonna go this way, right? And so the thinking is that person can now decide at a much faster rate because they could experiment and test hypotheses at a rate that could not be matched before. So I just wanted to give that take as well before we get into this super sick artifact you gave.

Yeah, I think we're living through this. You're describing what it's like to live through this and like how exactly should we adopt it? It. Yeah, and I wanted to kind of like put a lens on it. Uh, I think there's, there's a newer version of this that I linked to the actual, um, economics paper. Uh, yeah, so you can see it's adopted from, uh, Asimoglu, Autor, and Johnson.

That points to an economic, uh, paper, um, Building Pro-Worker Artificial Intelligence by Doron Asimoglu, David Autor, Simon Johnson. Hopefully I pronounced that right. This paper defines pro-worker technologies, including artificial intelligence, as technologies that make human skills and expertise more valuable by expanding worker capabilities. Our conceptual framework distinguishes amongst 5 categories of technological change: labor augmenting, capital augmenting, automating, expertise leveling, and new task creating. Only the last category is unambiguously pro-worker, generating demand for novel human expertise rather than commodifying it. So yeah, do you want to go over this, that paper at the intersection of this new narrative popping up around the product role?

I heard about this paper earlier this year when it was released, and I thought it was a good lens to look at that like, let's get to more nuance than just like, is AI going to take our jobs? Is AI going to automate everything? And I think that was, that was actually the hypothesis that OpenAI and Anthropic maybe considered core, what they were doing, like that, that's what AI would do. But I think that the paper gives a more nuanced lens and it says every technology that comes out has these kind of like different ways that it can augment labor, augment capital, it can create new tasks. And so, um, let's take a look at what's actually going on through that lens. And what this artifact does, it shows the baseline, like what did product teams look like before AI? And then if you click on the Copilot era, we start to see the ways that AI is impacting the tasks that we do in these jobs. And this is by the way, like a task-level analysis. So we're saying for each of these roles, what is the task that's being done? And then the key there, yeah, gray is human judgment is retained. There's augmented technology with AI is the purple, automated is the red, and then new task is green. And so this is the Copilot era. We all remember that, you know, and you can see kind of like the different roles in the product organization. The Harness era, uh, is the one where we got Cloud Code and we got the early versions of Codex, you know, and obviously that's a step function. And all of a sudden you're seeing, oh, I've got more time and more capability to do new tasks. And if you look at like the product manager, uh, prototypes over PRDs and file design, you know, like these were things that, uh, we maybe didn't have to do before we had so much like power through the AI tools, but now we can, we can allocate additional bandwidth in our work and this is value-adding activity. So what's being automated, like, right? The engineer first-pass review, right? So I think looking at pull requests and saying like, is this code good? Like nobody has to do that the first time anymore. You can, you can ask Claude Code, is this ready for my review? You know, it can do the first couple of reviews itself. So those are the kind of things that you're seeing being automated through this, the harness era. But then like there's the Agentic shift. This is what we all really felt when Cloud Code Plus Open 4.5 came out in December. And this is what the agentic AI sort of like built for us. And we start to map here to Boris's framework where it says recomposes towards. So this is actually like now the AI is so impactful in our job that it's causing job dissolution or role dissolution on the teams. The designers can code, right? Um, the, the junior engineers can code better.

The product managers could research like product managers could prototype, product managers could ship to production, like. It's a different world.

Yeah. And so that empowers like a different shape of the job. And I think this is just an interesting way to look at it, is to take these different like, uh, task workflow vectors. How did technology— is it still a human, uh, you know, judgment retained? Is it augmented? Is it automated? Is it new task? You can see the nature of change over time. And what you see is a mix of things, right? It's not just things being automated. It's that things get automated and the new tasks arrive that we can add more value, right? So there's no like sort of like upper limit on the amount of value that a human could add with these tasks, given the capacity to do more tasks, right? We're not seeing that like ceiling yet. And so I think that's the exciting thing is that, um, you know, like we're, we can augment ability and we can create new tasks. And that's how like these product teams are, are probably gonna continue to evolve. So I thought this really helpful lens, and I think we can take a look at the transition here. Okay. So if you're a product manager, now you're, you're recomposing towards a prototyper slash grower, right? Those are the like your, your job augmentation is kind of like making you most fit for those type of roles. The engineer is going towards the builder, sweeper, maintainer. The designer is towards the prototype or sweeper. Data scientist is towards a grower and engineering manager towards the maintainer sweeper. This is kind of like a conjecture or brainstorm, but it's taking the traditional roles, saying how they're changed and then how they're going to meet, you know, this like kind of world Boris is seeing on the front lines of the frontier models. And then the last section is of course, like, what does the world look like in the way that Boris described it, uh, in the recomposition era? And so now you get, like, you know, these, these different kind of, like, role divisions, um, and new capabilities people are capable— augmented capabilities people are capable of doing. And I think we're seeing this in our own work too, right? Like, where the role of prototyper and builder are definitely, like, bifurcating in ways that we need, like, more nuanced language to say, like, you you know, which, which state is our product at? And our latest agents, you know, like kind of orientate around the artifacts that each of these roles create. You know, as in like we actually have like a literal, we have an agent that literally shapes requirements, right? Literally shapes story tickets and those artifact driven sort of tasks that, that are being performed kind of reshape the, the nature of the work in the product team. And just one more like data point I wanted to show you. There's some headlines on the right-hand side if you scroll down and these are a collation of like what are, how are companies describing their workforce decisions about AI? And like, I think Klarna is a really interesting one because earlier, like in 2024, they said, guys, automation is here. We're laying off all the customer service people. You know, we got this with the 2024 AI, which they, they since have come back a couple of times and like restated their position about like, oh, we actually need customer service people for the more complex tasks. And, and now even like now, now they're kind of like even using the language of recomposition and new task skills. Um, and you're seeing this across different announcements. So I thought this was cool. And the artifact, as you go through it, it kind of tells us the changing narrative about like announcements that companies are making given decisions to like manage talent through this technology adoption. So it's super interesting. And so the other end of the bar, like one is like there's definitely like narratives coming through from Anthropic and OpenAI about product team recomposition. On the other side, there's this debate that is kind of low nuanced, is AI taking jobs? And I think both of these are converging into like, no, let's just use this task augmentation lens. And let's understand how the roles are changing. Where are we going to see automation? Where are we going to see new task creation? Um, that's the productivity vector that, like, we should be talking about. And sort of like, you know, the All In podcast, they have this, like, weekly debate about, is AI taking jobs or not? And they're sort of like all pulling data points that we just talked about, but using it to support, like, a yes or no position when the position is actually more nuanced. It is, you know, like, what is the composition of task skills and how is AI meeting each one of those? And I think that's what we want to be doing in organizations is like using that more nuanced lens.

No, that was really good, Jonathan. And to riff on that a little bit more, we're going to be riffing on this a lot for the pod audience. Cause this is, this is the talk. If you're a head of product or a product manager or an engineer, engineering manager, head of engineering, head of design, designer, like if you are in software development, you're on tech team, you're shipping product, you need to be thinking about these things because you're probably doing these things like parallel prototyping, idea generation, killing things like, you know, unshipping, uh, production hardening, fleet orchestrate. You're doing these things and you probably are just sort of doing it, and now you've drifted off into exercising skills and capabilities to meet the new demand of the capacity of work, uh, of being able to serve your, your roadmap and your customers. Because you have more capacity, people could do more, and you've probably drifted away from what the career framework looks like at your company, right? Like, unless they updated it and are thinking about this new language to describe the capabilities and skill sets in these— it's not even like roles anymore. Like, if we think about the archetypes, it's kind of just expressions of the roles in different scenarios to meet the demand of whatever is needed to keep moving the supply of we are baking the bread that is a better software product. So what I mean by that is, right, like the whole goal before was built around this fundamental assumption that engineering time is very expensive. So we want to measure many times and cut once. Now code generation is practically free compared to what it was a year and a half ago, 2 years ago, right? 3 years ago. And so code generation we're now finding is a way to prototype. You could code gen something and you're no longer seeing that as like, oh, that was a super expensive thing. You could generate code to build UI that you have no actual commitment to bring to production just to get something out there like this artifact that me and Jonathan are talking to, to align on an idea. To then get more experimentation and more hypotheses out to get more stakeholder alignment and more learning, right? So the whole job of a product team from product management to designer to engineer is to ship world-class products to capture market share, to build products that ultimately you can compound and create value in the world. And, uh, that creates economic value for the business, right? So it's the intersection of those things. Customer value is the product useful and innovating, creating value in the world. And, and so that should meet your bottom line for the business. That is the goal. So what are the things in between that goal? That has shifted, right? And it's really interesting to see like some companies are being like, ah, actually we're totally wrong about this. Like there's more demand now and now we can have our people serve higher level valuable decision making versus like lower level things like being a backlog junkie or, you know, doing the code review first time for a pull request. It's like, no, bring me in for the second time, do an adversarial code review, like save the valuable human judgment. For further along in the higher leverage areas of baking the bread. There is another thing I'll say there, which is really interesting. When we take a step back around technology lifecycle adoption, right? Every new technology that has ever been introduced is disruptive. Not gonna lie, this is disruptive. It's like Uber disruptive to taxi drivers, right? Like Airbnb disruptive to lots of local communities and like creating lots of change in like tourism and supply and demand around those markets. And it all goes back to like the telescope, the steam engine, like all these things. And one story, I wanted to share. I just read this, so I'm gonna, I'm gonna read it on the pod. I thought it was really interesting and very apt. During World War II, significant advancements were made in the engineering of hardware that could process information faster than humans. One of these machines was the ENIAC, ENIAC, the Electronic Numerical Integrator and Computer, located at the University of Pennsylvania's Moore School of Electrical Engineering. It was a giant machine made up of about 18,000 vacuum tubes, each with switches to communicate electrical pulses and signals. Uh, I thought this was such a cool story that explained a parallel in event happening, which is the event of software programming. And this is what it looked like back then, where women were the first computer programmers. And I think that's such a cool story because of all the, you know, discrimination and biases that they face as an uphill battle in the field today in STEM. But women were the first programmers and they were doing this by hand. And the story goes that these women effectively became some of the first computer programmers in the world. It was not an easy task, partly because the concept itself was so new. Programming, the original team learned, would not be a desk job. The Women would stand inside the ENIAC to plug in each problem, stringing the units together in sequences using hundreds of cables and some 3,000 switches. There were no instructions to read, no courses to take. So those original women that were operating this in World War II wrote the instructions by figuring out how to use the system and developed the courses for the people that would come after. And so their work was part of the early efforts that led to role specialization where software programming became divorced from hardware engineering. And so there, just from this alone, this dissolution of roles that we're sort of seeing now in this era, this happened before. This has happened every time and there's ever been a new technology. When you dig into the story, there were people that had a job and then a new technology or tool came out and their job kind of changed. And guess what? The frontline people that played that arbitrage to figure out the new emerging skillset and capabilities to plug into the economically valuable workflows or the work to be done, uh, those people defined what the roles look like. And I'm starting to see that, like, the people coming on Leti's podcast, they're the ones that are acting as these 5 archetypes, or, or many of them in combination across the traditional software lifecycle that's evolved to an AI development lifecycle. And they're just like the, the earliest, the first computer programmers, these women that have paved the way for what does software engineering look like. So the story of the early female computer programmers is valuable to consider through the lens of job specialization. And, um, we could see that it basically became from this, what you see on the screen, to now we're talking cloud code. And it's generating spec. We don't have to manually move around switches and stuff like that. There's no electrical engineering I have to think about. That's very abstract. And that all came from, if you think about the evolution of how valuable decision-making, where do I put my critical capacity, is because these people pioneered that, built abstractions and built abstractions and built abstractions from assembly, from here all the way to like C++ all the way to Swift or React Native or whatever. And now it's like, you don't even have to know those things. You just talk to Cloud Code and it knows those things, right? So if you went to these people back then and you told them, hey, you're going to be out of job, but you could do like 100x more. So it wouldn't be out of a job. It would be like, the job has just changed. That's it. Like you're processing data and you're processing information. That's what you're doing. And hey, we gave you a tool that does that millions of times faster. Do you think they'd be like, like, nah, fuck it, I don't want that. No, like, you're still getting the job done. So anyway, I just wanted to side quest that little story. I read it recently, uh, give a citation. It's from this book, The Great Mental Models by Shane Parrish. It's about economics and art. And all we're really seeing from a systematic lens is there is demand for valuable software that solves economically valuable workflows. That's true. There traditionally has not been enough engineering to do that across, you know, the limited constraint of talent we have in the world to meet that demand. What we're seeing now is the supply has gone up, more people could do these things, and the demand I think is meeting it. And I think that's, that's what really what we're seeing here in these headlines changing, um, where the role recomps are happening because ultimately people will still pay for good software. Now, whether or not that's going to be for everything, I don't know, but the demand is certainly there right now. And we're seeing that in these, uh, role recompositions. Anyway, just want to share that. I thought it was such a cool story I read last night on the plane back from the acquisition.com workshop.

So that, that dynamic, like, you know, you have a, a job and your, your job is to, you know, kind of plug in cables to vacuum tubes that are making calculations. That's, uh, that's a role that existed for some amount of time. But so as, as the laborer, as the person who has the job, like, you need reskilling then to co-evolve with the technology. Um, as the people that are building the systems, um, you know, like the, the engineers, the architects, the, the, the business decision makers, right? Like, you're just looking for more efficiency, more cost optimization. And so I think, you know, it's kind of like technology's form of of, uh, labor and capital having tension with one another. And I think like the ability to reskill quickly takes away the anxiety from labor, is like, okay, wherever the technology goes, I'll have a new job. But I think one of the problems is that there hasn't really been a lot of precedent of fast reskilling and showing people like how they can go down a career path and then follow along with the technology. And I think that is one of the fears. And even if there's recomposition here, um, you know, like you see companies like, uh, like Block and Cloudflare that are taking the opportunity during the recomposition to lay people off and to say like, you know, we don't need these analysts anymore. We don't need these middle managers anymore. And in theory, those analysts and middle managers can be reskilled to use the AI in other contexts. In practicality, there's fear. We don't know exactly what that looks like. And it's like, you know, going to create some uncertainty about like how you should drive your career. So I made sure to like analyze these through the lens on that, on the artifact. Because I think, yeah, you can see there's some really credible CEOs saying like, look, we have automation now. In some cases, like in Block, people are saying no, actually, like Block just overhired during the pandemic and they just use this as an opportunity to kind of clean things up and they have a vision for automation, but like it's not really in the data. So the job replacement or job loss story isn't like really well supported in, in the data yet, you know, but like we're just at the beginning of, of this story. Um, and what we see more of is this recomposition, right? Which I think is the exciting thing. And, uh, towards the whole history of technology, like, in— and we just need ways to imagine how job evolution follows the recomposition, you know, and how, like, creates a better future for everybody.

Totally agreed. I think for the product managers, engineers, designers out there that are fearful for your jobs, look at this as there is evidence that if you lean into it, you will be defining the jobs and the roles of this new era. That's the way I see it, is if you are a pioneer. You are pioneering what the new roles will be, similar to the very first computer programmers that had to do this by hand, those women. And I think that's such an exciting thing. That's like to reframe from scarcity, fear, to we're in a new world of abundance, right? And yes, it's going to be disruptive. Yes, it's going to be a roller coaster because change happens like that at every level, micro scale, macro scale. And if you're, to me though, if you're a product manager, engineer, designer, you should be leaning into these things if you care about job security. And furthermore, if you want to care about creating wealth, like definitely lean into these things and sit at the arbitrage of the latest tactics applied to the problems that are economically valuable to solve for the business. To just lean on the point of this, the Cloudflare CEO thing of like cutting 20% of the workforce, there is something for, you know, people to understand in the R&D departments is he's not talking about R&D. He's talking about people in middle management, finance, legal, internal auditing, revenue recognition, where those are valuable roles. To keep the business compliant, auditable, a good system that makes money and then takes that capital and makes more capital in the future through allocation of human resources and now agent resources and capital. So he's saying like we could delete the measure role. That's what he's saying. And I think that's also important for people to understand is some of these things are just like operational excellence type roles, but back to product management, design, engineering, like These are roles where there is a lot of criticality in your judgment and taste, and it's ambiguous. Like I talk to this customer, I have an interview, I could throw that to the AI gods to synthesize. What I find in practice though, is that AI gods are not as empathetic in listening, especially if you're trained to, to have the skill of empathetic listening and do interviews with stakeholders, customers, and you're trained in that expertise, like your judgment of what to build coming out of a qualitative subjective analysis, that is your moat. Personally, and you can compound that moat by doing more of that work, by exercising your brain more that way. And guess what? AI is allowing you to do that because you no longer have to manage backlogs as much. Like you don't have to do the boring stuff as much, the measurer stuff. You could do more of the innovator stuff, this prototyper, builder, sweeper, grower, maintainer sort of role. Anyway, I wanted to just add that other lens that I think is important for the individual contributors on software teams, especially if, because I've talked to a lot and we have a customer with like 2,000 engineers, designers, product managers. And there's a lot of fear around, oh, am I being replaced? Am I just training these robots to replace me or whatever? And I could tell you that innovators, and when I use that word, I want to be precise. This is innovator from a technology lifecycle adoption curve perspective. So when I say innovators at your org, there's going to be people that are looking at the latest drops from Cloud Code, from Codex, They're on Twitter. They're looking at the latest open source repos or new skills that are efficacious. They're looking at like RSI. They're on top of it. In your org, these are the AI-pilled people. The goal is figure out how they're working. Then can you then take some of those people that are also good at educating and training others and then get their early adopters going and do the same thing and just keep going and just keep going and treat that AI adoption like a product. That's all you got to do. Then taking a step back, if I were management, head of product or head of engineer or head of design, I would think about what are the carrots and the sticks that I could use as managerial levers to then move my people to do more like the innovators, to take on and adopt more of these recomposition capabilities and skills and workflows that we're seeing with going from Copilot era to Harness era to Agentic shift to now this recomp. So I would ask every manager out there, look at your career framework. Have you named these things? If you have not, go talk about it with your colleagues and go figure out some changes because the people that are not leaning into the tools do not have a carrot to go figure this out. And now we have some great language from thought leaders like Boris from the Claude Code team. It's a long riff, but what do you think of that, Jonathan, or any additional thoughts to add?

Yeah, I think that's great. And I, I'd say, you know, you can take it farther now and like capture it into your harness design. So, you know, like if we're looking at the prototyper role here, right? There's now bringing this augmented capability on top of Claude Code, on top of Codex into your like domain-specific harness. You're going to have a prototyper agents that help you do your job. The purple there, those are your skills, right? I didn't, I did a generation, you want to like co-evolve like this augmented capability with the AI and like those are your agent skills. You can invest in those and you can make them better. Killing decisions, like that's actually like the audit trail that you want, your traces and your dataset so that you can like tune the harness so that it knows what a good decision is and what a bad decision is. You want to train it on that. And so that's the way to capture your domain expertise. So that's the gray. The gray is like make judgments as a human and capture those, train the agents and try to like, you know, push it towards the point where you have a good LLM judge. And, um, and the green is the new task, and that's where you're learning. That's where there's like, hey, there's new unlock, I have new bandwidth, I'm going to be parallel, you know, parallel prototyping for the first time in human history, you know, like, let me figure out how to do this well. And like, that's where you want your, your actual attention, your cognition, your learning cycles to go towards learning. So, um, yeah, I'd say you can, you can take, you know, every— all this, this discussion you just talked about that you're having with your team and bring it into, um, a harness that can capture your learning and like facilitate learning, uh, for yourself and across the team.

Totally agree. Actually, this might be a good time for us to just kind of share how we're doing this on the team, because people are out there are probably like, okay, like I'm leaning in now. Okay. Like the training the agents isn't replacing me. I do have job security, Robert. Like, cool. Leaning into these tools will make me more valuable and I'm helping to define what this recomp era is about. And I think that's all good. Having tactics though, I think might be helpful as well. So if you're a product manager, Um, this is how we're evolving our work, I should say. Uh, what I'm about to show you is our own Clarity harness, um, that we're about to ship open source. And this harness takes the philosophy that every individual has their own individual harness. There should be domain-specific harnesses. There should also be department or company-wide harnesses. And the whole thinking is, if I'm a product manager, let's just take that persona with this particular case, I'm going to have like a story writer agent At the top, we have this Clarity Orchestrator agent. I have this interview agent that captures input. I have a requirements synthesizer agent. So I could open these up in this harness and see a view where I could actually very easily see, oh, this was from this transcript. Hey, we should think about a way to scale this as services stuff. We pulled this, my story writer agent worked with the requirements synthesizer agent to pull from the transcript, what are the requirements and how do we write this as a story? And then you as the judge could be like, okay, This was quoted from here. What was the chain that got me here? This is the transcript. Okay, great. What did we add? What was the diff from before? What's the full artifact look like? Cool. That's the requirement. I could approve it. I could annotate it. I could say this requirement was too narrow and I could save that and that saves in my Harness and then I could regenerate it. So the idea is your agents that you're working with, like if you're working with Cloud Code, it's gonna spin up some agents. If you're working with your own agents, you're going to point it to, you know, say, hey, like, use the skill, orchestrator calls the agent, calls the skill kind of thing. So with a harness like this where you're in the flow of work, you're writing your stories, you're making your prototypes, blah, blah, blah, you have agents doing stuff. The idea is there are agent runs and you want to know, did the input that I put in get me the expected output? And then how do I make that even better next time?, that's what you're doing. And then if you want to do that across 100 PMs, 200 PMs, 300 PMs, this is what it takes is you get those product managers to be able to annotate the agent runs. And I could see here, I could see, okay, we ingested transcript, we extracted requirements, we, the model made a decision, new requirement or updating an existing requirement.

Cool.

We didn't take the other path. There's a new requirement. We propose a diff and now it's waiting for me as a human to review. So this is how we're, we're dogfooding this open source harness for clarity as we get requirements from our own customers acting as Ford deployed engineers and AI founders. And we go in here and we're trying to annotate these stories and requirements, et cetera, so that our harnesses compound, so that as I talk to more prospects, more customers, all of it could be curated learning that is queryable. Even as the team grows, we have the provenance of why we made these decisions in the product, down to first-party data, down to this stakeholder, this customer, this prospect said this thing. And hey, Robert the human made this judgment call that this was the right requirement. This is how we formed the acceptance criteria for this story. And that informed from the input spec to this code change for this pull request. And hey, did that increase adoption? Did that move the needle on our business goals? And that's like the individual harness moves to agent harnesses, moves to the is the code good? So domain-specific codebase harness to now business outcomes. And that's how we see it is you need to have a stacked harness architecture if you want to harvest all of your accelerated learning from the edges of the org, the frontline workers, the PMs, the designers, the engineers that are learning and implementing every day. They're learning and now using agents agents to do it. And how do you get all those agents and people way better decision-making? Make all the data queryable, make all the learning queryable, make it curated, and tie it to real evidence. So anyway, just wanted to give a quick— I know we've been talking in abstracts and concepts, but this is literally how we think about looking at a run, going as an expert, reviewing the actual artifact that was created. In this case, this is like a story. I could annotate Next story is going to be better for the next PM or me, for the audience, just way more tactical. Get your people annotating data, get your experts annotating, and all these functions get better over time if they're driven by agents. And they are, and they're going to be increasingly more driven by agents over time.

Yeah, great demo. The UI is really coming together.

Dude, I've been dogfooding the shit out of it, man. I have been eating the dogfood. Food didn't taste good at first, and now it's tasting pretty great, I think. And I guess shout out for the pod. We don't try to advertise too much. But some people have heard of us, some people have asked for this. So yeah, if you, if you want to use the open source Clarity Harness, you want to be closer to us and evolve this with us, hit us up. Hit me up on socials. I'm on every platform. And we'll get you in the early waitlist. So we for clarity, if you're a head of product or a head of engineering and you're trying to solve the problem of, I have this employee where they have so much institutional knowledge, tribal knowledge in their brain. If they leave, I'm kind of screwed. If you have that problem, the real solution is how do you get them into a natural workflow where they're annotating data with the agents so that you can institutionalize that IP or that knowledge and compound it? Around those workflows, and then you can get another person to come in and immediately they have all that context, all that rich intelligence from that domain expert, and day one employee could be as effective as a day 365 employee simply because you've created a company brain more, uh, intentionally. That's what I'm seeing that every, every manager, every exec is thinking about in tech is like, especially if you're in a subjective, high compliance, high regulatory, domain, you're thinking about, oh my gosh, if my domain experts leave, we're screwed. And AI is making it so you can have a company brain. So the solve, we believe very strongly, is get into their workflows of where they're using their coding agents, Claude Code, Codex, or whatever, and then give them this mission control. So all of this just runs on local and we have a Postgres database and we're collecting all the agent runs, all the toolchain calls, and then showing the expert things they have to annotate. Then the next time the agents run, it takes those annotations, curates the datasets, does the data science work, Harness gets better. If you're interested, hit us up about the Harness engineering stuff. Happy to share more of our thought leadership on this.

To add to the domain, that super valuable domain expert, I mean, yes, you could create the container to help capture that expertise, but it also like, that's an A-player, right? Like, your company relies on them. Don't have them guard this stuff you've already figured out. Get them on the front lines getting to the new tasks, right? So when you enable other people to take on, like, the same responsibility that they've been with the, um, with domain expertise that they have, like, you could have them on the front line learning the new tasks, you know, that, that's probably, like, you know, some of the best, uh, intellectual property resources you have in the company are these people who've already proven themselves to the value of your company. Like, get them on the front line and get them figuring out what the new tasks are that they can use AI for.

Yeah, there is one other nugget that I got from an executive who has ran teams of 3,000 to 4,000+ product managers, designers, engineers, and has been on the board of Fortune 500, Fortune 1,000 companies. One of my mentors, Mike, he's really great. He talks about this, which is we know that that's the thing to do, which is get the experts to compound their intelligence. We know that they're the A players of the org. And then the question is, are we compensating them for that fairly? And I thought he brought up such a good point, which is if we're now, we're asking these people to go out of their way to annotate the data and say that's part of the job, job's changing. And hey, you have way more leverage by doing this thing, AKA I'm annotating these stories, I'm annotating these agent runs, I'm annotating the whatever to improve the harness. Now the harness has a multiplicative productivity J-curve effect on every employee in your org. You take that unicorn employee, you have them train the agent like I kind of showed you with annotating the data. And if their investment into that produces a J-curve sort of productivity multiplier for the rest of your org, aka a novice sort of product manager can come in and act as a staff-level PM, like, you know, they're not going to have the same judgment quality or whatever, but they can come in there and be effective way more than the before world where you had to shadow for 6 to 12 months and then you're maybe effective, right? You should, in my opinion, hot take maybe, and the take of other seasoned tech executives is think about your incentive structure for that behavior that you want to compound that investment for the org, for this intelligence layer, the company brain. Are you incenting your A-players to be on the front lines to do that? That's a really critical thing that I think every exec should be thinking about. If you aren't and they're not doing it, well then you haven't created the systems or the environment to encourage the behavior that you would like out of the and that's just full stop the issue. What's your take on that? Should experts be compensated for the compounding leverage that their time now gets by doing these activities, by doing the annotations, by doing the agent reviews?

Yeah, the world I'd like to see is one of knowledge attribution where, say, you're encoding expertise into a dataset and that's used to train prompts and agents that are used by other people. I mean, you can trace the attribution now. And like, that's really like, ideally becomes kind of a knowledge marketplace where you get credit, you get attribution. The agent I helped train and, you know, did so many different tasks today, you know, like it did 100 tasks today, you know, and that's your leaderboard, right? You don't want Telkey Maxine leaderboard. You want like, where is the knowledge attribution being used in context to provide company value leaderboard? That's how I like to see it. It'll be interesting to see, like you mentioned the incentive structures and if those can sort of align to the knowledge landscape that is rapidly evolving.

I guess to put it really bluntly, this is why I think the incentives should align economically. If I am your A-player and if I do have a fear that my job security is threatened by training these agents so that lower-skilled PMs that are earlier in their careers could take these agents and be as effective as me, if I believe that, then what is my incentive to do that for you? That's really meeting that objection. I think there are going to be people in orgs where the A-player sees it and they're like, "Oh, I'm just drawn to this. I want impact. I want leverage. I'll say, like, when I've worked in corporate, I'm like that. I'm like hungry for impact. Like, your A-players are hungry for impact. All I'm saying is there is going to be a percentage of those people that are— they're smart, they're intelligent. Don't come to them and, you know, ignore the elephant in the room. They've thought this through. They're like, are you training my replacement? Right? Like, and just confront the elephant in the room and say, like, here's what we think. The job profile is this way. This is a new responsibility. It is super impactful. And I believe it's every manager and executive's job to coach their team to see the vision that actually there's so much more leverage here. You can innovate so much more doing this. You could compound and help us all go faster. Like, I think those are the things that execs and leaders need to be giving their teams a vision that is way more positive versus the doom and gloom of like, you're gonna lose your job, we're gonna like train an agent to fucking replace you. It's like, no, like your job is to get your A-players to be activated on this. And then that is your advantage in market against the competition. So really exciting to see how this plays out. It's such an interesting era, and there's so many ways that we could talk about this and very, like, unlimited angles. But, uh, there's a lot we don't know because we just need time to happen and more experiments to happen across companies. But we'll definitely keep reporting on the progress of this. So Jonathan, any other thoughts on this and/or any additions what I just mentioned around the incentives?

I, I like the incentives narrative, and I think I think there's, you know, you look back 6 months and it was like, there was just so much uncertainty and fear. And I think that as the data is rolling in, um, like the potential is really exciting and it's a, it's a better narrative to hold whether you're an entrepreneur or you're working at a bigger company in terms of like what AI adoption means for you. I would like, uh, there was a couple insights I had just in our own, our own like, you know, like Linear project that we're working on, which was, is kind of interesting. I just like, you know, I don't know if this, this makes the cut for something that's like really useful, but that, you know, like working incrementally and trying to make progress on the goal of our dogfooding project. It was interesting, like Fable reflected back to me. It said, I've noticed this pattern 3 times that we've like, we've had a trunk towards the real path that we need to take and we've taken a fork by accident. And on one hand I was like, oh my God, like Fable just saw itself in the mirror. Like this is, I was like, this is what it's like to work with AI because you can't hold the whole context and like, you know, this happens constantly. But it also pointed at like the rapid iteration that you did in like ticket 320 where you did like 10 different prototypes and they were all quite diverse. Divergent. And it was saying, like, it says, look, we need to actually wrap each layer in a control plane and, and like make sure that we stay on the, on the, the real path. And this divergent activity is, is distracting. And I said, well, actually, like, read Boris's article, you know, your creator. I said, look, what Robert's trying to do is prototyping, and he should be taking divergent forks off of the trunk. And he, he, because he's like looking for where, where is the, uh, product market fit, like where is the utility. So you're in the, we have the prototyper hat on on, it's by definition, uh, a divergent fork from like the path. And you're trying to figure out like, okay, all of all these forks, which is the one that we want to double down on to like move into the building, right? And then in the building, I can't take divergent forks. I've got to wrap control plane on top of control plane on top of control plane, and each one like adds incremental complexity. And so that's where like, you know, this has been my experience as an engineer, just getting an idea out to market, like it'd be just like complexity explodes money, right? There's just like all these different cases you have to test. And, um, and so like that, the prototyper being, uh, like divergent and creating new forks, and the builder being somebody who's like wrapping control plane layers and like is stacking layers, is like, oh, that's really actually a really like interesting, uh, insight. And then the sweeper is like, okay, now we've got this like, you know, hypothesis wrapped in the control plane lanes. Now we gotta simplify this thing so it's maintainable. And that's like, I think, where you get the later stages of Elon's algorithm, you know, in terms of like, uh, the incremental automation. So that was kind of like an aha moment. It was both funny because like I feel like Fable saw itself in the mirror and it like, it's like, wait a second, that's horrifying. Let's fix this immediately. We now have a design, a named design principle that we use to address that. But also just like, you know, like how do we work together? And like the, I think having evolved names of like what the nature is of the knowledge work and how like, you know, the divergence and convergence and the stack layers, it's good language. And it like, I think it helps us understand, put ourselves in each other's shoes, understand like what the challenges that we're trying to solve solve at a given point. So it's just a cool experience with Fable.

Yeah, no, the reflection of like Fable— look at this tweet by your own creator, uh, like, actually the diversion prototyping makes sense. Like, that's so funny. But also this brings up such a gold nugget of a challenge that I believe other engineers and product managers probably have, which is you could go so divergent and, you know, you could evolve the tool so fast So the person that's prototyping, like I get something out there, there's an MVP and then I'm still thinking about it. Like I'm thinking, oh, like I want to like do more on top of this. So, but that changes the nature of the contract, right? Of what data do we expect? What's the schema? Like how do we meet the front end requirements? Like this prototype, like what are the actual features? And that's what has to get inculcated into server logic. And we have endpoints and then we have end-to-end testing. We have unit tests, we have integration tests. Like That doesn't compound if the contract changes all the time, if the data model changes all the time. You don't compound literally shit if everything changes all the time. You don't build a real product. I think that is actually a real tension point, which is we can go so fast on the ideation and prototyping. So you need to be really specific on what is idea land and innovation land and what is in the sprint. We're not perfect at this. I'll be honest. Me and Jonathan are still finding our rhythm here. We're finding that we ship a lot more, but like we definitely feel like, oh shit, like is that changing? Oh shit, I gotta change it. Like, how do you— oh, that ticket, what decision did you make in that session? Like, and so we've had some of this in the past 2 weeks and we're honestly now even thinking of we need better ways to work to meet the way that work has evolved. And I think that's such a key thing. So on that, a very tactical thing, to meet alignment around work. Number one, just do your sprint Agile ceremonies. Like I still find that effective and not going to go into that too far, but one-week sprints, two-week sprints, three-week sprints, whatever you want. We're on two-week sprints, but just having the intentional, we're locking in. Okay, if sprint scope changes, let me know. Okay, like make a ticket to document everything. Okay, figure out our sprint velocity, which is really weird because as the foundation models get better, your sprint velocity, what does it even mean, right? Like, but it's good to have some level of directionality on are we shipping more? Are we pounding the asset, that, that is the software product take to market. And we're still figuring out things. But the one thing that we're aligning on is build these sort of HTML artifacts to gain rapid visual informational contextual alignment between the humans that are driving agents, me and Jonathan, our startup's case, and then we get aligned. And one cool thing that we built recently because of the need was this tool, which is Clarity Align, where I was like, we're talking about these HTML files, uh, we throw them in Slack and stuff, and I'm like, does this make sense to you? Does this make sense to you? We talk about it in a meeting, we take the transcript of the meeting, we throw it in Claude Code, throw the HTML in the Claude Code, ask it to update it. So instead, now we're doing like, wait, if HTMLs are like the new alignment artifact, let's go ahead and just make a tool real quick, throw the HTML file in there, and Google Docs comments on it, and then have the AI iterate based on the annotations of the comments. So this is using that Dact Harness architecture. This is very new. I built this in a night, but we're going to dogfood this and get this working for us. But the main point of us bringing this up is saying the new ways of working are evolving the new ways of where is the constraint in the work? Where are we going slow? The bottlenecks are like, I'm going really fast on idea generation. Jonathan needs a foundation to build on. And clarity on what's the roadmap without tons of movement. So divorcing innovation ideation backlog from sprint commitment, and then being very rigorous around if we are changing sprint commitment scope, especially pre-PMF, which, which we are, to constantly generate hypotheses and validate them. And if you're dogfooding, you're doing that every minute, you know, hopefully, and being intentional and just like communicating that. So if other people have that problem, and have found that constraint in their workflows to go from, you know, prototyper, builder, sweeper, grower, maintainer, like these things, let me know how you're solving this. How are you solving the human and AI alignment on your team towards a goal to get shippable product out? I'd love to know how other people are doing that. But our meta has been stay locked in on the agile sprint cycles, be even more rigorous about scope and decision-making and then trying to communicate when things change and divorcing ideation and innovation from locked roadmap scope. Again, I'll say we're not perfect, shit's changing all the time, but we're trying to get there and the language definitely helps here from Anthropic. Okay, we have spent a good amount of time on this. Jonathan, any other things that you want to dig into or riff on when it comes to the topic of the roles of product management, engineering, design, the three legs of the stool of traditional software development lifecycle, how the roles are sort of blending together, dissolving. Any thoughts from a CTO perspective?

I think we covered a lot of it. It's constantly evolving. This is like a snapshot in time of since making this crazy world, and I think this is a pretty good lens to sort of continue to make sense of it. So more to come, I'm sure. Sure.

Cool. So we'll wrap the pod there on that note. Um, this was a very special pod. Uh, usually we go through, you know, AI news, AI memes, and what's the meta. Today we just focus on the meta of, you know, what are heads of product or heads of engineering or like your ICs, engineers, designers, PMs, like what are those people at tech companies doing? And in the new ways of working, how are they evolving their workflows and getting their teams to row the boat the same way? I'm sure everybody's thinking about this. We're thinking about it. We're living it. This language helps a lot. We'll add all the show notes of the links to these artifacts that we've shared and the news articles we shared in this tweet from Boris about the 5 emerging archetypes. And we will definitely keep reporting on this from the front lines of building our own company and our own product as Ford deployed engineers into some of these orgs that have thousands of engineers, designers, and product managers. We'll continue to keep the audience up to date with what we find and what we learn. And we don't know everything, but we'll tell you what we know, and we'll tell you what we don't know, and we'll give you our takes, hot or not. So thanks for listening to the pod, everybody, and give us a follow. That helps a lot. It helps us to keep doing the pod. So if you've listened this far and you haven't subscribed, please subscribe. Please, I'll give you like an emoji react if you like throw a comment in there. I'll, I'll comment back. Uh, it really helps us out. So until next time. Thanks a lot, Jonathan, for being awesome co-host and co-founder as usual. And thank you to the audience. Catch you later.

Bye-bye.