Paying For AI Twice | Sierra's Agent For Everything| The Task Economy | NY Halts Data Centers

We talked about this before. This is the first tech wave in my career where I've seen active, organized opposition. Other things, people have been like, I don't know, I'm neutral on mobile, or mobile's good. This is people are trying to stop something from happening as opposed to undoing something that's already happened. I think tech needs to take these things very seriously and ask ourselves our question: what is in it for the average person?

It's the week of July 13th. Welcome to the Learning Corner, a Precursor Ventures podcast. Welcome to the Learning Corner by Precursor Ventures Ventures. I'm Mia Farnam, and I'm joined by our GP and managing partner, Charles Hudson, where we will be talking through the articles and Goodreads that caught our team's attention over the past week. Our first Goodread this week is a post on X from Satya Nadella called The Reverse Information Paradox. Satya builds off of economist Kenneth Arrow's classic paradox where a seller of information risks giving it away for free just by trying to sell it. He argues AI flips that script. So now it's the buyer who risks giving away their most valuable knowledge, the contacts, the corrections, the internal know-how, just by using the tool that they paid for. Every time an employee corrects a model or feeds it proprietary information, the knowledge quietly becomes part of what the AI vendor learns. And over time, the vendor ends up knowing more about your business than you know about them. His fix is that every company needs a hard boundary around its own data feedback loops and institutional knowledge. Similar to how a patent lets an inventor disclose an idea without simply giving it away. He points to Alex Karp's line about customers wanting to own their compute models and data stack as proof that this is already on enterprise buyers' minds. I wanted your thoughts on this more generally, but we're obviously thinking about this good read from the standpoint of B2B companies wanting to protect their know-how while still capturing efficiencies from AI. But if you flip it to the standpoint of an employee sitting inside one of these organizations, if using these tools is making me faster and better at my job, does it actually matter what the seller is doing with my data on the backend? Is this a trade-off we're carrying about, or is it just noise compared to the productivity gain? And when does that start and stop?

I think the theme of this week's learning corner is like, be careful what you train and how you train it. Because there's some other things we'll talk about later that are related to data labeling and training. First of all, the fun thing, if you're Microsoft, is like, well, we win anyway. We're selling a lot of compute capacity to some of the big hyperscalers and hosted closed-source models. And if our customers decide they want to use open-source models, we're happy to sell them compute and hosting for those open source models too, if they're not gonna run them truly, truly locally. So then if you're Satya Nadella, you're like, as long as people are using AI and they're using our infrastructure, I don't really care whether it's OpenAI, Anthropic, Qwen, Kimee, whatever model they want to use. This to me feels like it was the inevitable pivot back from let's just trust the closed source hosted models. Yes, we have these existential concerns about security, but what's our alternative? Your alternatives in the beginning were use the hosted models and hope that they weren't using your data to train the system that you check that little box. And now the open source models are getting better and people are getting more comfortable saying, hey, maybe the right thing to do— we talked about this with Kirkland and Alice— maybe the right thing for us to do is to make the investment in training our own models. It's funny, I've been doing this this weekend with a fun side project, a video-related side project, training a video model, a version of Qwen on a bunch of video assets that I have that I'm trying to analyze. And it's fun and it's more for cost for me than for security. And I think we're gonna end up in a world where people are gonna use— it's the old, you don't drive a Ferrari to the grocery store analogy. Like people are gonna end up using the frontier models to do the frontier work, the really, really hard stuff where you want the most sophisticated models. There'll be some things for privacy, propriety, whatever people will want to keep inside their own firewall, whether it's actually worth protecting or not. It's probably more about a feeling of safety and a feeling that we're not sharing this information than it is that like having it out there on the model would really make a difference. And so I think you're going to see a pivot back towards maybe we should control and manage AI. It's funny, I told one of our portfolio company founders this: AI is just becoming part of corporate infrastructure again. And for a while it was this like wild, wild west rogue thing where people are like, well, we don't need controls or compliance or training or guardrails, just like spend, spend, spend and test, test, test. And now I think people are like, well, what if we just treated this like other parts of infrastructure where we care about ROI, where we care about compliance, where we care about data security and where we weigh the benefits of like openness versus this. And I think the pendulum's gonna swing back towards treating AI more like corporate IT. Will not be fun for the people who are like out here doing all these crazy cowboy things.

Yeah, well, that was my next question. Whose job will that be to determine what's absolutely has to be secure and compliant, and then what can you explore with frontier models and just kind of run amok on your own?

I'm just sure in 2 years you're gonna hear about an engineer who wasn't using all their tokens, who like swapped their tokens with some other bails of their team who needed 'em in exchange for a vacation. It's gonna get weird.

Bartering.

Yep. But I think the CFO is gonna have a big role in this. Yeah. And you're already seeing what I would describe as brute force attempts at this, that companies just saying engineers can only spend X amount in tokens per month. I'm like, but what if one person's really, really efficient at getting value out of tokens? Maybe that person should be allowed to spend more. And the person who's just wasting them translating PDFs into PowerPoint shouldn't get any tokens.

Yeah.

But we're not, we're not at that level of sophistication and nuance yet.

No, we're still token maxing.

Yeah.

One last piece I wanted to touch on this. Another comment that was in the threads a little bit further down the chain was how are workers' IP protected from the enterprises? So it's one click below that. We talked about this a little bit ago when we were reading the takes on if you got to take agents from one job to the next with you if you built them at a job, but the knowledge that you bring to a workspace and the know-how, that's training your enterprise's model on. How do you as the employee get to ring-fence what's trained on that versus what you get to keep?

Everybody involved in training is telling the people doing the training, don't worry, you'll be fine. You know, all the data labeling people are like, don't worry, you labeling all this data by your profession, you'll be fine. The companies are telling their employees, you teaching the AI how you work, don't worry, you'll be fine. I don't know if everybody's telling the truth.

More to come. Well, our next good read is a blog post from Neil Rahilly at Sierra called AI Pilling Our Company: Lessons Learned. After their engineering team started seeing 5x productivity gains from running coding agents in parallel, Sierra set up a 6-person AI acceleration team to figure out what it would actually take to get that same boost across the entire company, not just engineers. They started by building a separate agent for each department, one for support, one for sales, one for data, but found that setup actually got in the way since the most valuable work happens across teams, not within one function. So they collapsed everything into a single agent, called it Pinecone, one that persists across an entire project instead of resetting after each request. Since most real work doesn't actually get done in one sitting, it unfolds over days or weeks. One of Neil's bigger points is that the agent should really be thought of as just the interface. The actual system of record, your CRM, your ticketing tool, your codebase still lives underneath it, and an agent is only as good as the quality of the systems it's pulling from. Since launching, Pinecone has run over 75,000 sessions and now opens 70 of their engineering pull requests, but Neil is careful to point out that usage numbers alone don't prove the tool is actually making outcomes better, just that people are using it. So Sierra admits they can't quite measure whether Pinecone is actually improving outcomes versus just generating more activity. That's coming like from an agentic company themselves.

Yeah.

They build for a business. Would you have expected more from that experiment from them, or do you think the insights were kind of spot on?

I appreciate the honesty and the willingness to not AI wash what actually happened. I really enjoyed reading the article and I think what they're doing is this— everyone says, oh, there's no ROI. I'm like, yeah, because the gains aren't broadly distributed enough across the company to show up. And I think in the same way we've talked about companies that are beginning to take these open source models and use them instead of the closed source models, I think you're going to see more companies who say, you know, we have two choices. We could hire a very expensive consulting firm to come in and do an AI transformation, or we could do it ourselves. Or we could put together an internal team that will help us do an AI transformation. And I think you're going to see more companies say the answer is in the room already. What we really need is a team of people who understand our context, our workflow, and who can make the gains of this AI productivity extend beyond just— if it only lives in engineering, unless you're a software development firm, there won't be enough ROI for this stuff to pencil. It has to get into finance, ops, customer support. It has to get into all these other areas for at the organizational level for the ROI to make sense. Yeah.

And for Sierra, it's pretty obvious because because they have that technical talent that lives within the organization. But would you say that's the first phase is taking a good look at your organization and having an honest conversation of, do I have the talent that lives within this to run that experiment, or should I outsource that?

I think though, we're also at the interesting stage where, because the tools are relatively accessible, I'd argue what you really need are people who are curious and engaged with the tech. If there are people who are curious and engaged with the tech, they can come up with lots of interesting ideas. It doesn't have to be just the most senior people in your org. It doesn't actually have to be your most senior people leading this transformation. What you really need are the people who understand what the tools can do. I think the SaaSpocalypse, people are going to look back and say directionally they were right, but in the end, I think people are going to use headless versions of systems of record because those systems are very good at keeping track of things. But you won't interact with the UI and you won't use them in the same way, but they're still going to be a part of the tech stack and they're still going to be very valuable, important systems.

That was one of the most notable takeaways that I felt like was buried in the piece. Neil said, quote, we found it's best to work with your systems of record, not replace them. GitHub keeps the PR, Salesforce keeps the account, and Linear keeps the issue. The agent is a layer across them. So it's just essentially a big pushback on SaaS is dead argument. Well, our next good read is another post on X from Everett Randall at Benchmark called The Task Economy. Everett opens by contrasting two metrics: tokens, which measure how much a model gets run, and tasks, which measure whether a model is actually getting better at real work. Tokens have obviously become the default way everyone tracks AI's growth, but he argues that just telling your usage is up, not that the model is improving. His core argument is that the internet is basically tapped out as the training source, so the next real unlock for AI has to come from somewhere else— the knowledge sitting in professionals' heads that never got written down anywhere online. He estimates 99% of the expertise we actually want AI to learn— how a lawyer reasons through a contract, how a doctor works a diagnosis— lives only in people's heads. It's not on the internet, and his answer is what he calls the task economy, paying real experts to attempt actual work tasks and then grading or correcting their output so that judgment gets fed back into the model as training data. He points to impressive growth numbers as proof that this is already happening at scale, labs increasing their data spend 10x year over year, and Benchmark's own portfolio company Merkur going from $1 billion to $2 billion in ARR in just 4 months. Charles, do you like the task metric better than token usage? I assume yes, but wanted to hear what you think the limitations could be on this one or how it could be gamified.

I think when people told me about Oh, all of this AI stuff is going to be outcome-based pricing. I'm like, well, what's an outcome? And they're like, what do you mean? I go, well, some things are easier to quantify, like a support ticket when it's marked closed and resolved. That's, that's an outcome. But if you're closing the books for the quarter, there's a million little micro outcomes that roll up to the, like, closing the books. So I was like, look, for outcome pricing to work, you have to have a clear, easy-to-measure outcome. I don't hear people talking about outcome-based pricing as much as they did than they did maybe a year ago, much more usage-based pricing. You know, you look at like Handshake and some of these businesses that were not in the data labeling business that now are, they're ripping, which tells me that the models have an almost insatiable demand for trained labeled data. And it reminds me a lot of some of the things that Sarah Guo brought up in her post about untrainable, which I think we have a lot of things that are in people's heads that are not yet in these models. And to our earlier conversation in this episode, the question is like, how willing will people be to part with that knowledge? 'Cause you can kind of only get paid for it once. At some point you've contributed to the model, the model knows, and you, you can't resell that knowledge or insight again. And so I think if I were a model provider, I'd be doing the same thing. This is probably a, an ongoing expense for you to capture real-time, real-world training data on these tasks. But my guess is that like in a year it won't be 99%. It might be 80%, it might be 60%. This is like a one-way transfer of knowledge from humans to machines. And I think there's money to be made. When I first heard about this market, I was skeptical that it had longevity. I'm like, isn't this that's just arbitrage. Don't you eventually, in a fairly short period of time, transfer all of that human knowledge to the machines and then this is like, isn't a thing anymore? And I don't know that I believe that anymore.

Yeah.

But that was my initial thought was like, this is a one-time, one-way transfer. How is this like really a thing?

And it's not one-time transfer in your brain just because you continuously need to update, improve the models themselves so that it's actually ingesting the things that your brain would actually wanna put on paper. Right.

I think at some point law will be solved, for example, but then we'll have to say, well, what does a physicist do? Or what does a biologist do? Or what does an air traffic controller do, or what does an architect? We will move across domains and there will be new datasets to label and new information to contribute. But I think it will be like Locus to some extent. We will come and destroy all of the, like, offline knowledge in a given field. We'll get enough of it into the models that they're good enough, and then people will go say, well, what's the next area of expertise that we should go mine and acquire?

Someone down in the threads also made the point that the task economy can come in the form of AI creating tasks for humans to do.

Yeah.

So like Uber fares or DoorDash tasks, which calculate the value based on how complex each task is. So the argument from their standpoint was if AI is now just selling work, then surely the previous ways of how you bill it has to change. That, I think those are really specific examples because those are total gig economy works. We've mostly been talking in the examples of white collar professionals.

Yeah. I mean, people keep saying it's going to upend consulting. I'm like, maybe, maybe not. I don't know. I think the consultants are going to have something to say about that. That too. And you know, every pricing change has all these weird dynamics. And I don't know, my experience has been I spend a lot more on software than I used to. And even though software has gotten cheaper, I just use it more and buy more of it. So I have a lot of questions right now about does all of the math on this stuff pencil in the end?

Do you think there's a world in which white-collar professional people are doing this purposefully as a side gig, inputting some of their knowledge, know-how into the models themselves? Or are they just doing it passively just by using the models themselves day to day?

I think There are certainly people who are doing it as a side hustle, a GLG-style expert network type. This is a way to make some money on the side. And other people I think maybe don't realize that they're helping to train these models by asking questions and contributing data. As they get better, we're just gonna use them more.

Well, we have a fourth good read almost as a mini news flash. New York this week became the first state to fully pause new large data center construction, imposing a 1-year moratorium on any facility using 50 megawatts or more of power. Governor Kathy Hochul cited rising utility bills, strained water supplies, and pressure on local communities as the reasoning. And she's also pushing to repeal the sales tax exemption hyperscalers currently get on these facilities. During the pause, the state's environmental regulator won't issue new permits while officials work out consistent standards for how data centers get reviewed and built going forward. New York already has more than 12 gigawatts of large power-hungry projects, including data centers, waiting to connect to its grid. And nationally, only about a third of Americans say they actually support the pace of data center construction happening around them. Wanted to just get your reactions if you think this type of legislature will actually do anything.

I I think a lot of the job of a politician is to read public sentiment and to pass legislation that addresses the needs of their constituents. I think tech is totally losing the communications battle. I won't even go into the data around the claims on energy and water because your mileage may vary depending on where you live. I don't want to go down that road. Tech has totally lost the narrative battle here. And I think people say like, well, what happens when you live in a bubble? I think if you live in the tech world that we live in, this AI feels mostly good. It's probably not going to take your job. It's making your job more fun and interesting. But for a lot of other people, they're like, what's in it for me? I use the free plan of ChatGPT. Hasn't changed my life. I see this big ugly structure being built in my backyard. My electric bill's up, whether that's due to inflation or due to the presence of these, these things. I'm reading things online about the water consumption, whether, and some of that is motivated reasoning. And I think if you ask the average person like, what's in it for me? I don't know what I would tell them. Why you should have a data center in your backyard or why like And the new job thing, I think, isn't landing.

Like, I don't think people are like, oh, I, and I want to work at a data center.

Yeah, I want to be a construction guy pulling cable, or— I think because within tech AI has felt so inevitable and so magical, I don't think we've asked enough questions about why are so many other people so uninterested or active? Again, we talked about this before. This is the first tech wave in my career where I've seen active, organized opposition. Other things, people have been like, I don't know, I'm neutral on mobile, or mobile's good. And maybe you could argue social media in its twilight era, people have come to see it through a different light. This is people are trying to stop something from happening as opposed to undoing something that's already happened. And I think tech needs to take these things very seriously and ask ourselves our question, what is in it for the average person? And it's funny, like some of the AI people are just like, your job's going away. I'm like, well, that's also not helping the narrative, but it also might be true. So we have this interesting tension between what is the narrative that we want to tell people about why AI is good and what's the truth in reality. And we really don't know what's going to happen to jobs. Saying that you shouldn't worry feels like a very privileged argument.

And do you think the communications battle being so poorly lost by tech is a result of them just moving too quickly and not thinking it's important, or just assuming those people just don't have any voices or skin in the game to matter?

I think most people in tech view AI advances as inevitable, so trying to slow them down or stop them feels like very decel and like a waste of time. And that instead of engaging with people about their real concerns, they're just like, don't be a Luddite, get over it, this is gonna happen. That's not how you win people over to an argument. That's makes people feel like you just don't care about them. And I don't hear many people addressing the needs of these folks. Some of these data centers, like, they're off the grid. They are like separately powered by other things. They're not taxing the grid, but that's like a nuanced argument. I just feel like tech has not engaged with people on the issues that they seem to care about, which are these things are big, they consume a lot of power, and they are a very visible— you know, AI is very abstract. A data center is a physical representation of this abstract fear that I think people have. And seeing 3 of them go up in your community makes think, oh, the walls are literally closing in on me. This future state that I'm afraid of is happening even faster than I thought.

Well, we will certainly see more discourse about that over the weekend, but we are all out of time today. Thank you, Charles, for stopping in. Thank you to our listeners for hopping in as well. Please like and subscribe, and we will talk with you all next week. Thank you.

Bye.