Why Most AI Apps Are Dying, and How One Founder Is Fixing It | Nic Baird, CEO @Koah Labs

ChatGPT is losing $4 to every dollar or something crazy like that, right? Makes it impossible to sustain that at scale unless they come up with a strong monetization method. And we saw all these companies that were getting a lot of engagement and growing really quickly just shut down because they look at the projections of unit economics going forward with subscriptions only converting at 3 to 5% and say, this is never going to make money. With Character AI, the average user engagement per day is over 3 hours. That's twice YouTube and TikTok. It's insane. Like, Character AI tried to bring in basically like a traditional interstitial ad, so like a full-screen ad that comes up once you switch talking to a different character. And they got really bad backlash from their Reddit community because that's just not the way that people want to see ads, right?

Yeah, that's terrible.

We're trying to keep it native and relevant. It's in the same font, the same coloring, not something that's like a huge banner ad. It's like flashing and trying to get your attention. It's really just trying to solve the user's problem. We're doing 2% CTR across the entire network. Traditional kind of display ads, they're usually in like the 0.4 or 0.5% CTR. And Open Evidence makes $150 million a year in revenue on 1 million users, which are all doctors. There's no payment option available to them. These people are having so much fun and their users are so engaged. Like, why don't they deserve to live?

Today I'm talking with Nick Barrett, co-founder and CEO of CoreLabs. They're the company building what they call AdSense for GenAI. Nick, thanks for joining.

Thanks, Angelina. Happy to be here.

Tell me, tell us a little bit more about yourself.

Yeah, so I'm, I'm 28. I moved to San Francisco a few years ago. I'm from St. Petersburg, Florida. I went to school at Yale and most of my life I spent as my main goal was competitive sailing. And so before I graduated college, I had won a few national championships and went to the world championships a few times. And when I graduated, I thought the first thing I was gonna do was like go be a professional. My dad's in the Hall of Fame and it's big in the community and in our family. And so very, that's, that was the path that was obvious for me. And anyways, and I fell over myself multiple times and stumbled my way into startup land, which is where we obviously are now. But yeah, now I'm the founder of Co-Labs, having a lot of fun, enjoying my life here in San Francisco and working on hard things with a great team.

Maybe you'll tell me a little bit more about how the, how that pro athlete to startup founder transition happened, but We can talk about all that later.

Yeah.

I love to start with this question to help our audience understand better what your product does. Can you explain what KOA does at 3 levels? Let's say, how would you explain it to an elementary schooler, a college student, and maybe a professional in the industry?

Yeah. So if you're like an elementary school student, it's simple. It's, you know how you use ChatGPT and sometimes ChatGPT kind of recommends products? We are the layer behind recommending those products. Those products for folks that use the ChatGPT technology. It's very simplistic, but it's like, that will explain what we do. We're a recommendation system for products.

Are you seeing a lot of elementary schoolers using ChatGPT, by the way?

Hopefully we don't see a lot of elementary schoolers because we don't want to be serving ads to elementary schoolers just yet. Maybe when they're older.

The toys? Target?

Yeah. Eventually, there's lots of kids stuff. I think that A couple of good friends of mine are actually building products for storytelling for young children. And there's a lot of really interesting stuff happening in the AI space for really just like letting kids' imaginations run free, right? Within AI, which is great. You can generate visuals, you can generate like stories, you can have your own characters and you can invent these characters and they can be included in the story. And so we do have friends who are building products like that. And so in that case, we do see a lot of young, like very young people, elementary schoolers using AI in that way. I think ChatGPT in particular, maybe not so much, more of a professional use case, but there are definitely derivatives. That, that are being used by young people for sure.

Yeah, college student, more relevant.

Yeah, definitely more relevant to college students. The college students will be easier to explain effectively what KOA is from the college student perspective is we're a full-service ad network for AI-native companies. So basically anyone who's leveraging an AI API to build a conversational interface or a dynamic interface as a part of their core user experience component, we, we help those people serve advertisements and monetize that service area.

Hmm. What about for fratinals?

This is, uh, more fun because I realized that the way we normally describe our company to professionals is the way I would also describe it to a college student. So we can try to keep it very simple and high level. But basically, yeah, we're a full-service ad network for AI companies. Deploy via SDK into each publisher that we work with. We use their first-party data combined with basically traditional ad targeting methods to find the most relevant kind of sponsored content for them. And then we show that user, the, the end user, a personalized one-to-one relevant advertisement on their behalf. And then we also build all of the advertiser side of the marketplace as well. So the targeting tooling for them and the sales process for them, and we're a full-service network. So if you're an app, we deploy the SDK and then we do everything else on the advertiser side as well, including the ad server.

Awesome. So how about let's start with recent news. So we know that OpenAI had quite a week recently. So when we chatted last time, remember, OpenAI just got caught with some ad code in their Android beta. And then I learned that in early December, some users saw that and they, they, some potential ads for Target or Peloton or something, and then just immediately declared that we're not going to do this. They turned the whole thing down. So see, OpenAI is now actively backing away from ads.

Yeah.

Because of this backlash. What do you think?

Yeah, I think it's like, the good analogy is to draw back to like the Google days, right? Google early on, the way that all the search engines were monetizing was via display ads. And that was like something that the Google team didn't feel very excited about. And so they they've strayed away from it for a long time and focus really hard on delivering the best possible core experience to their users. And that's, I think, where OpenAI is right now. And frankly, where most of the foundational labs are in terms of just trying to create the best possible product and advertising and sponsored experiences and bringing the brands in. That to them is a way that you can bring that in later as a monetization effort. Right. And so for now it's like focus on the user experience. And there's merit to that conversation for sure. There's merit to that strategy. I think the economics of the situation are impossible to ignore that you're going to have to create more than one revenue stream than just subscription on the consumer side. Obviously there's the APIs and all the other business units that they have, but ChatGPT is losing $4 to every dollar or something crazy like that. That is just, makes it impossible to sustain that at scale unless they come up with a strong monetization method. And it will look something like when Google did search ads for the first time. It was very innovative on top of display ads, which is the only way that people knew how to monetize web surface areas, right? ChatGPT, I expect, will come out with something similar, right? When that happens and whether they can figure it out is a question mark because it's going to be really hard and difficult to figure out how we improve upon the search ad ecosystem and what that looks like inside of AI, which is like search, but also not really. It's used for tasks and used for solving problems and used for thinking and strategic partnership. And then sometimes it's used for helping write. And there's just, there's so many different use cases that are not commercial use cases. Right. And so figuring out how to, how to monetize that surface area is going to be something that requires probably years of work to get right, even with a very strong team.

So you're saying it's inevitable?

Basically, yeah.

If you were OpenAI, when do you think is the right time?

If I were OpenAI, I would say the right time is as early as possible because you have to show this. But I think that's because I'm thinking of OpenAI as like a business that has to make money and survive, right? I think if I have hundreds of billions of dollars in venture money to burn, then it's okay to delay that because you're focused very much on kind of making sure that you cement your leadership in this space that's obviously going to be very big. And so in that case, I certainly don't fault them. And I think that there's, I can really argue both sides, right? The earlier the better for them in terms of designing the product around commercial experiences. I do think that that will be a struggle to add on later for them and they'll fumble it and it'll be difficult and they'll probably get it wrong the first few times. That'll be okay. That's normal for businesses. At the same time, I totally understand if you have this huge war chest just saying, hey, we're going to focus really hard on making sure that we deliver the best possible product. Not to say that bringing in sponsored content inhibits their ability to do that. It's just a focus question.

I think this podcast, we're going to, we're going to talk for the rest of the remaining of the founders, not an OpenAI side of the world. We're going to cover the rest of us. Tell me a little bit, why did you start Koa? Yeah.

Yeah. So basically the story is that when we were, my co-founders, Mike and Eric and I were all members of South Park Commons and within South Park Commons, we were looking at all these consumer apps that were being built and people were leveraging what we talked about before. You've got all these people who are doing children's storytelling and all these people who are vertical search for construction workers and all these people who are like trying to build product recommendation tools. And there were all these different kind of AI, core AI experiences, these generative experiences that were popping up and people were, they would get 40 or 50,000 pretty engaged users and then they'd realize how expensive it is to maintain 40 or 50,000 engaged users when your core experience is generative, right? That's a much more expensive. Business to run, especially when your users are very engaged than a traditional app or a static web app, right? Because you have this inference cost. And so we saw all these companies that were getting a lot of engagement and growing really quickly just shut down because they look at the projections of unit economics going forward with subscriptions only converting at 3 to 5% and say, this is never going to make money. We're only just going to lose money forever. And I think even some folks were throwing in the AdMob or the AppLovins of the world and trying to make that work. And sometimes it was okay. And sometimes you could do a dropshipping thing and you could tweak buy, but it just, it was clear that there was not the right monetization solutions available for these consumer developers in the market for AI, right? And so when we saw that, we started to try to solve that problem. And originally we thought, can we get better at subscriptions? Can we increase the conversion rate from 3% to 4%? That was the original way we started thinking about it. But simultaneously, we were exploring more on the marketing side. And so I have a big network in there. My wife works in marketing. And so we were asking a lot of marketers what was top of mind for them. And the answer was always the same. What do we do about AI? What's going to happen in this world? How do we prepare ourselves? How do we take advantage of the opportunity to win in this new channel and this changes? And so we really just put two and two together. We're like, wait a minute, advertisers want to get into AI and AI companies need to figure out how to get paid. This seems like an obvious solution to just bring them together. And so that, that's basically how we started.

That's a very interesting story. But thinking of the— I don't think anybody have experience yet, right? Except for maybe Perplexity. That you were seeing some ads on Perplexity, but this is a new experience for sure. We've gotten used to ChatGPT interface for the past 2 years or so. The first thing I would think when I hear AI ads is like intrusive experience. Can you show us? Do you have a demo you can show us what a core ad actually looks like?

Yeah.

And I'm assuming the average person haven't experienced that.

Oh yeah, for sure not. So hopefully you can see here and then I'll go and show you one of our publishers, Liner. Can you see this?

Yeah, I can see it.

Okay, great. So this is one of our publishers, Liner, and this is just a great example of who are all these apps that exist. Liner is a search, AI search tool for students. And so you can see on this agent side, I think on the side, they're really like aimed a lot at like higher level education students and things like that. And that's their main core demographic. And we are, we power the advertising. Liner and help them monetize their search inventory. And for example, just to be clear about how things are non-intrusive, we're trying to keep it native and relevant. Let's say, and you can see some of my demos on the side here. Let's say I have prom coming up and I have acne. How can I deal with that? Something that, like, let's say like a high schooler would ask, and there's going to be this organic response. That comes together. And then we have our ad format here for one of our advertisers, Honeydew. And so the idea here is you can see that it's very native to the platform. It's in the same font, the same coloring, like brand associated with liner, not something that's like a huge banner ad. It's like flashing and trying to get your attention. It's really just trying to solve the user's problem, right? It's relevant to the query. It's bringing in some personalization, right? It's mentioning prom in the copy. And so it's trying really hard to make sure that the product is being recommended to the user is something that they actually find value in, right? And so if you look at some of the organic sources on the side, right, it's like, these are informational sources, which is very useful, but very few of them are going to be commercial sources. And so this is an area where Liner's organic responses are great for learning. And this is what most people use AI for, but when it comes to actually purchasing a product or like solving the solution immediately, this is where our recommendation system will come in and bring in the right small sort of experience.

Interesting. So it's standalone, so it's not embedded in the chat.

Correct.

Like one of the— you have 5 bullet points, it's one of it, and it says like a little sponsor tag on it. It's not like that.

That's right. And in fact, we actually— some publishers wanted us to bring into one of these and test that. And we looked at it, but the fact is that one, I don't think we want to get into manipulating the response itself and showing people that we're bringing brands into organic response. People want to read this information and hear what the model has to And so I think creating a separate standalone experience that is clearly demarcated as an ad, but something that is very highly relevant and hopefully useful to the user is both the best way to actually monetize that surface area and get paid as a publisher, but also the best way to make sure that you're giving the right recommendation to the user that doesn't feel like it's degrading the product itself.

Great experience. Yeah. Yeah. I actually envisioned, I thought it's going to be embedded in the chat itself. Yeah. Good to see that it's actually separate. Correct. Yeah. And do you know what, I don't know if you have any example right now, but can you show or can you talk about what traditional ads look like? If I'm a liner, if I just drop AdMods or AdSense on it, what would it look like?

I don't know. I don't have a great example for you here in terms of demo because I'd have to figure out how to turn ad blockers off and stuff, but it would look like display ads. And so maybe what I can do is just go to, okay, so basically I just went onto the New York Times, right? And it's here I am trying to read about news and like all I can do is look at this giant iPhone ad, right? And this is like an, this is basically an example of being like, wow, this is just like not additive to my New York Times experience in any way whatsoever, right? This is not what I'm here to do or anything like that. So you might expect that if we were to work with the New York Times, we would be, obviously we're embedded in the chat experience, but like we would expect to be something more native inside of here that is trying to add value, right? It's, these are all stories that are bringing on real information. And so we would try to allow the brands to enter in a similar way. As opposed to just like this, basically what is effectively a giant billboard.

Yeah. Thanks for sharing this. I remember seeing some sites, I was looking to buy a font for my design.

Yeah.

And those sites were full of those display ads. And I think over time I lose trust with the sites and I'm not sure if there's so many ads, is this trustworthy source?

Yeah, 100%.

That becomes a problem. How do you compare with your style with Perplexity? Perplexity is already running some ads with some follow-up questions, so sponsored follow-up. Different way of handling it?

Yep. Yeah, we tested and tried sponsored follow-up and a lot of our publishers didn't like it. And I think like one of the things to look at with Perplexity, and we really hope to work with them one day and think we could add a lot of value, sure, they tried ads and they went out and they went to market at a $50 CPM, which is really high. And they got a lot of interest and people were super excited to give it a test. And recently they shut the project down, right? And it's because basically they came back and said, this was really hard and we didn't get it right. And so we're going to have to revisit it again in 6 months or 12 months or whatever that looks like. I think that is a great indicator of where this, where we're at in terms of sponsored experiences in this environment. This is a really hard problem to solve. Even teams like Perplexity are struggling to figure it out. It's not something that you can just say, oh great, let's do sponsored follow-up questions and we'll do a— they used to have a video format in the upper right corner as well. And it is contextually relevant. There's accuracy there, but that's not sufficient, right? You have to. You have to understand what data goes into the targeting mechanism. You have to understand how to make sure that brands are well aligned to where if they get the clicks that they're looking for and they get the traffic they're looking for, do they know how to convert that traffic? Do they know what was the prompt that sent that traffic to them? Do they know like what the user's mindspace is in if they're shopping or if they're more on the informational side? There's so many things that are complicated about doing this that in order to justify an actually high CPM and a great experience, it's a very difficult product to build across both technical aspects as well as like go-to-market aspects in terms of bringing in advertisers and everything. And so I think there are others that are trying and you can even see like Character AI tried to bring in basically like a traditional interstitial ad. So like a full screen ad that comes up once you switch character, talking to a different character. And they got really bad backlash from their Reddit community because that's just not the way that people want to see ads. And so this is a, it's a hard problem to get right. And something that frankly is like going to be very difficult even for companies like Perplexity to build themselves.

Yeah, it sounds like everybody is in the, in the testing phase.

That's right.

We're just trying things out because it's a new experience, right?

Yeah.

I'm used to ads-free experience with chat. Why are you putting ads online in front of me? That's a big question from the consumer side.

Yeah.

How do you find good ads to show the users?

Yeah.

It has to be relevant, right?

Yeah, it has to be relevant. And so this is a search problem. And so it becomes really interesting, right? Because really what search is, right, is you say I have this incoming request and then I have all all this information, how do I surface the most relevant information? And that's basically the same way that we think about it. We have this request coming in, we have to determine and interpret the request, and then we have to figure out against our database of information, and in this case it's advertisements, right? What is the right thing to show? And is there a right thing to show is the big question, right? And basically like, you break it down into a traditional sort of like, really, it's a search question across all of it, and then you sprinkle in like the AI component. So there is some embedding involved, there is some semantic search involved, there is also keywords involved and static stuff involved, and you've got different columns that you're pulling against. So there are disqualifying columns, like if they're not in the right country, right? We shouldn't show an advertisement for United States based home insurance to somebody who's in the Philippines asking about. So location is a big piece of it. Demographic information is a piece of it. There's a buy publisher targeting that we can do. Some publishers blacklist advertisers. And so there's a lot of complication that happens there. And then there's a semantic component as well, right? So basically as the query comes in, We run multiple layers of analysis on that query to understand, one, is it a commercial intent query? Does the person have any interest in actually seeing a product right now, or are they just in learning mode? And if they are in learning mode, is there something different we can show them? The next thing we ask is like, basically, are there anything— is there any reason why we definitely shouldn't show a sponsor experience here? Is there profanity? Is it content that people— brands wouldn't be excited to be associated with? So there's some brand safety and protections there. And then the third component is basically, great, let's go see if we have a relevant if we have a relevant match, then we generate the copy for it and through one more layer that basically checks to make sure before the copy goes out, is the copy going to sound right or make sense given the rest of the content that's around it? And so we do all of that basically in many cases, as you saw in the Liner demo, before the answer loads. Because it can do most of that based on the query alone.

Do you always feed something back? Do you always return something to show?

We don't always return something. We have— About 30 to 35% of the time we'll return an ad right now. And the reason for that is basically that a lot of the queries are either not something that makes sense to show any kind of sponsored content in front of, or it's an area where, you know, frankly, like what we're focused on is user experience. What we're focused on is partnering with the publishers to grow their audience, to grow their application without, with a sustainable monetization method, but not, we're not trying, we're not thinking like, how do we juice as many dollars out of this? 6 months as we can. We're thinking, how can we build together over the next 5 to 10 years with the publishers in the network, right? And so we are extremely sensitive to making sure that we're only sending very relevant responses. And so when we think about it that way, like, even if something is, has a chance that it maybe could be relevant, but it also has a chance it could be very irrelevant, we will never show that. And that's basically the way that we think about it is how do we make sure that we're protecting the user experience above everything else, because that's how we build long-term with our customers.

Because I, because if I'm building an app, I don't want to having this feature and increase my user churn.

Correct. Correct. And that's a number that we care a lot about. And I think even when we work with, for example, Lucía, one of the biggest AI publishers in Latin America, they test user engagement metrics and they put us head to head against AdMob versus us. And for them, the user experience metric where, when we're putting against the test bucket is negligible, right? Whereas it's over 10% when they put it in something like AdMob, these display ads, these full interstitial video ads, like they're very disruptive to the flow of the user. And when you have AI applications, dynamic interfaces, all of these things that are popping up, like the user engagement is really high. With Character AI, the average user engagement per day is over 3 hours, right? That's twice YouTube and TikTok, right? It's insane. And so what you really don't want to do is throw a full interstitial ad in there and then lose the user because the user might be on there for another 2.5 hours, right? So if breaking them in the early part and like creating this very disruptive experience can be a very negative outcome for the publishers. And so it's something that we're extremely sensitive.

Uh, what do your users actually say when they see these ads? Do you have any feedback from your users? Have you heard of user complaining at all?

We've never heard of users complaining. We do some light user research and we definitely hear from our publishers as well. I think like the biggest thing is that we care a lot about relevancy and we get that wrong sometimes. And so sometimes people will say, hey, this is strange. This doesn't quite make sense. And so we continue to improve on that. We care a lot about it and we're doing everything that we can to really protect, again, the user experience. But of course, there are some times that we, that we get that wrong. And so we, we hear that from the users and we try to correct. But for the most part, I think because of, and you've seen the demo and understand where we're coming from, it's because of the way in which we interact with these applications, like it's what we're doing is really easy to just say, okay, this is not bothering me at all. Let me just like skip past it and go to the next thing. And so I think that we have not received like any particularly negative feedback from users thus far.

I would say that it's not a big pop-up window in front of my face. I could totally ignore it. It's just like the follow-up questions on Perplexity. I'm like, I never click on one.

Yeah, exactly. But you might, like, it's useful to see it. And sometimes you might say, okay, if you do click on it, it's because it's actually relevant. And so the really important piece of that is that one, it's fairly easy to say, hey, I'm just going to move on from this. But if you do click on it, it's on purpose and with a lot of intent. And so this is why we're able to have such strong metrics on our network, right? We're doing 2% CTR across the entire network. How is our CTR so high? Because we deliver relevant conversations, right? We deliver relevant recommendations and that's a really big component of it.

2% is to compare with AdMob or AdSense?

Yeah. So, if you look at traditional kind of display ads, they're usually in like the 0.4 or 0.5% CTR depending on where they're shown and how. And so, on mobile, the CTR is a little bit higher, on web, it's a little bit lower. Depends on where things are placed, but yeah, industry average is somewhere around half a percent. And so we're 4 times industry average on the display side because we're not really a display ad. We're somewhere in between display and search and we're native in, in, in the component as well.

Is it hard to set it up? Can you show?

Oh, you want me to walk through a setup?

Yeah, walk through a setup.

I can, I can show you our docs page. Yeah, like we, we've worked really hard on making it fairly easy to set up. And so our documentation is really simple and you can, anyone can access this docs page. I'm happy to share it in the show notes or whatever. But yeah, like we have SDKs for basically every mobile SDK that we need. And then JavaScript works for most people on web. And so the docs page is pretty short. And so you'll see it's a simple single line SDK. We'll give you a publisher ID, you place it in. And then basically all you have to do as a publisher is just tell us where you want the ad, right? You'll just give us the location that you want to place the ad within the call. And then, and then you'll send us your chat data. And so once we have the input. From the user, the LLM's response, and basically the, we have a number of different formats that we can use. So you would specify which format you would want to use in that particular location as well. It's a fairly simple process. And so that's, those are the 3 steps of, of installing the SDK, very simple. And then we have a styling guide as well in the documentation. So you can look into that and you'll have some adjustments you can make as a publisher to how you style it. As you saw in Liner, you can style it to be brand aligned with your application. And so. It's very simple. I think like the best way I can describe this is multiple kind of our customers have just come by our office and sat down with us for 2 hours and 2 hours later we're live. And so it is a pretty, pretty simple process.

Yeah. Thinking of how to set up AdSense, it's like the amount of documentation to go through and figuring that out. This is like elementary school.

Yeah, exactly. And so it should be really easy. And the funniest thing about all of this is like we get a lot of people excited about how simple our documentation is and we worked hard on design and things like that. And so that people are getting excited about that, but The funniest part about it is that actually doesn't end up mattering because what do people do? They just copy paste the entire page and throw it in the cloud and say, how do I do this?

Yes.

And so we've also designed some of our documentation around that as well to just make it very easy. So you can just like vibe code the integration. And of course that's not what we recommend people do. You should definitely be a strong engineer and actually do the tests and make sure that there's nothing else breaking. It is an SDK after all, but it is, it's yeah, most of our customers have been able to do it in a couple of hours.

So who is this for? Who should know Koa exists?

Yeah, definitely the people we're most excited about working with are the folks who have built something fairly niche with an audience that's really engaged, that are really struggling to just figure out like, how is this thing going to become sustainable? And if you are a developer that has 100,000 users or 50,000 users or even a million users, right? And you're just like, wow, this is great. I've created this thing that this audience is really engaged with that really, they really, but we don't, we're not converting enough on the subscription side. That is exactly the right type of person to work with us. Like we, we love helping smaller companies grow and become sustainable on the monetization front. We solve so many of the problems that come with advertising, all of the direct sales for you. We bring in the programmatic demand as well. We do all of the ad targeting, all of the serving, and we care so much about the user experience at the end of the day. So we'll work together on designs and we'll work together on like how to set this up in a way that facilitates the experience that you as a publisher, as a developer are looking for your users. And so. I, I'm, we're most excited about working with effectively what is like the indie developer, like small shops, right? That are building apps that are specifically very useful for a smaller audience.

If I'm building a live coding app this weekend, and when do you think I should think about monetization? Like, this is definitely not the first thing I think about.

Definitely not. I think the way to think about it is you probably want to plan from the beginning whether you want to run advertisements as a core monetization effort. And so in that case, we're happy to talk to you or give resources that help understand like, hey, how much money could I make from advertising is the right thing for me. You definitely want to think about that while you're vibecoding the app or while you're building something. But then I would say focus on the user experience, get 10,000 users that are really engaged first, right? And then figure out how you can deliver value to your users. That's definitely the most important thing to do right away. And then I think once you've figured out a way to get people really engaged as you grow and as you get back into a growth mindset and okay, great, how can I get from 10,000 to 100,000 engaged users? Now is the right time to bring us in. And build your sort of growth playbook around a monetization lever.

Do you think, because I think as a founder, I would think about how I make money, right? When I start this weekend, let's say. The first thing that comes to my mind is the traditional subscription model. Are you saying, should I think about giving, giving it out for free and just grow the audience first? Or what's your suggestion for these indie founders?

Yeah, again, I think my suggestion, especially if you're building something consumer or even prosumer, I would just focus on making something that people really engage with, right? And so I think the great example of this is, and everybody should just look this up, it's like this company Open Evidence, right? Open Evidence makes $150 million a year in revenue on a million users, which are all doctors, right? And doctors are very highly valuable as an audience because pharmaceutical companies all advertise to them. There's high income, so you can advertise Rolex and season tickets for whatever. And so it's a very valuable audience and, and Open Evidence has made a product that is very useful and loved by many doctors. I mean, when I talk to my friends who are doctors, they're like, oh, I use Open Evidence every day. Right. And so on a million users, they're making $150 million in revenue, all only from ads. There's no payment option available to them. And so I think that's, that's the best example I can give where of course subscriptions are a low effort way to do it. There are ways to kind of get that off the shelf and everybody, like there's no reason not to have a subscription model, right? That makes total sense. Makes sense for most people. But I do think that advertising is an underrated way to, to monetize. And the reason it's so underrated is because in the past, so many people have been burned by really bad advertising experiences, and we're here hoping to solve that. And then the second piece is that people have expected the advertising revenue to be fairly low in the past, because most of the time you don't have enough information to, to know a lot about your audience, how to target them effectively, how to give them relevant recommendations that they actually click on and then buy. And that's an area where we can help a lot too. And so not only do we feel like we can create an experience that's better for the users, we can also create one that's higher revenue per user, revenue per thousand impressions relative to the legacy players because of the ability to like target against these dynamic inputs.

Hmm. Can you share like average— I, I'm just representing an average audience who would want to build an application or become an indie founder.

Yeah.

They've been on the way. What's the— can you break down economics? Yeah.

Yeah.

What's the cost? What's the revenue model that they could think about factoring ads potential?

Yep. So the way that we basically think about it is every time you have a query, right, the user asks a question, that's an opportunity for us to show an ad, right? Some sponsor experience. So it could be in some cases you could show 3 or 4 sponsor experiences within one query, and sometimes you might not show one, right? And you can think about that in terms of like how closely related to productivity or search or something that like has a commercial experience is that query. The closer it is to something that has some commercialization component, the more likely it is that we can show a highly relevant and like cost-effective ad, right? The industry kind of standard on display. So let's say you just dropped in AdMob or you dropped in AppLovin, it ranges quite a bit, but let's say it's somewhere between 50 cents, like on web or on like smaller display ads, like in mobile. And then if you go up to full interstitial videos and things like that, you can enter the sort of $5, $6, $7 range in the most disruptive, like full, you've got to watch this 25-second video, otherwise you can't move through the app, right? And so people, so there's like ranges, right? That makes sense. For us, our average CPM across the network is floating around $5 right now. And so you get this like very close to the maximum you can get from the industry with the least disruptive format, right? And that's a format that you can fill at a very high rate. If you have a lot of relevant contextual content. And so the way we think about it is you would break it down into terms of what you think your percentage fill rate would be in terms of how many times, like how often do you think the query makes sense to show us an ad in, right? And then you would multiply that by the cost per thousand impressions. That's a $5 CPM. And so basically in that sense, you would just do that calculation and then figure out how many queries you think you're going to be processing every day. And that's the economics of the whole thing. And so. It'd be very similar to the way you would think about subscriptions, right? Which is like, how many users am I going to have? And then what's my conversion rate, right? And so you have to predict how many users you're going to have. In this case, you're just predicting how many queries you'll have. And then you just multiply that by— you can just use $5 CPM from us as a sort of average number.

So if one query sponsorship in an ad can cover like 3 queries cost, then I break even faster than a subscription model, correct?

If we're doing a 30% fill rate, exactly. That's exactly right. And I think in that case, like for most people, unless you're doing videos or images or things like that, it should well cover the cost of this. And so effectively you should be able to have sustainable growth at whatever scale that you need, because we have the demand on that scale that as we build out, we'll, we'll, we'll be able to fill in more and more inventory. We have a ton of demand side interest and people who are working with us already and big brands like Progressive and Choice Hotels and folks that are really excited about about what's going on in the AI world. And that's the beauty of Koa, right? Is like we can scale with the business from very small to very large, and we're happy with all types of folks as customers.

Is the best feature to integrate with Koa a chat interface?

If you use today, I think the way I would think about it is really just anything dynamic. And so we actually work with folks that have more of a forum type of application, or they're creating like generative news feeds, right? If you're thinking about news articles or just even just recommendation flows and, and we also work with, yeah, people who are really creating any kind of dynamic experience. And so chat is where we started and obviously there's a lot of user intent within chat and text-based is like how we started, but anywhere that there's a dynamic experience where the user can give us some level of input into what they're looking for, right? I would say a good example of this would be like things that are not a great fit for us is purely entertainment-based platforms where the user is just there to waste some time or hang out or just socialize or whatever, things like that is not super useful. The things that are very much closer to areas where I think we would have a lot of success with developers is if there is some kind of productivity element to the application, right? And so if that's a newsfeed, that is productivity. People are learning, they're gathering information. If it's an AI email tool, that's more of a prosumer tool. It's clearly has some productivity in mind. If it's a search tool, that's really great as well. I think anywhere where the kind of the user is there to learn something or get better at something or solve a task, right? That's an area where we could enter in. And I think within a dynamic experience, the thing that really matters is the user giving us an input that we can use as proprietary first-party data. That is how we are able to deliver such high CPMs, such relevant content, all this stuff, because we actually know what the user is there for.

What do you mean? What is the data exactly you're using?

Yeah, just any, anything that, that the user can give input to the application. So if the user can come in and say, here's what I'm looking for. I mean, that's search. Or if the user can say, hey, can you help me write this email to my boss? That's like a text. Task, right? If the user is able to come in and say, hey, I'm reading these articles because they're very engaging and these ones aren't, that doesn't have to be a text input, right? But we have some other input of what the user is like excited about or not, right? So some, something dynamic where the user is able to interact with the website or interact with the application that you're building and give either direct text-based feedback is the easiest way to do it. But if they're able to indicate in other ways what they like, what they don't like, what they're looking for, what they're not looking for, that is also So an area where I think we can succeed.

I was thinking, are you also factoring in the publishers, right? The developer side of things, whatever the product that they're building, do you factor in like their brand and their product, whatever the service they're trying to provide as well?

What do you mean by that?

Like Liner, right? Liner is servicing students.

Yeah.

So you actually factor that in as well, that this is the audience for this topic. Correct.

Yeah, exactly. And so we know that's an audience of students. And so then, yeah, of course. So that's an area where like—

Consumers, right?

Yeah. And so that's one of the things that's really interesting working with Liner is like we start with search and then we have this sort of dynamic interface of just AI search, right? But then are there other ways that we can bring relevant content to students, right? Student might not be using Liner, for example, to buy a computer or order food, right? But we might be able to bring in brands to give discounts on like new laptops or give discounts on ordering DoorDash or whatever it is. And so That might be something that audience is very excited about because they get additional value that is not core to that app. So that's like an area we're moving into and experimenting with some of our publishers depending on if that's something that they're excited about or not.

So if you existed a year ago or 2 years ago, then some of your friends will be saved.

That was our idea. And that's why I think all great companies are built from serious pain points. And like the serious pain point here is, wow, these companies, just these people are having so much fun and their users are so engaged. Like, why don't they deserve to live? And so hopefully we're going to be able to make it so that Not just can our friends' companies in that case have survived a year ago, but could we take every 1 and make it 10? Because people realize that, yeah, I can buy Foodbin and Nappa over the weekend and I can make a few thousand bucks a week. That's amazing. That's amazing. Yeah, exactly.

So does the developer side, do I have any control? Like in terms of what ads to display? Because my core purpose is to monetize, but in the meantime, protect my user experience.

Yeah.

So I want to know, like you mentioned a lot of things, relevancy is a, is a top priority, but I'm wondering if there are any other measures that you do to protect my brand reputation, including do I have any control?

Yeah. So right now you do have control over like the visual components, how it looks, where it's placed. We also have like moderation abilities. So if you can take down an ad after the fact, if it's like reported by a user and we have feedback forms and things like that, or like the little feedback icons, the user can tell us that this is like a problematic ad and we can take it down. And there are safety controls around that. In terms of just the way that, can you choose exactly which advertisers are shown and things like that? Yeah, you can if you have a great relationship with us, but the product is not really built for that just yet because it's changing so quickly. Like we're growing so fast on both sides of the marketplace that it's just like too much to keep up with as a publisher. And so the way that most of our publishers work with us, and we have Slack channels or email channels or even like weekly calls with a lot of folks that we work with to get feedback still, right? And so even though we have dozens of publishers, we still want to meet with them all the time to hear how we're doing and get feedback and the, the control you certainly have is just telling us if there's something that's wrong. And so we'll work with you on that. But I think the easy thing is most folks are just like, hey, this is awesome that we don't have to worry about this. And we've established a lot of trust with our publishers that this focus around user experience is important to us. So typically the way that we work with our publishers is we tell them when there's an issue before they've realized there's an issue, because it matters so much to us that we get this right. And we'll send them a note and say, hey, we're really sorry, but we noticed that this thing was a problem. Here's how we either resolved it or are planning to resolve it. And most of our publishers love that we communicate with them that way.

How do you know? How do you know that there's a problem?

Yeah, of course we have all of our own internal flags. And so if we see volume drop on a publisher for whatever reason, or we see maybe like an advertiser not succeeding in the way that they were before, or perhaps we see like a lot of reports coming in for specific ads. So there's different flags that we've created inside the network to tell us is there something that we should be looking into a little bit deeper to see if there's a problem? And so sometimes it's even things like all of a sudden our conversion rate from click to page view dropped significantly because let's say something on the publisher SDK side was breaking when we broke our— brought our browser up, right? This is something our publishers would typically not be able to see because they would have not only had to see the ad, but then click on the ad and then try to load it. And they would have to be in the same environment as the user, right? And so we can say, hey, on all of these device types, in-app browser is broken. That's creating a bad experience for users. We tell the publisher about that, right? And then resolve it.

Do you report to me any metrics?

Of course.

To help me understand, like, if the ads are helping or hurting?

Yeah, I think we really want to get to the point where we are measuring our side, the ad quality, right? And the user experience metrics. And so we do a little bit of proxying around that in terms of session time and follow-up questions and things like that. And so we do a little bit of proxy measurement on our side and we'll get better at that. For now, we do report things like CPM and click-through rate, which is the easiest indicator, right? So how do we know if ads are relevant? Click-through rate is high, right? So if we see an advertisement or one advertiser all of a sudden have a very low click-through rate relative to other advertisers, it's something for us to look into, right? It's sometimes that the advertiser is very irrelevant. It's very frequently the case that the copy that's being generated based on the way that they set up the ad is, doesn't make sense or is like in the wrong language sometimes or things like that, right? So. Most of the time CTR is the right metric to look at in terms of like how a global metric that indicates whether we're doing a good job or not.

So you do use LLMs in the process.

We do.

So if that's the case, is hallucination a problem?

It can be. I'm never going to say that hallucination is never a problem. We have never seen it become a huge issue. Occasionally the copy that's being generated will say like something like, it'll say something that doesn't really make sense for the product. Because for example, an example I can give of this is that we had this supplements product that was Or not supplements, but more of like athletic food. So think protein bars and little energy chews and things like that. And the user was asking about what are the best national parks, right, in the area. And our system flagged this and saying, hey, this person's asking about national parks, then we could also give a recommendation for that. Hey, if you're planning on going for a national park, like, you should use this product, right, if you're going to go out, because it's a great way to make sure you have the provisions that you need. But the way that the copy was actually generated is it was like, hey, this is the best national park. So our ad was coming out and giving an opinion on what the best national park in Brazil was in this case., and then later we said the things that make sense about you should consider using this product. But it was just funny that you wouldn't expect the advertisement to come in and say, here's the opinion that I have on what the best national park is. And so that's an example of where the thing, the system doesn't work quite right. And so we usually flag that.

Okay. And I want to ask you, like, why can't, why shouldn't I, or why cannot I do, just build my own ad system? Here's what I'm thinking, right? If the problem we're discussing is related to user experience. Can I just take the AdMob data and then make it look better myself?

Yeah, with AdMob in particular and some of the mobile ad networks, the simple answer is you cannot because they won't just give you like the advertisements, right? The way that you would have to do this if you want to build it yourself is you would have to build a connection to the programmatic demand side, right? And is this super hard? No, you could do it in, you could probably do it in a couple of months. Great developers could do it in a couple of weeks. There's documentation public on prebid and like the OpenRTB spec, and you could definitely build out publisher-side ad servers and all these things. The problem is that while you might be able to stand up something that, you know, works, there'll be a lot of issues around brand safety, a lot of issues around verification, a lot of issues around, you know, what we're talking about, like hallucinating. So it's not just the format piece, it's a lot of the targeting ad server piece as well that is a very complicated and deeply technical product, right? So there is a component of just, it is possible, but it also, and it might take only a few weeks to stand up a simple product, but to do it And to be able to justify the CPMs that we have and bring in really high quality advertisers, you need to have a very strong product. And that's something that takes— it's taken us a year plus to build something that is functional in the way that we have now. And that's our sole focus. So I would say like—

How many people on the team?

So we have 12 on the team now.

So it took 12 people a year to build it.

That's right.

That's right. Maybe.

Okay. Yeah.

I'm convinced.

And then the other piece of this is— yeah, yeah. The other piece of this. So there's a technical reason why, and then there's of course a go-to-market reason why too. To do, right? Going out and actually making the deals with all these demand-side players, whether it's programmatic players, and you'd have to make deals with a dozen programmatic players. Or if you wanted to go direct, which is how you get the best CPMs and the best deals and the most relevant content, that means having an entire advertising sales team to go out and sell these different brands. Whereas we bring in Progressive and Choice and Intuit and like all these big brands, right? Like otherwise your team would have to go and sell those people directly, which is obviously something that's very difficult to do. So I would say, yeah, exactly. There's two reasons why not to do this yourself is because one, it's hard and two, it's really hard to sell. And then of course the last piece is, and even if you can do it all, you might still make less money by yourself, even though you're saving. Obviously we take a platform fee, but it's so great. You can take the platform fee, go past the platform fee. Hopefully we're driving enough efficiency that even with our platform fee, you're making more money with us than you would by yourself. That's the goal.

I'm convinced this doesn't make sense for me to build it. Yeah. So I'm working on AI visibility kind of thing. We talked a little bit about it. So generative engine optimization.

Yep.

So here's something that I see in my work, and I think this is commonly agreed in the industry. People discover things in AI chats, but they go somewhere else to actually buy. So how do you prove to your advertisers or your publisher that the ad actually worked?

Yeah, this is a really interesting problem that we're trying to figure out. So one is our long-term goal is to make the right commercial experiences so the users don't have to do that, right? The users feel like the best possible way for them to complete the purchase journey is to just do it from within the app. This is the best thing for the publishers because it means they got full attribution for that purchase, which means our CPMs will go even higher than they are now. So how do we justify the advertisers today? I think we explain to the advertisers that frankly, and this is something we were talking about before, like AI is not a shopping platform yet, right? Google Search is a shopping platform, but also has all these other use cases, but eventually they got so large and became so well known for having like very strong intent and going into the commercial aspect that they were able to monetize that way. AI hasn't really done that yet. The main use case for AI is basically something between roleplay, which you can see is like a huge component of Gemini use and ChatGPT use, and like programming, right? It's like when you're having like AI help you write code, like you're not really trying to buy anything, right? So this is different. It's a different opportunity for the brands, right? And so redefining what success looks like aside from just did this person buy a thing, it's not even necessarily an e-commerce opportunity. Opportunity, right? This is an opportunity more for the brands to show up in the chat interface or dynamic interface and actually add value to the user and earn trust, right? And so this is more of a trying to add value as a brand to these end users to create a community of people who support that brand, right? And then hopefully that leads to organic purchases later. And then eventually, hopefully it leads to on-platform purchases. And obviously that's an area we want to get to. But the opportunity for the brands right now is what we call more of a middle of funnel opportunity, right? So it's one where the user is in the information stage, they're gathering information, they're learning, they're researching. Doing. And that— how do you show up in a way that adds value to the user when they're in that headspace is the problem that we're trying to solve. And that's something that we communicate with our advertisers to make sure that they're aware as brands that the opportunity here is not one that they should expect to click into a purchase at some percentage rate. That's just not where we are yet in terms of consumer behavior on AI.

I feel we should collaborate. I'm working on this exactly with a slightly different angle.

Yeah.

What are your thoughts on the DAO space.

Yeah, we know a lot of those companies really well. Good friends with a few folks that, like, leaders in the space. We love that, like, environment and think that they're doing really good work. I think, I think brands desperately want to know more about what's going on in AI, and so it's like a great solution for a lot of marketers and advertisers. And almost every single advertiser we work with has a GEO partner, right? And I think it's an awesome space, and those guys are crushing it and doing really great work. And I hope to collaborate with more of them someday so we can do organic and sponsored stuff together, because that'll be really fun. But yeah, I think the geo space is really interesting. I think what people will eventually move towards is that analytics is a great place to start. You have to, as a company, move on from just there and help the brands actually solve the problem. And so I think the best companies in the space are doing that, helping generate blog posts and help create the content that's necessary and helping restructure things for the crawlers. And there's a lot of things that can be done to make sure that Brands organically show up better. And then I think the most obvious kind of step for the geo folks, and hopefully this is an area where we can collaborate, is go find somebody who's actually aggregating on the supply side and actually help, help create the sponsored experiences within AI as well. I think when ChatGPT inevitably opens up ads, some of the geo players will probably pivot into buying agencies for advertisements on ChatGPT and others. And the best folks hopefully will partner with us and then be able to have not only access to more real-time data for analytics themselves,, but also an opportunity to upsell their customers into sponsored placements on different applications across the world that gives them a sandbox in which they can test different messaging, understand their users a bit better.

Yeah, I've came to the conclusion that this is a data science project. There's a lot of data that we are working with to really understand where people are showing up and also where people are transacting.

Yep.

So there are two things, right, people can do. One is the ad side, which is working with you. The other side is working on the SEO side so that they can show up organically.

That's right.

Are there any other objections that I haven't covered?

No, I think this is great. I think it's a, it's funny, like it's a simple product and we win on metrics. So usually it's pretty easy as long as people, anyways, I think when we show people how non-intrusive it is, it's very hard for people to say no. They're like, there's just, let's just give it a shot, especially when the numbers back it up.

Yeah, that makes sense. So what's the next for Koa? Anything you can share about what you're building?

Yeah, so next up is big focuses for us in the next couple of quarters basically is we built out a lot of the team. We'll continue to grow the team. We will focus a lot on brand safety and advertiser success. So we're really excited to figure out this middle of funnel stuff that we talked about. How do we get the advertisers to really set up for this? How do we make sure the post-click experience makes the most sense for when we're sending the traffic with these like generated copy? Can we make sure that the, when they land on the advertiser's website, it makes continuity, right? And so we have to figure a lot of that stuff out on the advertiser side and just like adding more sophistication to the product for the advertisers. So that's a big focus for us for the next few months. And then the other big focus is basically figuring out how we can expand out through channel partnerships and through other partnerships with publishers. So one of the really exciting things that's happening for the business is a lot of traditional publishers are creating these dynamic experiences, chatbots within their websites, AI experiences within their apps. Whole new apps that are distributed to their audience that are just AI-focused entirely. And examples of this would be like Quizlet or, InMobi has this like weather company called Skyla and it's becoming like, like an AI surface area. And so there's a lot of, uh, really exciting stuff happening on the publisher side in terms of people moving into the dynamic internet, as we call it, that is going to be a core focus for us in Q1, Q2 next year as well.

What is the dynamic— what you just mentioned about dynamic internet?

Yeah, just if you think about everybody was excited about the Web3 world of like crypto becoming the sort of infrastructure layer of how we do transactions online and you can pay with each other and edit the internet and things like that. I think what's happening now is the entire internet's going to go from something that's like a passive experience to an active experience, right? One where you're no longer just reading the New York Times. Now you're interacting with the New York Times. You're telling the New York Times, what about this? And can you tell me more about this story? And let's go deeper in this topic. And that's the way that people are going to want to interact with the internet as opposed to just like passively reading stuff. And so that's anytime we have an experience where the user is able to tell us with like more than just a click, what they're looking for, why they're here, like what they're excited about, what they want to learn more about, things like that. That's an area where we can come in and really help monetize that surface area. The early adopters on the traditional media side are starting to do stuff like this, right? Create AI search bots on their platform, on their website, create full, fully like dynamic experiences within like an AI search experience or even one where like they make some recommendations and then ask for user feedback and the user can tell type in feedback or things like that. So that's what we think about in terms of dynamic internet is that this is a trend that's likely to change, right? Where we—

it's like we're meeting in the middle, right? The traditional companies are coming towards the AI side and AI company is thinking about let's make some money from the ad spend.

That's right. That's right. And I think we can talk a whole nother conversation point around agents, but it's like agents facilitate a lot of this, right? Because now as a user of the New York Times, for example, I can go on and say, hey, New York Times, do this thing for me, right? And then New York Times can deploy an agents on their backend to go and figure out how to solve that problem for you, and then they can come back and present the information to you, right? And that's the way we expect the internet to change.

Yeah, let's do that. Let's, let me pencil it in. I know like I always say it.

Yeah.

I have a traditional closing question. If you were to recommend one book to a 20-year-old or one to a 40-year-old and one to a 60-year-old, what would they be and why?

Yeah. So for a 20-year-old, That's the only age of these different age groups that I've actually been. So I can tell you from my own experience, when I was 20, I was definitely like in search mode of who am I and what do I want to do with my life and this kind of stuff. So one of my favorite sort of coming of age books, and it's fairly short, is The Alchemist, which is a fairly old, very short book, but it's sort of, and it's very abstract. And I know that it's like a polarizing book, but I really loved it in terms of just, it's a main character who's really searching for who they are in the universe. And I think that's like something that I would definitely recommend. For 20-year-olds that are trying to search for why do you exist on this planet? What's the thing that you want to dedicate at least the next decade to? I'm definitely more of like a fiction and science fiction kind of guy. I have great like nonfiction books I can recommend too, but my list here is definitely more on that side. And then for the 40-year-olds, I think of a lot of 40-year-olds as having oftentimes like new parents and people who are settling into more of their life. And so one of the books that I've read that I've definitely recommended to a lot of my, my 40-ish-year-old friends is Tomorrow and Tomorrow. And the reason that I am excited about that is it is really like this, a similar story to so many stories that have been said before, but it's so relevant to like the Gen Z audience, right? Or so relevant to like the Gen Z story of, hey, like my, like games and like the internet is like a core part of my experience as a person. And so that it does a really good job of explaining that book, like walking through the story of a person who is ambitious and trying different things and like learning about life and and going to college and starting a business and all this stuff, all, all, all around video games, which is a huge portion of so many people's lives. That is something that a lot of 40-year-olds I know don't understand cuz they didn't grow up with video games. And then yeah, my last recommendation, which is a book I've really liked, it's a bit dark, but it's like worth reading, is, and for the sort of 6-year-old audience, would be a book called Educated. It's a book about a woman who was basically not allowed to go to school by her family in Utah until she basically broke out and found a way to, to go to college. And then her entire life changed and she wrote about that experience. And so it's based on true story. She wrote it herself and it's based on her own experience. And it's one that I think for a lot of folks that are like in that reflecting on life stage and is really interesting read in terms of just like, hey, what really matters is that you get to choose your own destiny and that you actually have access to information and can learn and try different things. And I think like for a lot of my, my parents are in their 60s and I think as somebody like recommendation to them would be like, yeah, a reminder that there's still lots to learn and lots of things to do. So go out and get it. That's what freedom really Yes, I will recommend to my mom as well.

And I think I'm not there yet, so I'm, I'm, I can read all these 3 new books. Thank you for your recommendation. Really enjoy chatting with you, Nick. You are, you seem to be one person who's choosing your own destination. Sorry, choosing your own destiny.

Yeah, 100%. Thanks, Angelina Yang. I really enjoyed the chat. I, I believe in freedom and I believe in choosing what you wanna do with your life, and I'd rather do the thing that I wanna do and that I feel like is important for the world to have and fail than do something different. So. Happy to be here. I love your attitude.

Yeah, thanks. Thanks for being here. You speak so fast. I feel like hard to catch up with you sometimes. Fighting myself.

My brain moves even faster.

I can tell. This is a jam session. You shared a lot of insights. I feel very useful for founders as well.

Yeah.

Whoever is thinking of building something and trying to monetize it. It's really important.

Awesome. Yeah.

Thanks for sharing all these tips.