The Example Engine: How Exa Is Creating the AI Librarian for the Web with Will Bryk, CEO of Exa

I think there's an interesting distinction between search and research. Google is a search engine. You know, you kind of know what you're looking for, but when you don't know what you're looking for, you're more doing research. And I think that's where Exa shines. What we're doing at Exa is we're kind of like trying to take all the world's knowledge and putting it into a new type of database, like a neural database. I like this database analogy because it's not really search. It's like you're kind of like with every query, you're filtering the database of all the knowledge into just what you need. Because the problem with, you know, search engines like Google is like you search something And they say, you know, 33 million results at the top. Like, what am I supposed to do with 33 million results? There's no way all these results are actually what I'm asking for. So it's just like, you just feel overwhelmed. From the product perspective, like, not every query should require the same amount of compute. Like, Google has kind of made this assumption that, like, no matter what query you type in, it takes a few hundred milliseconds. But certain queries are extremely complex and might require, like, scouring the internet for, you know, maybe even seconds or minutes. Thinking of it as more the optimal trade-off is cool. It's like you're always optimizing for exactly the amount of effort to put into the thing.

Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week we'll explore their revolutionary ideas, and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Thornberg. Hello, and welcome back to The Cognitive Revolution. Today, I'm excited to introduce you to Will Brick, founder of Exa AI, a company building a new kind of search engine designed with AI systems and workflows in mind. Noting that Google has limited people's imaginations about how to search the internet, Exa aims to help people surface things that were previously impossible to find. Now, if you've been following the show, you know that I've been exploring a variety of exciting new information and knowledge retrieval tools over the last few months. Today, ChatGPT remains my go-to workhorse for ad hoc coding and other random tasks, though recently Gemini Advanced and now Claude 3 are both gaining share. Meanwhile, Gemini 1.5 Pro has become my favorite for all write-as-me tasks, including creating the first draft of this introductory essay, which I did edit quite extensively, but nevertheless saved me a lot of time. Perplexity is still my go-to for quick and accurate answers to specific questions, but u.com research mode has now taken the prize for deep-dive, multi-page research reports. And meanwhile, Elicit is the most useful for structured, systematic academic literature reviews. So, in this increasingly crowded landscape, where does Exa fit in? Well, for one thing, while most of these products are aiming to provide answers to questions, Exa still returns links like a more traditional search engine would. After playing around with it for a while, I've realized that it's the quality, the controllability, and the depth of results that really sets Exa apart. And I've come to think of it as an example engine. To understand how valuable this can be, consider that just about every company I talk to would like to scale highly personalized communications, whether for lead generation, recruiting, or something else. Today, language models make this dream realistic. Given a list of leads or candidates and a bit of information about them, a language model can personalize your outreach at roughly human quality and far more scalably than was ever possible before. But where does the list itself come from? If the inputs to such a process are low quality, then the whole process will be garbage in, garbage out. And this is really where Exa shines. Given a complicated multi-part query, which can be up to a full paragraph in length, something that you wouldn't even bother to try with Google, but which language models can very quickly generate, And optionally an example of what you're looking for, Exa returns lists of a certain type of result, whether they be companies, people, how-to articles, you name it. And then you can feed those examples into your broader AI workflows. In this conversation, we get into not only how Exa supports previously impossible-to-scale use cases, but also how it works under the hood, including the nature of their web index, the reasons they are rolling their own vector database, and how they are planning to extend their functionality with knowledge graph-like structures in the future. Will's ambitions for Exa are huge. He aims to build a leader in the new information landscape. And my experience with the product suggests that Exa's API business is likely to be booming for the foreseeable future. And also that the ways in which information is found and processed across the economy may change even more dramatically than all the new web experiences suggest. As always, if you're enjoying the show, we appreciate it when you take a moment to share it with friends. I expect that everyone will find value in this episode, but I would particularly encourage you to send it to anyone you know who's building AI task automations and who would benefit from bigger, higher quality input lists. And of course, we always welcome your feedback. I have accounts everywhere, so please feel free to DM me on the network of your choice. Recently, a listener named Ashwin DM'd me on Twitter to offer to send me a piece of original art. And if you're watching the video on YouTube, you can see it right over my shoulder. He did not ask for any promotional consideration, and I certainly can't promise to promote everything that folks might send me, but I was super flattered to get such an offer, and I'm happy to point you to his website, ashwinsartstudio.com, for more of his detailed and intricate drawings. Now here's my conversation with Will Brick, founder of Exa AI.

Will Brick, founder of Exa AI, welcome to The Cognitive Revolution.

How you doing? Glad to be here.

I'm excited to talk to you about what you are building with Exa. It is an AI-first search product, and we're going to get into all the nuances and details of that. And really a quite different one that has caused me to think deeply about how I think about the different ways that I go about searching for, retrieving, and processing information. So I think this is going to be a really fun conversation. For starters, you want to just kind of give us a little bit of context on who you are, what Exa is, and how you are thinking about the world of information and how that contrasts against knowledge.

So Exa is a search engine built for AI systems. Really, we're trying to redesign the search algorithm itself. You know, like people have been used to Google for a long time, but now we have technologies like GPT-3, GPT-4 that understand text at the level of a human. And so the inspiration for Exa was what if we combine the power of language models, which feel like they understand text at near human level, with search, which feel like it hasn't changed in like, you know, a decade. And that's what we set off to do. And we've been working on it for a couple years. We're really like a research startup. So we have our own GPU cluster. We train our own foundation models for search, really trying to just like ultimately solve the hardest queries, the most complex ones that Google is really bad at.

I read extensively through your website. One of the quotes that jumped out at me there was, one thing we've realized is that Google has actually limited people's imaginations. When it comes to what you can find on the internet. And we are still expanding our conceptions ourselves. I'd love to hear a little bit about that kind of intellectual journey. Maybe you could start with what are some of the queries or questions or information needs that you think are not well served by Google or the last generation of search options? And then maybe tell a couple of stories along the way of kind of moments or use cases that have changed how you think about it?

It's funny, people think that search is solved, like Google basically solved web search. When you start thinking about it more, you realize there are all sorts of types of queries that are just like completely unsolved. Like for example, you don't go to Google to find a list of startups that are applying AI to law, uh, because when you search that on Google, a list like startups applying AI to law, you don't just get a list of startups. And then certainly when you start adding more modifiers like startups applying AI to law in the Bay Area, you don't just get a list of startups applying AI to law in the Bay Area. You keep adding, making the query more and more complex. You realize like, wait, Google's just not doing that. And then more broadly, anytime you try to get information and you're not using Google, it's because Google has failed you. So if you're going on LinkedIn to find people or PitchBook to find startups or Twitter to find high quality news, it's because you can't find that on Google. You don't know the right keywords to search. Like, even if you're trying to find a date, like, you're not going to Google because Google is not like a complex— it's like not a powerful enough search engine to like filter all the world's information because a lot is out there into exactly the knowledge you want. And so our vision is be able to, you know, in real time, take the petabytes of information that's out there and filter to exactly what you asked for, exactly the knowledge that you want as fast as we can.

So the example of startups applying AI to law in the Bay Area is one good example. What other kind of key queries or sort of, you know, canonical use cases come to mind for you? Especially I'm interested in this, this process of expanding your own conception.

There are so many. Okay. So like, I want to find a designer and I, I, I know of one that has a really cool style. So I have like, maybe, okay, let's say I have a webpage. of a designer that I really like and I want to paste it into Google and find similar designers. You can't do that. Like similarity search is just something that's really hard when you're— when you don't, like, again, like when you don't know the right keywords, Google fails you. And so when you try to do similarity search, it's like, you know, searching with, you know, a huge webpage as a search itself and Google just can't handle that. Whereas Exa can. Yeah. Like searching for people, like you want to find like researchers like in the Bay Area that have Rust experience. Like Google does not give you like a list of researchers. So you want to find people, you want to find companies, you want to find really high quality articles that have a certain argument. So like articles that really go deep into like the power of Rust over Python for multi-processed applications. Like you don't think to type that into Google because you know, it's just not going to work. Yeah, it's really just like just any sort of complex query that you might want to make, Google fails.

Yeah. Okay, interesting. So I have been kind of on my own personal journey of trying as many of the new information products as I can get my hands on over the last several months. We've actually had a few founders of other notable companies on the show as well. So we've had Arvind from Perplexity, We have an episode with Richard Socher from You.com, and we've had the founders of Elicit on as well. And maybe I would just kind of compare and contrast a little bit my experience with Exa against not just Google, but against those. And I'm going to try to come up with kind of the right label or sort of category name for each of these products. They're all a little bit different, and it's quite exciting, actually, to see that there are just people going in conceptually different directions and not just competing one for one. So when I think of Perplexity, I think of kind of answer engine. It's like I go there if I have a question and I want the fastest, most kind of clear, accurate answer to it. I think they do really well with that and they give you an answer in paragraph form. I go to you.com these days and I think of that a little bit more as an autonomous research assistant, where especially in research mode, which is my favorite mode from u.com. I can give it kind of a multipart question and it'll go out and do multiple rounds of searching, processing the pages, trying to synthesize the results. And then with Elicit, I think of that as more like a structured literature review tool where it brings results back not in paragraph form, but in tabular form. And it's like, here's all these papers and here's these different dimensions and here's how— and they're very focused on academic content. And here's how we sort of assess each of these papers on each of these dimensions. And you can kind of add columns and you end up with something that's like a very structured sort of layout of the information that you're engaged with. Exa, I am using the term example engine so far. And I think what is really interesting about the Exa experience is that the input is very different and the nature of the results are very different from what you would see with Google. But the sort of presentation is actually maybe in some sense the most traditional in that it gives you kind of a list. But I guess I have a number of questions like that about this. One, how do you think about the example engine label? You can compare and contrast to the other ones as much as you'd like. And I'm interested in kind of a number of the design choices, but let me just let you react to all that first.

The label is difficult because even we who've been building it for a long time don't really know the exact thing to call it. I'd say it is particularly good for research. So maybe a research engine might work because if you think about it, Google and a lot of these, the tools you mentioned are built on top of Google and they add some sort of post-processing step. But ultimately, the results they're using are based on a keyword-like search engine that Google uses. And I think there's an interesting distinction between search and research. Google is a search engine. You kind of know what you're looking for. Taylor Swift boyfriend, you type into Google, you get that thing. Or you search my name, you get content surrounding my name. You know what you're looking for. But when you don't know what you're looking for, you're more doing research. And I think that's where Exa shines. You type in a natural language prompt, we call it, explaining the type of thing you want, and you get a list of things that match that. So I think research engine, It's kind of like EXA feels more like a library. You go up to the librarian, you say, hey, I'm looking for startups applying AI to law. You don't know the right keywords to use there. Some of the startups applying AI to law might not mention AI or law. They might say, we take a certain machine learning algorithm and apply it to legal documents or something even less keywordy. And the librarian knows a lot, very smart, and is able to point you to the things that that match that. It's more like a research-like experience. And yeah, in terms of the UI, we're really focused on the API and the UI. We always want people to be able to use Exa, and we are figuring out the right user interface. Right now it's a list of links, but that's not ultimately the right user interface for consumer search. I certainly like LLMs will and are a huge part of the right consumer interface. And I think there are really interesting things coming there that we're going to explore. But we're, yeah, right now it's just, we want to like just show people exactly what the API returns. But I think more we're going to move towards a consumer interface that is more like the products we want people to build with Exa. So more like a researcher type, type interface.

Hey, we'll continue our interview in a moment after a word from our sponsors. Okay, so that's maybe a great segue to what are some of the things that you have seen people build or want people to build that you think are most exciting? And I can say, in just kind of testing this and working with a friend of mine who introduced us, who he and I are working together on a number of different kind of prototype applications for this company, Athena, where we're both advisors, he has done some really cool stuff with similar companies, similar candidates, similar articles, kind of creating synthetic thought leadership or synthetic plans. He uses it for, if you want to, because the assistants a lot of times don't have the expertise, but the theory is it's out there, right? So he showed a great example the other day of, I want to increase my oxygen capacity in my lungs, and what do I do? That's one of my goals. But how can the assistant help? Unfortunately, the assistant in general is not going to have a lot of expertise there. But what we're starting to patch together is like, well, let's go find some expert content about that. That's where Exa comes in. Like, here's a list of articles about how to improve your lung capacity. And then further processing downstream with language models, you can get to practical steps. Well, maybe we could order this machine, or maybe we could sign you up for this. online exercise course or whatever, right? You can start to go from the goal to unpacking the goal with hopefully expert content and then starting to create concrete actions that the assistant can actually do for you and advance, help you advance toward your goals. That's the sort of thing that we've been experimenting with, but I'm sure you have many more scenarios. I'd love to hear about some of your favorites.

Yeah, people build all sorts of things from like the very businesslike and, and, you know, professional to the wacky. And so just an example, like one tool I, I'm particularly proud of is it's a paper writing tool that helps people write papers and it uses ChatGPT to present so many different ideas for what to write next and can unblock you and may inspire you to think of like new ideas. And, but ChatGPT, you know, is not smart enough to do that on its own, so it needs to rely on a search engine. And if it relies on a traditional search engine, it's pretty limited. It can't search in all these powerful ways. But when it uses Exa, it's able to recommend really good papers that really exactly match the type of thing you're writing about or exactly follow the type. You could say, what's a paper that follows this idea? And you could paste the idea. That's a crazy type of search that you can't do on Google. Here's a paragraph or 2, and then like, what's an idea that would naturally follow this? And you could do that with Exa. We're helping academics learn about their field because they've for 2 decades been using a tool that was limited in certain ways, and now we're like opening up the world's knowledge. Like, that's, that's exactly what I got into this for. Um, and okay, so that's just one example. Like, another tool that I'm happy about is, you know, a tool for VCs to like source companies. So it's like, again, like a ChatGPT Like product, like interface really, it's like a chatbot, but the ChatGPT is recommending like startups to go investigate. And the VCs would not find these startups if it weren't for Exa. So it's kind of like we're connecting in a way. It's like we're enabling a marketplace of startups to VCs. Like, yeah, so you could enable new marketplaces through better search. And then wacky things, you know, like people have made apps where they can meet people in real life because of Exa. thinking dating app, you paste someone's profile and then you get 20 different people who are similar. So people search is very powerful. I know someone who has met several people in real life because of Exa. So that's cool to see the digital to the real. And Exa is just a very fun search engine. So beyond just tools people are building, people are just finding— they're seeing the world in a new way because of Exa. At one point we had Rediscover the Internet as the the title on the consumer page, because that's really what's happening is like, there was this internet, it was always there. There was all these things, like really interesting things that were there and you just didn't know about them. They were hidden because of the algorithm. And because we changed the algorithm, those things are now revealed. And so someone described it as uncovering a hidden temple of knowledge or something, which, yeah, it's amazing. And there are infinite insights on the web from, you know, documents that you haven't already found, but also from combining documents. ChatGPT can do the combining, but it needs the, the right inputs. And, uh, so really when you combine Exa to like find all those like 100 documents that are relevant, and then you have— you combine Exa with like an AI like GPT-4 or eventually GPT-5, that's when you get like absolutely magical experience. We're the knowledge engine, ChatGPT is the combining engine, which I like why a lot of our products like they use AI, right? Again, we're a search built for AI. So all these products I described, they're like Exa is being used by an AI system. That's when you get the most juice from what we're doing.

So what does that mean, a search built for AI? I mean, I think that's a very provocative phrase and it definitely has a strong point of view around where the world is going. I don't immediately know what that means though, in terms of how that cashes out. How are the results different or how are you What are the demands of AI as a consumer of search results? How is that different from the human needs as a user of search results?

If an AI is doing the searching, it's very different from a human doing the searching. So, I mean, for a few reasons. One, a human is lazy and types in a few keywords max, right? A human doesn't want to type in a paragraph explaining what exactly the human wants, whereas an AI could instantly output a paragraph or 2 paragraphs depicting exactly what it wants. So you want a search engine that can handle the complex queries that an AI can make. Like Google was optimized for very simple queries like Walmart homepage, like it's things like that. And okay, so that's one. Also, you know, AIs can take 100 results and instantly gobble up all that information, whereas humans, you know, we search and then we click on a couple links. And read them and move on. So that means like you want a search engine that can return like not just a few links and a few quality results, but like 100. You also like humans when they search something like, again, like Taylor Swift, we don't really know what we want. We kind of just want like a diverse array of things related to Taylor Swift. Whereas when an AI searches, it knows exactly what it wants and it asks for that thing. So if it wants a list of companies, it'll ask for that and then It wants a list of 100 companies. It doesn't want like one company and then another blog post about companies and then a LinkedIn post and then some Reddit forum. It knows what it wants. You just want a search engine that is like controllable and can return it exactly what it wants, exactly the number of results, handle complex queries. And then the last thing is that AIs, they don't want links and titles. They want like full page content. They want like exactly the content itself because— They don't want to, they don't want to get a link and then have to like go wait in a browser and read that thing. They want to just like get all the information at once. So you need all these features if you're going to be serving an AI like really well. So you could, you could connect an AI to like Google, but now you're just connecting this like completely new creature that has like all these different properties to something that was designed for, you know, organic humans. And it's just like not a good pairing. Like this is such an important part of the human experience, the way we find information. Why are we, why are we using some old technology for some super new thing?

So the 3 things that I heard there, tell me if I missed anything. One is because the language models are just so quick at generating text, they can do much deeper, more robust queries, you know, with no effort, right? As opposed to humans just don't sit there and type paragraphs. So you want to be able to handle that. Second, you can handle a lot more information coming back. So you want to return more. And third, instead of links, you want to return actual content. You also mentioned something about types, basically controlling the type of the response as opposed to just kind of whatever. So that's all really interesting. How does that compare to, in terms of starting to get into a little bit of how it works? I know that Google obviously has moved somewhat in this direction, right? I mean, the original thing was PageRank, and I actually don't know if you're using any sort of PageRank-type algorithm in your system, but they certainly have moved toward some sort of BERT or whatever, the successor to BERT, where they are embedding content. You can do at least some amount of semantic querying there, right? But how would you contrast I guess on those key performance dimensions that you outlined, how would you contrast Exa against Google today? I actually don't know what really— it's an interesting point. I don't know what would happen if I put in a full paragraph into Google. Does it just still say like, sorry, there are no results for that? That's an interesting experiment that I've not done.

It doesn't do what you ask it to do. If you paste an abstract of a paper, for example, and you say similar papers, at the bottom, like it's gonna give you a bunch of results that are the same paper. It's like, it's like finding like a bunch of examples on the web where people have, where that paper exists, uh, as opposed to finding similar papers. Um, so it's not like, it's definitely not handling those kinds of queries. I mean, you could experiment with more of those, but like, it's just not meant for that. In terms of how the Google algorithm like contrasts with our algorithm, it is totally different. So with Google, I mean, no one knows exactly how it works, but in terms of their BERT stuff, Like it seems more like a re-ranking at the end. So it's like a post-processing step. So maybe you do an initial filtering of the entire web to like some number of results and then you re-rank with some neural method. And that's very different from going neural from the beginning because, you know, you will just miss things that are not in that first filtered set. Like if you filter to 10,000 and, you know, if you miss things that are relevant, it just, the re-ranking won't Won't help. But yeah, I mean, their algorithm is of course like more— when I say a keyword-based algorithm, it's more complex than that. It's Google. I mean, it is very effective for what it's trying to do. And they have like a knowledge graph that they, you know, when you mention Obama in your query, it's gonna like know about— it's gonna take some information from Obama from the knowledge graph and input that into your query. So it's more complicated, but it's fundamentally like a keyword-based algorithm. It's like looking at keywords. And whereas our algorithm is, we train like a transformer-like model to predict links. So the way it works is we, we find places on the web where people talk about links. So imagine, imagine like a Reddit comment where someone says, you know, yo, check out this cool aerospace startup, and they paste a link after. We, we hide the link and ask the model, you know, predict, take that text and predict which link followed. And if you do that like hundreds of millions of times, you get a really good search engine out of that because at search time, someone's like aerospace startup and the model predicts the most likely links to follow. And so in some sense, this is like PageRank. It's more like a neural PageRank. It's like learned PageRank. If the model often sees people, when people say a great fundraising essay, they often refer to a Paul Graham essay, then the model learns, okay, when people talk about a great fundraising essay, when someone searches for a great fundraising essay, let's hit them with the Paul Graham one. So it is like a learned PageRank, but I mean, that's just V1. For V2 and V3, we have some really, really cool, exciting things coming up.

To the degree that you want to share more about that, I'm keen to hear. But before we even get to V2 and beyond, to just be a little bit more concrete about exactly how it's working, Very interesting that you are starting with this conversation about links when you're actually— you also said transformer-like model. I'd be interested to know more about that and what exactly the like part of that transformer-like means, what sort of deviations there are. But I imagine you're not doing— because you could do that in ChatGPT as well, right? You could be like, complete the following. here is a great essay on fundraising, https:// and see what it comes up with. And a lot of times there would predict token by token, www.whatever.com/whatever, and sometimes it might generate real links. A lot of times it would generate made up links and link you to nowhere. So I assume you're not doing literal token by token link prediction that way. Can you give us a little bit more on what makes the model transformer-like and a little bit more concretely what exactly is being predicted at runtime? Hey, we'll continue our interview in a moment after a word from our sponsors.

That's correct. We're not predicting it token by token. We have an index of documents and the model is trying to predict which documents. So instead of predicting a link, it predicts just like a document ID. And so it's not generating the link. And it's transformer-like because you can't just use a— if you take a transformer and try to do what you're doing with predicting the next token of the link, it's just not going to work because you're forcing the model to memorize URLs and that just doesn't work. So you can't just naively take a transformer and fine-tune it for this objective. You have to create a new type of objective, So we have, it's an embeddings-based approach. So like our, the transformer ultimately outputs an embedding and you search using that embedding, but like doing that at scale and very precisely, that's where a lot of the research that we did for like really like a year when we started this, we could talk about, it was like really exciting. Like very few startups actually do like research into what they're doing for a year before they ship product on top of it.

Yeah, Google comes to mind as one that did at least some serious work before launching back in the day. So it makes sense that, I mean, I think the embeddings, the future of embeddings, I think is going to be extremely interesting in many ways. And, you know, I've had this kind of hypothesis for a while that more and more communication is going to be mediated by embeddings as opposed to being mediated by natural language. As systems come to talk to other systems, I think in many cases it's just not going to make a lot of sense for that to be reduced to a string, a linear string of natural language tokens, and then kind of reprocessed. If on both ends you have AI systems that natively work in some sort of embedding space, then some sort of translation from embedding space to embedding space Seems like a lot more efficient, a lot less lossy, also a lot more problematic in some ways, in that it becomes very hard to know what is going on. We can't interpret that always very well, or we can't interpret it in the same way that the systems are interpreting it, certainly. So I think that's really interesting. And this sort of feels like a little bit of a glimpse of this aspect of the future that I've been kind of anticipating.

Embeddings actually feel to me like the past. I think embeddings are just the beginning. I would be disappointed if embeddings are the way we're passing around knowledge in the future because embeddings really do lose a lot of information. Particularly, it depends how many embeddings you— obviously, if you take text and you output 100 embeddings, then you're not losing information. But if you're taking text and outputting an embedding and then doing a dot product with other, the document embeddings you have in whatever index you're talking about, That dot product operation is just like lossy. Like, it's not going to be able to handle all the types of complex queries you might want to do. So our thing is embeddings-based right now, and it's really, really powerful embeddings. Like, we really, you know, squeezed out as much juice as you can from embeddings. But embeddings are still fundamentally limited. Like, they can't do complex operations. For example, like startups applying AI to law in the Bay Area where the founders, you know, have experience in Rust and blah, blah, blah. You keep adding more modifiers. You cannot expect a single embedding and a single dot product to handle that kind of query. So when I talk about V2 and V3, that's what we're talking about, moving beyond embeddings. Embeddings are definitely a step up from no neural, but at least right now, the best systems are probably going to combine keyword-based approaches and embeddings approaches. But ultimately, you could do Way more powerful things. Maybe you could think of it as there's a spectrum between an embedding dot product and a full transformer output. An embedding dot product is a very simple operation and a full— so, okay, imagine you take, just more specifically, you take the embedding of a query and the embedding of a document and you take a dot product, you get a score. On the other side of the spectrum is you put the query text and the document text in the context window of the transformer. and it goes through all the weights of the transformer and outputs a score. This one took a huge, way more compute than this one. So it's a much more powerful operation. It's just hard to do this one over the whole web. This one you could do over the whole web. So are there things in the middle that are unexplored? I think so.

And it's probably worth kind of describing a little bit how Exa is best prompted, which I noticed it's interesting. You kind of have your best practices and then you also have like an auto layer that allows people to kind of use the sort of keyword search that they're used to using and translates it into the best practices for Exa prompting. Maybe just give a little account of that first, and then that'll, I think, shed light on the underlying technology.

Sure. Yeah. Because the model was trained to predict that, let's say, document ID. So check out this aerospace startup link. That means that the best way to search Exa right now is to search in a way such that a link follows your text. It's a little strange, but like, you know, type a query, type a prompt such that a link, the link that you want follows. So if you really want the model to output startups, the best thing you could do is like say like at the end of your prompt, do like parentheses startup colon. Like that is the end of your prompt because what, what follows that on the web when someone Puts that, obviously a link to a startup homepage. And so you're like really biasing the model to return exactly what you want. This is something we were actually considering putting on the homepage, just like that. It's a really powerful way to search Exa is to prompt it in that way. And there are all sorts of prompting tricks that make it, that allow you to squeeze out the model. We're in a very similar state to like GPT-3 before our, before ChatGPT, before RLHF. It was a text completion model. And so you had to prompt it in these weird ways and it was hard to use. And then they, you know, RLHF'd it to make ChatGPT. And that's great because it's easier to interact with. And now my mom could use ChatGPT, but it also, you lose things because RLHF makes it less controllable. And from our perspective, it is a little harder to use for sure in this, when it's, you're using this model that requires like the text to— it requires this weird format, but it also makes it a very powerful and controllable search engine. So we haven't put a huge amount of resources into RLHFing the model. Yeah. Also, a lot of our users are businesses who are happy to learn how to prompt well, or they might even not be prompting themselves. They have like, you just tell ChatGPT to prompt it correctly and it does that. So I think ChatGPT made our weird prompting scheme less Less of a negative.

Yeah. Well, you certainly have to be willing to step outside what people are familiar with to create something new. No doubt about that. So under the hood, then, if I understand correctly, the input is this text prompt that sort of tees up, here's the kind of thing that I'm looking for. The model predicts an embedding. That embedding is then through a vector database, find the K nearest neighbors or whatever from your existing index. And then those of course correspond to source links, and that's how you ultimately bring links back. Its prediction is in this high-dimensional embedding space. Is that basically how it works today? So then in the future, you imagine kind of elaborating that in multiple ways. I guess just hearing your example of startup, law, Bay Area, these sort of conjunctive composite queries, my first naive reaction to that would be maybe I want to kind of decompose that and have multiple different embedding guesses and then run multiple different k-nearest neighbours and then maybe look for the overlap in those resulting sets or have some sort of post-processing heuristic on how to combine those results at the end. I guess you could also just have language models if your language models get fast enough, they could start to do the post-processing for you. But how do you see that kind of evolving as we go? And what are the bottlenecks? Is it just that we're still inventing this stuff? Is it that the compute right now is not there to power all the things that you maybe you know how to do them, but you just can't either afford it or it takes too long? I'd be interested to hear both where you think we're going and what are the hard parts that we need to get over to get there?

That was a great intuition, and we are hiring. Breaking down the query definitely is a way to do this. It's subtle how you break it down, but for sure, that's, that's, I mean, certainly, I mean, that's how humans do it, right? Like when you have this very complex query with lots of modifiers in your head, you break it down into the right components and then you make sure everything matches each of those components. So yes, that is a very good direction. Then there are other things that even that will not help with, particularly if the knowledge is spread out across different documents. So again, like just taking that example we've been working with, like startups applying AI to law that are in the Bay Area where the founders have Rust experience. Now on law and AI law websites, they don't usually talk about the founders and their Rust experience. So you need to combine information from other parts of the web. And so doing that is where, that's where you start unlocking like super high value. And we have, you know, pretty clear ideas for how to do this. don't want to get too deep into that, but basically you can imagine knowledge graph type approaches. But you don't ever want it to be very discrete. You always want to be working in a neural fuzzy world because then it allows for arbitrarily complex queries. Whereas if you have a knowledge graph that literally has Obama, books that he's written, those books have authors, everything has very discrete metadata, you're limited to just that metadata where if everything is like in like an embedding, you know, fuzzy space, you could make complex queries over that graph.

Training models to output things in embedding space is, I think, just a, there's like a massive set of unlocks there coming for us. It, that seems pretty clear to me. I still quite underexplored.

This is such a big unexplored space. Language models have been explored for the generative, in the generative area, generating text, generating images, generating video, but really have not been explored too much in the improving the search algorithm area, surprisingly, right? Like there's a lot of value in it. And so it's just like a lot of really intuitive things like you thought of, like haven't really been fully explored. You don't get a lot of PhD students writing papers about web search anymore. So it's just like there's this whole space of possible solutions. You need a company with the right incentives to go pursue it.

Yeah, the best minds of our generation are elaborating chain of thought still. So we've got to finish that up before we can get on to the next thing. So the index is interesting in and of itself, right? The couple of things I've wondered about as I've been researching the company, using the product, one, from what I could tell online, you've raised $5 million. I don't, I'm not trying to ask you to break any news here or share anything new, but Either you're managing to do a lot of indexing on a relatively small budget, or maybe there's more funding behind that that is not public yet. But I was kind of just wondering, how are you building a web-scale index and doing what— have a compute cluster? Is it really on that small of a budget, or are there more resources that you have accumulated to make that possible?

Yeah, I mean, we try to be frugal, and there are lots of optimizations you can make. that make this stuff not too crazy expensive. But yeah, we're not also, we're not like crawling the entire web right now. We're crawling a very high quality subset of the web, the subset that our customers care about. And because we're not crawling the whole web, that allows us, you know, as a lower resource startup to, you know, pursue algorithms that are more complex and to just do it all on a budget that makes sense. But yeah, like crawling like hundreds of billions of web pages continuously, which is what Google has to do. That's quite expensive. Crawling a lot fewer and not on such a regular cadence, that's totally possible for a startup. I think this is a problem that other startups have had in the past, other search startups. They've tried to, too early on, be comprehensive when you're just going to be okay at searching over everything as opposed to really good at searching over a few things. It's kind of like the advice of do something that makes a few people love you as opposed to Do things that make a bunch of people like you for startups.

I've recently seen Brave, the browser company, has an interesting offering where they basically have, I believe, like a web index as well, but it's based on where people are browsing. I don't know if you've seen that, but that kind of reminds me of what you are describing. And you could do something much more heuristic-based if you just said, hey, let's look at Reddit and let's look at Archive and let's look at, you know, pick your favorite 50 websites and like the things that people link to from there. I could see something like that also really taking you pretty far. But I don't know if you've looked at the Brave thing, but it is specifically based on where people are going, where they are spending their time. And that seems like a really interesting way to get a view on what is actual real content that people value versus— and I imagine it's got to be You know, just given the proliferation of stuff in general, I imagine, you know, 90+% of time on the web must be spent on like under 10% of the pages, right?

Yeah. Yeah. I don't know the exact number there, but certainly the links that people share is an interesting metric for what is high quality. So like our model is trained to predict what people share and people often, no one shares SEO blog posts. People share like that really good recipe. They don't share the SEO blog post about a recipe that's like trying to, you know, that has all sorts of ads on it. So our model is really trained to predict links that are just give you like raw content as opposed to like putting all sorts of ads and like stupid information in it. So it's like, yeah, it's like our model is trained to not predict SEO. And I like with Brave, it's like people often spend time on the things that are not SEO.

Let's talk about the business for a second. I was initially kind of, I'm just looking at the pricing. It starts at basically $10 per 1,000 searches. And that is if you want to get up to 25 results per, and then there's a higher price if you want to get up to 100 results. And then you can, you know, things can go on and get more elaborate from there. You've got the custom enterprise option for people that want to, you know, to go beyond what the standard plans offer. My initial reaction to that was like, seems a little expensive. Typically, an API product these days is like $1 to $2 per 1,000 calls. However, as I'm talking to you and I'm thinking about this, I'm realizing that one thing I've said in the context of Athena and the apps that we're building and the things that we're trying to equip our assistants to do for their clients is almost every client that we have, has an opportunity, if not a pressing need, to scale some sort of lookalike search. Here's some good sales targets that we've sold to successfully. I want more like this. Not easy to do in today's world. You could go search LinkedIn, you could do a number of different things, but definitely not easy. Similarly, here's a great candidate that we want more resumes like this. Again, you can do that kind of on LinkedIn, but also not super easy. So I'm struck by those 2 use cases being super ubiquitous and imagining the more we talk, I'm like, that sounds like a significant part of the business. And if the results are good, first of all, for us at Athena with the assistants, it could really unlock their ability to do that effectively, because a lot of times they struggle. It's like, yeah, we need an engineer with a certain background and skill set or whatever. And it's like a lot of times that flies right over the head of the assistant leaser when they're first getting started. They just don't know. And similarly, sales target, what makes it good? What makes it not good? I imagine that there could be— I wonder how you think about this when you're talking to customers about it, but it seems to me like investing a little more on the quality of the results, saves you potentially a lot in terms of the iteration time to actually get something working, and perhaps also the downstream language model processing of the results. So anyway, that's all speculation. You can react to it. But I guess what are the kind of main business-wise, what are people really just coming back to and doing over and over again? How are they reacting to the price? And how do you think about kind of your price versus other, you know, kind of complements in the overall systems that they're building.

I mean, one thing that's interesting is a lot of customers are searching over a particular domain of things. So it's like, you know, they're searching over only people, or they're searching over only companies, or searching over only news articles. And so they want a search engine that just returns them those things. So you just, you just want a list of people that match your criteria, and it's just really hard to get Google or Bing to do that. And so customers come to us because they use Bing, the Bing API or wrappers on Google, and they're like, this just doesn't give us what we want. And so we just, we need you. So a lot, a lot of the customers are coming to us because they just need this. There's no other way to do it. And no one really complains about the price because it's just something they need. We can always work something out, you know, at high volume with, with the enterprise plans. And we've done that. But yeah, it's just such a unique experience that we really, we really see ourselves as like a premium search, right? It's search that you can't get anywhere else. So yeah, I mean, customers are using us for those domains that I mentioned often, but in various ways. It's not just similarity search. It's all sorts of downstream use cases. It's too hard to put them into particular patterns. Maybe you could put them into patterns of some retrieval augmented generation use cases. Then there are automated analysis use cases. And then there are even crazier ones like creating like, again, like hundreds of thousands of results or even millions of results for fine-tuning models. Really, there are a lot of different types of use cases of the same product.

Yeah. Interesting. I'm coming up with some more ideas just listening to you describe that. And yeah, I mean, it's interesting, right? What's expensive? What's cheap is where I think we're very much like getting used to a new regime here because On the one hand, 1 cent per API call could be considered expensive. On the other hand, if you're comparing it to human labor, it's extremely cheap. So the quality is obviously the real determining factor there. I'm not one, by the way, to emphasize the cost too much. My general advice to people building apps is costs are coming down across the board, and Right now, the first question usually is, can you make something that meets your needs? And I would do that using the very best tools, whether it's GPT-4 or Exa, whatever the best tool is, use that, maximize your performance first, and then later on you can upgrade to an enterprise plan for savings, or you can wait for savings to kind of just generally secularly arrive, or you can optimize in any number of ways. But don't do that too early because then you may miss the fact that maybe you could have actually built something that worked and you didn't even get there because you were too budget conscious too soon. So I don't mean to suggest that I'd be too concerned about the price. My advice would be to anyone listening, definitely use it if it is the best thing for you. And I think for a lot of these use cases that we've been discussing, it seems like it very likely would be. How do you handle evaluations? This seems like a challenge. It's a challenge for every AI startup, but it seems like you have kind of maybe a unique flavor of that challenge.

Yeah, this is a difficult problem because it's very subjective. So if you evaluate Exa on the types of queries that people use for Google, then obviously Google's going to be better. And if you evaluate Google on types of queries that are great for Exa, then Exa is going to be better. So like, what distribution of queries are you sampling from? And we have anecdotal evidence that a lot of our customers are going to Google if it's failing them, and then they come to us, but that's not a quantitative metric. There are web retrieval datasets, but again, they're often designed for the old way of searching. So we've really had to come up with our own internal metrics for comparing our search to other methods. And one way to do it, one interesting way is to break down, instead of just sampling queries and being like, is this good? It's more you break down the model into different abilities that it could have that you want it to have. So for example, you want it to be able to handle authorship. So if you say like essays by Paul Graham, you get essays by Paul Graham. That's better because that's objective. Like there's no subjectivity. Like does the result match that query? Like authors by Paul Graham or that part of the query. And so then you could do other things like not just authorship, but topic. You know, this is about space exploration and make sure it's about that. So if you break it down to abilities, you've turned a subjective problem into an objective problem. There's, well, but it's still like a lot of internal datasets, so nothing like You can't find this kind of thing on the web or sorry, on open source datasets that you could then compare benchmarks to.

Interesting. Definitely a challenge. So internal evals basically. How about the vector database technology? That's something that a lot of people are 1 to 2 years behind you on where obviously you've been working on this stuff for a while and have probably seen pros and cons and different trade-offs. A lot of people right now are like, Maybe are either about to or have just done their first vector database prototype. How would you, I guess if you'd be open to sharing, I'd be interested to know specifically what are you using and what are the kind of most relevant trade-offs that you have encountered? If you don't want to go that deep, then I'd love to just hear kind of what you have observed broadly, what you think people should be considering when they're choosing a vector database and kind of where you think that class of technology is going.

Yeah, sure. So we build our own vector databases, just following the trend that everyone builds vector databases. But I think we have a good reason because we're doing web-scale, uh, search. And, but it's particularly, we have all sorts of filters that we want to apply to the query. So it's, we're not just doing a simple, uh, you know, nearest neighbor lookup. We're doing more complex things than that. And so like these vector DBs were not designed for all the different types of filters we want to apply. We, I mean, we, of course we have looked into the open source vector DB, like offerings and I just compared ours and, you know, ours works way better for the type of searches that we're doing. Definitely. Yeah. I mean, it's gone, it's gone through like multiple versions. There is a, it's a really interesting problem. We've had a lot of fun building our own vector database. But one way you could think of what we're doing at Exa is we're kind of like trying to take all the world's knowledge and putting it into a new type of database, like a neural database. I like this database analogy because it's not really search. It's like you're kind of like with every query, you're filtering the database of all the knowledge into just what you need. And so like when you search for startups applying AI to law, you should get like 367 results. These are all the startups applying AI to law. And when you add Bay Area, it's like a filter to the database. And now you get like instead of 367, it filters it to 54. And so just thinking about the world's knowledge as a database, it makes it It makes it feel like everything is actually like, like you have comprehensive knowledge of what's out there. Because the problem with, you know, search engines like Google is like you search something and they say, you know, 33 million results at the top. Like, what am I supposed to do with 33 million results? There's no way all these results are actually what I'm asking for. So it's just like, you just feel overwhelmed. And then like the things that are returned are not actually exactly what you're asking for. So yeah, anyway, so like we're trying to build like a new type of database. That combines neural and non-neural methods. And if we're trying to do that, then it makes sense to build our own.

So combining a couple of the threads, it sounds like you're almost, if I had to guess it, it would be like your vector database maybe has multiple vector representations of each thing where they're kind of projections in various directions. Like you have a certain essay, but then you could project it in the authorship direction, or you could project startups into a geographical direction, and then that maybe starts to open up this multidimensional query, but still staying in the fuzzy embedding space, as you said, is important earlier. Is that kind of the direction that this is going?

I think there would be a problem there because there are a huge number of possible directions. So are you going to make an embedding for authorship, an embedding for topic? That would be Like that would be, that wouldn't work. So you need to be, you need a more general method. But yeah, these are definitely the types of things we think about all the time. And so let's say you did do it that way, then you now have like multiple embeddings per document and like, okay, so you want to search with the first embedding, but then you also want to use the second embedding as a filter. Like a lot of these open source libraries don't support that type of thing. So that's, this is exactly the type of thinking that made us want to build our own and I have really have full control over, you know, okay, now we want to introduce keyword filters and we want it to be, you know, just as performant. And we don't have to, we don't want to, we don't want to have to deal with all like the problems that they faced because they were building a more general thing, like, or like the slowness that, that introduces using an open source library because they're building a library that's meant for the very general use case. Whereas we have a very specific type of use case. Like when you have a very specific use case, you're going to be able to optimize your system more than the general use case. And in our case, optimizing it 2x or 5x has massive returns.

So do you think you would ever sell the database as a product? I mean, there's obviously huge opportunity just in that element of the stack, right? So it's interesting to— obviously you guys have a high level of ambition for what you're doing, and I wanted to ask a little bit about the company culture as well. But you're not that big of a team, not that many financial resources brought on so far. And I guess I'm a little surprised you're doing your own vector database, although I understand the rationale behind it. But one might say, like, geez, could that be a product unto itself?

I suppose. But I think our team is really focused on this mission, this particular mission of we want to filter all that information to the knowledge you need. And we feel that that's going to be magical and have huge implications for the world. And so we don't want to get distracted by like selling vector DBs or stuff like that. It is true that to build what we're building, like we had to make some like super performance systems all across the stack. You know, we crawl the web, we have really clever ways for crawling the web. We parse it well. We train, you know, again, train our own models for search and then we serve it in production extremely quickly. So all those parts are like valuable in their own way, but We just want to focus on that mission, which is just like, oh yeah, allow all the world's information to be filtered to the knowledge that you need as fast as possible. Companies our size have to focus their product on one thing. We can't be selling vector DBs and then also selling search and selling the crawler or something like that. It makes sense to focus. We are a small team. We work extremely hard. I mean, every startup's going to say that, but we work very intensely. We're people who are just super excited about the technology. Like, it just doesn't feel like work, you know, it's just really fun. It's just, these are, we happen to be working on a problem that requires very interesting technologies and very unexplored space. So yeah, it's just a great time. And I think that's, that's how small teams are able to move so fast is when they're just like having a blast. And like thinking about it in the shower. And, you know, you just, if you're a 50-person team and you treat it like work, you're going to move way slower than a 7-person team that treats it like play.

Yeah, I can totally relate to that. I have probably never worked harder than I have over the last couple of years. And it has felt much less effortful than various times in the past because I just love learning about this technology and building stuff. And, you know, there's always, ever get tired of one, at least in the approach that I take, which is very kind of interdisciplinary, and I call it scouting. So I'm always kind of looking for new angles. If I ever get tired of one angle, there are plenty of different angles that I can switch over to, and there's always something interesting to discover. From the careers page, interesting sentences. We are a growing family of highly technical idealists on a very practical mission. And then you've got 5 core values as well. One thing that stands out also is you're a pretty young team. You guys even are like, some of you are living together in a shared house, which is also, I guess, kind of the office. So that's interesting. And that's like, maybe was out of fashion for a minute, maybe starting to come back into fashion. But you've heard Sam Altman say the founders are getting older, so you're kind of bucking that trend a little bit with this. Like, let's do it in our 20s. Let's be super intense and just eat and breathe this stuff nonstop. You can talk more about that if you want, but the 2 core values also that jumped out to me were perform with an optimal speed accuracy trade-off and also wield great power responsibly. You can correct again any of that however you'd like, but I'd love to know what is the optimal speed accuracy trade-off, or do you have a framework for thinking about it? I'd love to know that for myself. And I'm also curious about The power, you know, what sort of power do you expect to have? You know, what are the concerns that you're starting to prepare for?

Yeah. So you're referring to the values. So it was an interesting thing to think of values for the company culture that match the product we're trying to build. So yeah. So, and the 2 you mentioned are interesting ones. So like compute speed trade-off is something that matters a lot. And in our system, because ultimately, like not every, so from the product perspective, like not every query should require the same amount of compute. Like Google has kind of made this assumption that like no matter what query you type in, it takes a few hundred milliseconds, but certain queries are extremely complex and might require like scouring the internet for, you know, maybe even seconds or minutes. And so you want, you do, if you want to have an optimal search engine, you will have to make this trade-off between speed and compute. And then from, okay, but how does that relate to company culture? Well, I mean, constantly as a startup, you are facing problems that, you know, you could spend a year on. or you could spend a day on. So you could like, there's a speed compute in this sense is like how much brainpower or effort you put into it. And often the right approach in a startup is just like, you know, like, let's do that. Like, it's just move fast. It's really the move fast, break things type mentality. But I think thinking of it as more the optimal trade-off is cool. It's like you're always optimizing for exactly the amount of effort to put into the thing as opposed to like just always moving fast and breaking things. It's like, no, sometimes we need, we do need to think for a day about the right approach. So I like optimize the trade-off better than like just move fast, break things, but it is the same type of idea. And then the wield power responsibly. Yeah, I mean, we are, as builders of a search engine, we are like, we are the doorways to the world's knowledge for people. And, you know, imagine everyone was using Exa or every business was using Exa. We control the information, the knowledge they see. That is like extremely powerful. I mean, like people who control knowledge control power. And so we have to think very carefully about how do we do that in a way that isn't biased? How do we do that in a way that doesn't create negative downstream effects? Because Google has had huge downstream effects on the web. It influences the type of content people even create. And much more important is it influences the type of content people are aware of. Our philosophy is give users full power to get what they want. So as opposed to like, you know, biasing the web, biasing the results towards like what we think is right, we want to give users the power to explicitly say what they want. And so, I mean, there are interesting moral questions there, but I do think that's, that's the right approach. And it's— that's now possible, like, with, you know, transformer-like technology, you can like actually just give people exactly what they asked for. And you could like, you know, for every search, like recommend like, hey, like, by the way, you know, you're searching for this thing, but like a lot of people have noticed that this has all sorts of problems kind of like with Twitter and like the reader suggestions or whatever it's called. That's really helpful. But yeah, ultimately like give users the power to find anything they want, no matter how complex, is I think the best way to wield that power.

Do you expect there to be a dominant player in the future or do you expect kind of a more multipolar information and knowledge finding ecosystem?

I think we're going to be the dominant player. I think certainly the market is going to expand. So the market for knowledge, for information retrieval is actually going to get way bigger. And so there's definitely room for more players in a way that there wasn't before. So I could see it, I could see there being multiple players, but, but still you do get benefits of scale. I would say it's more likely than before that there'll be multiple players.

Anything else you want to touch on though, briefly before we break?

If you're listening to this, you know, we're hiring for all sorts of super cool roles and we build some of the coolest technology in the world as a small startup. So would love to just get the best people to join us and build, build the future of, of information retrieval.

Love it. Will Brick, founder of Exa.ai. Thank you for being part of the Cognitive Revolution.

All right, thanks.

It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co, or you can DM me on the social media platform of your choice.