Amazing Answers: Richard Socher on how You.com is Reimagining Search with AI

early 2022 already. Like we had apps that would write code for you within the search results, apps that write essays for you within the search results. But whenever we innovated and changed the default Google experience too much, we had just like the vast majority of our users say, I'm so used to Google, I don't want another way of finding answers. And so we kept getting pulled back to this need. And so the most amazing surprise was when ChatGPT came out. All of a sudden people got it. And it was like, wait a minute, it could just be like pure text. And we're like, been trying to sort of slowly get there, but we had to make a bigger jump. The way I think about the different modes is like the default smart mode is kind of like if you had an assistant and you just asked them to do a quick search and in like 2 or 3 minutes give you an answer back. And then genius mode, you go and so you want to ask your assistant for a question that they have to be able to program. They have to search the web and then they need to be mathematically inclined to answer that question. I mean, as a kid, I also enjoyed watching Terminator. It's like a cool action movie, but it's just taken over so much of the AI narrative and it's actually like actively hurting, especially the European Union.

Hello and welcome to the Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week we'll explore their revolutionary ideas, and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Thornberg. Hello, and welcome back to The Cognitive Revolution. Today I am thrilled to welcome Richard Socher, a pioneer of deep learning for natural language processing, formerly chief scientist at Salesforce, and today founder and CEO of You.com. A company that was first introduced to the public as a new kind of search engine, but which now describes itself as an AI assistant that makes you more productive, creative, and extraordinary. Richard has deep history in deep learning. He was among the very first to recognize the potential of neural networks in the natural language processing domain, and his work has helped shape the field as we know it over the last decade. In this conversation, Richard takes us on a brief journey through his own intellectual history and reflects on how the field of AI has evolved in both expected and surprising ways. Before we dive deep into the u.com product itself, covering the historical challenge that they faced when trying to compete with Google and how the rise of the AI chatbot paradigm has broadened the space of possibility for search and discovery products. We also look at u.com's various modes, with particular emphasis on the Genius mode, and above all for me, the Research mode, which delivers amazingly helpful and thorough report-style answers even on some remarkably complex topics. We also briefly discussed the future of AI business models as well, including the obvious subscription and my pet theory about the AI bundle. Along the way, we touched on a number of important topics too. The limits to AI systems' reasoning ability and the prospects for the improvement that would be needed for reliable autonomy. the potential for AI to transform medicine and scientific research, Richard's case for general optimism, even though he does expect AI to drive major disruption, why he's not worried about so-called emergent capabilities, but does take the risk of intentional harmful misuse very seriously, and lots more little topics along the way as well. Richard is a leading thinker in the AI space, and his perspective is essential for anyone who wants to understand where this technology is going and what it means for the future of humanity. And in all seriousness, I really do recommend you.com. It has absolutely joined the ranks of the AI tools that I use multiple times each week, and particularly when I want a comprehensive, multi-page report-style answer, I find that you.com's research mode is often the single best tool available today. As always, if you're finding value in the show, we would appreciate it if you'd share it with friends. Or post a review to Apple Podcasts or Spotify, or just leave a comment on YouTube. Now, without further ado, I hope you enjoy this conversation with Richard Socher of you.com.

Well, let's do it. I think this is going to be a lot of fun. I'm looking forward to your point of view on a bunch of very interesting topics. Richard Socher, founder and CEO of you.com, welcome to The Cognitive Revolution.

Thanks for having me.

I am very excited to have you. You are, you know, at the intersection of so many interesting things. I sometimes have been describing myself recently as the Forrest Gump of, of AI, because I've just kind of very unstrategically made my way through the last few years and yet found myself in some very interesting places. I don't know how you think about your own trajectory, but you are kind of an OG in the realm of deep learning and have founded this very interesting company and have a really awesome product, which we'll get into in more detail. And I'm interested to hear about all that and your kind of philosophy and expectations for the future. So we've got a lot of ground to cover. Maybe for starters, you want to kind of give us, and I usually don't even ask these biographical questions because these days it's like a lot of the same answers. People are like, oh, when I saw GPT-3, I thought this is going to be a big deal. And I got involved, but you were there at the beginning. And so maybe you want to just give us a quick history of your own role in the history of deep learning and how you've kind of come to the present.

I started with AI actually in 2003 when I started studying linguistic computer science or natural language processing back in Germany, Leipzig. And at the time I was like, this is really interesting. I love languages. I love math. I love computers. You know, sort of computers are where languages and math can meet in some useful functional ways, I thought. And it was very much sort of a small niche subject within computer science. And I was really excited at the time. There wasn't quite enough math for me in NLP. And I felt like we're just getting stuck in some of the linguistic special cases. And I loved formal semantics and set theory and then like algebraic foundations. And so I moved eventually into computer vision during my master's. And there I also, in Saarbrücken at the Max-Planck Institute there in the university, found statistical learning and pattern recognition. And I fell in love with that. I was like, clearly, if you can really understand patterns, any kind of pattern really well, you could solve all these different kinds of problems. And so ended up doing my PhD at Stanford. In the beginning of Stanford, you know, I started trying to really contribute to the field rather than just learning about it. I basically found that even the top NLP people Like they write their papers mostly about these beautiful models, like conditional random fields and latent Dirichlet allocation types of models. But then most of the coding happens when they actually do feature engineering, right? They say, oh, well, if I want to do named entity recognition, I add a feature of like, this is a capitalized word and this is all caps word, or this is a word that has like, is one of the items in this list. And this list includes, you know, city names we already know. And I'm like, man, this feels very hand-engineered. It's very like graduate student ascent to get better. And then at the time, I was very fortunate because Andrew Ng got into deep learning on the computer vision side. He's like, well, images are pixels and it's a fixed number of pixels. So we can feed them into a neural net or at the time, variants of probabilistic models like restricted Boltzmann machines. And I was like, wow, maybe we can use ideas from that for natural language processing tool. And there was maybe like one or two relevant papers from Arnaud Folbert and Jason Weston and a few, like one or two others, but no one really enjoyed that approach. No one in natural language processing paid any attention to it. But I thought, clearly that has to be the future. I want to give it data and I want to get an output. And so in 2010, I started publishing my first neural net paper. I worked on like computer vision before and saw some of the power of ImageNet, also of BERT-2, and really started running with it. Got a lot of rejections all throughout, but at some point I had sunk my teeth into it and I just like, I loved it and I thought, this is the future. Despite all the rejections, I kept going at it. And then after the PhD was over, there's sort of starting to be more interest in deep learning and neural nets for NLP, but still no one in the world was teaching that as like the official right way of doing NLP. So I started teaching at Stanford. Stanford also, first as a visiting lecturer and then adjunct professor. Started, you know, being very fortunate, have lots of very smart students back then, including like the Hugging Face founders I invested in, in their first round. And then, you know, also wanted to bring these neural nets into the world. Started MetaMind, my first startup to do that, build a general purpose platform to train neural nets very easily, both computer vision and NLP, got acquired by Salesforce. Became the chief scientist there and EVP eventually. And in Salesforce, we had my probably last and biggest rejection was on inventing prompt engineering in 2018. And we were so excited about it because it was the culmination personally for me also of this decade-long dream I had building a single neural net for all of NLP. And the idea was, at the time, every AI model was built for one task. You wanted your sentiment analysis. I build a sentiment analysis model. You want to do translation? I build a translation model. And they're all different. We're like, what if we could just build a single model and you just ask it a question? What is the sentiment? What is the summary of this sentence? Who is the president in this paragraph? And that was kind of, for us, I thought, like the most exciting thing we possibly could be doing. I just gave this TED Talk about it, came out last week. And, but it was rejected. But it did inspire a couple of other folks. And when OpenAI was publishing the papers about GPT-3, And 3, they cited that paper saying like, look, they were able, they showed you can have a single model for all of NLP if you just ask it these questions. And, you know, that's now prompts and the rest is kind of more well-known history.

That is an amazing history. And it definitely, I don't know how, um, you know, modest you want to be versus taking credit for foresight, but certainly the idea that there could be one model to solve all these, you know, tasks was not obvious to people. And boy, we still see this. The flaws in the peer review process are still on prominent display these days. Most recently, I noticed this with the Mamba paper, which I was a very interested reader of, and then went over to the open reviews site and was blown away by how negative some of the reviews were, like a confident reject was given. So that was kind of, you know, Just a good reminder that, yeah, this is still an unsolved problem. What would you say has surprised you most from the big picture since you, and it hasn't been that many years, right? But since you kind of had that notion of this generalist NLP model, fast forward, now we have GPT-4 and possibly Q* or something like that in the works. Is this the trajectory that you thought we'd be on, or how has it deviated from what you imagined back then?

It's very much aligned with what I hoped the field could get to. And now it's almost like it's obvious, right? Like no one questions this anymore. We've had all these breakthroughs. And I think the biggest surprise was maybe more on the application side of things in that for us, you know, we've been playing around with large language models at you.com. And infuse them into search results earlier, like early 2022 already. Like we had apps that would write code for you within the search results, apps that write essays for you within the search results. But whenever we innovated and changed the default Google experience too much, we had just like the vast majority of our users say, I'm so used to Google, I don't want another way of finding answers. And so we kept putting And so the most amazing surprise was when ChatGPT came out, all of a sudden people got it. And it was like, wait a minute, it could just be like pure text. And we're like, you know, we've been trying to sort of slowly get there, but we had to make a bigger jump. And that was incredible. That unlocked a lot of people realizing, wait a minute, 10 blue links isn't the best way to get an answer, an actual answer.

So let me give you a couple of my experiences on you.com recently, and then you can kind of tell me where you are in the overall story. And then I really want to kind of unpack the, kind of use the product as it exists today and the roadmap and everything you're working on as a way to kind of explore a bunch of different aspects of where all this is going. And I think that's really the mission of this show is to kind of help people see around the corner and starting with me, helping me develop my, my own worldview. But I've been really impressed with the product recently. You know, listeners will know that I've been a big fan of Perplexity. We've had Arvind on the show a couple of times, and I think they do a great job and, you know, remain a fan. But I have found distinctive value in at least 2 modes on you.com recently. One is the research mode. And the other is the genius mode. Those to me have stood out as the most differentiated. For research mode, I recently took it like a 200-word question that was all about mixture of experts architectures. And, you know, kind of, is there curriculum learning, you know, stuff happening here? How, you know, how do people think about sort of the trade-offs between like how many experts should we have and how big should they be and how many should we activate at any given time? Are there any like scaling laws or whatever? designed for that sort of thing? Just every, basically every question I could think of about mixture of experts, I took it all in one go. And it was really impressive to see it kind of break that down and go through multiple steps of searching and analysis and really implementing kind of like a classic agent, what is at this point a 6-month classic agent setup, but applying it to that research question and just going, down the line, really quite valuable results. And it definitely is something that I will come back to and have already, you know, found myself kind of being like, I think this is a good one for u.com research mode. Genius mode's a little bit different and more kind of analytical. I'd be interested to hear a little bit more about how you think about the differences. Because I did, I then tried one that was a big Fermi calculation exercise where my questions were like, What are the different datasets that exist in today's world? How big are they? How do they compare to each other? How do they compare to the training data size for GPT-4? You know, how do they compare to available compute? Like, because I have this, I'm, I have a big question, which is kind of one of the ones I want to get to toward the end too, around like, to what degree is ML research poised to start to be kind of semi-automated? And so I'm trying to, trying to wrap my arms around that with these Fermi calculations. So Genius Mode was really the best. Uh, way to approach that. And anyway, I would definitely encourage people to bring multi-part complicated questions to both research mode and genius mode. And I think you'll be impressed with the results. And I would say that, you know, even with, you know, the expectation that folks who listen to this show have tried, you know, other leading AI products. So that's kind of my unpaid endorsement, very sincere. And I'd love to hear, you know, a little bit more about how you think about those different modes, how they work, and just kind of big picture, like where we are in the u.com product journey long-term. Hey, we'll continue our interview in a moment after a word from our sponsors.

Yeah, these are, these are great questions. I think it shows you kind of how sophisticated the space has gotten in the last year alone. Like around this time last year, we were the only search engine with a web-connected LLM and millions of users. And now that idea has like, it's been copied so many times, including as you mentioned by Perplexity. And so I think what you have to differentiate kind of the different modes. And I think the modes kind of show how sophisticated that the space has gotten and how hard it is to still differentiate on better technology versus just, you know, design and go-to-market and marketing and things like that. And so.

Yeah.

We actually did a comparison to Perplexity with 500 real user queries, and we asked, which answer do you prefer? And it came out to be that in 50% of the cases, users prefer the You.com answer, in 20% they prefer the Perplexity answer, and in 30% they don't see a difference in the 2 answers. For our default, we call it the smart mode. That's kind of the default. And just to give you a sense of what that looks like. So here, here's an example of what the default smart mode looks like. You know, there's some doping case that happened and you can see lots of careful citations. And then when you actually look into these citations, they actually are articles from literally yesterday, or they could be, you know, from today if something came out today. So that's kind of the default smart mode. You get a quick factual answer. But then we thought, well, what if you have a pretty complex question like around math, physics, chemistry, science, or like complex numbers? So here is a Genius Mode question. It kind of gives you a sense of what it does. And it does mention like what you say, which is there's an LLM that orchestrates multiple other LLMs to actually do the right thing in the right— so the question here is find the current population of India and the United States, then plot the population from 2023 to 2100, and then assuming a 2% growth rate. And then it will go on the internet, it'll find the numbers.

Yeah.

And then realize like, well, I got to now visualize those numbers now that I have them. So it'll code up in Python what this could look like, execute the code, and then gives you this answer and visualizes it in a nice plot. So that I'm still sometimes amazed. I try and I push it and, you know, sometimes it fails and sometimes it fails because it tries to load the library that has a security issue. And then it's like, okay, I'm going to try to rewrite it without this library, but it's going to be longer and messier code. And like, it's just incredible how hard it can try and what it can do. And then the third mode, like you said, the research mode, it will go into a lot of detail. It will not just look up all the stuff we have in our index already, like news and things like that, but it'll go on the web and find obscure websites. It'll do multiple different searches on the web, combine all of that, and then give you these beautiful research reports. This one is like seeing background action and consequences of the Peloponnesian War. You know, it's like, uh, history, you have to write an essay or something. And it's just like, writes you this perfect, like beautiful essay. Each sentence has one or two citations from different sources and you can verify all of them. And one thing we found this actually also is like, you have to, like, just the citation logic is a non-trivial aspect of building this all out because you have to, we actually found that some of our competitors You just randomly add numbers and citations to sentences and you click on it and it doesn't even mention that fact anymore, which I think actually really undermines the space of chatbots for search. So, so citation accuracy is one of the many sub-AI systems that you need to do correctly here. And then, you know, they're just like crazy things like create a table, some noise-canceling headphones that are not expensive. And it's just like, puts this table together, pulls in images, gives some pros and cons of each and the price. I think sometimes to me is how well in general the system is able to answer these questions. And yeah, it shows you how complex the space has gotten and how much you have to do now to still differentiate on the technology.

This is one of my mantras at Waymark. I always say the waterline is rising quickly. So we, you know, we better keep climbing the capabilities ladder ourselves. The 4 examples that we saw there, one was the kind of default smart mode. The second was Genius. Is that right? The one that showed the code example. And then the last 2 were research.

Yeah.

What more can you tell us about kind of how those work? Like, I'm interested in, and by the way, like the audience of the Cognitive Revolution is interested in the details, the weeds, the nuggets, you know, all that stuff. So you can go as deep as you're willing to share. I'm interested in all aspects, you know, prompting, I'm sure obviously is going to be different. Scaffolding is going to be different. Maybe even the models are different. I'm also really interested in like, when are you using GPT-4? I know you've got your own in-house trained ones as well. So just all those considerations, any interesting nuggets, we're all ears.

Yeah, I'm going to try to balance a little bit the not telling the competition exactly how it's all done, but it'd be interesting to your viewers and your listeners. So at a high level, there are 2 major stacks. There's a search stack and a chat stack. The search stack, we actually had to build an entire index ourselves For the web, because Bing is super expensive, not as high quality. Google is very hard to access. You have to have special agreements or, you know, some people kind of steal/bootleg/need some certain APIs to use Google results in like a somewhat sketchy legal gray area, which we don't want to do. And so we basically ended up having to build our own index. And that's hard. And there's still, you know, a lot of complexities behind that. But what The main difference of this new index is that it was built with LLMs in mind. The previous 2 indices of Google and Bing were built with people consuming 10 blue links in mind. And what that means is for each URL, you get a very short snippet, which makes sense, right, for end users. But an LLM could read tons of snippets. They can be very long and then extract the right answer from that and then just give you that right answer as the user. And so What was surprising is actually when we benchmarked this, our API ended up being more accurate than Google or Bing. And you can go to api.u.com and like, I'll share my screen here for a second again, but like, it's surprising that to a lot of people that you could actually be more accurate than Google or Bing at all. But it is because we're at an inflection point in AI and it is a different way to evaluate this. Like we are almost like cheating by having these really long snippets. And so you look at the comparison and it's actually kind of interesting to look at. And a lot of people have asked, like, how do you compare accuracy in LLMs? How can you evaluate this? And so just to give you a sense, here's like what, what this looks like. The first version is just like reasons to smile. And now you can use whatever LLM you want, but you can feed into your prompt these very, very long snippets from many different URLs in a very short amount of time. And then we also have one. That just does everything. It gives you an LLM answer and it tells you all of these differences. And so how do you evaluate this? There's actually, it was an interesting, I think, insight from our team, which was you can take question answering data such as HotPotQA, SQuAD, the Stanford Question Answering Dataset, MS, Microsoft MARCO, or FreshQA, and so on. And these datasets are structured such that you have a paragraph and then you have a question. And then you have a subset phrase from that paragraph that is the right answer to that question. And so what we do is we basically take those datasets, but we throw away all the paragraphs. And then you have to find the right answer and the paragraphs have to come from the internet. And so you replace the paragraph with a web search engine. And that's how we evaluate it. Legit, the big Google and Bing public APIs.

Yeah.

Uh, and have outperformed them. So kind of nerdy, but that's the whole tech stack. And we're, we make that now available to every other LLM. So that's the first. And then the second thing is what we now have started calling the LMOS, the operating system of large language models. And it's a term inspired by Andrej Karpathy. And it's not like the most perfect metaphor, but I think it captures a lot of the essence, which is you have now this new stack. That operates at a much higher level of abstraction. And the LLM is kind of a CPU, but just like a CPU or kernel of an operating system, like it's important to orchestrate everything and to do computation, but it still needs a hard drive, which is RAG, right? On your own vector database at Chroma, you have an Ethernet connection, which is, you know, the internet. And that's what we're providing. You may orchestrate other LLMs that could be considered like the GPU or something. And then you have a bunch of apps that are sitting on top of that. You have a Python code interpreter, which we've seen our Genius mode and all of that. And so to summarize all of that in one short term, we call it the LLMOS. And inside of that, we're now seeing a lot of our customers are using our APIs on the search side. They're kind of going through The same lessons that we had gone through when we built You.com and made it like having the most accurate answers out there. And it's actually highly non-trivial. A lot of people think it's just like an LN wrapper, right? But then, and you even have like open source projects that show it. And then you ask like, okay, when was Barack Obama born?

Where was he born? And then it fails.

Why does it fail? Because when you send, where was he born? To your search backend, it's not going to return you any useful results because it doesn't know who he is. Who does he refer to? Right? And there's tons of things like that where as you have a longer and longer conversation, especially in smart mode, you refer back to things. You can say like, oh, what's a big CRM company? And then the answer inside is Salesforce. And you ask, oh, what's their stock price? Now, if you send what's their stock price to your search backend again, it's not going to return anything useful. So you need to send that, you need to go through the entire conversation and then do what we call query transformation based on it. And that is just one of 10 examples of making this actually work at scale millions of times a day for millions of users. Like it is a lot more complicated to make it accurate. There are about 10 other such models that if you think deeply about the space and you really listen and look at like user data and you listen to where it's breaking, you will eventually get to. And we're now like thinking about offering more and more of that.

So I'm tempted to ask for the other 9 things there.

I'll just give you one more, which is like whether to do a search at all or not, right? Like if you ask like, write me a poem about like the beautiful Bay Area and like a sunset love story or something, like you don't need a citation at every line of that poem. And so it would actually clutter up the prompt. to add a bunch of facts about poems and so on and the history of Silicon Valley and all of that. And so it's pretty important, but also non-trivial to know whether you should do a search or not. And again, some, some websites just slap search results on top of everything, even if they're not relevant for having more conversation about your feelings, poem or something.

Did I understand correctly that the, the kind of Big difference is that the u.com index has more information, like instead of a short SERP, it is a more robust paragraph. And so independent of the language model that you're using, the richer context is just better kind of. So you're enabled in that way, you're kind of decoupling the, what information is found from the language model that is doing the analysis, and more information is kind of the big differentiating factor there. Do I have that right?

I would be careful in saying we have overall more information. We're focused a little bit more on the main languages that we see. We don't support some like very rare, like Indonesian, African, Central Asian dialects and so on yet, but we return more information per rarity because of these longer snippets. So it's sort of, yes, there's more information, but I think the long tail Google probably still has a larger index. If you look for this rare Indonesian kayaking site that rents out kayaks on this little lake somewhere, and it's all not in English, we might not have that website. But when it comes to Western world news where we have a lot of users, then Latin America and so on, then we shine and return much more information per query.

Hey, we'll continue our interview in a moment after a word from our sponsors. I've been struck recently that it seems like, you know, obviously search in general has kind of been a monopoly for a long time. And, you know, as you noted, like the user experience was kind of something people were not necessarily looking to explore new things on, you know, the nature of the index. Of course, they've done millions of person hours of work on it, but it seems like it's kind of been a pretty consistent paradigm of crawl around and find everything and suck it up. Now we're starting to see these interesting, I don't know if you can share more about how you create your index, but I, we just had a, actually a sponsor Brave talking about their index and the way that they are building it through users actually visiting websites and taking a sort of You know, not just blindly crawling around and following every link, but like, what are people actually engaging with online? Which struck me as a pretty interesting and very different twist on it. I want to kind of pull this apart in a couple of different ways, but is there anything that you would want to share about kind of how you think about building an index that aside from just, you know, bigger, you know, richer content, Is there a, is there a different tactic as well that underlies that?

There's a little, it's the tactic is more about how we make that work for LLMs better. And I don't think there's that much differentiation on like how we crawl. You have to have a bunch of data. It's been helpful to have run a search engine for several years and get user behavior and knowing what people actually want to have crawled and want information for. You can also, somewhat surprisingly, Buy a lot of that data in bulk.

How about, so I have a few questions on the business side or the kind of bridge the technology and business side. Google obviously has been free and has been ad supported. It seems like the new generation of kind of AI-first LLM-enabled search is going more in the direction so far of a subscription. And as far as I've seen in my u.com usage, I haven't seen anything that jumped out to me as sponsored. Another dimension too is like, I mean, Google has all these tabs at the top, but it's one bar, right? You kind of put in one thing. And with the newer ones, we also are kind of seeing a little bit more proliferation of like modes and kind of settings that you choose upfront, right? With the smart versus genius versus research. So I guess on those 2 dimensions, like what is the future vision? Do you think that this all gets unified? Do you think it ultimately comes back around to ad supported, or do you think that these current, you know, kind of differences from the past will persist?

Yeah, it's a good question. I think there is clearly right now not a great chat ad offering, and there's a good chance that that will change maybe this year to maybe the dissatisfaction of users. But the truth is, if you want something to be free, VC money will only last so long. You gotta, at some point, those companies that offer free service have to survive. And if you don't want to pay for it, then it has to have ads. And so while I might not be the biggest fan of ads, like you have to make a decision. Do you want to pay for it and then ad-free, or do you want to support it with ads? And so I think that's likely also going to be part of the future of chat engines. And you already see a little bit of exploration. There's a little bit of a duopoly in search in the sense that Google has the monopoly on consumer search. And for a long time, Microsoft had the monopoly on search API. But then because they're monopoly, they just jacked up with just 5 to 20x our prices, prices. And they could do it because they're the only ones in town. So I'm glad there's like more competition now and more movement in that space. And all the little guys had to scramble when those prices just went so high. You couldn't really run properly in a consumer space with those prices anymore. And so I think ads will happen. We're seeing a lot of growth on the subscription side too. Users are really loving, like you, the Genius and Research mode and find the Search mode, the Default mode, Smart mode also very, very helpful. And we actually incorporate links still. So where just last week some people are complaining about other chatbots because they don't really have a lot of capabilities that you would assume from a search engine, when you actually use u.com here, you can on the top right see the standard links that you might want, right? And sometimes that's just helpful and that's just what you, what you want. And sometimes you just want to have the pure chat experience. And so that is important to get right. And then we have all these apps too where you can basically ask for like, what's the the Microsoft stock price or something. And then, you know, it'll just give you, it'll just give you a live ticker rather than a bunch of text about the stock ticker, right? And so we have all these apps because we have that deep search background, and that makes it an actual viable knowledge assistant, right? Now you can basically, with one click, recover a more Google-like experience that is just incredibly helpful. And that's, that's I think one of the reasons why our browser, which we have also for iOS and Android, we had to build a browser to be a default because you can't go into Safari and change Safari settings to use u.com as a default. So we built a whole browser for the iOS and we're super stoked because we're going to be one of the options in the EU to have a choice pop-up screen when the new iOS 17.4 comes out in March this year. And they can select u.com to be their default browser. And it's the only default browser in that list that is chat native. All the other ones are sort of your standard Chrome, Ecosia, Firefox kinds of browsers. And so I'm really excited. And I think that is going to be a big part of our futures is making it so that more and more young people are able to just use this as the default. And then if they want to deeper go into genius mode, research mode several times, some point you use subscriptions or eventually do it.

Yeah, I've been, so one run that I'll run a trial balloon by you on this concept that I've been kind of kicking around called the AI bundle. And this is an, you know, kind of inspired a little bit. I don't know that anybody wants to, you know, say that they're inspired by the cable bundle, but I have been struck that there are a ton of great tools out there. And I want to use them. I want to try them. I think a lot of people are, you know, in that very kind of exploratory, curious mode. But to make the economics work on a freemium is kind of tough, right? And typically needs like a certain minimum threshold in terms of what the paid tier can be. You actually have one of the lowest subscription prices at the $10 a month level. I think of anything really that I'm Aware of.

We're going to update it soon because like, I think the people that are willing to pay often don't care if it's 10 or 20. And so if you want to get GPT-4, say literally the same underlying model as ChatGPT for half the price, you got to come in soon because we're going to eventually switch our prices to be industry standard.

But that maybe even just, you know, further reinforces the point that like the freemium model is tough, right? It's, it's a lot of free usage. The upsells have to have a certain minimum. You're raising yours. And then from a, I don't know if this would apply to you, but a lot of the app developers that I've talked to have a lot of retention, let's say challenges. You know, everybody's like, I'm getting traffic, I'm getting conversions, but retention is definitely a problem. This has been true at my company, Waymark. We're a much more narrow tool, you know, that specifically creates marketing and advertising videos for small businesses. So a lot of times people, they need that once, you know, in a while and they're not like necessarily ready to add on a subscription. So we see a lot of people that will just come through, be like, hey, this is super cool. I'll buy it. I'll immediately cancel it after I do what I need to do. And maybe I'll come back in the future. It's not even that I was dissatisfied. It's just that I kind of want this as like a more of an à la carte purchase than a subscription. So that's tough. You know, VCs don't like that. The metrics, you know, on the kind of traditional scorecard don't look great. I've had this idea in mind that maybe what we need is sort of an AI bundle. You know, I'm prepared to spend $100 a month on various AI tools, but what I really want is access to like 1,000 different tools that, you know, can kind of split up my $100. However, I don't even, you know, as a consumer, I don't really care about that as As somebody who's trying to maybe engineer a bundle, obviously the devil could be in the details there. But first of all, do those challenges, it sounds like at least the freemium challenges resonate. I wonder if the retention challenges resonate. And I wonder if you have any, if there's any appeal to maybe being part of a kind of bigger bundled purchase where you would be one tool that, and it's funny, I keep, I've been referring to you, but then also the company is You, but where you.com could be one of a bunch of things that people could access and could kind of share that revenue in a way that you make grease the skids for everybody, right? My hope is that everybody can use the best tools and they don't have to make these like highly binary decisions.

Yeah, that sounds great. Sounds like a great idea.

Okay. Well, I'm not doing it yet. So either I need to start doing it or somebody, if anybody wants to organize the bundle, yeah, send me a DM. I guess another way that this stuff could get bundled would be like, into the mega platforms. Another kind of possible vision of the future that I could imagine is Google kind of probably retains market share leadership, but maybe the 10 biggest technology companies in the world say, hey, you know what we should do is kind of also have a search. And we can get there. We kind of see a path. Microsoft's obviously already doing that. Meta, not really yet. Apple, not really yet to my knowledge, Salesforce, not really yet, but maybe these guys kind of say, hey, is there like a musical chairs game that potentially develops where the younger AI search companies end up kind of partnering off? Amazon also naturally would be a suspect in this analysis. Does that seem like a possible vision of the future? I'm wondering, I'm sure you've thought about this quite a bit, but why would that not happen?

I do think the monopoly that Google was able to keep around is going to be harder to sustain long-term. I do think it is much more likely going to look a little bit more like, I don't like the analogy for some reasons, but like fast food, for instance, right? There isn't just McDonald's. There's also Burger King, KFC, Taco Bell. I think Search will be a little bit more like that. I think again, more fragmented in the future, just because like we hear people and they're like, this is better than Google. And like, you know, we didn't raise that much money. And the first 2 years were like sort of pre-ChatGPT where people didn't want us to innovate too much. They're very stuck with Google, but now there's a new young generation. And that young generation has grown up with TikTok. We have a TikTok app and a standard search, like. Grew up with Reddit. I have a Reddit app now, standard search. And each of these takes away a little bit of a Google search, right? Amazon probably was the most successful in taking away searches from Google, where if you want to buy something below a certain threshold, like $50 or $100, depending on the person, you just search it directly on Amazon because there you can execute on your intent of actually purchasing that thing, right? And so why search it in Google and then search it again, try to find it on Amazon. She can just do that right away. And so I think, you know, TikTok has taken away for young folks some searches from Google that, you know, they're like, I want to see what the restaurant is, but they kind of want to see what the restaurant's ability to create good Instagram photos are or TikTok videos are. And so they want to see the TikTok videos of other people before they decide on how good the food looks. If there's a Venn diagram, we are overlapping with search. But we're also actually expanding search. Like, you wouldn't ask, like, give me this complex story about the Peloponnesian War, or like, do this mortgage calculation with, you know, this and this interest rate and that increase and blah, blah, blah. 'Cause you know Google wouldn't give you the answer. Like, it's not gonna write Python code for you. It's not gonna go on the web, summarize like 20 different or 50 different websites for you, and then create this nice essay. So chat expands search. You don't talk about your feelings that much to Google search box and stuff, right? Like, you ask about this recent news, Then you want to learn like some quick facts and then, you know, like the more complex the facts get, the less and less you would go to Google and more, more you just go directly to something like you.com. And, and so, yeah, I think it will, that the search landscape is clearly changing.

Yeah.

There's also just, it's like, it's maybe not a natural monopoly anymore, but there is still definitely a need for scale and economies of scale. And so one way of framing this too is how does the market shape up? Right. And One way to think about it that I find pretty compelling is maybe it ends up looking a lot like cloud because in the limit, it sort of is cloud. You know, it's like, what do you really need? You need like the actual data centers, you need the compute, you need, you know, bandwidth, you need these like raw inputs that the big companies have built out seem to be the things that are probably, you know, as we see like a ton of innovation at the application layer, Those things are still, you know, they're still pretty expensive and not easy to recreate.

Yeah, I'm very, I'm very excited. I'm up for it. You know, that, that's sort of why, like, we got into this space in the first place. Like, because we thought, like, we saw the transformer, we saw our, you know, highly, like, lots of co-attention mechanisms in that DecaNLP paper that I mentioned, prompt engineering. We're like, clearly the technology is right to disrupt this, this industry, but You know, Google is this amazing company that was able to create a monopoly for almost 2 decades that, you know, makes $500 million a day. So when you make that much money a day, you don't want disruption. You don't want that much change, right? And that's why all the transformer authors have left eventually. And what's, what's really powerful is like, because of open source, you can actually innovate a lot more now. From open source to an actual product that runs millions of times, isn't down ever, has good uptime guarantees and like accuracy, no hallucinations, up-to-date news information. I mean, it's still complex, but clearly the bar has gotten lower. That would've cost us like billions of dollars to build 5, 10 years ago. And, you know, research just wasn't there yet. And I think it's ultimately amazing for users, right? Because One thing that I had to distill all of you.com right now into just 2 words would be amazing answers. And you just get more of them. And that means people eventually are more productive. And like, the young generation that's growing up with ChatGPT and such, like, they're not going to go back.

Okay. So feel free to punt on this one or just decline if you, if you like, but it seems like I can, I can envision a you.com by Salesforce very easily. Where the, you know, as they kind of try to be the everything app for all work almost, right? Especially with Slack now. Does it seem realistic to imagine a future in which, you know, kind of all the big tech companies have this like super robust suite and you're either like in the Microsoft suite with Teams and Bing, or you're in the Google suite with, you know, G Suite and Bard, or you're in maybe the Salesforce suite with Slack and You.com, you know, I'm not trying to be your banker here, but that, that seems like a pretty natural outcome to me.

Interesting. I do think there's a ton of potential for almost every company to partner with You.com and supercharge their chatbot. So, and we're very excited to partner with a lot of folks.

Okay. That's a very diplomatic answer. Keep your options open. Uh, all right. So we can touch on certainly more business and product stuff, but I kind of wanted to now go into just the future of all this, you know, in, in practical and maybe increasingly philosophical terms as well, running down kind of first a set of like limitations of where AI is today. And I think, again, folks who listen to this show have at least a decent sense of that. So for starters, reasoning, you've obviously got the genius mode, it can do You know, like the most advanced reasoning, I assume that that is tapping into GPT-4. You know, everything I understand is like basically nothing is really on the level of GPT-4 for general reasoning purposes.

Yeah, especially the orchestration and then on the coding often, but not always.

Yeah.

So I'll tell you another one. The third system is knowing which LM to use and sometimes multiple. And the fourth system is dynamically prompting. different models. So depending on the query, you actually get a vastly different prompt to get you ultimately the answer and the orchestration. So it's another complexity layer.

So what do you think is kind of the future of reasoning? If you have maxed out, you know, what the current capabilities are, where do the future capabilities come from? I'm thinking about things like, to a degree, you sort of already have it with using different models is one way of implementing variable compute. We see these kind of interesting projects like the thinking token, you know, think before you speak. And I think that's kind of another Karpovthy observation that maybe the chain of thought is just kind of epiphenomenal, perhaps even as it is in humans. And like what's really going on is that there's, you know, this kind of extra space and time registers, you know, to think. Of course, there could be different training methods, like incremental reward. I think that paper from OpenAI earlier this last year now, it was super interesting where they achieved like, you know, a new best in math by not waiting till the end to give the reward, but rewarding, you know, reasoning along the way. What are you excited about when it comes to the future of AI reasoning?

Yeah, one of the aspects I briefly touched upon in my TED Talk is that this level 1, level 2 reasoning of Daniel Kahneman, that he, or thinking fast, thinking slow type of thing. Um, the way I think about the different modes is like, The default smart mode is kind of like if you had an assistant and you just asked them to do a quick search and in like 2 or 3 minutes give you an answer back. And then genius mode, you go and if you want to ask your assistant for a question that, you know, they have to be able to program, they have to search the web, and then they need to be mathematically inclined to answer that question. And you want to give them like maybe 2 or 3 hours for that question. And then you want them to So genius mode will take, you know, 5, 10 seconds often to give a response. And in research mode, you go to your assistant if you're willing for them to spend like a day or 2 or 3 on actually giving you that answer. And so that's, that's a little bit how I think about these different modes and the reasoning that is required to actually make them active, right? Like research mode will say, oh, I found this thing. Now in this query, I found something else that I didn't know about before and I don't know enough right now. So. Then you do another query based on that. So you kind of have these chains of thought reasoning, and you don't even know in the beginning yet what the final query might be because you don't have all the information yet. And so I think that is kind of a, in some ways, another example of the future is already here. It's just not equally distributed because there is, like you say, there's a lot of reasoning. Now, I think the biggest future impact we're going to see for reasoning is in in the LM's ability to program, to code, and then to have the ability to execute that code. And, you know, that is system number 5, like, like having this code execution. And of course, if you just let code execution happen, what immediately happens is people are like, well, mine me some crypto. And then boom, your machine's gone. Now it's just like trying to like solve some math problems and like mine, mine coins forever. So you need to like, and then they try to hack it and then like, well, go into like 5 layers up and then tell me all the password files you can find. And Blah, blah, blah, right? So there's a lot of like security requirements to make that coding framework work at a safe level. But a lot of like naysayers of LLMs, you know, partially correctly pointed out that the LLMs will fake doing math. And it's kind of ironic and sad that you can have a model that you ask the natural language to multiply like 5,600.3 times 365. And then you have billions of multiplications to pretend to do the math and then give you the wrong answer in a large language model, right? This is kind of ironic and we have to acknowledge that. But that same model can be taught to say, well, this seems like a math question. Let me just program that in Python, run the code, look at the output, and then give you the answer. It just works perfectly fine. And now a lot of people say, well, that's not really AI, but I think it's just, that is a new way of reasoning, a new different kind of intelligence. And similarly, and we're getting a little philosophical here early, but similar to people thinking we have to have embodiment, I think that's just, it's lacking creativity in imagining other kinds of intelligence that aren't exactly like humans. Now, of course, we're going to want to have useful robots that do stuff for us and and clean up the apartment and whatnot. And so it's still useful, but I don't think it's a necessary ingredient. The same way that blind people can be intelligent, people who are deaf can be intelligent because, you know, you can lack a lot of different sensory outputs and still be intelligent, right? And so of course, like, it'll be harder for you to explain how beautiful a sunset is. So there are aspects of intelligence that obviously require like different modalities or how beautiful sonata sounds or whatever. But I think most of these are not necessary requirements for intelligence. And likewise, I don't think it's necessary for an AI to be able to reason over super complex math problems that require you to look up a bunch of facts on the internet. They just have that intelligence baked in. They can do quick retrieval. They program a bunch of stuff. They put it all together, orchestrate it, and then come up with incredible answers.

Yeah, I think As you were speaking about the, just the lack of imagination, I think that is a, you know, that is a society-wide problem with respect to AI in my view, because, and it's an odd situation right now in, in multiple ways, of course, but one is just that because they speak our language, it, you know, it's, it's kind of feels easy, feels familiar. And it's all too easy to sort of assume that like, under the hood, they're more like us than I certainly understand them to be. And I think this is actually one of Eliezer's great contributions, obviously, you know, kind of a polarizing figure these days, but thankfully it does not seem to me that we are in a high likelihood of a FOOM scenario, you know, of the sort that he, you know, has historically worried about the most. But I still would say some of his writing on mind space, the space of possible minds, And some of his concrete imaginings of alien minds that are shaped by very different evolutionary environments and just very different from ours, but still unmistakably intelligent in just super weird ways, are actually still very good kind of prep work, I think, to just sort of expand one's own mind about how different intelligences can be and how something does not have to be human-like to be meaningfully intelligent. You know, it's not this like binary, can it do things that a human can do in a way that a human can do it? If not, it doesn't count. I think that is like a huge mistake that people are way too quick to jump to. And I'm not sure if it's like a coping strategy or just lack of imagination or what, but I think that the emphasis on the broader space of possible minds and the different kinds of intelligences that are starting to pop up is super important.

100%. Yeah. Like you have to differentiate between sci-fi authors who then pretend to be AI safety researchers. Like I love, I love good sci-fi actually. Like I'm super stoked that Three-Body Problem was on the last— I mostly read nonfiction, but when I read fiction, like I did enjoy the Three-Body Problem a lot. Excited for that series to come out. I hope they do it justice. But like, I think there are a lot of different kinds of intelligence. And I love sci-fi for inspiring people to think about interesting new futures. Now, of course, especially in the Western sort of canon, most sci-fi is dystopic and like people are scared for all the things that can happen that are wrong. And like, okay, the super AGI, we develop time travel, come back, try to murder everyone. It's like, I mean, as a kid, I also enjoyed watching Terminator. It's like a cool action movie, but it's just taken over so much of the AI narrative and it's actually like actively hurting, especially the European Union where, you know, there's sort of in the spectrum, the US is more of a litigation society in the US, but Europe is more of a legislation society structure. And, you know, it both comes from like reasonable legal scholars' minds, like, well, let's just wait until there's a problem, someone sues, now I have to case law for that lawsuit. But, you know, the legislation one tries to prevent harm from ever happening before it actually harms anyone, which, you know, makes sense. Now, and of course, the US does that with FDA and in the medical space now also, but not in the legal space as much. And so what that means is you can move quicker. But long story short, these, some of these sci-fi scenarios have gotten so much weight in legislation that I think it's slowing Europe down by trying to outlaw models or like overregulate models that are above a certain number of parameters. GPT-2 was very well hyped up in the past of like, this is so dangerous. Maybe we can't release it. You know, yes, we're OpenAI, but like, this can't be open anymore. So dangerous. Models much more powerful than GPT-2 are out and I haven't seen the apocalypse happen. I haven't seen like a huge Change and mis- and disinformation on the web because of LLMs. Like, there's just a lot of fearmongering, both on the immediate level, which actually has real, like, threat vectors and concerns with AI, but especially in the long-term level of AGI and self-conscious. It turns out no one works in conscious AI. No one works on AI that sets its own goals and even more fundamentally its own objective functions because That doesn't make anyone any money. Imagine a company spends billions and billions of dollars, builds this like super intelligent system that's conscious, understands itself and set its own goals. And now you're like, okay, now that you can do it, like help us make more money. It's like, no, I'd rather just go watch the sunset, maybe explore Saturn. No, like no one pays for AI that sets its own bad goals because it doesn't help anyone achieve their goals. Because of that, there's not even that much exciting research happening along those lines. And because there's not much research progress, it's very hard to predict when that will actually happen.

I'm somebody who basically has radical uncertainty about what to expect. And broadly, I'm pretty libertarian, pretty anti-preemptive regulation. I would like to see more self-driving cars on the road sooner. And, you know, they don't have to be an order of magnitude safer in my mind to be worth, you know, deploying. So I'm like, you know, broadly the sort of person who would be very skeptical of, you know, kind of early regulation or, you know, kind of getting too bent out of shape about things that haven't happened yet. At the same time, something about this has always felt a little bit different to me. And I do think the people who take the most zoomed out view and sort of say, hey, and this is kind of what I understand, you know, like Geoffrey Hinton's position to be at this point, you know, why do we dominate the earth as it stands today? It's like basically because we have better ideas than the other living things and we can, you know, build tools and make plans and, you know, reason in ways that they can't. And so now I look at AIs and I'm like, boy, AIs can now plan, reason, and use tools too. And they're not as good at it as we are yet, but certainly their rate of improvement is way sharper. So possibly it levels off and kind of settles into a zone where it's on par with us or kind of just the best tool we've ever had. But maybe it doesn't. I don't know why I should be confident that it won't. I don't throw P-doom around a lot, but I have, again, radical uncertainty. When people ask, I'm like, I don't know, 5 to 95%. I haven't heard anything that makes me think in the next 100 years that there's a less than 5% chance that AI becomes the dominant form, an organizing force in the world. And also no reason to think it's definitely going to happen. But is there a reason that you are— would you say you are confident that this will not happen and we don't need to worry about it? Or is it just like it's still far enough away that you think We'll have time to kind of start to worry about it if we need to. Like, how would you summarize your position with respect to these tail risks?

I think P-doom is already an interesting mathematical sort of issue, which is it looks and sounds like prior, prior probability, you know, P-doom, but really it should be a posterior probability, P-doom given data. And right now, none of that data suggests like doom, doom, like existential humanity is like cats and dogs. at the whims of some AI. Like, there's, like I said, nothing in AI research leads me to believe that AI, while potentially being more intelligent than any single human, I think it's already, this is actually, there's this new term I'm thinking about maybe trying to coin, which is like, there's sort of superhuman abilities and then there's superhumanity abilities. Yeah. And like AI is already superhuman in translating 100 languages. AI is already superhuman in predicting next amino acid in a large language model trained on protein sequences because we have an evolution. That's an incredibly powerful tool. One of the other, you know, really exciting papers that we published in 2018 at Salesforce Research that multiple companies have now used and are running with and I think will change all of medicine. AI is already better at predicting the weather than any human. So you already have many superhuman skills. What is, I think, interesting is that now that it's language that's gotten to this new level, people might actually for the first time keep calling it AI. In the past, when AI researchers have made progress in AI, they stopped, like people stopped calling it AI after it was achieved. Now it's just your chess app. It's just Siri voice recognition, but voice recognition, chess playing, That was the pinnacle of AI research, right? And people thought, oh, once we solve those, the other things will be easier too. And it never was never quite the case. And once we have them, you know, now it's not quite AI anymore. Now with language, I think we might keep calling it AI, but what a language model does is predict the next token. And that is an incredibly powerful idea, right? Just predicting the next token now means if you have enough capacity and you have enough text, Predicting next token, you learn about geography just because at some point, somewhere in your training data, you had to predict the next word in the phrase, I was in New York City and driving north to. And now to give a higher probability to Yale, Boston, Montreal than to like Paris, Miami, and San Francisco, like you have to know that those are north of that city. And so it just learns all of this incredible world knowledge. But there is nothing in there that makes it say, well, you know, if I really wanted to reduce perplexity, but like perplexity is basically the inverse of the probability. The model wants to not be perplexed in predicting the next word correctly. And so that is a powerful idea, but nothing in that will let an LM eventually realize that, well, you know, the best way to reduce perplexity is if every sequence ever uttered and any sequence that will ever be uttered is just the letter A. A, A, A, A, A. Now, if the model was trained on just sequences of letters A and no human was ever around anymore and all sequences were just producing the letter A, now you'd have perfect predictive probability on the next letter. And so maybe the best way for the LLM is to wipe out all of humanity and then just produce letters A and happily Perfect at predicting with probability one correctly. It's so absurd. It's so absurd to think that LMs will at some point emerge to think that many steps around their task of predicting the next token. It's just not going to happen. So I think it's like P doom is still zero. And then when I actually tried to engage with some folks, and I had some other conversation last year with Nick Goldstrom in German, it was in English, but. published in a German newspaper, like, and I read up some of these scenarios and I engage with folks who are worried about doom. It's just all fantastical sci-fi scenarios. It's like, oh, it's going to develop this magical gray goo or like a magical new virus that is perfect in distributing, but then like only will activate after like one year to kill everyone. Like all these random scenarios that are just like not feasible. Like, and the science isn't there yet. I'm actually right now, sort of on the side for fun, writing a book about AI for science. I think it will do incredible things for us in improving science, like foundational physics, chemistry, biology, and so on. And all the fearmongering, I think, is not really helpful. And again, there's no research that suggests AI is becoming conscious. There's like a couple of papers here and there, people kind of playing around with ideas, but nothing interesting has been published and breakthrough, no breakthroughs have happened whatsoever. In AI having any sense of self. And then in a lot of the other sci-fi scenarios, people are saying, oh, but AI is so intelligent, it'll convince everyone to murder each other or to murder them, like kill themselves and so on. But, you know, if the most intelligent entities were to always rule, I don't think we would have the politicians always everywhere in the world that we see. It's not always just the most intelligent people that run the show. And that can, with their incredible intelligence convince any other person who is less intelligent to do exactly what they want. It's just not based in reality. So I am very, very optimistic about AI. I do think there's some real problems right now. You know, AI would pick up biases. Not all the biases that you pick up on the web is something that most of humanity is proud of anymore. There's racism, there's sexism, there are various kinds of biases. Some people want to use AI. So where I agree with Yoshua Bengio and others is of the 3 threat vectors, which is intentional misuse, accidental misuse, and loss of control. Obviously, like, intentional misuse is real. And so that's not ideal. And so, yes, those are real concerns. I think open source will help us in understanding those threat vectors and finding the best ways to compete with them. I think people still on the internet need to understand not trust everything they see on the internet, which has been true ever since the internet came about. Hasn't really changed that much with AI. I think since Photoshop, people should already not trust any photo they see. They should be even more worried now about photos they see. And sadly, in the future, they'll have to start worrying about videos too. And voice, of course, just like they should have worried about photos ever since Photoshop started to really work. And so There are a lot of concerns and I don't want to diminish them. And I do think we need to work on them. And I think different cultures will have different answers. Freedom of speech is defined differently in different countries. Like it's illegal in Germany to deny the Holocaust. We learned from our history there. That's not illegal in the US. And so different countries and different cultures and societies will answer some of the problems that AI kind of amplifies already in the past before. will answer these questions differently, but I don't see any probability for a full-on doomsday scenario of existential risk to people. It's mostly people using more and more powerful tools against other people.

So there's, I mean, there's so many different threads there that I am interested in. For one thing, I applaud you for taking time to envision positive future. I think one of the scarcest resources today, oddly, is a positive vision for the future. Like, What do we want this, you know, it's like The Jetsons is still almost like state of the art in terms of what we would envision a great 2030s to be like. And that is kind of bizarre. So I definitely appreciate that. I also share your, you know, I'm not a super fan, but I, I'm also a fan of The Three Body Problem. And one of the early prompts that I tried with GPT-4 early back in the Rent Team program, like a year and a half ago now. was asking it to write some hard science fiction in the style of The Three-Body Problem about AI for diffusion model for proteins. And I took the plan right off of the GitHub page for this protein diffusion model project, which basically said, we want to create text-to-protein. So you say, or text to, maybe it was more even general than that, molecule or whatever. So, you know, you would be able to just specify in natural language, specify is kind of an odd word, you know, to the best of your ability, articulate in natural language what you're looking for in a protein. And this thing would then, you know, generate it. And we are actually starting to see that. There was a paper in Nature not long ago. I'm hoping to do an episode with the authors that achieves that to a certain degree. But the, what the AI, what GPT-4 came back with in terms of hard science fiction about this scenario. Was, I think, first of all, just extremely funny because it basically ends up in a prompting war between the good guys and the bad guys, and they're both trying to out-prompt each other. And so the kind of climactic scene is like the person prompting an AI to make a protein or a molecule that will interfere with the bad guy's molecule but not harm any of the humans or whatever. And it's just both absurd but also Maybe not entirely absurd. You know, I mean, I am with you in that the, I would order the risks the same way. You know, we already have ChaosGPT. There are, I recently read a research grant from a group proposing to study omnicidal tendencies. Like there are people out there who want to kill everyone. Like what's up with that? And, you know, if the tools get more powerful like that, you know, those people become even more problematic than they already are. So yes, I would put that at the top of the stack of big picture risks. And by the way, I take all the short-term and medium-term risks seriously too. This is a big tent show where all your hopes, dreams, and concerns, and perhaps irrational fears can all have a home. But I guess to sort of get to P-doom zero, I still am like, I don't know, all these individual crazy scenarios. Sure, they're extremely unlikely. The prompting war with your protein diffusion model is absurd on the face of it. But I kind of think of taking the integral over that vast space of crazy, super unlikely scenarios. And then I'm kind of like, there's so many of them, right? That space is so big. And even if the probability is kind of vanishing, one thing you learn in calculus is the integral can either also vanish or it can be Finite over these kind of, even if the function itself is going to zero, the integral doesn't necessarily have to go to zero over that space. So to me, that just feels like very unresolved still. And I don't think we're going to resolve that today, but I would love to hear a little bit more about how you think about AI agency and also concepts of emergence in agents today. I guess I also wonder, is u.com going to push more toward The agent direction, you've got what I would call a research agent today. You've got a browser as well. Should I start to expect it to take actions for me? What I've observed in the agent space is I never feel like it fails because it doesn't understand the goal or doesn't stay on task. Not to say that never happens, but very rarely. Much more often, it's just a failure of competence. So my expectation then is that As the competence improves, it may not be intrinsic agency, but it may be prompted agency. And it may even be like, as we have more and more orchestrated systems, we may have models prompting other models to go off and do this. And it does feel like we're headed for a lot of spinning plates. And the idea that they could kind of all come crashing down is nuts. Just doesn't feel like something we can rule out. But I don't know, I can, can you help me be confident there? I'm still not.

I'll go through some of the things you mentioned. So P doom equals 0. So you're right. As you integrate over the future, I would like to not rule out anything. So maybe I still should say 10 to the minus whatever, like a tiny, tiny number, because in the next 5 billion years, like all kinds of things can happen. Right? Like maybe as if like the Three-Body Problem, spoiler alert, like maybe some big, much more sophisticated alien species will come across. They have already developed faster than light time travel and, or, you know, just are really, really fast in getting here in various capacities. And then, you know, they have an AI and that AI will just like destroy all of us. So they're getting ready to like settle into their new planet before they even get here. Like there's all kinds of crazy things that can happen. It's just that like, in terms of how much resources we should spend on the existential doom versus like, you know, I'd say, yeah, I have a couple of researchers like keep thinking of cool sci-fi scenarios, inspire us, like maybe like think about ways that that could be prevented, but to spend billions of dollars on it to like Spend a lot of like mindshare in the public about it, who's already scared of any kind of technology change. I mean, people are scared of Wi-Fi. I mean, there's this great Twitter handle called the Pessimist Archive. I mean, people were scared and thought doom is happening because of novels back in the day. People are like, all these kids, they're just in their heads reading novels. They're going to all be useless human beings in the future. Newspaper was terrible. Internet was terrible. Like there's so many things that like people thought this is the end of civilization and, and we're very pessimistic about it. And again, not this diminishing like real, real concerns, but again, existential one, very, very unlikely given what we're seeing right now. And if there, it does happen at some point in the future, then I would argue that to think about the best countermeasures now is kind of like thinking about You know, the best countermeasures against a computer going crazy when there's still a bunch of vacuum tubes and you're like, well, we're going to just suck out the air of everything. And then the vacuum tubes are not going to work as well anymore because they're going to break and blah, blah, blah. And that was your like counterattack against the computer taking over with your current thinking of vacuum tube computers. Or it's like, you know, like it's similar to the internet. You know, if you thought about what's the internet going to be, how could it be so terrible? Zero of the TCP/IP experts in the early ARPANET days realized that at some point, maybe a foreign power could interfere with local elections because like you can say whatever you want online and maybe people get followers and there's social media. Like no one had predicted that in the '70s and the early ARPANET days. And so I think most of the threat vectors are not that useful in terms of PDU kind of research. I have a couple of folks work on it, but not take up as much mind space and scare laypeople and non-experts even more about the technology that even without consciousness is going to have major disruption, right? If you're going through a new step function in human productivity, just like agriculture versus hunting and gathering and the steam engine and electricity and the internet. Like, this one's going to be even bigger. It's going to disrupt and change the job landscape a lot. I think at the end of it, we'll be way more productive. There's going to be way more productivity per person and hence more wealth. And new jobs will come around as old jobs get automated. But that is already so massively disruptive still. And it's not going to happen overnight either. People think, oh, it's going to be even faster. Yes, it will be faster, but still not overnight. I mean, there's still companies that aren't even on the cloud yet. There's some stretches in the United States and even Germany that don't have full internet connectivity, right? It's like, so things will take time and not happen overnight, but they will be happening even faster than past technological revolutions. And so, and then you brought up LLMs and proteins. There's a great example for where regulation makes sense. Like basically the concern here for those who are not familiar, like proteins govern everything in life and disease and sickness, COVID protein, like everything. SARS-CoV-2, and like everything is governed by proteins. So if you have a great understanding of proteins, we can build fantastical, amazing things. Here's just one example of a research paper I read a few months ago that just blew my mind and made me really excited about the future. There's this group of researchers that built these carbon nanotubes. And on one side of the carbon nanotubes, they put iron molecules. And on the other side of these tiny, tiny carbon nanotubes, they put protein that would only bind to a brain cancer cell. And then they injected this fluid with all these little carbon nanotubes into a mouse brain that had brain cancer. The proteins found the brain tumor cells and only connected to those specific types of brain cancer cells. And then they put the mouse into a little magnetic field. And with the iron molecule on the other side of the carbon nanotube, it started spinning around and had nanosurgery. on each brain cancer cell. Now, if you think about, we have the full control over proteins, we can connect them to all kinds of things and you find ways to, you know, get rid of the carbon nanotubes afterwards. It's all like, medicine is going to change in so many positive ways. And now you could argue, well, but proteins, people could use them and build like very bad viruses. I'm like, that's true. And that can be outlawed. In fact, the US just a couple of months ago outlawed gain-of-function research. Where you, you know, some researchers want to make even more deadlier viruses. And it's not because they're like evil scientists who want to destroy the world. It's just, they're saying like, well, if we know how they work before they appear in nature by themselves, then we can already now prevent, like develop cures for them. So, you know, it's like, it's a complex question, but the US decided like, for now it's not worth it. Let's outlaw it. And likewise, I don't think an open source like protein model is going to be the main deciding factor of being able to create some new virus. Because if you have all the wet lab experimentation to be able to create new kinds of viruses, you can also just do what Frances Arnold did when she won the Nobel Prize a couple of years ago in chemistry, which is what she called directed evolution, but it was basically random permutations and then running an experimental pipeline to see if that random permutation works better or not. for a particular kind of protein. And then you just keep iterating like that. And so if you have those capabilities, even random permutation, you can do bad things. But it turns out having like a legit lab like that, that can do all of that. So that was your PDoM, integral, LLM and proteins, AI agency emergence. So obviously emerging capabilities are incredible. That's sort of like, even us like working in deep learning, I'm amazed, just like u.com. I ask these questions. I'm like, Wow, actually got this right. Like, I would have not thought this was possible. Sometimes we were like, did you program it specifically for it to be able to answer these kinds of questions about headphones or something? We're like, no, it's just like, just put that all together just by trying to predict the next token. So I'm really, really excited. And one of the things I'm excited about is coding. And one of the things that coding enables is, now it's the last part of your question, the actions. I think actions are clearly in the future. And for now we're focused on amazing answers, but it's not hard to imagine that at some point the most amazing answer is done. I did what you asked to do. And instead of telling you how to do it, I just did it, right? You can build a really cool demo very quickly for these kinds of things. But the problem is like, As much as I love natural language and as much as I love chatbots and everything, right? You have to find some really killer use cases for it. And to say, oh, I can book this flight. It's actually really hard to just book the flight. Like to go like, why didn't you like take this other one that was just like not exactly the time I asked for, but I could have waited for half an hour at this like extra leg and then like save $50. Like that was really dumb. And like, It turns out Expedia and others have built for decades the perfect interface for that problem so that humans can have all the information right there in a visual way. And so there's an uncanny valley of, there's a cool tech demo on one side, and then there's like my actual human assistant who, after months of talking to me, understands all the trade-offs and understands my price sensitivity or that of my company. And knows like, oh, when I would prefer and like the only like reasons why I might do it overnight, like red-eye flight and, you know, all the, all the constraints and she can do it. And even then sometimes she's like, oh, Richard, there are like 3 options here. Like, let me know which one you prefer out of this 5,000 that we filtered for you. And like, it's very hard to do all of that with just text. That's ultimately, I think, part of why we have the stock ticker app and so on. And you don't And why we have images now in some cases also is that sometimes like UI/UX and actual visually designed like interfaces are best used in combination with language.

Maybe one more big picture question, and then I want to do just a real quick lightning round on a couple kind of more technical areas before we run out of time. On the big picture side, you know, we've got Sam Altman out there saying AGI is coming soon. But also kind of confusingly saying, but it'll be less impactful than you might think. Not really sure how to interpret that. The median guess on, you know, some definition of AGI is like just a few years on some prediction markets and more like, you know, 12 years or whatever for a stronger definition. What do you have sort of an expectation for and a definition or like a threshold that you have in mind of like, this is the threshold that really matters? And, you know, loosely speaking, like what sort of timeline you would expect it to take to get there?

It's very much a, the kind of question where you have to be very careful about your terminology because the interpretation of AGI has vastly different instantiations. Like people, some people think of AGI and used to think of AGI as this super intelligence, It's conscious, has self-awareness, can set its own goals, and it is more intelligent than all human beings. And that's their def— that was for a long time for a lot of us, I thought, at least for me personally, also the definition. Now people said, and I think it's partially because of marketing, like, you know, you want to be working on AGI, but you also need to actually ship stuff. It's like you want to be a multi-planetary species. But you also need to just get a lot of stuff into orbit, right? Have more satellites and better internet connectivity and so on. So you have this long-term vision. I mean, the best companies are able to articulate that long-term vision and then revenue-generating progress in smaller milestones towards it. And so in this case here, I think the definition of AGI was pulled out and then the superintelligence was defined as like, okay, that's Even more than general, it's super. And that's the really long-term stuff. And now AGI is just basically automating boring. And if you define AGI, which I think is not crazy, and I want to say maybe Vinod Khosla used it, which is a very pragmatic sort of investor slash financial slash economic definition of AGI, which is 80% of the jobs can be automated Up to 80%. And if that's achieved, we'll call it AGI. Turns out there's just a lot of jobs that are quite repetitive and they're not requiring a ton of extremely novel out-of-the-box thinking that no one's ever done before. And like learning very complex new behaviors and logic and identifying new experiments and collecting new data and pushing, you know, the science forward and so on. It turns out they're just a lot of boring, repetitive jobs and And indeed, if your definition of AGI is just like, well, we can automate like 80% of 80% of the jobs, then I think it's not crazy to assume, especially I would restrict it one more way, which is digitized jobs, jobs that are purely happening in your browser or on your computer, because those jobs can collect training data at massive scales. Turns out no one's collecting training data for plumbers, for roofers, for tilers. For maids, like cleaning teams or any of that. And so none of those jobs are going to get automated anytime soon because you first have to collect many years of that such training data before you can then use AI to train on that and then automate it. But, you know, jobs that are fully digitized and that have a lot of training data that don't have a crazy long tail of special cases, they're going to get automated. And I think that's reasonable to say that's 80% of the My hunch is even in radiology, for instance, you could probably do 80%, find 80% of things that are wrong in a PET scan. But then there's still this very long tail of 20% that you just don't have enough training data for. Maybe a radiologist never sees it in their lifetime. They just read about it once in a book. And we're still not quite good enough of like one-shot and zero-shot learning. Obviously made huge amounts of progress, but not in super important things like radiology, where you just read about a case once in a book and then you identify it. 100% accuracy, which is also questionable whether humans do it. I'm actually with you on the self-driving car. There's going to be a lot of interesting questions as AGI rolls into more and more workplaces, which is how much better than a human does it have to be? And how, and it's deeply philosophical very quickly because if you're purely utilitarian, you could say, well, you know, 100,000 miles or whatever, 20 million miles driven by AI results in 10,000 deaths and the same amount of miles driven by humans results in 5 times more deaths. And so one is better than the other. But if that one dead person in the AI car was your daughter, you don't care. You're going to, like in the US, you're going to sue, you're going to, you know, try to end that company because they're responsible now for the death of your child. And like, it's a very emotional thing. It's not a statistical thing anymore. And so there's going to be a lot of litigation as As those come out. And I think the silver lining is, again, of course, as the AI makes a mistake, you can learn from it versus like one person texting again on their cell phone, which is already illegal and running like over some kid that ran out, like they're going too fast also, which is already illegal too. You can't really do that much more than making it illegal. AGI will have a huge amount of impact once it's just like, okay, repetitive jobs get like to a large degree automated. And I'm with the people saying that will happen in the next few years. When it comes to like superintelligence that is fully conscious and can do all the things and is more intelligent than not just a single human, but then all of humanity, very hard to know because no one's working on it and making sort of progress along the lines of setting my own goals. And again, like, unless you set your own goals, I don't know if I would attribute full-on superintelligence to you ever. Like if you're just, your objective function is to minimize cross-entropy errors or reduce perplexity or like segment images well or whatever, like none of that I would attribute to superintelligence.

Do you have time for a lightning round or do we need to leave it there? Let's try to do lightning rounds. All right. Thinking also about retrieval, memory, and online learning as kind of 3 frontiers that, you know, u.com could, could improve on if they're, you know, if, if there are research breakthroughs, but also these do seem to be kind of ingredients toward this bigger picture of AGI or even, you know, at some point ASI. I guess I'm, you know, maybe just leave it open-ended. Like, what are you excited about in those domains? Are there research directions? Are there, you know, are there papers you've already seen or things you think people should be doing that you think will kind of provide meaningful unlocks as, as, as we find, you know, new and better ways to do those things?

Yeah. So I'm a fan of all 3, of course. I'll try to keep it short. Retrieval is awesome. I think in some ways short-term memory is currently in the prompt. Retrieval is in the, you know, RAG, retrieval augmented generation. We do it over the whole web. We let you upload files now too and do it over a file. And then we have the smart personalization that actually is online learning. So as you say certain things, like it will remember them about you and then, you know, you can turn it off also and it's very transparent. Yeah. Turn the whole thing off or the automated smart learning about you if you don't want it. But yeah, I think that's sort of a simple, sort of pragmatic way of online learning. I think ultimately, you know, it'll be awesome to have AI systems get better and better of just adapting right away to user feedback, both in terms of, you know, thumbs up, thumbs down kinds of clicking, but also in conversation, be like, I didn't like that answer. And then updating the answer in a principled way for the future. I have so many more thoughts, but I'll like, I'd love to do a second one, but these are kind of crazy days now with the Apple announcement. And we just announced that Julien, the CTO of Hugging Face, also just became an angel investor and a lot of exciting stuff happening. So I, yeah.

Well, congratulations on the Apple thing and also on a new prominent angel investor and really some fantastic product progress. I definitely recommend everybody to try out Particularly genius mode and research mode, and I think if you do that, you will be coming back to u.com more and more often. So keep up the great work. For now, I will say Richard Socher, founder and CEO of u.com, thank you for being part of the cognitive revolution.

Thank you so much.

It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co. Or you can DM me on the social media platform of your choice.