The perplexities of information retrieval

Foreign.

Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI related tech is changing the world, this is the show for you. Thank you to our partners at Fly IO, the home of changelog.com Fly transforms containers into micro VMs that run on their hardware in 30 plus regions on six continents so you can launch your app near your users. Learn more@fly IO.

What's up friends? Do you remember when Chat GPT launched? I do. It felt like the LLM was this magical tool out of the box. However, the more you use it, the more you realize that's just not the case. The technology is brilliant, don't get me wrong, but it's prone to issues like hallucination on its own. But there's hope. There is still hope. Feed the LLM reliable current data, ground it in the right data in context. Then, and only then can it make the right connections and give the right answers. The team at Neo4J has been exploring how to get results by pairing LLMs with knowledge graphs and vector search. Check out their podcast episode about LLMs and knowledge graphs throughout 2023 at graph stuff.f. they share tips on retrieval methods, prompt engineering and so much more. Don't miss it. Find a link in our show notes. Yes, check it out. Graphstuff FM Episode 23.

Welcome to another episode of Practical AI. This is Daniel Whitenack. I am founder and CEO at Prediction Guard, where we're senior safeguarding private AI models and I'm joined as always by my co host Chris Benson, who is a principal AI Research engineer at Lockheed Martin. How you doing Chris?

Doing great today, Daniel. How's it going?

It's going great. I am sometimes perplexed throughout my workday, but generally the week has gone well and I'm really excited because hopefully our guest today will provide a lot of ways for us to navigate through the perplexity.

I hope he finds that punny.

Yes, yes, of course. We have with us today Dennis Yaritz, who is the CTO and co founder at Perplexity.

Welcome.

Thanks for inviting me and great to be on the show. Yeah, yeah.

Well we've been wanting to make this one happen for a good long while. Of course been following the things that Perplexity has been doing. Really impressive and inspiring work and yeah, just really excited to hear a little bit about that story and also this kind of space of answering, giving people knowledge with generative AI with language models search more generally. Maybe could you set us up by maybe talking through? Of course people have followed everyone's kind of followed this surge of generative AI over the last couple years, but there's been kind of a segment of that that has focused a lot on kind of answering questions, discovering knowledge, curiosity, exploring topics, an intersection with search more generally, I guess. So could you kind of help us understand maybe that journey, how that's kind of developed and how, how also you and the co founders of Perplexity kind of came upon your approach to that?

Yeah, that's, that's definitely been like very fascinating. Almost two years, I guess, I think general web search, right. So what do you want to ultimately get is an answer to your question, right? So the current iteration that we had with Google and all the other classical search engines is an approximation to get to this point, right? So you have your question, you ask it as a form of query and then you get a bunch of documents that are very relevant, but you have to still do ultimate. You have to do like additional work to get to the bottom of it, right? So you have to scan through the documents, you have to for yourself understand what is, is this a true answer? If it's a not true answer? Kind of like have to trust the information. And as you can imagine, it's a lot of work, especially if you trying to search for something very complicated, maybe things that are not so obvious. And wouldn't it be nice to kind of like avoid that step, right, where you just like answer a question and ask the question and you get an answer right away. So that's kind of like ultimate destination we're trying to get. Obviously there has been, it's been tough to get there over last decade or so. There's been like a lot of work, but it never quite worked. There is this like a much higher level of hallucinations, much higher level of maybe not perfect synthesis of the information. You kind of basically get like a Frankenstein. So instead of coherent, a nice, easily parsable readable answer, you get some just basically extracted pieces of the information and you just concatenated together. So not very pleasant. And it's funny that when we started, so one of our angel investor was Jeff Dean, requires no introduction and he was saying Google actually wanted to always build something like this, but it's just because they had like such high expectations for accuracy because like, you know, millions of billions of users using Google, right? And like if you like hallucinate like 1% of the time, you're going to get a lot of unhappy people. And so they were never able to because of the models were like not as strong as they are right now, they were never able to get to just like 99.9% of accuracy. Right. And that's why kind of like this work never panned out. But something great happened. 2022, right. So when we started our company we kind of both myself and Aravind, my co founder, we come from academics, right? We've been doing a lot of research on language modeling, reinforcement learning, stuff like that. And he was actually at OpenAI at that time. We've been very clearly following improvements of GPT models like GPT2 and then GPT3. That's where it's actually got very interesting. And it was kind of like became obvious and obvious there is going to be something there. Right. And this was the primarily motivation for us to start a company. We wanted to do build an answer engine from the get go. But it was kind of like very ambitious. I remember we would go to the investors and there would say like, oh, we're going to build a search engine. And they're like start looking at you like you're crazy. Which makes a lot of sense. They would say like, oh, there's Google already. And they had like a fair point. But we still kind of weren't very discouraged by that. We knew there is something there and we started prototyping. So the first version of Perplexity we actually created as a slack bot or like Discord bot where kind of like it was a very primitive combination of a search engine. Plus at that point it was like Davinci 2 models. It's still, you know, pre chatgpt and it's kind of like worked much better than I expected. It was like very quick. Like we put it up this demo in a couple of days and it was like, you can already see that in certain cases. It is very helpful because this was like very early company. We were trying to hire one of our first engineers and we didn't know how to organize the insurance for him or what it. And so we actually use this bot to ask those questions about insurance. Because if you go to Google and start asking the questions about insurance, you're going to get a lot of ads and you're just going to get very quickly disappointed.

I've appreciated that same thing in founding a company and using these models in that way. It's been useful.

Yeah, exactly. And then it was kind of useful. So we were kind of trying it out and playing with it. But then what really happened started happening is like, I think it was like a couple of weeks before ChatGPT OpenAI released this DaVinci3 model and I literally remember very clearly like changed the like literally the model name and start asking the same questions that I were asking before. And I could see right away just like so much better. It's just like so much better at understanding of your intent. So much better at like understanding where like what should it say, what it should not say, so much better like synthesizing the answer. And I was just like really like blown away. And you know, I was like, okay, so there is like definitely something there. And then obviously like chatgpt happens like few weeks after and we were like, okay, so we have to like our initial product was actually because as I mentioned, we came from academic background. Like our citations were like the core component because it was like very clear to us from the beginning. Like if you want to get an answer, you want to make sure it's accurate, you want to make sure you can verify it. And so the citation was like this sort of like first class citizen in our product. And then when ChatGPT came out it was like, okay. So one of the biggest point of feedback for them was okay, so I don't know if this is accurate information, if it's not, if it's hallucination, if it's not, how would I verify it? And that's why we decided, okay, so this seems like a good opportunity to release our product. We like literally in a matter of two days put up a website connected to our backend that we had and just obviously did not expect that it's going to be people going to use it and the usage is going to grow as much as possible. But coming back to your original question, I think what happened, it was just literally in a matter of days or month. I mean obviously it follows a lot of many years of research, but it was very clear step function in the quality of the generated answers. And you can like literally if you sort of like spend some time playing with it, you can clearly see that it now becomes very, very good. We also realized at that time, okay, so like model's only going to get better, things only going to get like faster and cheaper. So there is something, there is a lot of stuff to build here.

For those who are still kind of learning about your organization and what you're offering, could you step back for a second and like if you were talking to Jeff Dean or another investor, kind of giving them the elevator pitch about what you're doing specifically and you know how it's differentiated from the GPTs and stuff out there, how do you define that. How do you describe yourself in terms of the specific opportunity that you're pursuing.

The fastest way to get the most accurate answers to your questions? I think that that's essentially like answering. And we kind of like one of. One of the first people who coined this term.

And can you differentiate that from a search engine a little bit? There's.

Yeah. So basically in a search engine, right? So you get like all the way. As I mentioned before, in the search engine, you have to search first. You get the documents. Let's say you have like top 10 results and you have to scan through all of them and identify the information to get your answer. Here we kind of like, we do this step for you. So we take the first step of the retrieval of the relevant documents and then we synthesize them into a human readable, nicely formatted answer that you can then like, if needed, if you want to get more information, then you can click on the citations that are kind of like nicely attributed for each sentence and then you can learn more information. And so we kind of like wanted to do things like very simple. We were like, early on, I identified. So there is like two things we care about. This is accuracy and then speed. Because this is, you know, you want to get information fast. Google trained us all to get instant search results. I think that was very important. And early on it was kind of challenging because those models were very slow. Our infrastructure was not very advanced to do that. So I remember very first version you would have to wait for three seconds or five seconds to get an answer. It was very slow. But because it was such a better experience than just looking the search results at Google, so people still would use it. But our ultimate goal was like, okay, so can we be as fast as possible? The main differentiator is just we care a lot about quality, so we minimize the chance of things being inaccurate or like hallucinate and we want to do it as fast as possible. And so that's kind of like distinguish us from Google because Google doesn't, for example, generate the answers, even though more recently they started doing this, which is kind of like validated our idea. And ChatGPT probably primarily focuses on like different things, you know, but I guess also more recently they start doing a web search as well.

What's up, friends? I love backblaze and I'm happy to have them as a sponsor. Backblaze makes backing up and accessing your data astonishingly easy. This is a service I personally use. Go to backblaze.com PracticalAI you get unlimited cloud backups for Macs, PCs, businesses for just 99 a year. You can easily protect business data through a centrally managed admin. Protect all the data on your machines automatically, easily deploy across multiple workstations with various deployment options. You can add on enterprise control including granular access permissions, advanced single sign on group management controls and compliance support. They even offer multiple restore options including rapid recovery in the event of data loss or ransomware that sucks. You can access your backed up data from anywhere in the world using their web app or their iOS or Android app. You can even restore by mail. They'll give you a hard drive with all your data shipped to your door. You buy a hard drive restore, send the hard drive back within 30 days and get a full refund and get one year file retention and version history over 55 billion with A B files restored for customers so far, visit backblaze.com practically I so they know where you came from and continue to support the show. This is a service obviously recommended by me, but also by New York Times Inc. Magazine, Macworld, PD, PC World, LifeWire, Wired, Tom's Guide, 9to5, Mac, and just so many more. You receive a fully featured no risk trial@backblaze.com PracticalAI Again, they're supporting the show. Go there, play with it. Start protecting yourself from potential bad times. Start today.

So you mentioned a few things there. You mentioned web search, you mentioned retrieval, you mentioned the large language model. So at least in kind of how I think about it and maybe others categorize it differently, there's one element of information that you can get from an LLM which is I'm going to put in a prompt and it's going to generate text and that may contain some facts or made up facts or some text, but it may be informational, right? So there's, there's some sort of knowledge that can be gained there. And then in a second case there's a way to retrieve on the fly external data. So that could be like from your company's documents, it could be from the web, whatever, and then inject that into prompts into the model which kind of grounds it and like you say would give you a citation. There's also, you know, more agentic approaches to this where maybe there's multiple ways that you could get knowledge, maybe doing a web search or searching a certain database that you have access to or any handful of sources that you've kind of curated as tools and you call them in a more automated way. So I'm wondering from your perspective, obviously you are part of a team that have been exploring this very deeply from very early on. Like you say when these models made that jump from your perspective now both in terms of what you're building with perplexity and also kind of how you generally see the ability to get accurate information from these models. How do you view those kind of categories in terms of their utility and what you're relying on for the accuracy element specifically?

The way I see it's going to unfold, I think the tools and sort of like agentic behaviors. I think that's where it's going. I think it's going to be the main bottleneck for this right now is just like models are not smart enough yet to take into account and sort of like reason all of the information that is out there. But I think it's going to be like a main component. So there's going to be like models, they're like very powerful already right now. They're like trained on a lot of data, like basically Internet. They have a lot of internal information, internal knowledge and they can do already very good job of synthesizing information. There's like certain things that they don't do well and perhaps they never going to do well. Those things like for example, like computation, like when you need to do maybe around some code or do some sophisticated math computations that the LLM architectural, I guess transformers is going to struggle at that also, you know, because those models are like so big sometimes it's going to be very expensive to them update very frequently. So you need a way to ingest something, some new information that just like happened and it's still not part of the LLM weights. You have a way to. And this is what we sort of like specialized also some private documents as you mentioned. Right. So sometimes if it's like enterprise, you know, you have a, some of the documents that obviously the model was not trained on and you kind of like maybe want to reason about those documents. Right. And there's like all kinds of other tools that you can. Yeah, eventually there is going to be like agents that is going to do like actions. Maybe you're going to like book a ticket, like buy a ticket or something like that and stuff like that. So I think definitely where it's going, it's going to be like a synergy. Everything's going to come together. We just need a top level powerful model that's going to kind of reason behind multiple things and we'll have to need to have like long context windows, maybe like some memory as well and then you know, just like utilize those tools as much as possible. How we at perplexity thinking about it we are like okay so and there's like multiple ways and multiple sort of like applications, multiple use cases. Of this general approach we are primarily focusing on the information retrieval part of it. So kind of like initially web is the main component because there's lots of things to extract from vep. But also we're thinking about how to integrate different data sources, maybe some of the different databases, maybe there's some more complicated or more specialized documents, like maybe there's PDFs or something like that or some financial data, things like that. The other aspect that I'm very excited about and we're working on is kind of do you know right now we can answer, we can do like a great job of answering like complicated questions that you can get answers on Google but like still not something where you can ask expert and then can get an answer right? Like what if I have like a question that is just like requires like multi step of reasoning. So it requires like okay like searching web multiple times, analyzing information at during the issue retrieval and then maybe refining the search. So kind of like those things that maybe going to take some time but if you do it yourself you would spend let's say like an hour. So here maybe the system would spend like 30 seconds and it's going to save you a bunch of time. So kind of like those use cases that can answer very complicated questions and we believe that there is a world for those type of questions. Technology like this is going to be.

Useful as people are using an answer engine like yours more and more often going forward. And you kind of alluded a moment ago to the fact that LLMs are not the be all. There are things they don't do well like mathematics and such and a variety of other things I'm sure that we could all throw out there. But they're really powerful at what they do. But clearly there is a place and a need for both the LLMs. You know these largest models that get all the, they kind of suck up all the air in the news cycles as well as many smaller models that specialized, you know, a mathematics model that you know that you plug in. As we're looking at trying to use answer engines to retrieve information and that information is increasingly multimodal in nature. In terms of what you're asking, how does the architectures of those come together? This is a space. It's not the first time we've asked it here but it's evolving so rapidly. We're no longer hosting a model, you're now hosting a a whole collection. And they may be mixed with models that you're API ing out to and such. How does that look to you as you're building this company at this point.

It'S going to be always like trade off between if you have one powerful model. I mean yes, it can do lots of general things super well, but it's going to be slower, it's going to be more expensive. One of our key principle is just we want to do things very fast so we can get answers as fast as possible. That means you have to design your system like orchestration system in a way where certain things will have to rely on customized models, something that is much smaller, much faster. But it's like a specialist model. So it's not a model that knows how to do everything, but it knows how to do one task. And basically the challenge here is how do you balance between this general models and this specialist models. And I think we've been doing this from the very beginning. So when you send the request to Perplexity, it's just not one model. There is, I don't know, at least 10 different models trying to do lots of things with your request. It's like all kinds of ranking models, bunch of embeddings, all different classifiers and stuff like that. The other trade off here is just like with general model. Let's see. One of the big, and I think it was actually very critical component of why company like Perplexity in the first place became possible is it's like the speed of iteration. You literally can change the prompt and you can just get a new product in a matter of hours. Imagine a couple of years ago if you wanted to build something like Perplex or whatever other Genai product, you have to collect data first, have to train the model, launch the product, see if this product makes sense, does it have market feed or not? If it has market fit, then you start collecting data and then you just keep improving. So what being possible with GPT models like an API is just like this kind of flipped over. So you very quickly can build a product, see if there is any signs of life for this product and then you start collecting data, which is I think honestly the most important thing. And once you collect data, you can distill it, you can build many other smaller models and kind of optimize the experience so you can make the models faster, you can specialize them. I think this is the key. I think this honestly was One of the most fundamental changes in the development and we kind of took advantage of this thing early on and then still using, but it's still tricky. Kind of like imagine every time you have this specialized models and if you have tons of them, you have to then treat each model like care about each model separately. So if you want to retrain this model so you have to spend some time on it, so you have to evaluate it, so it becomes more difficult to manage. But on the other hand, you have some great benefits. So the key is just like you don't want to go too overboard with those models, like if everything is on customized models, but also you also clearly don't want to have just one model that's going to do everything. So.

Yeah, I have a question maybe related to that, which I think is a pain point a lot of people are feeling and I'm, I'm guessing your teams have felt, which you even mentioned this. Like you, you can have make a small change in your prompt or create a new prompt and all of a sudden it's almost like you have a new product which is sort of like amazing in one sense and really frustrating in another sense because as you were just alluding to, it's like, oh, maybe I have these like 17 different things chained together and they all have prompts that I've worked really hard on. And then like Tomorrow, you know, llama 17 comes out or something and now like it behaves. It has a different character of behavior than the previous model that I was using. I'd love to use it. But now I have all of this. It's almost like AI model debt that I've, that I've got in my system. Do you have any perspective on that or any, anything that sort of has happened in, in your experience in this regard?

Yeah, yeah. This has clearly been a thing and it's been happening quite often. And if I would guess that's going to continue happening. One thing we like realized early on is, okay, so this is going to be the case. There is going to be like there is not going to be like one model that rules them all. I mean, even though like for some time it was like GPT4, but like now we can see the, there's particular like anthropic Gemini, like Llama. There is going to be a future where there is several frontier models, right? Because of that we decided, okay, so let's design our infrastructure and our system in such a way that it's going to be model agnostic. And then that means, okay, so there's like a ways where you can evaluate each component independently. There is a way where you can quickly change things up to adapt for like a new model and stuff like that. And that's it took some time to get there but I feel like it was like very correct decision for us. And then so for example one of the advantage we have over let's say like things like ChatGPT or like Lord or like basically one model providers companies is just like we can seamlessly integrate many different models and like our users can okay decide they want to use this model or they want to do that more later on as we progress. I think we can even decide based on the complexity of the query or the type of the query we can route to a particular model that does the better job for those type of queries and minimize maybe some of the queries are super simple. You don't need to run a very large model for those to answer. So then you can optimize speed and things like that. So I feel like you have to just make a system in a way where it's like agnostic to the model.

What's up friends? Have you ever had trouble accessing that favorite sporting event or that awesome show or that film even because it's not in your region? Well, our friends at NordVPN can help you switch to a virtual location to a country where it is available, unlocking a world of entertainment. Plus it's not just about streaming, it's your go to for online security. Protect your bank details, your passwords and your entire online identity if you're traveling. They can shield your data on public WI fi, keeping you safe no matter where you're at. They also have this cool feature called threat protection that means you can say goodbye to viruses, malware and phishing sites because NordVPN will protect you no matter where you're at. One of the fastest VPNs globally. They ensure no buffering while streaming and no stops for your ISP from bandwidth throttling. It might sound costly, but think again. NORDVPN costs less than a cup of coffee a month and you can use one account on up to six six devices. To get the best discount off your NORDVPN plan, go to NordVPN.com PracticalAI Our link will also give you four extra months on the two year plan. There's no risk with our 30 day money back guarantee. Once again, go to NordVPN.comProficalAI.

So to follow up on what we were talking about before the break There I know that you were talking about really building around model agnosticism to be able to handle that. I couldn't help but wonder, occasionally as we get a new model out, it breaks new ground on modality being added in a whole new approach, that kind of thing. And so how do you, as a business builder who is having to try to accommodate all these different models when you have one that jumps out and has a completely new thing added in that was unexpected prior to the announcement, how do you in the organization, how do you guys kind of pivot to accommodate that and keep the agnosticism and yet provide that extra functionality that's now available? How do you all tackle that problem?

The most important thing to be is to not be caught off guard and try to anticipate what's going to happen. I think that's very important. And it's kind of like for the most part, I wouldn't say it was too hard to predict what's going to happen, but after that I think it's primarily product decision. Do you. Does this new feature benefit your product or not? Do we want to build something in product or not? So for example, one great example was image uploaded, kind of like multimodality, right? So we knew it was like last year, we knew for sure that this is going to happen at some point because there was already some smaller models that kind of like supporting that. We knew that much better model is going to come out. And then because of that, we kind of like in advance start, first of all understood, okay, so this is going to be important for our project so we can support this and this and that, those use cases. And we decided to build infrastructure in advance and kind of like anticipated. Obviously we didn't fully predict how exactly it's going to operate, but it was very close. So it required literally a couple of days to adjust. And then we were like one of the very quickly can release it as a product. So it's having the system that kind of general enough that can support those new modalities. It's very important, get some great deal of anticipation. But also sometimes certain features that maybe come out, maybe that are not useful for your product, you also don't want to put everything into, okay, so this is a cool feature. Let me just add it to your products. That's always not a great idea in general, only things that make sense. And if those things makes sense for your product, you likely already thought about how would you implement them in advance. It just makes it a bit easier.

While you were talking, I Was thinking there's one axis that you have to navigate here around model releases and functionality and modalities, all of that stuff. There's another maybe around UI and user experience. Sort of multiple people making the comment, oh well the chat interface came out with ChatGPT, so everyone's sort of like focused around the chat interface. Is that the best way to utilize this sort of technology in the long run? There's probably a lot of exploration that's still open around UI and user experience with this type of technology. And certainly chat is relevant and you know, we're using it a lot already. I'm wondering from your perspective, especially as we see this functionality maybe more embedded in the physical world around that, whether that be like in our glasses with, with meta glasses or in like kiosks in airports or whatever, whatever those things are. What is your perspective on how important it is to explore new types of UI or user experience with this technology?

I believe like, and I mean we kind of like, we're like very confident early on is just like chat interfaces is a temporary thing, it's just too limiting. It has a lot of constraints and that's why we didn't follow the usual route. It was all of the chatbots, they were literally copied ChatGPT and kind of put chat interviews. We kind of thought a little bit more about this and we designed it. I feel like ultimately right now we're still in this early stage where people care about the model itself. So the model is the thing. But as this thing get more advanced, like as more people start using Genai products, I feel like the main thing is going to be product itself. Like what kind of things can product do? Do you do it better than this? Do you have the best ui? Do you have the best ux? And that's why we kind of like early on was like been thinking about those things and we kind of design our product in a way that is the mostly suitable for the things that we wanted to do, right? So if it's like search, we knew that chat doesn't make sense for search. It's just like that's not how people search for information. That was a very big factor I think in our success. The other thing, I guess also even last year we started prototyping and experimenting with this concept of generative ui. So something where LLM can guide what kind of UI elements you can generate. And then sometimes one of the things in chat interface, if you want to ask a follow up question, sometimes it doesn't make sense to, you know, ask it as a sentence, right? So maybe you want to like, show like a checkbox, like a button or whatever. Like if there's like, it's just like especially on the mobile, like everybody uses phones, right? It's just like not very convenient to type, especially you if you're on the run. So you'd rather like press a button. That's why, like, maybe speech and I guess, like voice technology is going to be one of the interesting modalities for sure for an interesting interface because it's kind of. It has a lot of advantages. Obviously it has lots of disadvantages too, but definitely going to be interesting. And I think going forward as we go towards agentic behaviors and more things going to become possible. I think it's definitely not going to be like chat interface. It has to be something else.

I'm really fascinated by this topic and I think something that both Daniel and I have some passion for just in, you know, daily use and stuff. I'm wondering, with you thinking about that kind of productization, do you think that's something that Perplexity engages in in a direct way or supports other companies through that? And then there are so many times in the course of a typical day where I'm wishing I had other ways of interfacing with these capabilities that we're talking about. And I'll give you just a. Just a tried example. I will take my dog for a walk at a nearby park every day. And that's my thinking time. That's where I'm really trying to be creative. And I'm walking and I have to keep walking. You know, I don't want to stand right now. I have to. I stop and I pull out my phone and it's frustrating and people are going by me and I'm trying to hold my dog and. But I want this experience. It might be while driving, might be while walking the dog where this seamless way of utilizing these capabilities that we've grown accustomed to come about. Are you, A, how do you see yourselves being part of that next journey on the interface side? And B, do you have any ideas on how to get there? It's just. That's the next thing I want definitely.

Looking into this, I think. And we consider like, multiple options. Either, like, certain things I think we can do ourselves for certain other, like, things we probably have to work with some. Some other, like, partners. Because, I mean. Yeah, but I truly share your kind of, like, experience and I think it's. Yeah, it's just like if you sit in front of a computer I think they are the by far the best interface is keyboard. I don't think you can do better than that. But yeah, if you, you know, occupied with something else. If you're driving a car, maybe you're walking. There has to be something else. Like even like phone is, you know, it's okay. Maybe like even like taking notes. Maybe you like say a command and something like that but. And you get voice back. That's already like something but it misses like visual information. So you kind of like want to add that so that that means you probably have to have some sort of like glasses on. I think it will definitely happen and we will try for sure. We spend a lot of time this year improving our mobile app to do voice to voice and we can invest it a lot into voice generation. So for example, you can ask various questions. If you need something quick, you walk in you. I want to have a quick lookup of information. So we support that. There is something for example for if you drive a car, we have this. You can read up the stories or discover from perplexity. That's also like AI generated voice. So it's kind of like you listen to a podcast. So that's super important. Yeah, I think that the next step is vision and sort of like how do you get there?

Maybe one challenge that I've been thinking about this entire time while we've been talking relates to definitely a danger that I think people have identified as related to this new technology, which is you've already mentioned that you're kind of doing web retrieval or retrieval from certain sources as kind of a primary way of grounding answers, of ensuring accuracy of citations. But I know a lot of people are concerned about and thinking about this sort of idea of data poisoning where we're putting out actually a lot of generated content on the web.

Right.

And that proportion of human and generated content is going to change over time. Which means even for retrieval systems, especially if you're doing web related searches, there's a potential that you could retrieve generated content itself and get in this kind of weird loop. Of course there's a separate problem for like the models and how they're training. And I mean this affects a lot of different areas. But I know probably you've been doing a lot of thinking about this because it's kind of key to how you operate as a system. Any perspective on that that you feel like people should keep in mind kind of moving forward or things that you're thinking about in terms of whether that's data curation or Validation or. I know a lot of people are talking about detecting generated content and that's maybe hit or miss. So there's all of this kind of connected stuff, but generally around this idea of data poisoning or generated content on the web. Any thoughts?

To me, it's kind of like a technological problem. I feel like it's very reminiscent of spam classifiers. Right. Like just like whole another level or like, but let's say like 20 years ago when you like receive emails. Yeah, you would receive a lot of spam. And then eventually people develop technology that can detect it. I feel like something like this will happen. So it's always going to be like a constant battle between generators and discriminators. So at some point generators maybe is going to be better.

It's like fighting malware.

Yeah, exactly. It's the same concept. It's definitely going to be an issue for sure. But my hope is that and my belief that is the good guys, the good generator is going to be just like it's from machine learning fundamentals discrimination is much easier problem than generation. It's like much easier to tell what is good, what is not, than generated. And usually seems like we've been more successful in detecting this stuff and I don't see any reasons why it's not going to continue.

So this has been really fascinating as we wind up here and have you here for one more question. As we've talked about the future and have been kind of talking about what our expectations might be and how those might be fulfilled, are there any other areas that we haven't addressed that you're interested in and possibly as part of that, any way of kind of summarizing your own vision without it being just answering questions that Daniel and I have thrown at you, but your own vision for what the future looks like to you and what you want it to be and what perplexity is trying to realize it as to kind of paint a picture of what we might see over whatever time frame you want to address.

Yeah, yeah. So I'm very excited basically to get to a point where I have any question, any problem I want to have, and then just go to Perplexity and get an answer or suggestion or even perform an action for this. I already mentioned this thing where kind of implementation, increasing the quality or like the complexity of the type of questions you can ask and then making sure that the system will be able to handle those. I think this is definitely going to be the future and think we work hard on that. It kind of like opens up Another dimension, you know, you can just, you can ask like lots of simpler questions or you can just like one. Have like one hard question. So it's kind of a, you know, complexity versus quality. And I think ultimately when you get information, you usually use it for some sort of like decision making, right? And so if we can then take this information that we've retrieved for you and synthesize in a form of answer, can we also then do like some decision making for you and can you perform actions on your behalf? So like, imagine, I don't know, like, you imagine like you're researching something, maybe you want to like research to buy like best running shoes, right? So it's pretty painful procedure right now because there's like so much stuff in Internet, like you don't know what you trust. So that's why, like, you know, if you identify usually like running shoes, you stick with them forever because it's just you trust. But like, things like that. So imagine if somebody will do like this like research for you, like really nails it down like weights all the pros and cons, and then suggest, okay, do you want to, this is the possible like variance. Do you want to buy this thing stuff? And then you say yes. And then it goes like automatically buys it for you. And then like two days later it's delivered and you, you only need to type like questions once. You don't have to push your credit card in stuff like that. And there's like tons of examples. So kind of, I would say the future is like, first you would have to make sure that you can nail sort of like the information retrieval part and kind of like generating the most useful information. The next step would be like decision making. So like make decision based on this information and then third step to actions. So that's, that's where I think I would be excited to get to that point.

That's great. Yeah. Thank you for being willing to take time to dig into a number of topics that I know, I know our listeners are exploring themselves and also interested in and certainly Perplexity has been leading in a lot of these areas. So keep up the good work, work and thank you for, for the work and the perspective and taking time to talk.

Appreciate it. Yeah, thanks for the great questions and thanks for having me.

All right, that is Practical AI for this week. Subscribe now if you haven't Already, head to PracticalAI FM for all the ways and join our free Slack team where you can hang out with Daniel, Chris and the entire Changelog community. Sign up today at PracticalAI FM Community. Thanks again to our partners at Fly IO, to our beat Freakin residents, Breakmaster Cylinder, and to you for listening. We appreciate you spending time with us. That's all for now. We'll talk to you next time.