EP 465: Deep Research Throwdown - Perplexity vs. Google vs. OpenAI

This is the Everyday AI show, the Everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

One of the biggest no brainer use cases for AI right now is Deep Research tools. But there's also a problem. As obvious as it is that we should be using these AI powered Deep Research tools, it is equally as confusing over which one to use because now Deep research is just a type of AI tool and there's already three of the biggest players in the large language model space that have a tool literally called Deep Research. So today we're going to be doing a somewhat deep dive on Deep Research in our Deep Research throwdown, looking at perplexity, Google and OpenAI's version of deep Research. All right, I'm excited to go deep. I hope you are too. What's going on y'? All? My name is Jordan Wilson and I'm the host of Everyday AI. That's what you're listening to. This is going to be, I think, one of those episodes. If you listen to it, if you watch it, you're going to save a lot of time. So if you're listening on the podcast, this might be one of those ones where you check out your show note, show notes and come watch the video. We're going to have that on our website. That is your everydayai.com. so on that website you can sign up for the free daily newsletter. We're going to be recapping today's episode and also in today's newsletter, keep you up to date with everything else that you need to be the smartest person in AI at your company. So thank you for listening. If you are on the podcast, please make sure to subscribe. Follow us. Leave us a rating, all that good stuff. We appreciate it. Live stream audience love to see you here. I'm gonna keep reminding you all. This is so relevant. Go on our website. Go listen to. Go listen to episode 443 to 447. It's all free. They're short episodes, about 25 minutes. These are our 2025 AI predictions and roadmap series. Trust me, you need to listen to them. All right, we're gonna have the AI news for today in the newsletter. If you normally tune in, this is going to be a longer show. I'm actually going to challenge myself to keep it concise because I want to give you all a pretty deep dive into the three different kind of Deep Research tools. So, you know, go check out our daily news on the newsletter. All right, let's get into it. Perplexity, Google, OpenAI. I'm curious, live stream audience, which one of these deep research tools are you all using? So I use them. All right, and we just did a show yesterday specifically on the Perplexity Deep Research, right? So theirs is the newest of the three challengers in this new deep research space. So I'm curious, live stream audience, let me know which one are you using? Are you using them all? Are you just using one, Are you using two, which one are you using? And which one are you liking? All right, so I'm going to say at the end which one I'm using the most and which one I'm liking the most. And I am think I'm going to use them all for different purposes. All right, so also if this is helpful at the end, hopefully I've earned your repost. So if you're listening on LinkedIn, please, this is helpful, repost this and I'm going to send you our guide that is 10 business use cases for deep research. All right, so like I said, there's three big players right now in the deep research game and I wouldn't be surprised if by next week maybe there's another. And I know it's confusing, they're all using the same name, Deep Research. So Perplexity is the newest and their Deep Research, right, just released within the last couple of days, February, in January, OpenAI released their deep Research. The month before that, in December, Google released there. So Google was the first company to have a deep research product to market. Although previously I talked about this yesterday, it was actually OpenAI had been tied to this deep research tool in reporting from, I believe May of last year. So a lot of people are like, oh, you know, Google was first. Everyone else is copying. I don't know about that. I'm not saying OpenAI was first, right? But OpenAI had been tied to a deep research tool as of last year. It doesn't matter who was first. I think it matters who is best. Not just who is best, but who is best for your company and your specific use case. All right, so originally when I was planning on this, planning out this episode, I said I'm going to do a bunch of live demos, right? But then I thought our biggest audience is actually the podcast audience. And I thought I would be all over the freaking place if I do multiple. I was going to do a series of three to five deep researches across these three different platforms, score them up, add them up. So I actually kind of Just did that offline right before this, so I can tell you kind of the results or just, you know, to put everything kind of in a nice chart. And I thought that might be a little more helpful for you all because I know that this is important. Right. Especially now, since I think this is one of the best use cases for getting a positive ROI on Gen AI. Right. I've said since the very beginning. And this is why I've always been, you know, know, pretty big on Chat GBT and, you know, kind of lukewarm on some of the others. Right. I think Google has gotten so much better. But the ability to query the Internet is so important. All right. If you're using a large language model, number one, you should never copy and paste anything. Number two, human in the loop is important. And I think the human in the loop responsibility is changing. And I think we're going to see that specifically here with Deep research. But the ability to be connected to the Internet is so important because a lot of the information that goes into the training data of these large language models can sometimes be very old. Right. So that's why I think OpenAI and ChatGPT have been a leader in this space, because I think they've provided the best ability. Right? So even before they had Chat GPT Search, which I believe they released in October, they had Browse with Bing, which, you know, when it first came out was actually kind of bad and there was better GPTs that connected to the Internet. But Browse with Bing actually became amazing before they replaced it, you know, at least officially with Chat GPT Search. All right, so let's, let's just quickly talk about what the heck is deep research in general, because like I said, I just laid out the timeline. So we've had December, Google, January, OpenAI, February, perplexity. So what the heck is Deep, a deep research tool? Well, it does the most basic human research work and they work in different ways, but essentially these three tools, one thing that they share in common, they go to multiple websites and they try to take your input and they try to answer it in the best way possible. Oftentimes it's in the form of a long report. The qualities, the quality of these reports vary. The process in which these three different tools actually do the deep research is much different. Right? But this is, this is something, and this is why I think it's going to change a lot of industries, right? Maybe not two of these tools, but I think one of these tools is going to be incredibly disruptive. Presumably, they're all Going to continue to get some, some good updates. But this is what so many of us humans do on a day to day basis. If you are a knowledge worker, right, Especially here in the US that means you get paid to sit in front of a computer and use your brain and create value for a company. So much of what you're probably doing is you're probably doing a lot of research, right? So what does that mean? Well, you're probably going to multiple websites, you're reading, you're synthesizing, sometimes you're summarizing and you are trying to extract information and use that information to create new business value or to better understand something. All right. And that's what these deep research tools do, right? Again, this is never something. Even though they create these really long reports and they're thorough, a lot of people just think, oh, now I can just, you know, copy and paste these and use these. That's not what I think they're necessarily good for. I think these are to help us research better, to help us come to a point where we can use our strategic minds as humans, our creative minds as human, as humans, a little faster. Right? So this is where like, and this hits me, right, because I used to, I worked at a non profit for 10 years. We essentially just became a marketing and activation agency for Nike and Jordan Brand. And I spent so much of my time, I would say I spent probably at least 60% of my time most of those years doing a combination of Internet research, grabbing information from, you know, dozens of websites and compiling it into reports, right. Whether these were reports to send to a partner like Nike and Jordan Brand, or maybe it was to, you know, to work on a, on an rfp, right. We had to gather all this information, this data, this statistical trends forecasting for a proposal, right? But this is something I've spent honestly thousands of hours of my professional career, right. I was previously a journalist, you know, even in the earlier days of the Internet, right. This is what I would do if I was working on a big story. I would do as much Internet research as I possibly could. And you know, let's, let's be real here. We have to talk about SEO because it's terrible. Using the Internet is terrible. You know, not only is the Internet made to distract you, but it is just distracting. And because of AI in large language models, the Internet has become more distracting. Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year. Or more, but can't really get traction to find ROI on gen AI. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use gen AI. So whether you're looking for chat GPT training for thousands, or just need help building your front end AI strategy, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everydayai.com partner to get in contact with our team or you can, you can just click on the partner section of our website. We'll help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on Geni. Right? When was the last time you went through an entire workday? Be honest with yourself. When was the last time you went through an entire workday? You, you researched a bunch of in depth topics, stayed on track, didn't go down any rabbit holes, didn't accidentally have 882 tabs open, right? It doesn't happen. The Internet is made to be distracting and it distracts us, right? And there's just a bunch of garbage out there now, right? Because it's a lot of SEO stuff, right? So a lot of these companies have figured something out and you know, they have a blog post from 10 years ago and they just update it once every year and it's actually might be bad and not very good information, right? But sometimes they trick the algorithm, right? You trick a Google algorithm into thinking that you're an authoritative source on something when it's actually you're not. And that's what a lot of times as a human, when you're researching something, that's what you're trying to do. You're trying to look at all these web pages and you're like, all right, what's, what's real, what's not, what's good, what's garbage, right? And that's kind of the initial or the 10% promise of these tools. So who should use them? Well, who shouldn't, right? Let's be honest. Like I said, if you are an everyday professional, right, if you are a knowledge worker, especially here in the US where, you know, it's the Wild west and there's no real rules on AI, you need to be using these tools whether it's, it's it's perplexity. Deep Research, Google Deep Research, OpenAI Deep Research, you need to be using them. So a lot of people are thinking, oh, these are great for students. It is, it's great for students to learn things. It's great for, for researchers, it's great. If you are working in a highly technical field as well, that would generally require you to look at many, many, many long documents, right? But if I'm being honest, when I think who should use Deep Research, I see more use cases for who should use it versus who shouldn't. I honestly don't know anyone that sits in front of a computer all day uses the Internet that shouldn't be using one of these three tools. They are, I think, one of the biggest places of impact that we'll see AI have in 2025. Personally, I think the management consultant industry, if they're not using like this as their starting point, they're screwed, right? Or they're just going to be over billing their clients and those clients are going to end up going with someone else that, you know, is essentially honest, right? Hey, we used to bill you, you know, for 50 hours of research. Now we can do it in five, right? Or, or two, right? So I, I do think this is really going to change, especially some of those high priced services that require, that require just such an, like, such a high level understanding to, to be able to absorb and make use and synthesize, you know, hundreds or thousands of pages of data. That's what these tools do. So what types of tasks are they best for? So number one, learning new topics. It's a great thing to do, right? Instead of, like I said, getting lost. Because not only do you have to go to all of those web pages, right? 5, 10, 20, 30 web pages to learn this new topic, you also have to be able to synthesize it. You have to be able to connect the dots between the different points and these different points might happen across different points on your journey of trying to learn it, right? So even, even the general way of doing this, using a deep research tool is so much better because not only does it, you know, give you points A, B and C, but it builds the bridge and it segues and it creates a narrative that helps explain those missing points, right? Because otherwise, again, that's just something. You just have to hope that there's a resource out there, one all encompassing resource that does it all. But there's usually not, right? Because everyone's learning experiences is different. Everyone needs something different out of their research, right? So learning New topics, number one, great use case market analysis, obviously doing competitive research. Competitive analysis, right. That's huge. Keeping up with recent news and trends and how that impacts your type of work, problem solving and discovery. I mean, think, think like any, like I always, like, I always think it's like, okay, if, if you're on an airplane, right? And, and maybe there's not going to be good WI fi. You have a type of work that you can do, right? Deep writing, you know, deep reading. Right. And then it's like, oh, okay, well, hey, when I get back to WI fi, need to go research these things, right? There's almost that, that, that list of things that you're like, oh man, I gotta go do that. That's what these Deep Research tools are great at. All right, so podcast audience, this is one of those. You might just want to just check out the, the little screen that I have here. All right, so just, this is from. I've, I've been using, right, all these tools since the day they came out. Literally since the day they came out. I've been using Google Deep Research, Perplexity Deep Research and OpenAI Deep Research. So Perplexity is the newest one, but I've, I've used it for many hours already, even though it's, it's only been out a couple of days. Same thing. OpenAI deep research, I use this extensively. Extensively. Luckily, I haven't hit my 100amonth limit, although I think I'm almost there in, in Google Deep Research, I'm using it non stop. So these are all three tools that I use a lot. And I've been kind of keeping track on how they all work and kind of how they're built, because that's important. So now let's start the differentiation process across these three different tools. All right, so I have them tiered off. So free users, that's number one. So right now, open AI Nope, no free users. Although in the future you'll get two queries a month. A month. Yikes. All right, anyways, yeah, right now, Nothing free for OpenAI deep research or Google Deep Research. But with Perplexity Deep Research free users have five queries a day. So automatically there's a big plus in the column for perplexity. So here, let's look at our normal paid plan. We what you have to pay to use these different Deep research tools. So right now, the only tier it's available on for OpenAI deep research is the Pro plan. That's $200 a month for 100 deep research queries all right, again, OpenAI did say that they'll be rolling this out to the normal ChatGPT plus plan and I believe that will be 10 queries a month. But that's not out yet. Right now only on that $200 a month pro plan. All right, so for perplexity, if you're a paid user, it's unlimited usage. I think they do a cap at like 500 a day, right? But no one's, no one's getting that. So it's essentially unlimited usage if you are on a paid perplexity account. Same thing for Google Deep Research, it's essentially unlimited, although you can only run two queries at a time. I haven't run into any maximum number on the others. All right, model used, this is important. All right, so I'm actually going to layer these two together, model use and reasoning. Because OpenAI uses the most powerful model on this list, which is OpenAI's O3. So from what OpenAI said, this is not the same O3 that we have access to in chat GBT. This is a fine tuned version of the full version of O3. All right, so right now, even if you're on the Chat GPT Pro plan, you only have access right now to O3 mini and O3 mini high. So this is a fine tuned version of the full version of O3 which is according to all benchmarks, it's scary good. Right? So it's a reasoning model. Perplexity, Deep Research, they have a reasoning model as well that powers this. However, they did not disclose what the model is, so hopefully they do pretty soon. But there's no information on what, what reasoning model they're actually using. Right? Are they using R1 from Deep Seek? Are they using O3 mini? Are they using a reasoning version of their sonar? I don't know. No one knows. They didn't, they didn't release it, so not a huge fan of that. And then with Google Deep Research, they're using Gemini 1.5 Pro, which is not a reasoning model. So again, we talk about this on the show a lot right now there's kind of two different families of models, right? And they actually might all be merging in the future. So we'll have a story on that sometime soon. But essentially you have your quote unquote old school LLM transformer models and then you have your reasoner model. So your reasoning models, they kind of show this step by step logic, right? It kind of reasons under the hood. So that's why, you know, I think we'll See some examples here. It's two different approaches, right? Because I think just because Google Deep Research isn't powered by a reasoning model doesn't make it less powerful necessarily for some people. Depending on your use case because they take a different approach which I kind of like. All right, so file uploads OpenAI yes. Perplexity, yes. Google Deep Research, no. So why does that matter? Well, maybe you have a huge report, right? And you want to do some research, right? Maybe this is a report from fourth quarter of 2024. Maybe you need to do a bunch of research and find all the new laws, regulations, competitors. Everything they're doing related to this big report. Well, you can just upload it, say, hey, this is the report from quarter four, 2024. You know, do all research on any new laws, regulations, competitor movement in any of, in anything that this report touches as an example. Right? Boom, done. So big advantage there for Open AI and Perplexity. No go right now for Google Deep Research again. That's all. This is all as of today, right? This could all change tomorrow, next week. Follow up questions. Okay, so this is the model asking you follow up questions. Not yet. You, you obviously once Deep Research is done, you can keep talking to that chat and get more out of it, refine it, etc, but this is the mod or the mode. Does the Deep Research mode ask you questions? And you might be saying like, why does that matter? Well, some of these take many minutes, right? So you might put in a prompt and let's, let's be honest, y'. All. The weakest link of artificial intelligence is the human in the loop. The human is always the one that doesn't do a good enough job. Because if you do a good enough job with AI, let's be honest, it's way smarter than any human out there right now. You can't argue with benchmarks in science. You can't. All right, I don't care who you are, you're not smarter than in O3 deep research. You're not, period. Right. At least not across multiple categories. You're not. So it's important to have that opportunity to clarify and refine the query before one of these Deeper Deep Research models goes and spends 5, 10, 30 minutes on it. So that's extremely important. So OpenAI does ask a full follow up. Perplexity does absolutely nothing. And Google Deep Research essentially outlines its plan ahead of time and then has you. You approve the plan. Okay, so different. Let's talk about sources use. And this is where I think Google Deep Research goes in A little different path because this is more of the method, the, the research methodology. Right. So for the most part both Open AI and Perplexity use this reasoning and logic. So what that means is they might have a certain path set, right. But they might start out on that path. Let's just say, hey, we're going to go to the, you know, this is going to be a ten step research plan, let's just say. And then they get to step three and they see something that they didn't anticipate or they didn't know. So Open AI Deep Research and Perplexity Deep Research because they use reasoning step by step between all of these steps of research they can pivot, which is huge. That is huge. Right, but what that means, and that's, and that's what you get when it's a reasoning model. What that means is it can take fewer sources, right, because they're making hopefully educated decisions and they usually start first with a broad or high level source. Where Google Deep Research takes a completely different approach. Google Deep Research because it is not powered by a reasoning model, so presumably it doesn't cost them as much. And Google essentially serves you up cached versions of these pages. So for OpenAI deep research, I've seen it usually go between 15 to 30 sources. Perplexity deep research varies wildly. I'd say like 20 to 200 and then Google deep research I'll say 50 to 500. Right. I've had Google deep research go to like 1300 before, but it uses cached pages. So as far as I know in Google friends I know there's some of you out there listening from everything I understand and people I've talked to, it essentially is not visiting, quote unquote, visiting these web pages in real time per se. It's working with a cached. Right. Or you know, Google essentially crawls websites usually multiple times a day. So it essentially just synthesizes all of that information. Whereas OpenAI is, is actually visiting the pages. So it takes way longer. But Google's is still takes longer as well because when you look at speed, OpenAI deep research is 5 to 30 minutes, perplexity is 1 to 3. So perplexity is crazy fast. And it does go. So Perplexity in terms of like pages per minute, I would say it's usually the most. Right. But cool quality. We got to talk about that here in a second. Right. So sources use Google's the most time. OpenAI Deep Research is the slowest. Perplexity is the fastest. Google Deep Research is in the middle. Then what does it give you? Right, an output. So all the outputs that it's essentially a combination of, you know, intros, breaking it down by sections and essentially gives you a pretty long report. And I, I've seen in different, in different instances where, you know, it'll usually give you a comparison chart if you're asking for comparisons on something, which is really nice. So it'll put together a very comprehensive, helpful, insightful report for you to use and even give you information that you might not have even thought of. Right. Like breaking things down in a chart, chronologically, competitors, etc, so even it's going to usually go a little above and beyond what you think it may deliver. So the output, the number of words. So OpenAI about 1000 to 2000 words. Again, this varies. I've obviously had, you know, plenty that are less than a thousand, plenty that are more than 2,000. Right. But I'd say for the most part, you're in that 1000 to 2000 words perplexity. Pretty short. 400 to 800. Again, I've had plenty come in at more than 800, but seems most are in that range. And then Google Deep Research are generally the longest. I'd say about 1500 to 3000 words. But this is where it's important. All right, we have to talk about hallucinations because that's what this is ultimately all about. Because what we're doing when we are using a deep research tool, whether you are using it for personal purposes or you're using it for business use cases, you are essentially putting your trust in a deep research AI tool to go research things and tell you the truth. Right. And this is one of those instances where accuracy matters. This is one of those instances where it's better to be more factual than first. Right. I'm fine with using OpenAI deep research, even though it's the slowest because in my experience it is the most factual. All right, so these are my arbitrary kind of rankings and I have an actual example that I'm going to share with you all on the screen here. Hallucinations, I'll say OpenAI Deep Research. Very low. Very low. I don't see hallucinations a lot. All right, Google Deep Research. I'm saying low. Some, some hallucinations. Not a ton. It's very, very, very accurate. Perplexity is perplexing. It is concerning in some of my use cases, in some of my examples, how many hallucinations there are. Right. So I mean, you might look at it on paper and be like, oh, wow. Perplexity is free, it's fast, it still uses a bunch of sources. This is amazing. Well, you gotta know your stuff. I wouldn't be copying and pasting anything Perplexity Deep Research unless it's something you already have an intimate understanding of, it's concerning. I'm going to show you examples. So the example I'm going to show next and then we're going to jump in, we're going to begin a Deep research. Actually no, let's, let's just go ahead and do, do a Deep research query first. We'll start it and then we'll look at this example of some of the hallucinations or accuracy rate first. All right, so let's go ahead, we're going to do a simple one here. So I'm saying what are the latest LLM large language model developments this week and what will the impact be on knowledge workers in the US and how can businesses take advantage across. Okay, so again I am starting off in OpenAI. It's the one that takes the longest. I do have the PRO plan, So I have 100 queries of this. So to use Deep Research, it's a little confusing, right? Because you can technically access it in almost any mode or model. And so like right now I'm in chat GPT01Pro and I can click the Deep Research button. That does not mean I'm using 01 Pro. That just means I'm using Deep Research, which is the fine tuned version of O3. So it was a little confusing and I wish OpenAI kind of fixed this. All right, so anyways, I'm going to go ahead and start and what you're going to see here, if I share my screen correctly. There we go. All right, so I started it. Here's my query and it's asking me questions. It says could you specify which areas of LLM development you are most interested in? For example, are you looking for updates on new models, advancements and training techniques, ethical concerns, regulatory changes or applications in business? Also, do you want insights on specific industries regarding the impact of knowledge workers and business strategy? So I'm going to answer that. I'm going to say mainly interested in updates on new models and their applications in business. And then I'm going to say focus anything that impacts US knowledge workers. All right, so now once I click Enter, then Open AI, Deep Research is like, all right, cool. And then it's going to say starting research. And then there's this little box here. I'm going to give it a second to Propagate here and I'll be able to click on it if I want to. I can kind of watch it work step by step. All right, let's just go ahead. We're going to jump over into Perplexity Deep Research. Exact same thing. Make sure that you have the Deep Research. If you are using Perplexity, right? And it is free. So anyone can go run out there right now and try this. Make sure that you have the Deep Research mode on and I'm going to click the Enter button and you'll see right away it's not going to ask me anything. It's just, it's just preparing the research plan on its own. It's, it's running off by itself generally. So, okay. And I said this yesterday in my Perplexity Deep Research episode, which, you know, if you are interested in this one in particular, make sure to go, go watch or listen to that. One thing you need to be doing is reading the chain of thought. Not only is it pretty fascinating, but it's going to make you better at using these tools and it's going to ultimately give you better outputs. Also. Never, never, ever, ever do a Deep Research query once, period. Always do it once. First go look at the chain of thought, then look at the output and then improve it. All right, so as an example, here's the way that Perplexity is tackling it. It says I need to search the web to find the latest developments in large language models for February 2025. Good job, Perplexity. Getting that month in there is extremely important because what happens essentially these models, they break it down step by step and then they run different queries at different points in this step by step process. So Perplexity got this right. The first thing they're doing is they're just googling or they're searching LLM developments February 2025. Getting that date and year extremely important because I'm asking about the latest LLM developments. So I don't want anything from January. I don't want a 2024 year in review recap. So even though Perplexity didn't ask me a follow up question, it's at least starting down the right path. All right, so let's quickly jump back over into OpenAI deep research. And now you'll see, I, I can click on this sources thing and it's giving me a progress bar. All right, and then I can go see the activity and the sources. All right, so we're going to go visit that here in a second. So unfortunately I Have to have a different instance of Google Chrome Chrome open to share Gemini because it's not accessible via my workspace account. All right, so now, same thing, I'm asking the same thing and I am on the pro plan of, of Gemini Advance. All right, so this is Gemini 1.5 Pro with deep research. I'm asking the exact same prompt. What are the latest LLM developments this week and what will the impact on knowledge workers in the US be? And how can businesses take advantage of. So you'll see what Google does is it gave me a plan. So it said, you know, kind of gave me a, let's see, eight step plan. I'm not going to read all eight of these steps. But it's finding articles, then it's going to find research papers, then it's looking at, you know, how businesses are using large language models, etc. So these are kind of general. Right. Whereas Perplexity and OpenAI just went much more specifically. So it says it's giving me the option, then I can either edit the plan or start the research. So I'm going to click to start the research. All right, so now let's jump back because I am guessing that we are going to be done with perplexity, because perplexity is pretty fast. All right, I should probably go. Wait, I think I navigated away from that page. Give me a second. That's the bad part of doing these, doing these live. So unfortunately, I don't know what happened. It looks like it just timed out. Yeah, that's strange. So, yeah, the Perplexity one just timed out and started over in the middle of that. So I'm not sure what happened there. So we're going to restart this Perplexity one and hopefully this time it doesn't just randomly time out. So I don't know why I did that. It's the first time that's happened to me. But all right, it's going down the same path here. So now what? Now here's what we're going to do. We're going to hit pause on this. So we have the three different tools that take anywhere between, you know, two minutes to 30 minutes. This is a simpler query. So this, none of these should, I don't think, take more than like 10 minutes. And it looks like some of them are. It looks like OpenAI is almost done and it looks like Google Gemini is going to take a little bit longer here. So let's go back to this comparison chart that I had up here because I talked about the hallucination rate. And then I did an example in. I essentially asked all three of these to do a, a deep research on myself and Everyday AI. All right, so here's the actual, here's the actual prompt that I used. All right, let's see. Here it is. I said, tell me everything about Jordan Wilson who does Everyday AI from birth until today. Make it creepy in depth. Right? All right, so let's, let's look, let's look at how these different ones did because I, I literally wrote down the number of hallucinations that I found per platform. All right, OpenAI Deep Research 0. Google Deep Research 2, which I think is still pretty impressive. Perplexity got the majority of everything wrong. It just hallucinated off the chain. I have no clue why. But you can go back. I, I actually covered this yesterday in the Perplexity Deep Research, the dedicated episode on that. So if you want to see more information about where it went off the rails, you can go do that. Essentially, it brought in way too many sources. It brought in like 300 web pages and I would say more than half of them were irrelevant, were not relevant to either myself, Jordan Wilson or Everyday AI. It was a bunch of random Reddit threads and it just started pulling information. None of that was related. So some problems with perplexity. They did say that they were trying to improve the accuracy here. All right, but let's just go ahead and look at the, at these examples. Okay, so now, so live stream audience, I'm sharing the reports that each of these created with that simple prompt, right? I said, give me a report on Jordan Wilson from Everyday AI from beginning, the end, right? So this is the Google one, which did pretty, pretty well. So live stream audience, anything that I thought was, you know, impressive or, you know, noteworthy that in a good way, I highlighted in green. Anything that was a hallucination, I highlighted in red. So one hallucination that Google got, it said, I co founded Everyday AI with Chris Ball. No clue who Chris Ball is, but. And then it cited that as well to a random page that had nothing to do with Chris Ball. So one hallucination, not sure where that came from, but Google did a pretty good job. It got some of the details right. Some things that were only on like one place on the Internet. So it did a pretty good job of looking at all the nooks and crannies, you know, looked at every page on my old business website, my current business website, interviews I had done in the past. So did, did a pretty good job. Pulled out some, some key highlights There. There's also another hallucination here that says I was a college professor. So where that hallucination came from is it was from an interview I did with an actual college professor. And I'm guessing it got mixed up in the transcript. Each speaker was labeled. So for whatever reason it thought I was the college professor. I was not. Although, hey, coming. Coming to a university near you. Working on it. All right, going through the rest of the Google, it did a really good job. So Google Deep Research, aside from those two hallucinations, which, I mean hallucinations aren't good, it really did a good job. Formatted everything kind of throughout my career, right? Non profit work, strategic partnerships, starting my other company, Accelerant AI, then starting or accelerant agency, then starting everyday AI a little bit on different topics that I touch on in my viewpoints on these topics. So I mean, overall it did a pretty good job. It even found something that I forgot this was on the Internet. Like I'm an avid basketball player, having met a key mentor during a game. Yeah, that was like 20 years ago. So Google Deep Research did a really good job. So now let's look at. Here we go. So this is OpenAI's deep research. So the formatting didn't really work very well when I copy and pasted it because it inserted all of these citations in here in the middle. So not great from a copy and paste perspective. But from the actual report inside ChatGPT looks great. It did really, really well. Got my birth year, my birthplace, when I started working in, when I started delivering newspapers. Right. I started delivering newspapers as a 13 year old. I don't even know where it got that I forgot, but it was correct that I started working as a sports writer at my hometown daily paper by the time I was 17, got some information about the Pulitzer fellowship, you know, that I traveled abroad. None of the other ones got that. Yeah, so just a lot. It did a really good job and at pulling out specific details that I didn't see really very much in the other two reports. You know, something even found on the podcast that I think this is. I did another podcast interview. Someone interviewed me about the podcast and you know, I mentioned that, you know, community interaction is a big piece of, of everyday AI and this was the only deep research that talked about that. So I'm like, okay, that's. That's pretty unique because I only really shared that in one place. So yeah, I mean overall, I mean a lot of information here that I'm scrolling through on the screen, but you'll notice zero red, no hallucinations. I read this thing top to bottom twice. Nothing is incorrect. And it did a pretty freaky good job pulling out details from me all over the web. So even though it didn't go to, you know, 500 websites, I think this ended up only going to like 12. It did a good job of logically going step by step, starting broad. Right. Making those connections. It said, oh, Jordan, Everyday AI. Okay, let me go do a little bit more research on Jordan. Oh, he has another company. This is the right Jordan. Right. And then it's kind of working its way backwards, but you can go through and see step by step. It's journey, right? Which having a reasoning model and being able to look at it is extremely important. So perplexity. Yikes. Okay, so it did find a couple things that were good, right? It said, By 13, I built my first website, a GeoCities page, which is correct. But then it just, I mean, analyzing cereal box marketing tactics. Although that's something right up my alley. That's not correct. That's not what I, I did on my first website. It was actually. What was it? It was like covering the best like rap and hip hop music or something in the 1990s. It wasn't anything about marketing tactics for cereal boxes, which is honestly something I love. But that's not correct. But if you're looking at my screen right now, I'm not going to take the time and poo poo on Perplexity. But the majority of this report is red. It is concerning. It made up some of the craziest things ever. Right. So it's, it's, it's saying everyday AI 2021 through present. Which is wrong. It's 2023. It says content cadence, 5:30am Central Standard Time, live stream. Wrong. It says the 1734 phenomenon. Listeners noted eerie vocal modulation, shifts in episodes around the 17 minute mark, correlating with spikes in chat, GPT, API usage. Wilson dismisses this as audio compression artifacts. Right. It made up some of the weirdest things I had ever seen. Right. Look at this one. It says Chicago AI Week 2024. Yes, I presented there at Chicago AI Week. But then it says keynote speech included subliminal messaging in the form of steganographic QR codes hidden in slide decks. Scan codes led to an unlisted YouTube video of Wilson reading passages from Parable of the Sower backwards. Like Perplexity. Where did you come up with this? How like you, you saw the prompt that I gave everyone. Everyone else gave me Pretty much factual report. Perplexity went nuts. I don't understand it. All right. Yeah, I mean you can see everything's, everything's in red here. Everything's in red. Here we go. Projected 2026 Leaked internal roadmaps hint at everyday AI v point v4.0 A neural lace interface bypassing vocal cords to implant thoughts directly into listeners brokers areas. I don't, I don't know what this is y'. All. This, this is, this is wild. This is wild. I don't know what perplexity did. It hallucinated off the chain. Now I will say this, this is the one that it hallucinated in the most. I've actually run this two times since this one, but I had to share this one because this is the first round that I did and I was comparing it, first round, first take with everyone. It's actually gotten a little bit better. But even today when I run it, there's still a lot of hallucinations. So it's not giving me the same high hallucination rate when I ask about other things. And maybe it did it because I'm like, yo, take it to a creepy in depth level. So maybe it thought I'm just gonna make a bunch of stuff up because none of that stuff even existed on those 200 irrelevant sources either. It was just all a bunch of made up garbage. All right, so we're going to wrap this up quickly so I have that screen back on here. So I, I encourage you, if you're listening on the podcast, check your show notes. I, I made a nice little chart comparing everything because you need to be able to make the right decision for you. Right. And, and here's kind of what I boiled it down to. Which Deep Research service should I choose? So OpenAI deep research offers the lowest hallucination rates and it has a fine tuned O3 model, but it is slower and way more expensive. All right. Google Deep Research provides extensive sources. Way more sources, pretty low hallucination. But it does require a paid plan. So you know, OpenAI and Google right now, no freebies. Right. I think Google's like right there in the middle. Perplexity. There's tons of promise and perplexity. If they can get the hallucination thing in check, I think they got a banger. But the last year perplexity has been going in. In my use cases, Perplexity has been making stuff up at a wildly unacceptable rate. Wildly unacceptable, right. So it is the fastest, it has the best limits. It's free. Right. Five a Day is a great free plan. So. But the hallucinations, I can't honestly recommend anyone use it. Right? That's my experience. But what what I say do the exact same thing I just did, right? Have it run a creepily in depth biography of yourself or of your department if you work at a big company, right. Something that you know better than anyone, have it run a report on that, see if it's factually accurate. So maybe my experience is is more of a, you know, fringe use case but I followed it up. I'm not seeing accuracy doesn't look like is a strong suit right now of Perplexity Deep Research. OpenAI Deep Research crushes it. This is the one I, I use on a daily basis. I use Google deep research in OpenAI deep research non stop. I'm not going to use Perplexity Deep Research much right now because I can't personally trust the outputs because what it's shown me, it goes off the rails sometimes. Which you can't have, right. So I hope this was helpful. Doing a deep research throwdown in a deep dive. So I'm gonna probably follow up in this maybe in like six months because I assume they're not the only three players that are going to get into this, right? So I'm sure we're going to see something in the near future from Meta or Microsoft or maybe a startup out of nowhere that does this exact same thing, right? And I, I, I, I do think, you know, there's going to be open source versions of this, there's going to be improvements from these three big players. But I will say this, if you're still not sure on where to start with generative AI and you just get confused. Oh, all these large language models, all these features, all these things, right? Start here. People are always like, where do I start? And I'm always like, well, find the right use case. You know, depending on what type of work you do. You know, here's the different large no start here because every single person, pretty much if you get paid to sit in front of a computer and create business value, which means you're a knowledge worker, you use the Internet, which is, I'm guessing 95% of the people listening to this podcast, this is your best, one of your best use cases. You're spending way too much time researching the Internet. The Internet stinks. The Internet has become an eyesore the past two years. Mainly it's large language models fault, right? They probably didn't really have the legal access to go scrape all this information. Anything anyways, but that's going to be, you know, adjudicated in the courts in the years and decades to come. But you know, right now the Internet stinks. It is so hard to find good quality information at scale on the Internet. This is something, if you do it right, follow my advice, always follow it up a second time. Make sure you test it out. Keep human in the loop. But I think, I think if you do that, you're going to be blown away right away by the power of these deep research tools. All right, if this was helpful, I hope it was. Please repost this show on LinkedIn or Twitter. You know, tag me if you do that. We have a guide that we put together 10 business use cases for deep research. All right, so go click that repost button. If this was helpful, I hope it was. Thank you for listening. Please, if you haven't already, go to your everyday AI.com Sign up for the free daily newsletter. We're going to be recapping this show as well as keeping you up to date with everything else that you need to know. Thank you for tuning in. Hope to see you back tomorrow and every day for more Everyday AI. Thanks, y'. All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.