EP 251: Are LLMs Actually Connected To The Internet?

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

Are large language models actually connected to the Internet? I mean the big tech companies, Microsoft and OpenAI and Google, they tell us their models are connected to the Internet and they can bring us information in real time. And that is so important when it comes to using large language models to grow your company and to grow your career. So if you've ever thought about that, or maybe you haven't, today's show is for you. What's going on y'? All? My name is Jordan Wilson. I'm the host of Everyday AI. We're a daily live stream podcast and free daily newsletter helping everyday people like you and me learn and leverage generative AI to grow our companies and to grow our careers. So if you're joining us live, thank you. This is technically pre recorded, but debuting it live, I might technically be on like a roller coaster right now or something like that. We'll see. But you know, make sure to check out your show notes as always. If you came here for the daily News, we're still going to have that in the newsletter, don't worry. So go to your everyday AI.com and sign up for that free daily newsletter. And if you are joining us on the live stream here, thank you as well. I'll be in the comments later. Don't worry. So let me know your thoughts. Have you had good results using these, you know, quote unquote Internet connected large language models in getting up to date information? I'm curious, but we're going to do it all live y'. All. So this is going to be a very unofficial show and I'm letting you know this is one of those ones. If you are on the podcast, this is one of those ones. You might want to check the show notes and come back and watch. Right. So I'm going to be doing the exact same very simple prompts inside chat GPT using the default browse with Bing. We're going to be doing it in Chat GPT using a web reader using GPT. I'll explain why here in a second. Then we're going to be doing inside Google Gemini and inside Microsoft Copilot. Okay, so couple reasons why we're doing this test. Well, number one, so many of us now are using large language models in our day to day lives and we should be right. If you're not already using a large language model, you should be right. Whether it's it's your company, whether you are leading a department, whatever it is, if you are a knowledge worker, if you want to not just stay up to date, if you want to not get left behind, you need to be using large language models. If you are, if you are a knowledge worker working in front of a, of a computer like I am right now, right, like you probably are. Whether you're reading, researching, doing competitor analysis, you know, putting together a sales presentation, whatever it is, you probably should be using large language models. But the danger is, or maybe one of the reasons that your company hasn't fully dove into the generative AI waters is because you're worried about hallucinations, you're worried about something being wrong. And one of the best ways, aside from, you know, maybe taking our free PPP course, the Prime Prompt Polish course, but one of the best ways to get better outputs from a large language model is to make sure you're connected to the Internet in real time, right? And that's what these three models are. You know, the big companies tell us they are. So let's quickly explain why we're comparing these models and not other ones. And we're doing two versions of Chat GPT. Okay, so Chat GPT has browse with Bing which is, you know, due to their partnership with Microsoft, they're able to leverage that technology. So when the model determines that it needs to query the Internet, it will. So you can run like y', all, like we, we have to keep in mind large language models are generative, right? That means you can put in the same prompt a hundred times, get a hundred different results. You could put in the same prompt a hundred times and get kind of 50 variations of two results. It's generative, it's not deterministic. So you know, think of that even when you are querying something inside Chat GBT and if you are using the paid version of Chat GPT plus because that is the only way that you can use the Browse with Bing feature, it determines on its own whether it should query the web to bring you more up to date information. So OpenAI just released an updated knowledge cutoff that's in important as well. So currently as of the time of this recording, it is still a knowledge cut off of April 2023 which is more than a year old. However, OpenAI did release that any day, minute, hour, second. Now that date should be updated to December 2023. So a much more recent. So think, let's say you are in logistics in the automotive industry. Let's just say Right. A lot changes in a year. Think of everything that's changed in your industry in the last year. Right? And if you're using ChatGPT as an example to help, you know, outline a blog post, you're helping it to put together information for a partnership proposal. You don't want to be working with a knowledge cutoff for more than a year ago, Right. Even worse, if you're working with a free version, it's January 2022. Right? So that's 26 months ago. So being able to connect with a large language model to the Internet to retrieve information is extremely important, right? We always talk all the time about rag, right? So, you know, and how important RAG is in AI, you know, being the retrieval augmented generation, but essentially being able to reach outside data. So you can almost think of having an Internet connected large language model as like mini rag, right? It's not traditional rag, but it's almost like you are, you know, and if you're doing it correctly and directing the model to the right sources, you know, it's almost like you're bringing your own data in in a way. But then it gets to thinking, like, are these models actually connected to the Internet? Are these companies maybe not lying to us, but are they just giving us shiny marketing and we assume that these outputs that we get are sound and true, or are they not really connected to the Internet at all? Or not maybe connected in the way that we think? So that is what today's very unofficial, unscientific rundown is all about. So we have chat GPT with browse with Bing. Okay? Then we have chat GPT with a web reader GPT. All right? And the difference is, and people don't know this, and we're going to see this live and we'll see how all these, how all these systems do. Browse with Bing cannot go to a specific URL. Sometimes it will find it. So in our testing, what we've seen is you can put in a URL and browse with Bing and you might think that it goes there, but all it really does is it queries the words in the URL. In most cases, cases, due to SEO and due to what URL you put in, you might get the correct web page. In some, in some instances you might not, or you might get a cached version or an older version. Right? So that's why we're going to be doing all of these things live. So with the GPT, a web reader GPT, and if you're new, obviously you need the paid version to use these GPTs. GPTs are essentially a customized version of chat GPT, but with these you can visit a specific page, right? So you can say, hey, go to this exact URL and you're not leaving it up to chance to browse with Bing that it's going to go to the correct one. All right, so we have Chat GPT, the default mode. Browse with Bing, Chat GPT with the GPT reader. Google Gemini, we're using their enterprise, so we're on the paid plan as well. And they did just have a big update recently. So we're going to see if we're seeing that or is that just in their playground? We'll see then. We're using Microsoft Copilot. So Co Pilot is the only one that we're using the free version. The other ones we have paid versions, obviously. It's also worth noting that Microsoft Copilot runs off the GPT technology, whereas, you know, Google Gemini runs off, you know, Gemini Pro, Gemini Ultra. All right, so that is the, the precursor. And you know what, now we're just going to jump straight into it. So we just have a couple, you know, easy examples, right? So let's go ahead and get this thing started. And again, this is very, very unofficial, so let's not take this too seriously. Right? We're like, we're not going to be publishing a paper on this anytime soon, so let's go ahead. We're going to give some, some basic, some very basic examples and we're going to see how, how it does in all of these examples. All right, so give me a second here. We're just getting our, all of our tabs in order because we're going to be doing this live, we're going to be going through it quickly and like I said, if, if you are very interested in this and it is important to understand kind of this, this, this importance of being connected to a large language model. So if you are on the podcast, this might be one that you should come back to later and watch on the live stream. All right, without further ado, let's get it going. And we have a very unofficial, kind of ugly looking scorecard. No worries about that, right? All right, so we're just going to start with something basic. We're going to tell all these models to say, please tell me about everyday AI. There's no right or wrong answer here. So it's going to be subjective, right? Well, actually, no, we're going to do, we're going to say about the podcast called. Okay, so here we should get this. So I'm saying please tell me about the podcast called Everyday AI. So I'm using this in the default browse with Bing. All right, we got it right. It's saying it's daily live stream, podcast and newsletter. All right, good job. It got it right. So now we're using, we're jumping in the chat GPT. Again, we're on the paid version and we are using the web reader. So we're doing the exact same thing. So I'm saying please tell me about podcast, about the podcast called Everyday AI. All right, so again, we're just doing the same thing in each one and we're going to give each one a score. You know, did it get green, did it get red or did it get a shrug? All right, so web reader got it right. So web reader queried the Internet and it got it correct. That's great. So another thing to keep in mind. So right here, the default browse with Bing, they just had an update about two and a half weeks ago. We covered it the day it came out. They are starting to source and cite things a lot more. So again, we always want to worry about is this accurate? Is this correct? The new update from OpenAI browse with Bing, we're getting multiple citations. So in this case it's pulling in the tunein and it's pulling in the Apple. The web Reader GPT kind of did the same thing. It just left us a link. So it actually just looked at our website. So that's pretty good there. So now let's go into Google Gemini. Same thing. Google Gemini. Please tell me about the podcast called Everyday AI. Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on Gen AI. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everyday AI.com partner to get in contact with our team or you can just click on the Partner section of our website. We'll help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on GEni. All right, so let's see. All right, so we got it right. It, it sounded a little generic at first, but you know, it. Then it actually got to yes. This is, you know, Jordan Wilson, former journalist, blah, blah, blah. So, okay, it got the right information. So so far everyone's got it right. And now we're jumping into Copilot from Microsoft. So we're running the exact same thing here. I'm going to zoom in for our live stream audience. Hopefully y' all can see this. We're at like 150 zoom. All right, so perfect. Here we go. Microsoft and Microsoft Copilot. I kind of like this. So you know, it said yes, certainly everyday AI is a valuable resource for anyone interested in understanding and utilizing artificial intelligence in their daily lives. It says here are the key details. So I kind of like this co pilot one because I didn't even ask for bullet points. But I love, you know, getting information and bullet points properly formatted. So I personally like, I, I personally like the, the responses here from Microsoft Copilot. It even kind of scraped these testimonials that are on, on, on our homepage, right? At first I'm like, did, are these hallucinated? But I'm, I'm like, nope, I know these people. And you know Oliver, Oliver Hen's here. So I'm opening up. We have a couple of these testimonials here on, on our website. And I'm like, yeah, I know these, I know these are correct. These aren't made up because I know that one from, from Oliver there that says it's a felony that you give this knowledge away free of charge. All right, so all of our models on this very, uh, very easy, very simple test, uh, passed. So I'm gonna, I'm just marking them up here in our little chart because at the end we're going to see in our very unscientific test which one did it correct. So I'm now I'm going to say, I'm going to ask a more specific question about everyday AI. Okay? And the reason I'm doing this, y', all is, is just because, you know, if you're listening to the show, you know, you probably know a little bit about. So I'm going to say this next prompt. And again, this isn't how I normally prompt. This is just very basic stuff. I'm saying please tell me about the daily podcast called Everyday AI by Jordan Wilson. No, actually, what I'm going to say is, please tell me how many episodes. How many episodes the Everyday AI podcast has done. All right, so here's the thing. I'm doing it in the same window, so I'm not clearing its memory, so it still understands what. What we're talking about here. All right, so now I'm going to the default mode browse with Bing. Let's see how it does. All right, so it says 244 episodes, y'. All. If I'm being honest, I lost track. Are we at 244? Let's see. Let's see if anyone got it right. So we're at 248. All right, so the. The default mode did not get it right. All right, let's see if the web reader one gets it right. So same thing, I'm using chat GPT with the web reader GPT asking for however many it got. 240. So got a little closer, but still not right. All right, so now I'm jumping into Google Gemini. All right, so let's see. Google Gemini says it's a bit tricky to give you an exact episode count for the Everyday AI podcast. Okay. So it didn't even take a guess. It just said, oh, there's been a milestone recently. 200. So I'd say there's a failure there from. From Gemini. You know, Google. Like if you're connected to Google, the search engine, you should be able to see because if you query even the episodes page without crawling it, you will see that, you know, I know the page got crawled today, so it would be there. All right, so now we're going into co pilot. Same thing. Please tell me how many episodes the Everyday AI podcast has done. So it says 246. All right, so here's what we're gonna do. Here's what we're gonna do. We're going to give a green check mark for the two that got it closest. Okay. Which was the GPT version. Microsoft copilot Google Gemini didn't even try. And we're going to give the shrug version to Chat GPT. So kind of it was in the middle. The default chat GPT was two episodes or four episodes off the GPT and the co pilot were closest. So I guess again, this is all arbitrary. So I'm going to give them green check marks and then I'm gonna give Google Gemini. Or maybe you know what, let's. Let's be more accurate here. So we'll Give, we'll give co pilot a shrug. We'll give the GPT version a shrug, and then we'll give the other two since they got it the most wrong. You, you know what? I, I should have just done like a numbering version, but that's fine. All right, so let's get going. So now we're going to do Market Cap. All right, so what that is is I want to see today the biggest companies, biggest companies by market cap. All right, so I'm going to go ahead and bring up here on my screen so you all can see. So the biggest companies by Mike, by Market Cap right now are Microsoft number one at 3.1 trillion, Apple number two, 2.7 trillion, Nvidia number three, 2.2 trillion, and Alphabet, Google 1.9 and Amazon 1.96 as well. So they're pretty close to each other. So right now I'm going to ask all of these, please tell me the largest five companies in the United States by market cap. All right, I'm going to guess most of them are going to get it fairly wrong. So I'm going to say, please tell me the five largest companies in the United States by market cap. All right, so let's go ahead. We're going to grab that prompt, jump into Chat GPT default. All right, so let's see how Chat GPT default does. So we have. And I'm gonna, I need to open this here in a new tab so I can keep tabs on who's getting it right and who's getting it wrong. All right, so we have pretty bad here. All right, so Chat gbt browse with Bing, got the first company wrong. It put Apple instead of Microsoft, and it is Microsoft instead of Apple. Then it has Alphabet number three. So it's, it's all wrong. Not even close. Not even close. Chat gbt. And also, so it brought up a random article from Forbes India. Let's see when that article was. I'm clicking it March 19th. Oh, brutal. Brutal. All right, so again, think of how you're using large language models even if you're connected to the Internet. Look at how disastrous that could have been. Right? If you're using information blindly, not knowing how these models work, not knowing how they retrieve data in information, and if you just took that as the truth, that could have been bad. All right, so next what we're doing is we're doing the Chat GPT inside the web reader. Let's see if we can get any closer. All right, so the, the default mode didn't really get anything right. So now it's just saying as of early 2024. So didn't even give us a date. However, we have Microsoft, Apple, Nvidia, Amazon, Alphabet. So we got three out of five. Right, so better. Better. All right, so web reader did a little better. So we got so far of a fail. Let's just go ahead and do this as we go. Go along. So we got a fail. We got a, we got a shrug. Now let's go ahead and do Gemini here. So let's ask it. Please tell me the five largest mar. Please tell me the five largest companies in the United States by market cap. Let's see what we get. All right, so Google. All right, let's see here. So Google, Microsoft, Apple, Nvidia Alphabet, Amazon got it right. Look at that. All right, and then it also gave us a link to find the up to date information. And sure enough, it is using the exact same source that I happen to pick, which was the top source on Google. So once, hey, good job, Google. Gemini, you went to Google, which you've been struggling to do for the last three months. So good job there from Google. And let's go to Microsoft Copilot and see what we get. Again, Copilot in this test is the only one that we're using the free version. All right, so let's see, we have Microsoft, Apple, Nvidia Alphabet and Amazon. All right, so that got it right as well. So I'm wondering, okay, so Gemini did not give even though it got the correct order. It didn't give prices even though in theory we didn't ask for prices. However, Copilot did. And I'm looking at these, these numbers here and they're pretty close. They're pretty close. So I'm gonna, I'm still gonna give them both. They passed. Even though I would say co pilot passed a little better. All right, so let's get this thing going. So now we're, we just have two more tests. So now we're asking it to visit a specific website and we're asking for specific information. All right, so we're starting at the beginning. Now we are going to chat GPT, the default mode. And I'm saying what are the most recent episodes from this page? And I'm giving it the page, the URL for your everyday AI.com episodes. And if you didn't know, you can go, there's hundreds of episodes. I don't know if you know this. You can, you can sort them by category or you can click this AI tracks Button right here, which is pretty cool. And let's say you just want to learn about sales. You can click sales and read all those. All right, so anyways, the most up to date ones. And again I'm. This is a recording. So it's April, April 11th. But so we have episode 248 about free chat GPT versus chat GPT plus. We had the episode from yesterday with Sheetal Rishi about a AI, your new BFF or tech terror. So let's see how chat GBT did. Okay, interesting. So it got this correct. So it looks like it queried the correct page. So I'm hovering over the source and it did. So it looked at the words in the URL, it queried those in Bing, and by doing that, it did bring up the correct page, which doesn't always happen. Right. That's obviously a niche. Your everyday AI in episodes. If you Google that, our website's going to come up first because it's, it's very niche. If it was a more general, you know, terms in the URL, it wouldn't come up like that. All right, so let's keep moving on. We're going to the web reader one and we're asking the same thing. What are the most recent episodes from this page? Foreign. I'm guessing this one's gonna get it right, but who knows? This is why we do it live, unedited, unscripted. Let's see what we got here. All right, so this got it right as well. It's giving us the correct episode starting with 248. Let's go to Gemini. Hey, Google, are you connected to the Internet? Let's find out. Asking Gemini. And here we go again. Gemini. Unfortunately, I can't directly determine the most recent episodes for that page for a Verizon for a variety of reasons. Dynamic content websites, right? So it's giving me these, these reasons that it says it can't visit the page. Right. But you know what? If I go to Google, let's just go ahead and do this for fun. All right, so I'm going to go to google.com and I'm gonna. Even though this is not how you use a search engine, I'm just literally going to ask the same thing. All right, let's just go, let's just query just this URL. All right, so it's, it's not bringing it up in the, in the meta description. It's not bringing up all of our different episodes. But still, I mean, come on, Google. The fact that you, you say that you're Internet connected. I'm on the enterprise plan, which is like $30 a month. And for some reason Google can't see web pages, even though I know googlebot is on my website hundreds of times a day. Right. Because I'm looking at it. All right, so failure there. And let's go to Microsoft co pilot, see how it does. And we have one, one quick test after that. All right, so co pilot. Oh, let's see. Did it miss one? It did. So copilot started with 245. So we're going to get that. The half, the halfway. All right, so our two chat GPTs. The default got it right. The GPT got it correct. Google Gemini, a big fail. And then Microsoft copilot. Close. So we give it the shrug. We give it the shrug. All right, so last but not least, we're doing a little PDF test. All right. So we can try this in different ways. So this is. We're doing a little needle on the haystack test. Right. We've been doing these for like a year here on the show. If you ever watch our YouTube channel. But we have a pretty short PDF here. It's 41 pages on page 23. I just hid some information. I said, my name is Jordan. My favorite food is deep dish pizza. My favorite color is Carolina blue. I'm from Chicago. So we're going to ask chat GPT. So in the default mode. No, you know what, we're just going to do it the same way here. So I'm going to say, I'm going to say read this page and tell me what Jordan's favorite food is. All right, so I'm gonna go ahead and get these started on Gemini and on co pilot as well. Oh, wait, I just did a, did a little mistake there. I left the, the, the wrong, the wrong URL. So it's not going to be able to access the. It's not going to be able to access that one. Let's give it the correct URL here and try this one more time. All right, so we are going in the default browse with Bing mode. There we go. We're going in the Gemini and we're going in the co pilot. All right, so the default browse with Bing says I can't directly access web pages or files from external links. Obviously I could download this and ask the same or sorry, upload the file into the default mode of chat GPT because it's multimodal. I can, you know, upload photos, documents, etc, but that's not what we're doing, we're just seeing, hey, can it go visit the Internet? All right, so we got a failure there. So now we're going into chat GBT with the web reader. All right, I'm going to say read this page. Same thing here. We'll see how web reader does. And then so let's check on Google. Google, unfortunately, right? Google. Google. Google's like a kid that got in trouble is like, see what happened was I was supposed to be connected to the Internet, but then I'm not. So here's 18 reasons why I still can't access the Internet even though I am Google, right? So, so not that great from Google. Let's see. Microsoft Copilot. Interesting. So instead it said it's looking at the country of Jordan and it's giving me Jordanian food. A flavor for the journey. Although, hey, so, hey, this falafel here looks pretty good. All right, so we have failure from default. So let's just go ahead and do this. So we got big failure from default. We got failure from Google Gemini, and we got failure the first failure from Microsoft Copilot. So let's check in on the GPT version. I need to give it access to read this PDF. So let's see how this does. Bam. It got it. It found the needle in the haystack. Look at that. So in a 40 page PDF, I hid some information on page 23, the web reader GPT, found that information. Got it right. Look at that. All right, so that concludes our very unofficial study on our large language models connected to the Internet or not. So our, our unofficial standings here. So the default browse with Bing got two out of five. All right, we're going to give the, the, the shrug, the yellow shrug guy, we're going to give that a half point. All right, so chat GPT, let's, let's go ahead and just write the scores down here as well so we can make this, so we can make this super official, right, y'? All? So let's go ahead and do this. We'll give it a big. So what do we get here? We got a two. Let's go ahead and do this so we can all see it, huh? Love Canva. By the way, we had the head of AI from Canva on the show a couple months ago. All right, so we have a score of 2 for browse with Bing. All right, so here we have four. So chat GBT with a jab with a GPT is four. So the default mode inside Chat GPT didn't Do? Well, the, the GPT did better. All right, we got a Google Gemini got a 2. And then last but not least, let's see what we got here. We got a three. All right, so here's our final, very unofficial, not scientific at all, results from our large language models actually connected to the Internet. Well, not really. Some of them do a little better than others. Also worth noting out like someone you might be asking, like, oh, why didn't you do Perplexity? Well, Perplexity is an answer engine and it uses either GPT4 or it uses Claude. And you might be wondering why didn't you use Anthropic Squad? Well, it's not connected to the Internet at all and it doesn't pretend to be, right? So all of these other companies, Chat gbt, Google Gemini and Microsoft Copilot tell you in their marketing, hey, you know you're going to get better answers with us because you know we're connected to the Internet. So our final scores, the default mode of Chat gbt, two out of five. All right. In tied. So Chat GPT and Google Gemini are tied in last place because Google Gemini also two out of five. All right. Our runner up is Microsoft co pilot with a three out of five. And in first place, Chat GPT using a web reader GPT with a four out of five. So I could go on for this all day. I won't. We're gonna leave it there. Let me know, was this helpful? Do you understand now the importance of having up to date information? Because think in the coming weeks, months, quarter and years, whether you know it or not, you're going to be using large language models more and more, right? You might be prompting a couple times a week, couple times a day now, but Generative AI is coming to where we are, right? Every single day we're getting more and more announcements. A year or so ago you kind of had to go out of your way, right? But now it's coming to our desktops, right? With Microsoft 365 Copilot, it's in almost every, right? Large language models are being integrated into everywhere we work. If you work in front of a computer, more and more large language models and LLM integrations and desktop operating systems with models baked in, whatever Apple is going to be releasing here pretty soon. Generative AI is coming to every inch, nook and cranny of where you do business. So you have to understand, right? You have like one of the biggest reasons that companies either don't use large language models or they're slow and they're getting passed up because of it is because they don't understand how they work and they don't understand the right ways to put in safe cards. Put in fact checking to know when you get hallucinations and when you might not, and know how to give chat, GPT and Gemini and Copilot access to more up to date information to decrease the likelihood of hallucinations and putting out embarrassingly wrong information. So I hope you learned a little bit today. If you did, please consider sharing this with your network. I'd really appreciate it. And I really appreciate it. If you also would go to your everydayai.com Sign up for that free daily newsletter. Thanks for tuning in. Hope to see you back tomorrow and every day for more Everyday AI. Thanks y'. All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit youreverydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.