A Massive Upgrade To ChatGPT! (This is Crazy)

Today, OpenAI just announced something that is completely game changing. This is going to completely change the way you use ChatGPT. It's going to give you Internet access inside of ChatGPT. It's going to allow you to upload images and video and audio and CSV files and connect it to your website and do pretty much anything that you could possibly imagine with ChatGPT. This is that game changing and I'm going to show you all about it right now. So the first big announcement came from OpenAI. They announced that ChatGPT will implement support for plugins. Plugins essentially extend the functionality of ChatGPT and allow you to sort of bolt on additional tools to what ChatGPT is already capable of. For example, they're starting with Expedia, Fiscal Note, Instacart, Kayak, Klarna, shopping, Milo Family, AI, OpenTable, Shop, Speak, Wolfram and Zapier. They also have two of their own plugins, a web browser and a code interpreter. And they've open sourced the code for knowledge based retrieval plugins so that other people can actually create plugins for their own knowledge bases to attach to Chat GPT. Now, when I heard this news, I immediately went over to ChatGPT and saw that I was not one of the people that had access to plugins yet. And I don't think this is widely rolled out. In fact, if we look at their website it says we will begin extending plugin alpha access to users and developers from our wait list. While we initially prioritize a small number of developers and ChatGPT plus users, we plan to roll out larger scale access over time. Fairly soon we'll be able to access this stuff inside of our own chatgpt. But they did release some videos that kind of give you an example of the types of things you'll be able to do with plugins. So here's an example of the browsing plugin here. Down at the bottom they type how do this year's Oscar winners compare to recently released movies for box office sales? It now browses the web. Because this has a browsing plugin for it, it reads the information that it got from browsing and and then gives a response. The 2023 Oscar winner for Best picture Everything Everywhere all at once earned 100 million at the box office. Recently released movies such as Scream 6 have grossed 58,529,284 in the first seven days, while Scream 6 had a strong start at the box office. It's worth noting that Everything Everywhere all at Once has had a longer run in theaters to accumulate its 100 million in box office sales. So with this additional plugin, even though ChatGPT itself is only trained through 2021, using the browsing plugin, it will actually be able to search the Internet and augment our results with what it found on the Internet. Now, there's a lot of tools out there, a lot of Chrome extensions that claim to extend the functionality of ChatGPT for you by connecting it to the web on your behalf. And well, a lot of those extensions are going to become obsolete thanks to plugins like this. They also show in this demo that it does cite its sources. They hover their finger over the little citation there and it shows. The link is to Box Office Mojo. And here's another demo where they install the Wolfram Alpha, the OpenTable and the Instacart plugin. And then they ask Looking to eat vegan food in San Francisco this weekend? Can you get me one great restaurant suggestion for Saturday and a simple recipe for Sunday? Just the ingredients? Please calculate the calories for the recipe using Wolfram Alpha. Finally, order the ingredients on Instacart. So they connected OpenTable plugin, which will help them find a vegan food restaurant in San Francisco. They connected the Wolfram plugin, which does all sorts of complex calculations for them to help them find the calories of their meals. And then they connected the Instacart plugin to help them find and order the ingredients. You can see that it uses the Open Table plugin. I found a great vegan restaurant in San Francisco for you to try on Saturday. Greens Restaurant. You can make a reservation for your desired time through the provided link. It gave them a link there to make a reservation for Sunday. Here's a simple vegan recipe with the ingredients. It moved on to that next instruction where it gives them ingredients for a chickpea salad. And then it says, now let's calculate the calories for this recipe using Wolfram Alpha. The total calories for the chickpea salad recipe are approximately 862 dietary calories. Here's the breakdown of the ingredients and their calorie content. So it's using Wolfram to figure out the calories of the chickpeas, the cucumber, the red bell peppers, the red onions, and and it gives them a full breakdown of the calories of that meal. And then it says, now let's order the ingredients from Instacart. And it uses the Instacart plugin. Here's your shopping list for the chickpea salad and it adds all the items to an Instacart shopping list that you can just click on. And it takes you over to the Instacart app and allows you to order all of those ingredients. It put them all in a shopping cart for you already so that you can order those ingredients with one click of a button. You have that order. And it did all of those from a single prompt connected to all of these various plugins that were added to ChatGPT. And then they also have a demo here using the code interpreter plugin. So the prompt down at the bottom plot function 1 sin x. Its code interpreter is running. And figuring that out, you can see here's a graph of the function 1 sine x. Please note that the graph has vertical asymptotes where sin x is 0. For clarity, the Y axis range has been limited to negative 10 and 10. Now this kind of stuff is over my head. But the code interpreter plugin added onto gpt4 instead of chatgpt was able to figure this stuff out. The next prompt zoom into range of x values between 0 and 1. And then it drew a new chart. Here's the graph of the functions 1 sine x zoomed into the range of x values between 0 and 1. Plot A tangent line to the graph at x equals 0.3. And there you go. It drew the tangent line. So it can do all sorts of graphing and complex charting. And then check this out. It actually opens up a CSV file directly inside of ChatGPT. And it says, great, you've uploaded and file named Music CSV how can I help you with this file? And then it writes the prompt. What are the columns of this data set? Here are the columns in the Music CSV data set. And then it lists all of the columns because it's actually able to read the CSV file inside of ChatGPT. How many rows does it have? The Music CSV Data set has 10,000 rows. Can you run some basic visualizations? And then check this out. It just created all those charts right there. Here are some of the basic visualizations based on the Music CSV dataset. Histogram of song durations. This shows the distribution of song durations in seconds, a scatter plot of song hotness versus artist familiarity, and the bar chart of the top 10 most frequent artists names. And it pulled all that data from the CSV that they uploaded. Now let's do the following analysis. Let's exclude year zero. Let's consider only years with over 100 songs from the set of Years. Let's find a quantity that is the most positively correlated with time. The quantity that is most positively correlated with time in the filtered data set is song loudness. Can you plot that? And then they plot that on the graph. Here's a scatter plot of song loudness versus year in the filter data set. The plot shows the relationship between the loudness of songs and the year they were released with data points representing individual songs. Make an aggregated plot. Here's a plot of the mean song loudness by year in the filtered data set. The plot shows the trend of the average loudness of songs over time, with data points representing the mean loudness for each year. So this is all using this code interpreter model. Here's another one. They just uploaded a PNG file. Great, it looks like you've uploaded a file. Let's take a look. So here's the PNG file that was uploaded into ChatGPT and then the description it gave is. This is a neo modernist Python icon in the form of a digital painting. It features a stylized depiction of the Python logo with bold lines, geometric shapes. The color palette is modern and sophisticated with shades of blue, green and yellow. So it actually looked at the image and pulled all that information about that image. And then the next prompt they give. Can you make it 4x smaller? Here's the neo modernist Python icon image resized to be 4x smaller. Can you turn it to grayscale? Here it is in grayscale. Please tint it a little light green. Here it is a little light green. Now use OpenCV to select foreground only. So now it's using computer vision to find just the foreground of the image. Here's the neo modernist Python icon image with the foreground selected and the background removed. I mean, it's kind of hard to tell, but you can tell it's just a black background in the upper image. Still has a little bit of a green shaded background to it. So it actually removed the background using OpenCV. Can you make a download link for me? Sure. Here's your download link. Like, this is crazy. And that's all using the code interpreter that's going to just be an available plugin in ChatGPT+ pretty soon. Now here it is showing off the retrieval plugin where you can sync it up to other data sources and it will use those data sources as the context for its responses. So this is synced up to a United nations data source. And they asked what human rights were introduced last year in 2022, one of the human rights that was introduced and emphasized was the human right to clean, healthy and sustainable environment, et cetera, et cetera. How has thinking on climate change evolved over the years? You have access to use a table and then it's using the United nations data source again. And then it starts to just draw a chart of all of the data it has access to inside of that database. Notice that the database is beyond 2021. That's because it's pulling from the United nations database and not ChatGPT's knowledge base. And these are just the demos that they show on OpenAI's website right now. Some other really cool demos started popping up on Twitter as well. Here's one that Sam Altman, the CEO of OpenAI posted himself. He connected ChatGPT to Wolfram and then he asked, what is the current distance from Earth to Jupiter? Wolfram does whatever computations it does. The current distance from Earth to Jupiter is approximately 5.908 astronomical units. This is equivalent to all of these big numbers. 549.2 million miles. The light travel time between Earth and Jupiter at this distance is about 49 minutes. And then here's another demo posted by Greg Brockman on Twitter. He uploaded an MP4 file, an actual video file, and he's using the code interpreter plugin here. And then he asked, can you extract the first five seconds of this video? And then it works on it, runs some code, does some computations here. The first five seconds of the video have been successfully extracted. You can download the extracted video clip from the following link and then it gives him a link to where he can Download the first 5 seconds from the video that he just uploaded. And then he shows playing a five second clip of that same exact video there. How mind blowing is this? This is just insane. This adds sort of unlimited functionality to ChatGPT. And once again, this is just going to totally blow so many other companies that are building on top of GPT3 out of the water. Because now you're just gonna be able to do it with ChatGPT. And here's another demo from Greg Brockman using the browsing plugin where it actually connects ChatGPT to the Internet. He types what is the response on HN to the ChatGPT plugins announcement? It knew that HN he was referring to Hacker News. It's now searching Hacker News, it's reading the content on Hacker News and it's responding. The response to the ChatGPT plugin announcement on Hacker News includes a variety of comments from the community. Many users are excited about the ability to create their own plugins, with some mentioning the retrieval plugin as a starter kit that offers built in integrations to the Pinecone Vector database, et cetera, et cetera. And it goes off to show the sort of reception that people had towards this announcement on Hacker News. Overall, the response on Hacker News seems to be a mix of excitement, skepticism and anticipation for the potential impact of ChatGPT plugins on various domains. And then Ben Tassel, he actually works with Zapier. He was able to show off some of the stuff that you'll be able to do with Zapier. Now these are gifs, so I can't really go full screen, so I have to zoom in as best as I can here. But here's this example. Now he's setting up a chatgpt action with Zapier. He connected his Gmail account. You can see the Enable plugin. Up here is the Zapier logo. Find my last email from Lars using zapier. It found the last email and it starts to paste in the content of the email and then it says you can also view the email directly in your Gmail inbox by clicking here. Now the implications of these ChatGPT plugins are huge. It extends the functionality of ChatGPT by so much. ChatGPT is gonna have access to the web. It's going to have access to Zapier. So you can connect it with all sorts of different tools like your email autoresponders or Slack or Discord or any other tool that connects with Zapier. You'll be able to upload CSV files, upload image files, upload audio or video files and have it do the editing on them for you and cut out clips from it and remove backgrounds and do all sorts of crazy stuff with images. Once again, we're seeing a tool, we're seeing a big company come in and just add features to tool that are literally entire products that other companies have been selling. So other companies that are out there that are based on GPT3 or that add web connectivity to GPT3 or do pretty much anything on top of GPT3. If you have chat GPT plus and you install some plugins, you could pretty much just do it with ChatGPT. Now this sort of infinitely extends the functionality of ChatGPT Plus. Now it'll be interesting to see if some plugins become like premium plugins, similar like WordPress, where some of these plugins you actually have to buy to get extended functionality. We'll see if a whole new industry sprouts up from this. I'd say that's kind of likely, but also you're building in somebody else's playground. And if other plugins start to pop up and people sell them, well then other plugins are probably going to pop up and people are going to give them away for free. Or ChatGPT is just going to build it in. So we're going to see this kind of back and forth with smaller companies trying to build and innovate and then the bigger companies like OpenAI, Microsoft, Google, companies like that. Just saying it's a feature now in our bigger tool. So it's going to be a real, real interesting game in the AI world. Now if you want to stay in the loop with all of the latest AI tech that's coming out, all the latest cool tools, all that sort of stuff, check out FutureTools IO. This is the website where I curate all of the coolest tools that I come across and I put them on the website in an easy to organize and easy to find manner. And if you don't really want to spend the time going through the tools, join the free newsletter. Every Friday I send you basically the TLDR of the Week where I show my five favorite tools that came across my desk. I share a handful of news articles, a handful of YouTube videos, and one cool way to make money with AI. Check out the newsletter Future Tools IO. And if you like videos like this, make sure you hit the thumbs up button and click the little subscribe button down below. That'll make sure you see more videos just like this in your feed. Thanks so much. Appreciate you for tuning in. I'll see you guys in the next one. Bye.

Sam.