GPTs and GPT-4 Vision API - 10 INSANE Use Cases

During the first ever OpenAI's developers conference they had some huge announcements. I think the two most prominent were the introduction of GPTs or agents and the release of a new GPT4 Turbo model with 128k context window. Now you can use GPT4 with vision through the API, which is amazing and we have already seen some really cool demos that people have built on top of it. I'm still waiting for my access to GPTs, but I wanted to show you some really cool demos that people have been able to put together using both the GPT4 with Vision API as well as custom GPTs. So first I'll show you a few examples of some really cool projects that are being put together using the GPT4 Vision API. And later I'll show you a few examples of really cool custom GPTs. The first video that we are going to be looking at is by Peter. So what he did was he used the GPT4 Vision API to do a screen capture of this League of Legend games and then use the text to speech converter from OpenAI to actually generate a commentary of the game that is going on. So let's have a listen.

As we hit the 18 minute mark, T1 has a slight lead with a 3k gold advantage and one turret in their favor against LNG. The vision game around the mid lane is intensifying as LNG scout.

This is pretty cool because GPT4 is able to not only understand what is happening in the game, but it's also able to provide a pretty good commentary. Now this is not only limited to video games, but you can actually do this on real soccer as well. So here's a demo that was put together by Gonzalo.

All like a magician on the field, dodging. One, two, three. Unstoppable. Look at him go. The crowd is roaring. Can you believe this? He's taken on the whole defense. He's a one man show, ladies and gentlemen. He shoots.

Go.

Messy, messy, messy. Unbelievable. What a goal. What a goal. Glorious. Absolutely glorious. The stadium explodes in joy.

Okay, you can see it's actually able to recognize the action now. I think probably it would be better if we make it a bit more enthusiastic. But both the vision recognition using GPT4 Vision as well as the text to speech is working pretty great here. Let me know in the comment section below if you want me to create a tutorial on how to do that on your own. Okay, so here we have yet another one. This is from a user called Robert. And this is pretty exciting and very different than what we have seen so far. So Robert put together a system that is reading everything on the browser. So he's simply selecting areas in the image or on the Web page and GPT4 is able to recognize what that is. So for example, he is selecting Spine section and it's able to determine that, right? It's able to read equations, tables, and so on and so forth. So all he's doing is reading or recording a video of his browser screen, sending it to GPT4 with vision. And GPT4 with vision is able to recognize everything that is happening on the screen. So this is absolutely brilliant. All right, so here is yet another one. This is called Webcam GPT and this is developed by Benjamin. Basically, Benjamin gave access of the webcam to GPT4 with vision, and now it's able to recognize whatever he's showing. There are a couple of demos that I saw where you can use this as a coach or trainer for your workouts. So basically, GPT4 can look at the video feed and then determine whether you're performing work out correctly, or it can actually give you recommendation on how to improve it as well. Okay, so here is actually an example of that. This project is again from Roberts. So basically it's looking at different posture or yoga posture and then giving you feedback on what your form looks like. Now keep in mind, all this is happening within 24 hours of the release of GPT4 Vision API. I also wanted to show you some example demos of custom GPTs. As you can see, I don't really have access to it yet, so I'm not able to create one of my own. But GPT4 comes with a whole bunch of different custom GPTs that I'm going to probably review in another video. But I'll show you a few example demos or projects that some users on X have put together. The first one that I'm going to show you is from Ethan. He's a professor of computer science. Now he put together a pretty nice blog post on what you can do with GPTs or these agents. An example that he has presented in here is how to convert your data into a paper. So this is from his blog post. So what he did was he asked the GPT builder that cannot write papers, right? Then the GPT builder is asking about the data set and field scope and guidelines, right? So here Ethan provided data set in a CSV format. And this is regarding nationalism and corruption data set. And the field is supposed to be economic psychology. Now based on this data set, the GPT Builder is able to use both the code interpreter along with access to the Internet to put together a pretty nice abstract introduction, a corresponding theory and even analysis of the results within the data set. So this is pretty huge. Now there are definitely some hallucination, but just to think about it, that you can build a tool like this and then you can potentially use it to generate reports. It's pretty awesome. Now here's another project from Ravan. So what he did was he created what he's calling X Optimizer GPT. And this is fine tune on his posts. And he's using this to figure out what are the optimum times to post on X so that it can increase the engagement. Now, I believe you're going to see a lot of tools like this coming up. So essentially what he has done in here, he actually downloaded all his previous analytics data from X and uploaded that he's enabling web browsing Dall E for image generation as well as code interpreter. Then he's creating a custom GPT that not only optimizes his posts, but will also determine what is the best time to post this based on the historical data that he has given to the code interpreter. I personally see some really huge business opportunities for something like this. Okay, here's a fun one from Brett. So he has combined the web browsing with GPTs and he is asking to browse the web to find this year's Coachella lineup and make a playlist of the highlights. So basically this is GPT that he's calling a playlist AI Spotify. Now, since it has access to the Internet so it can see what was the line of Coachella. And then based on those, it can create a playlist in Spotify of all the songs. Right. So this is pretty neat as well. Now there are a couple of examples from Nick Dobos that I really like. So for example, in this case he's using the custom GPTs to create a baby AGI GPT, which is pretty awesome. So there's a link, there's going to be a link to all these different demos in the description of the video and I'm going to put this one specifically. So I would really recommend everybody to check this out. All you need to do is just provide the task that you want to do and just like baby AGI, it will create the task list for you, right? And you can give it different commands to execute it. So it is pretty awesome. There is a link to that specific GPT in here. So I'm going to put this in the description. Now along with the GPT4 Vision API and custom GPTs, OpenAI also released the OpenAI API assistant. So here's a demo using the API assistant. This was put together by Mayo. Now in this demo he's creating a system that will score candidate resumes based on the job description. Now you can simply provide the job description as well as a candidate's resume and this specific API assistant figures out which one is the most relevant resume or most relevant candidate for a specific job description. Right? So it's just an example of a use case that you can develop on top of the GPT4 API. So these were some of the cool demos that I found on X. I hope you found them useful. I'm going to be creating some videos on reproducing some of these demos so make sure that you look out for those tutorials. I hope you found this video useful. Consider subscribing to the channel. Thanks for watching and as always see you in the next one.