The Most Insane Week of AI News So Far This Year!

This week has legitimately been one of the most insane weeks in AI. Some of the announcements that we got this week literally will change the world and just raise the bar for all AI companies. Now I know there's probably going to be a lot of comments of people saying oh, why you clickbait me? Or whatever. This video is justified. Just wait until you see the insanity that was this week in the world of AI. And I'm going to start with Thursday's announcement from Google's DeepMind where they announced Gemini 1.5. It was only last week that we got Gemini Ultra and this new Gemini 1.5 is actually pretty dang crazy. Now to start this new model from Google, DeepMind uses the mixture of Experts architecture. Now I talked about this in a previous video. Basically Mixture of Experts is where you have a handful of smaller language models. When you give a prompt into the model, it will then decide which of the experts to send that prompt to before getting a response back. This is just a much more efficient way of doing these large language models because as it sends the prompt to these smaller language models, it only has to process it off of a much smaller set of data. I even put out this tweet back in December that breaks down how the Mixture of Experts concept works. So I'll link below if you want to learn more about it. But that's not really the part that's so crazy about Gemini 1.5. Gemini 1.0 had a 32,000 token context window, giving you a combined 24,000 words of input and output from the large language model. Gemini 1.5, on the other hand, can now run up to 1 million tokens in production. 1 million tokens is around 750,000 words of input and output text. Put that in some more context. All seven of the Harry Potter books contain 1,084,170 words. With a 1 million context window, we are this close to being able to upload the entire Harry Potter book series and ask questions about it. It's also got better understanding across modalities. It says here it was given a 44 minute silent buster Keaton movie and the model was able to accurately analyze various plot points and events and even reason about small details in the movie that could have easily been missed. Small so it analyzed a 44 minute silent film. No text, no transcript, nothing for it to work with on the text side, and was able to analyze the plot points based on the video alone. With that large of a context window and the ability to upload that much text all at once. A lot of large language models tend to lose stuff within the text. With Claude, we have a 200,000 context window, or roughly 150,000 words. Researchers do this test called the needle in a haystack test, where they will take, let's say 150,000 word piece of text and bury a sentence or a word or something somewhere in the middle of that text. And then they will ask the large language model to answer a question about that little piece of text that they placed somewhere in the middle of this large bank of text. Now, a lot of tools like Claude and chatgpt struggle to find that little nuanced piece of information in this large amount of text. However, Gemini 1.5, it says right here in a needle in a haystack evaluation, where a small piece of text containing a particular fact or statement is purposely placed within a long block of text. 1.5 Pro found the embedded text 99% of the time in blocks of data as long as 1 million tokens. So again, to put it in the Harry Potter context, it's like putting a single sentence in book 4 of 7 on page 213 and asking the large language model to answer a question about that little snippet that you put in. And Gemini 1.5 was able to find that 99% of the time in its various tests. So this announcement of Gemini 1.5 alone would have been huge news for this week. I mean, we don't have access to it yet, but it's a big announcement that I'm imagining we're going to have access to really soon. But on the day that this dropped, this was the biggest news in the AI world for like two hours. Because then OpenAI stepped in and said, we're not going to let Google have a moment today. We're going to unveil something that is just going to blow everybody's minds. And we got sora. SORA is the most insane AI text to video model anybody has ever seen. It can generate up to 60 minute videos and some of the videos it generates are just insanely realistic. Like, this is mind blowing. So mind blowing in fact, that I felt it deserved its own dedicated video. If you look for this video called AI video from OpenAI just blew everyone's minds. I did a much, much deeper dive and showed off a lot of the demo videos that came out. Since I recorded that video, OpenAI released even more information about Sora. They actually released their entire research paper on it. Now the research paper has a lot of the demos that we've already seen in it, as well as a technical explanation of how this model actually works. But it also has some new demo videos that we haven't seen yet, with some additional context around what we're looking at. For example, these three videos of sea turtles swimming around in different aspect ratios, as well as showing off what it looks like when you generate videos from image prompts. Like, here's a still image of a Shiba Inu, and here's how it animated it. Here's a little monster illustration, and here's what it looks like when it was illustrated. An image of a realistic cloud that spells Sora. Here's the image as an animation where the cloud just sort of poofs into existence. They uploaded this image of a wave inside of a museum. And here's a video of the wave crashing and a surfer actually riding the wave. You can extend generated videos and make them longer. Here's a video that was extended to create this infinite loop of a bike rider where you can't even really tell when it starts over. It just loops through the same thing. It could even do video to video. On the left, you see this input video of a Porsche here. And on the right, you can see it says, change the setting to be a lush jungle. Change the setting to the 1920s with an older school car. Make sure to keep the red color. Make it go underwater. Now it looks like the car's driving underwater. Change the video setting to be different than a mountain, perhaps Josh. And now it looks like we've got a car driving through Joshua Tree. It can also connect videos where it sort of seamlessly merges from one video to another. Watch this video here where we see a drone flying around what looks like the Coliseum. And then kind of out of nowhere, the drone converts into a butterfly. And next thing we know, that butterfly looks like it's swimming around underwater. You can see here's an input video of a drone flying around. Here's an input video of a butterfly underwater, and it sort of seamlessly merged between the two videos inside of this center video. Here's another one of, like, a cliffside house sort of thing seamlessly transitioning into this little snowman village here. Or a chameleon seamlessly transitioning into this bird that we see here. And look at how realistic this is, too. I mean, I'm still blown away by how realistic these chameleons and these birds look. Here's some more example videos that they put out down here. They even use the AI to make simulated Minecraft videos. If you Just saw this browsing Twitter. You would assume this was just somebody filming themselves play Minecraft. Nope, this was generated with Sora. Sora can also generate images. This is something not many people have talked about yet, but Sora is also capable of generating images. We do this by arranging patches of Gaussian noise in a spatial grid. With a temporal extent of one frame. The model can generate images of variable sizes up to 2048 by 2048 resolution. You can see some of the AI generated images that are here. I would argue that this is more realistic than anything Dall E3 can create up here on the top left. Makes me wonder if this is the predecessor to Dall? E, if this could generate images just as well, if not better than Dall? E. And it's capable of generating full on realistic 1 minute long videos. Why would people go back to Dall? E? Just do it all with this one platform once we have access to it. Speaking of which, we don't technically have access to, but there is a way to sort of play around with it and get some variations. For example, on the research paper, there is a page here where you can play with this video a little bit. You can see it says adorable kangaroo. I can actually click on an adorable kangaroo, change it to a woman. Change it to an old man. Change it to a toy robot wearing a green dress and sun hat. I can change this to blue jeans and a white T shirt, or purple overalls and cowboy boots. Let's do an old man in purple overall and cowboy boots. Or an old man wearing a green dress and sun hat. Let's change it from Johannesburg, South Africa to Antarctica. Now we have an old man in a green dress walking in Antarctica. Here's an old man in a green dress in Mumbai, India. And then we've got during a beautiful sunset, let's do during a winter storm or during a colorful festival. You could go in here and play around with all of these variations and see what kind of videos it generates for you. We can't really do our own text to prompting yet, but this is a cool way to sort of see what it's capable of. If you haven't checked out the Sora research paper from OpenAI, I will link it up below because it's definitely worth clicking through, seeing some of the videos it generated, playing around with this little generator tool that they've got here and putting it through its motions. Because this is some of the most mind blowing AI anybody's ever seen. Period. End of statement. And here's something cool that I saw from Metamike on X as well. He actually took the AI generated video of Santorini here, plugged it into Polycam 3D and turned it into a Gaussian splat that you can actually move around in. This might be the future of how AI generated 3D objects are created. Use a text to video model to prompt what you're looking to create. It creates a video of that thing from different angles and then we use gauss and splats or nerfs or something like that to convert it into a 3D object. I know I'm getting into the weeds a little bit, but this whole thing just really, really excited me. Got me all pumped up. If you saw my video yesterday, I was going insane over this Sora tech. It is mind blowing and I cannot wait to get my hands on it. But I did say that this was the most insane week I've seen in a long time in AI, and so far I've only talked about two announcements. So as you can probably guess, there's more. And since we're on the topic of video, today's video is sponsored by InVideo AI. Now, I've talked about InVideo AI a lot in past videos because it's a tool that I use and love myself. If you're not familiar with Nvidia AI, it's a tool where you can enter a single text prompt and it will generate a long video for you. You can generate 10, 15, 20 minute videos with this tool that include stock video, a script, a voiceover, sound effects, music. All of it is created in this single video. They recently added the ability to upload your own voice into the platform and have your videos use your voice. It's real simple. Log into Invideo, click on the Voices tab on the left side, click on add voice. Upload 30 seconds of your voice. Now, when I'm ready to make a video, I come to my homepage inside of the Invideo dashboard. Click on Create AI Video and then enter my prompt. Now this actually works well with long prompts. So I'm going to use the prompt, create a long YouTube video about how important lighting is in videos. It is one of the first things early filmmakers and YouTubers can work on to get much better output. Provide some practical ways to get great lighting on a budget. Keep the language super simple, easy to understand. Start the video very matter of factly and make it progressively deranged. Humor as you go on. Use the Matt wolf voice. About 12 minutes long would be nice. And add subtitles you can see I'm directing it to use my voice and let's generate the video. It asked me a few questions to better dial in the video. So I'm going to set the audience as YouTub look and feel. Let's go ahead and use Bright. Just a couple minutes later, I have a full video made with script, with audio, with subtitles, with voiceover. It's almost 12 minutes long and it uses my voice. Check this out. So you've got your light source. Now what? Well, it's time to manipulate that light like a puppet master. And what's even cooler, I can edit this video completely with text prompts. I can tell it to do things like change the intro and start with a joke, or add a call to action at the end to ask them to subscribe to the channel. I can tell it to change the background music, tell it to change the voice, Anything you can think of. I can just ask it in a text prompt. And if I want to get even more granular with my edit, I can click on Edit here. I can change any of the footage in the entire video. Simply select it, search out what I'm looking for here, and replace it with any stock video that they have available in their library. I can also edit any piece of the script by coming up to the top, clicking on Edit script and adding to the script however I like. Simply apply the changes and after a minute or so the script has been changed and the new voiceover added. Now you can create up to four videos for free with InVideo AI. But if you're really serious about video creation, you're going to want to upgrade to the plus plan. That's what I have that allows you to export the video without watermarks. To quickly learn more, I've made sure the link is in the description and you can check it out today. Thank you so much to Nvidia for sponsoring this video. Outside of Sora, OpenAI was in the news for some other reasons as well, including the fact that that they are rolling out memory inside of Chat GPT. Basically, when you use Chat GPT it will remember previous conversations and details from those conversations, store them for you, and then remember that information when you have future discussions. We can kind of see in this screenshot here some of the types of things it Remembered has a two year old daughter named Lena. Daughter Lena, loves jellyfish, loves to travel, is interested in traveling to Mexico for April vacation. So it remembers a lot of this stuff so it can use it as context in your future conversations. We can also See in a demo video here that we can go to our settings inside of Chat GPT, go to a personalization tab and we can turn memory on or off if we don't want to use it. Also by going to Settings and the personalization section, we have the option to manage the memories. See all the memories that are in there and if there's any specific memories we don't want it to hold on to, we can delete those memories. They're also adding the ability to do temporary chat. So if we come up to the top click on temporary chat. It's basically saying that when we use this temporary chat, this discussion won't appear in our history, it won't be used to create memories, and it won't be used to train the models. It's basically incognito mode for ChatGPT. Memory is also going to be implemented inside of GPTs as well. Most of us don't have access to this memory feature inside of ChatGPT. It's something that still being rolled out, but if you want to double check if you have it, you can always go down to your name in the bottom left click on Settings. And if you don't see the personalization tab here yet, just like me, you don't have it yet. And while we're on the topic of OpenAI, Andrej Karpathy, one of the original founding members of OpenAI who was there at the beginning of the company, left for a little while, joined Tesla, helped develop the full self driving for Tesla, then came back to OpenAI. Well, this week he quit OpenAI again. Of course this led to a lot of people speculating. Is he worried about how far AI is getting? Is he pulling a Geoffrey Hinton and is now going to speak out about the dangers of AI instead? What is going on here? Well, Andre took to X himself to explain. Hey everyone. Yes, I left OpenAI yesterday. First of all, nothing happened and it's not a result of any particular event, issue or drama. But please keep the conspiracy theories coming as they are highly entertaining. Actually, being at OpenAI over the last year has been really great. The team is really strong, the people are wonderful and the roadmap is very exciting and I think we all have a lot to look forward to. My immediate plan is to work on my personal projects and see what happens. Those of you who have followed me for a while may have a sense for what that might look like. I don't know exactly what his personal projects are, but I do know he has a YouTube channel and he is one of the best people on YouTube at explaining complex topics. If you want to get a deep dive understanding of how a lot of these large language models and AI tools work, Andre's YouTube channel is amazing for that. His video called One Hour Talk Intro to Large Language Models is the best explanation of how these LLMs work I have ever watched. So I think he might be leaning more into content creation and and helping people understand how these models work. I'm hoping we see a lot more from him on his YouTube channel because the stuff he has put out has been really, really good. It was also reported this week that Sam Altman from OpenAI is seeking as much as $7 trillion for a new AI chip project. He wants to release the reliance on Nvidia. At the moment, there's not enough GPUs to go around and if he builds a company that's actually manufacturing them, then his company can control the power flow of GPUs. However, this article came out from the information this week also that says no, Sam Altman isn't raising trillions of dollars for chips. A Wall Street Journal report last week that implied Altman was seeking up to 7 trillion worth of capital for what sounds like a joint venture to end all joint ventures caused quite a stir in reality. Altman privately has told people that figure represents the sum total of investments that participants in such a venture would need to make in everything from real estate and power for data centers to the manufacturing of the chips over some period of years. He's not going out trying to get companies to give him upwards of $7 trillion combined. This is the sum total over years and years and years with everything being taken into account how much it might eventually end up costing all right, so I'm done with the OpenAI news, but there's still quite a bit more to talk about that happened this week, including the introduction of Stable Cascade from ST AI, something that we actually do get to use. If you do want to deep dive into Cascade, I made a video about this called how to Make Professional Logos with AI for Free. Title is focused on AI logos, but the entire video is all around using Stable Cascade from Stability AI. I just use logo creation as the examples for using Stable Cascade. And you know, it's a crazy week when something like Stable Cascade that generates really, really good generative art with legible text in it is like the ninth thing I'm talking about in a video. But again, I did do a full breakdown on it as far as prompt alignment and how well it adheres to the prompt. It beats out Playground V2SDXL, Turbo SDXL and Worst Gen V2, which is actually the model that this was originally built off of as far as aesthetic quality and how well it generates. Playground V2 edged it out slightly, but stable cascade beat out SDXL turbo, SDXL and worst gen V2 and when it comes to the speed of generations it is much faster than SDXL and Playground V2. You can see it right here on the left, but not quite as fast as SDXL Turbo. It can do things like upload an image prompt and generate other similar images. It seems to be able to do inpainting where you can see. It gave this image of a cat and then put a dog's head on the cat. It seems to be able to work with a lot of the control nets. Here's one with canny edge where this sort of drawing of a lighthouse was given and then here's the output images. It can do 2x super resolution up to 2048 by 2048 and it's all around just a really good model and anyone can play with it right now. Over on Hugging Face, go to Hugging Face, type in Stable Cascade into the search box, select Stability AI Stable Cascade. Over on the right you can see the spaces that it's being used in. Let's go ahead and use the Multimodal Art Stable Cascade and from this page I can prompt any image I want, like a Mecha robot and favela by Tarsila do Amaral. Why do I always pick the prompts I can't pronounce? Or an astronaut riding a green horse? Or a delicious Feahota ramen dish? Or my favorite, a penguin holding a sign that says Wolf. Again, it's a really good model, probably the best model we've seen at being able to generate text within the images. But I did do a full video all about this, so make sure you check that out again. This would be big news on its own if it wasn't for the fact that so many crazy things happened this week. We also got chat with RTX from Nvidia. This is one that I want to make a dedicated video around. I just haven't gotten around to it yet because of all of the other crazy stuff that happened this week. If you look at this week, I've dropped more videos this week than most previous weeks because there's so much happening. So this is probably like a next week video. The chat with RTX is a large language model user interface that lives on your computer. It will work if you're offline. You don't have to be connected to the Internet. All of the files are purely on your own local system. It can use models like Llama and Mistral and I'm imagining more models coming in the future and you can add your own data to it. So you select a folder path with your data set that has text, PDF or doc files and then you can ask questions of the doc files that are inside of that folder, like what was the name of the restaurant that Sarah recommended? It looks through all of the docs that are in the folder that you specified and tells you the name of the restaurant that Sarah recommended is the Red Lei Bernardin. And then it gives you the reference text files of where that information was originally stored. You can also grab YouTube videos, plug in the YouTube videos in the YouTube URL data set, and then ask questions about that YouTube video. Like what GPUs did Nvidia announce at CES 2024? It it searches through that YouTube video and finds that Nvidia announced the RTX 4080 Super RTX.