AI News: ChatGPT Gets Overhauled & Microsoft Reveals Massive Updates!

So it's been an absolutely insane week in the world of AI. We've got announcements out of OpenAI, out of Google, out of Microsoft, out of Meta, out of all of the AI art generators and all of the video generators. Everybody decided to drop new announcements and roll out new features this week. So there is a ton in this video. I'm going to try my best to rapid fire it, not ramble too much and share all the crazy AI news that happened with you this week. So let's just get right into it. Let's start right here with OpenAI. This week was OpenAI's Dev Day. For the most part, most of the announcements were more geared towards developers and not really end users of Chat GPT. But there was still some interesting stuff that came out of Dev Day. Sam Altman did a little Fireside chat at the end of Dev Day and allowed the audience to ask questions. And a lot of the questions were about when are we going to get AI agents? And according to this article over on tomsguy. Com, it says OpenAI confirms AI agents are coming next year. OpenAI is on target to launch agents next year. These are independent artificial intelligence models capable of performing a range of tasks without human input and could be available in ChatGPT soon. Now, as far as I know, OpenAI hasn't actually released the recordings from Dev Day yet. However, I did find this YouTube channel called Kyle Cabasaras. I'm sorry if I mispronounced your name, where he actually filmed the entire sort of Fireside chat with Sam Altman. And the closest thing I can find to them saying that agents will be here by 2025 was this little clip here.

Maybe talk to us a bit more about how you see agents fitting into OpenAI's long term plans. Are you a huge part of the.

I think the exciting thing is this set of models 01 in particular and all of its successors are going to be what makes this possible because you finally have the ability to reason, to take hard problems, break them into simpler problems and act on them. I mean, I think 2025 is going to be the year. This really goes big.

Yeah. So they did mention from stage that they think 2025 is going to be the year and Sam Altman kind of sort of confirmed it. That was the only clip that I can find that seemed to kind of confirm that they think agents by 2025. Again, I don't think the full video is online, but if you do want to watch this entire Fireside Chat, they did check out Kyle's channel here. I'll make sure it's linked up in the description as well. I'm going to talk more about the Dev Day announcements in a second, but before I do, let's talk about Canvas, which is actually a ChatGPT feature that they just rolled out this week as well. And Canvas is kind of a complete overhaul of the UI. Inside of ChatGPT, we can see that Chat GPT will start suggesting edits. It'll adjust the length to be shorter or longer chat, change the reading level, add final polish and check for grammar and clarity, and you can even ask it to add emojis. They've also added some coding features with this new Canvas, like review code, add logs, add comments, fix bugs, or port to a different language so you can Go from like JavaScript to Python or whatever. And according to Sam Altman here over on X, he said that the new Canvas feature is Now Live for 100% of Chat GPT plus subscribers. So if you are a paid subscriber to ChatGPT, you should have this feature now. Now, when I pop into my ChatGPT account, it defaults still to ChatGPT4O, but if I click this dropdown, you can see we now have an option to switch to GPT4O with Canvas. We don't get Canvas with the new Zero1 preview mode that thinks through things, but we do have it with the GPT4O model. So if we click into here, we now have the ability to call on the Canvas. If I just give it a prompt like write a short story about wolves learning to use a computer and hit Enter, you can see that it just completely changed whole interface and it put my chat over here on the left sidebar and it put the story that it just wrote over here in the right window. Now, if you use Claude and you've seen Claude's artifacts, this feels very similar. The biggest difference is in Claude artifacts you can actually have it generate code, and then you can actually preview what the code does without actually going to another screen. This will just output the code, but. But you still have to copy and paste the code over somewhere else to actually execute the code. Now, what this did when it created this new right side of Chat GPT is it made it so I can sort of select text here, and when I select text, I can ask Chat GPT about just that paragraph. I can tell it to rewrite just this paragraph, and if I submit it, you'll see that it will actually automatically fix just that paragraph and leave the rest of the document Alone. In the past, you, you would have basically had to tell it to rewrite the whole thing, but just fix the first paragraph and it would have completely done the whole thing over again. If I select this text, I can also format it, you know, make it bold, italic, change it to headings, things like that. And then over on the right sidebar, you can see this little pencil icon down here. If I hover over it, it opens up a new menu here with Suggest Edits, adjust the length, reading level, add final polish, and add emojis. So if I click Suggest Edits, it actually reads through the story that it wrote and then suggests its own edits to its own story. And if you like the edits that it's suggesting, just like in a Google Doc or something like that, you click Apply and it actually tweaks it and edits it with the new version that it suggested to itself. I can click on Adjust Length. And we've got a little slider here to make it super long or super short. So if I go to shortest and just let go, you'll see it'll just rewrite the whole thing, but in a much shorter version. If I click adjust the length, bring it all the way up to longest, it reworks the entire story, this time making it quite a bit longer. We can change the reading level with a slider as well, all the way down to a kindergarten level, up to a graduate school level. So if I bring it down to a kindergarten level here, you can actually see that the writing style is more like a kid friendly writing style. Deep in the forest, under the tall trees, the wolf pack found something strange, et cetera, et cetera. If I change the reading level all the way up to a graduate stud school level, you can see it's much more descriptive. Deep within the dense forest ecosystem, beneath the expansive canopy of towering conifers, a wolf pack encountered an anomalous object. So it completely changes the reading level. So if you're trying to get it to explain some sort of complex concept to you and you're not quite understanding it, you can essentially tell it to dumb it down for you and keep on dumbing it down until you finally get it. You can add some final polish. You can see it goes through and formats it. We got like a headline and some sub headlines and it just sort of broke it up and cleaned it up so it's a little bit easier to read. And then if we click Add Emojis, it will do exactly what you expect it to do and just fling some emojis in there as well. And yeah, make it look like that. I'm gonna go ahead and clear this and go to a new chat window because there's something else that they've recently added that's pretty cool. If I come down to the chat window here, they actually have like quick shortcuts. Now if I type slash, you can see I've got reason, search and picture. So if I select picture, anything I put after this, it will call on Dall E3 to generate. If I do search, it will make sure that it searches the web before it finishes the prompt. And if I click on Reason, it's going to make sure it uses the new Zero1 model that really thinks things through. So some pretty handy new updates to Chat GPT this week. All right, jumping back to some of the other stuff they talked about at Dev Day, I'm going to kind of go quickly through it because this is more designed for the developers that are using the API. But they did introduce vision to fine tuning the API, so developers can now fine tune GPT4O with images and text to improve the vision capabilities. So if you're using the OpenAI API and you want to build a tool that uses their vision models, you can actually upload some of your own images and give it some additional context and it will actually get better on the specific type of images that you train into it. Again, something more focused towards developers. Another thing focused towards developers is that they introduced the Real Time API. They recently rolled out the advanced Voice mode where you can have a much more conversational chat with Chat GPT. Well, they rolled out the ability to use those conversational bots inside the API. So now other apps can use that same technology within their app. And if you want, you can even test this yourself outside of the Chat GPT app. You can actually go to the OpenAI playground here, platform.OpenAI.com playground and over on the left they added a real time box here. And I can actually start a session and have a conversation with the GPT4 model here using this sort of new advanced voice features, but outside of ChatGPT. Hey, how are you doing today?

I'm fantastic, thanks for asking. How about you?

I'm doing great. Just recording a video, breaking down all the news for the week in the AI world.

That sounds exciting. There's always so much happening in the AI world. Any particularly big stories you're covering this week?

Oh, wouldn't you like to know?

Fair enough. I'll just have to wait for the video then.

They also rolled out model distillation in the API. This lets developers easily use the outputs of frontier models like Zero1 Preview and GPT4O to fine tune and improve the performance of more cost efficient models like GPT4O mini. They also added prompt caching into the API. This is something that significantly reduces the cost of using the API if you're a developer. This is something that Claude's had for a little bit now, but we're finally getting it in the OpenAI API. And those were pretty much the big announcements that came out of OpenAI's dev day. But there's other OpenAI news this week in the fact that they also got new funding to scale OpenAI. Now it's looking more and more likely that OpenAI is going to convert from a nonprofit to a for profit company. And in this past week they managed to raise $6.6 billion in funding at $157 billion post money valuation. This makes them the third largest startup on the planet, I believe. Last week we got Meta Connect and they showed off some of the new Meta Ray Ban sunglasses. During Meta Connect, they announced a new feature that these glasses were going to be getting memory so you can look at something and say like remember where I parked? Or hey, remind me in 10 minutes to call my mom or whatever. Well, those features are rolling out right now into the sunglasses and glasses that you have. If you've got a pair, the new update will also allow them to recognize QR codes and open them on your phone and make phone calls based on a phone number that is seen in front of the camera. I actually tested these features. They work great. You look at a QR code and just by telling your glasses to scan this QR code, you can pull out your phone and it just opens the app on your phone real quick. Pretty handy. The memory feature is really cool too. I actually haven't tried it with an image yet where I take a picture of like a parking spot number, but I have tried it where I told it to remind me to do something in five minutes and then that reminder came through. Pretty handy feature. And while we're on the topic of.

Meta for today's video, I partnered with Meta because they just released llama 3.2 and it's a significant leap forward in AI technology.

Whether you're a developer or you're just.

Simply interested in AI, this update is worth your attention.

One question I get asked constantly is.

How can I use an AI model.

Without actually sending my data to like the big corporations?

Well, Llama 3.2 has a pretty solid answer for you. You can run it directly on your device. In fact, you can use it without even being connected to the Internet if you want. So what's new in llama 3.2? Well, first the larger models 11B and 90B, they both now have vision capabilities. This means that not only can they understand text, but they can understand images now as well.

You can ask the AI about a chart in your report or to describe.

A photo and it would understand the visual context. But Meta actually put out some lighter weight models as well with 1B and 3B text only models. These are perfect for on device AI, AI applications, and even on mobile phones. Imagine a personal assistant that can summarize your messages or manage your schedule, all while keeping your data on your device. The 8,000 token context window. That's like being able to put a whole book's worth of information inside of a single conversation. This llama 3.2 is actually optimized for Qualcomm and Media Tech hardware right out of the gate. Now this is super important for anybody that wants to actually develop AI powered applications for mobile phones. And one of the most important aspects of these Llama models is that they're open source. So you can download these models from llama.com or from hugging Face and start building immediately. They're compatible with platforms like AWS and Google Cloud and Microsoft Azure and a ton of others. Now personally, I believe open sourcing these models is really, really important. It encourages innovation and it allows for more diverse applications. This year alone, llama has seen 10x the growth and has become one of.

The preferred large language models for AI development.

And if you're a developer, this means.

That you have access to the most.

Cutting edge models that you can modify and adapt to your specific needs. And while if you're not a developer, it means that the apps and services.

That you're going to use could be.

Significantly more intelligent and more helpful. I've also got to mention that Meta really prioritize safety.

With this new release, they've introduced new safeguards including Llama Guard 3, which is.

Designed to ensure that these powerful models.

Are actually used responsibly.

So whether you're aiming to create innovative AI applications or you're simply excited about the future of technology, llama 3.2 is actually really worth exploring. It's powerful, it's really flexible, and it's available for everybody to use and improve upon and iterate on. So if you're ready to get started, you can go to llama.com or you can go to Hugging Face and download the models to begin your journey with llama 3.2.

The future of AI is open source.

And it's here now. Thank you so much to Meta for sponsoring this video.

All right, moving on to Microsoft now. There was a ton of announcements to come out of the world of Microsoft, especially if you have one of the new Copilot Plus PCs with the NPU Neural Processing Unit built into them. These are pretty much all of the new laptops and computers that are coming out from Microsoft these days are this new version of the copilot PCs. Some of the new features you're going to get are the recall feature. This was a feature that was supposed to roll out when the new copilot PCs rolled out, but a lot of security and privacy concerns popped up and they sort of put it on the back burner for a little bit to fix it and improve some things. And now they're finally rolling out this recall feature, which is essentially like your Internet browsing history, but for everything you do on your computer. It remembers you editing videos or writing documents in Word or browsing through your photos. Anything that you did throughout the day, it sort of saves it as a history so you can go back to that moment and remember what you were doing on your computer. And yes, you have the option to turn it on and off. And no, they don't actually send the information that they're collecting back to Microsoft. This is all just on device. They're also adding this click to do feature where if you have like an image open on your computer, you can click on it and you can see. It gives the option to visual search with Bing, Blur background with photos, erase objects with photos, remove background with paint. So just by clicking on the image you get a whole bunch of new sort of AI related options. It says here. It also assists with text related actions such as rewrite, summarize or explain text in line, opening in a text editor, sending an email, web searches and opening websites click to do is context aware and accessible from any Copilot plus PC screen. They're also improving the Windows search with some AI. You can see they did a search up in the top here where they searched Barbecue party. Notice that all of these images are just titled like Image 1123 Image 1111. Windows figured out what the context of these images were and pulled up all the images related to a barbecue party that were on this computer and it says here it works even when you're not connected to the Internet. So this isn't like an online feature. This is just going to use your laptop's NPU that's built into it. I'm not sure if this works with videos or if it's really only for pictures. I really want this for video. That would help so much with organizing B roll, but getting it with pictures, I imagine it's only a matter of time before we're getting it with videos as well. They're adding a feature called Super Resolution Inside of Photos. So you can open a image inside of Photos on Windows and and actually upscale the image. They're adding Generative Fill and Erase inside of Microsoft Paint so you can erase things in the background and Generative fill inside the image just like you can in Adobe Photoshop. But now you could do it in Microsoft Paint as well. So a lot of cool new features that are going to be available on these Copilot Plus PCs. Microsoft also introduced Copilot Labs and Copilot Vision. The first feature available in Copilot Labs is Think Deeper, which gives Copilot the ability to reason through more complex problems. It sounds to me like this thinking Think Deeper is essentially going to use the new OpenAI01 model that uses that chain of thought prompting where it really thinks things through. But it looks like we're going to get that inside of Copilot Labs. There's also Copilot Vision. It says if you want it to, it can understand the page you're viewing and answer questions about its content. It can suggest next steps, answer questions, help navigate whatever it is you want to do and assist with tasks. All the while you simply speak to it in natural language. And they say it's an entirely opt in feature. So it only works if you turn it on. But here's a little demo that they put out of what that looks like. They say, hey, Copilot, I'm looking for a place to stay. They're on this website, Stay Nest.com copilot starts making recommendations. What do you think of this loft house? Now in the video, they're speaking back and forth, but there's music on the video. I don't know the copyright status of that music. So I'm not playing the video for that reason. But this is an audio conversation that's happening. The user says, hmm, it's a bit pricey. The AI calls that person Bougie and they say, I'm not. I'm just looking for something nice, a little color on the walls, you know. And then the AI says, this one definitely has some color. The user says, wow, it's giving me a headache. Haha. We don't want that. Wait, this one looks perfect.

Minimal, modern.

Ew, the user says, you're right. I love it. We're booking it. So that's their little demo of what this Microsoft Copilot vision looks like. Microsoft also updated the Bing Generative Search feature. They say Today we're rolling out an expansion of Generative Search to cover informational queries such as how to effectively run a one on one and how can I remove background noise from my podcast recordings. Whether you're looking for a detailed explanation, solving a complex problem, or doing deep research, Generative AI helps deliver a more profound level of answers that goes beyond surface level results. To use it, you simply type Bing Generative Search into the search bar and you're met with some queries that you can use. There's also a deep search button on the results page and they do say it might be a bit slow right now. Let's just try Bing Generative Search. And sure enough, when we test it out here, we get a whole bunch of different potential prompts that we can use. I click on Reduce Podcasting Noise. You can see we get an AI generated response here along with a table of contents. Another thing Microsoft is doing is they're starting to pay publishers if their content is surfaced in some of these generative search results. Right now it looks like it's just big companies like Reuters, Axel Springer, Hearst Magazine, USA Today, and the Financial Times. I'm not clear if this is going to be something they roll out for smaller content creators, because that'd be kind of cool. If you write blog posts or make YouTube videos or something like that, and it responds with information that it pulled from smaller creators, it'd be cool for them to get compensated as well. I don't know if that's on the roadmap or not, though. Right now it looks like all of the big news media outlets, though, they're trying to work with them so that they can show results from their websites and also pay them when they do. And in the last bit of Microsoft News for the week, the head of Microsoft AI, Mustafa Solomon here wrote a letter sharing his thoughts on where he thinks all of this is going. And what he describes is essentially Copilot turning into more and more of an agent for you. He says Copilot will be there for you in your corner, by your side, and always strongly aligned with your interests. It understands the context of your life while safeguarding your privacy, data and security, remembering the details that are most helpful in any situation. It gives you access to a universe of knowledge, simplifying and decluttering the daily barrage of information and offering support and encouragement when you want it. Over time, it'll adapt to your mannerisms and develop capabilities built around your preferences and needs. We are not creating a static tool so much as establishing a dynamic, emergent, and evolving interaction. It will provide you with unwavering support to help you show up the way you really want in your everyday life, a new means of facilitating human connections and accomplishments alike. Copilot will ultimately be able to act on your behalf, smoothing life's complexities and giving you more time to focus on what matters to you. So what he's describing essentially sounds like an AI agent that's trained on you and what you want to use it for most, which I think is something that most people can probably get behind as long as it's done in a safe and ethical way without impeding too much on your privacy or sharing too much personal data with the big companies. All right, moving on to Google, because Google Google had a handful of announcements this week as well. They've been making some updates to the Google Lens tool, a tool where you can upload images and have it sort of search out those images of where they are in the web and give extra information around those images, things like that. Well, now it can actually understand videos as well. We can see in this demo here some fish schooling, and they talk to it and they say, why are they swimming together? And it looks at the video, understands what's in the video, and then actually gives an AI response based on what it saw in the video. They're also adding that voice questions feature where you can talk to Google Lens. So they take a picture of the sky here and then they say, what kind of clouds are these? And then it gives them an AI response. But they asked that question vocally. It wasn't them typing the question. They're also adding this feature to shop what you see. So you see a backpack, you take a picture of it, and then it finds where you can actually purchase that backpack online. They're adding the ability to identify songs in their circle to search. Kind of like the Shazam app, where you just hold it open, listen to a song, and then it tells you what the song is. It sounds like that exact feature is going to be rolled out into Android devices. They're also going to organize your search results using AI. And if you're still using Google to do a lot of your searches, you're probably going to start to see some of these changes take effect pretty dang soon. But Google makes most of its money through advertising, and so in this new world of AI, they have to figure out how to make money off of the AI responses as well. So we're going to actually start seeing ads inside of the AI overviews. We can see right here that it showed some sponsored messages. So somebody searches how do I get a grass stain out of jeans? And then it gives an AI response of how to do it. And then when they scroll down a little bit, you can see right below the response there's some sponsored results for things like tide pen and OxiClean related to their search. And in large language model news this week we also got a new version of Gemini Gemini 1.5 flash 8B. This is a new small large language model that's 50% cheaper, 2 times higher rate limits and lower latency on small prompts. This is really for developers that use the API here and on benchmark tests it looks like it performs pretty well compared to other models in a similar size. Since we're talking about large language models, Nvidia announced a new large language model this week called NVLM D72B and this is an open source large language model that is also capable of vision tasks. And according to this article, it rivals the leading proprietary models like GPT4O. And if we look at the benchmarks here, we can see that this NVLMD 1.072B is actually pretty on par with GPT4 Vision model. And in one benchmark even outperforms GPT4O and Claude Sonnet here, which is pretty impressive given the fact that this is an open source model and not a closed model like anthropic or OpenAI or Gemini. Pinterest is rolling out generative AI tools for product imagery to advertisers. Pretty much the same thing we've seen in tools like Shopify and Amazon. You can upload an image of your product and it can remove the background or put it in a different scene, things like that. We've seen this roll out in all sorts of e commerce platforms at this point. Well, now you're going to get it directly inside of Pinterest. There was some huge news in the world of AI imagery this week. Black Forest Labs released a new model called Flux 1.1 Pro and they also made their API available. So Flux 1.1 Pro, you can use it right now over on Together AI Replicate, File AI and FreePik, and it's quite a bit improved if you've Seen things on like Twitter or X where people are referring to Blueberry. Blueberry was sort of the code name for Flux 1.1. Here's a little comparison that my buddy Angry Penguin put together. You should definitely be following him over on Twitter if you're not already. He shares all sorts of cool AI announcements, but we can see the difference here of Flux Pro. The old model Find me the stars align. The new model, Find me where the stars align seems to be much better with text. You can actually see his prompt here if you want to duplicate that. Here's another one. Sky's the limit. Sky's schlint. Here's another one. Sky's the limit. So it's much more understanding of what you're looking for, at least from the tech side of things. Here's another example. Feel free to pause on any of these if you want to look at them more closely and grab the prompt. But we can see that the text and even the image is quite a bit better. In this image we can see the barbell is kind of going into the cat's head or maybe behind it. I can't really tell this one. You got the whole thing on the screen and the text is exactly what was asked for. Here's some more examples. Here's another example of a Ghibli style old Japanese city. Blue sky, sunny background. Japanese temple, Japanese traditional. Here's the original one. Here's the new one. Looks quite a bit better. A lot better color palette in my opinion. Obviously what's aesthetically pleasing is very subjective, but I find this to be a little bit more aesthetically pleasing. Here's another example. We can just really see how prompted here it is because look at how big this prompt is. A vector illustration. A group of adorable smiling ghosts wearing different color witch hats. We're getting that each ghost has a unique expression. Cheerful pumpkins with carved faces. Background should be dark purple. It pretty much nailed every single element from this long prompt here. Here's another example. In the first one, it didn't even get the eye lift to eat. And the second one, it kind of nailed it. Maybe one of these straws is meant to be the eye. I don't know. A handwritten letter written in Old English and signed Flux Pro at the bottom. Look at that. Here's another example and another example. And one final example. So thanks again to Angry Penguin for sharing all of those with me and giving me permission to share them in this video. Feel free to go back and pause on any of those if you want to See the specific prompt that I didn't cover? Angry Penguin also gave me a quick tip. He said that you can actually use Flux Pro 1.1 right now for free if you use it over on the Glyph app website. Some of the other sites that are allowing you to use it for free are only allowing you to use it for free for like a day. This one, it's free for now and from what I understand it'll be free for a few weeks, but I don't know how long it's going to be free for. But at the time this video is going live, you can actually play around with Flux Pro 1.1 for free on Glyph. So let's go ahead and sign in real quick. Let's just try a monkey holding a sign that says subscribe to Matt Wolf. It got the To Matt Wolf part but kind of missed the subscribe. Let's run it one more time. This time it got it right with no issues. I actually love making images over here on Glyph because they show up in this feed down here. And now anybody that goes and looks at this feed is going to have a monkey telling them to subscribe to Matt Wolf. But That's Flex Pro 1.1 and I will link up where you can use that below. You can go use Angry Penguins integration of it, or you can go build your own glyph with it in the workflow. We also got some updates out of Leonardo AI this week. Now this is a company that I am an advisor for, so just keep that in mind when I talk about them. But when they do really good stuff, I talk about the good stuff. When they do stuff that I don't really like, I point out the stuff that I don't really like. So I try to remain fairly unbiased. But this is just the news. This week they rolled out a new style reference feature. You can upload up to four reference images to direct the esthetics of your image output. You can also adjust the strength of the reference image. They also rolled out a new Image to Image feature using the Filter Phoenix preset. They've had Image to Image in Leonardo for a while, but the feature wasn't available to use with the Phoenix model, which is the model that's probably the best inside of Leonardo. But now Image to Image is available using the Phoenix model. So if I jump in here, I.