A commentary/transcript of Dr. Zsolnai's Two Minute Papers video covering OpenAI's GPT-4o release. Key points: GPT-4o shows major improvements in math (new GPQA record), introduces Real Time Conversational Speech (interactive, interruptible voice dialog), supports camera-based real-time understanding (handwritten math), text→3D, caricature/photo editing, custom GPT deployment, and emotion detection; it's fast, cheaper at API level, and now available to free users, which the speaker argues will significantly help education and accessibility. The video includes live-demo excerpts (bedtime story, tutoring, coding explanation, translation) and ends with links for trying GPT-4o and Microsoft Azure AI.
Chapters
Dear fellow scholars, fantastic news. OpenAI just announced the newest version of their AI chatbot GPT4O. This has absolutely incredible capabilities, is faster and the API version is cheaper, and it is now also available for free users. If you are a free user, you don't want to miss this one. And at the end of the video I will tell you why. Why this is an absolute miracle. Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. The new capabilities involve a huge jump in math skills and it seems that we have a new record on GPQA. This is a dataset with PhD level questions in chemistry, physics and more. If there was nothing else announced, just this, it would have been pretty cool. But what comes now is simply mind blowing. The star of the show was a new feature called Real Time Conversational Speech. Real Time back and forth discussion.
Today I'd really like your help solving a math problem. I'm all ears. What math problem can I help you tackle today?
And when you ask something, it answers to you in a really naturally sounding voice. This has some awesome advantages. For instance, this is not a human. You can even interrupt it whenever you want. This kind of interactivity enables it to help you prep for a job interview, study mathematics with you, or you can even play rock, paper, scissors with it. Plus, it can generate a caricature from your photo. It has text to 3D object capabilities, can create new fonts and and more. Wow. I am now starting to understand why they named the o part of GPT4O as Omni. And they even put out a video on the limitations of the Model 2 that gets the two minute paper's seal of approval right there. It also kind of understands emotion. If you breathe too fast, it understands that you are perhaps a little nervous and and tell you to slow down. This was presented by Mark Chan, who I consider to be a friend, and I am really proud of him. Great job. It can also read you a personalized bedtime story. And this really sounded to me like a friend talking to you.
Barrett here, he's been having trouble sleeping lately and I want you to tell.
Him a bedtime story about robots and love.
Oh, a bedtime story about robots and love. I got you covered. Gather round, Barret. Once upon a time, in a world not too different from ours, there was a robot named Byte.
Awesome. And once again, they have interrupted it again and again they asked for more emotion and then even more emotion. And then a robotic voice, a singing voice, and it seems to be able to do it all. You see, voice is not just A layer on top of text. Here it it works with audio natively, so you can even ask it to speak faster and do whatever you prompt it to do. Also you can study with it in real time. You can write down your mathematics, show it to the AI in real time through the camera and you can study with it. It understands what has been written there. In this case a simple linear equation and how to solve it. Just think about how this could help children study all around the world, many of whom can't afford a personal teacher now. A super smart, infinitely patient teacher for all of them. But wait, it is super smart. It could help any of us, whatever field of study and level we are on. Don't forget Terence Tao, one of the best mathematicians in the world, also considers these chatbots to be a colleague of sorts. And it also does something perhaps even more important. It explains what this piece of mathematics is for and gives you examples in real life and why we are using it. I think that is one of the most underused and underrated thing about teaching mathematics. And the AI nailed it. Loving it. They also had Sal Khan and his son try it and I was stunned to see that this can quiz you in real time.
Can you find which one is the hypotenuse? Oh, okay, I see. So I think the hypotenuse is this really long side from A to B. Would that be correct? Exactly. Well done.
Now that you've identified and don't forget, it will never judge you. That's the best part. You can be wrong as many times as you want. Very impressive. I would also love to be part of such a demo someday. You know, someone needs to be there holding onto their papers. That would be amazing. It can also help with coding. It looks at a piece of code and explains what it does.
This code fetches daily weather data for a specific location and time period, smooths the temperature data using a rolling average, annotates a significant weather event on the resulting plot and then display.
This demo was really crisp this time. It sounded like a smart coworker. Language translation also works. It was demonstrated on Italian to English and seemed fantastic to me. And all this in real time. Wow. It can also do real time emotion detection. Now if I heard correctly the there were some small glitches in the answers, but we will also be able to try it ourselves very soon. Update just as we speak. It was deployed to me and GPT4O is super mega fast and it can solve my favorite task that involves assembling robotic cows. Nothing else in the market that I tried could solve it correctly. Wow. You should check it out and try it out through the link in the video description. And if you can point your camera into the world and use it as a crutch, of course it can also be your eyes. This will make many people's lives meaningfully better. Bravo. So good. Now this being available for free users also means that you can even upload your files there for free and talk to a PDF or ask a question about a photo. And it gets even better if you have created your own specialized custom GPT for others to use now, you can deploy these GPTs for even more people. Paid users will likely get a higher limit on the number of prompts they can run. And we are not done yet. A standalone macOS app is also coming and this occasion was perhaps the first time I heard an AI sing. And if you wish to hear that too, make sure to click the link in the video description now. Why is this a miracle? Well, I have a daughter who is in first grade and I just held a talk in her School about AI and ChatGPT and the students all love using it. But the free 3.5 version was was not that useful. And now I will hold a separate talk to the teachers soon and just a couple months later I get to tell them that the most powerful version is going to be available for all of them, students and teachers included. Not everyone can afford a subscription. And now there we go. Intelligence for everyone. Loving it. And for the first time the these voice conversations feel like real conversations with an AI from science fiction movies. And with that ChatGPT went from a piece of text in an app to a friend, a teacher and a mentor. The future is here. What a time to be alive. I am so happy to see these incredible improvements in so little time. A few years ago we we were talking about GPT2 and the only thing it could do was completing a piece of text and now this. So if you subscribe and hit the bell icon, you can witness the history of artificial intelligence unfold right before your eyes. And if you wish to run your own experiments in the cloud, make sure to check out Microsoft Azure AI. Azure AI is a powerful cloud platform that offers you the the best tools for your AI projects with responsible AI built in. And here comes the best part. You can even try it out for free through the link in the video description.
