Dr. Zsolnai breaks down Apple Intelligence, the on-device AI features Apple just announced. You can remove objects from photos, generate images in Notes, get email summaries across apps, and have Siri read your calendar to handle tasks. Most runs locally on your phone, though you can also tap ChatGPT from within apps.
Wow. Apple just put up a masterclass on AI or what they call Apple intelligence and goodness. I did not plan a video on this at all, but they absolutely knocked it out of the park and you simply have to see this. Dear fellow scholars, this is two minute papers with Dr. Zsolnai. It was such a joy to see how the papers that we talk about here are finally coming alive. Now they put up a clinic on how to take these amazing tools and showed how to integrate it into a system. For instance, you will be able to invoke image inpainting to remove unwanted details from your photos right in your photos app. When writing up your class notes, you will be able to have an AI create new images based on the context of what you have written. That is is an absolutely amazing idea. If only I had access to a tool like that back in school and the new generation will be able to study like this. Wow. And it gets better. It even learns your style and smooths your writing. I wrote a paper on doing a similar thing before. It is not easy. Or what about asking the text to image AI to look @ who you are talking to and make a sketch of them. And whenever you read an article in your browser with one click, it summarizes it for you and you can decide whether it has what you are looking for. But the point is that it works on text anywhere really, because it is in your email and other apps as well. You see, this is not AI on top of the operating system. It is part of the operating system and that changes everything. You can ask if you're gonna make it in time for your daughter's performance. It reads the email for you, understands who your daughter is, where and when the performance is from the calendar and use maps to estimate whether you will make it in time. When filling out an online form, it pulls up your driver's license from your photos and types the data in for you. And hold on to your papers, fellow scholars, because it might even make your email experience a little more joyful. You see, when you get a new email, you will see a one liner summary. So you will know what the email is about without even opening it. Wait, let's not stop there. If it can understand and summarize these emails, then it can also prioritize them and tell which one is urgent and highlight them. That was reading, but it helps with writing too. This is the best one. If you feel a little grumpy and wrote something that you shouldn't have, that's okay, it will rewrite it in a much more professional manner. Or if you are that one person who always has those perfectly worded emails, you can just ask it to proofread it and it explains exactly what it did and and why. And if you're on the go and you have to whip up an answer really quickly, just look at this way of doing that. A couple taps. This is a thing of beauty. Loving it. And when you get a pile of notifications it can summarize them and prioritize those too across several apps. I also love how it understands context and what you are looking at currently. You saw it with the notes example, but for instance if you are talking to someone you can also say add this address into his contact card. So this is an AI that doesn't just answer in text, it actually performs actions for us. That is huge. Why is that? Well, because you don't need to think in terms of apps anymore. You just say what you want to your phone and it does it and it will make life a little bit easier for all of us. Now note that all of this is Advertisement Take everything with a small pinch of salt. We actually have to see how well it works in practice. And when it comes to text to image capabilities, it will be able to create AI generated emoji for you or even fun little images of yourself too. My understanding is that all of this happens on your device. I keep talking about running the AI in your pocket and here it is. What a time to be alive. And if you have asked for something a little more complex, you might get an offer to invoke ChatGPT itself from your iPhone and you will even be able to compose text with it right from your apps. Other AI systems I would imagine Gemini will also be available later down the line. This is an important step forward. Previously you had to come to the AI to enjoy it. You had to register and download an app. Not anymore. Here the AI comes to you and it comes to help millions and millions of people all around the world. OpenAI figured out the technology and and Apple figured out the usability. And this is the point where I truly feel that helpful AIs are brought to the masses. And yes, the new iPad calculator is absolutely magical. And if you wish to run your own experiments in the cloud, make sure to check out Microsoft Azure AI. Azure AI is a powerful cloud platform that offers you the best tools for your AI projects with responsible AI built in. And here comes the best part. You can even try it out for free through the link in the video.
