The Hidden AI in Apple's new VISION PRO - Spatial Computing

So Apple has finally unveiled its long awaited mixed reality headset. But what you may not know is how much AI is involved in making this device possible. In this video, I'm going to break down everything you need to know about Vision Pro and how Apple is using AI in a subtle but powerful way to create the future of computing. Let's get started. So Vision Pro is Apple's new device that seamlessly blends digital content with your physical space. You can can see the world clearly through the headset, but also interact with apps, games, movies and more using your eyes, hands and voice. Apple calls it a spatial computer. It combines AR and VR experiences in one headset, letting you switch between the two using a digital crown on the side. You can also control your level of immersion while using environments which are immersive 3D spaces that you can customize and share with others. For example, you can create a cozy living room environment with a fireplace and a couch, or a few futuristic cityscape with flying cars and neon lights. Vision Pro is Also Apple's first 3D camera. You can capture spatial photos and spatial videos in 3D, then relive those cherished moments like never before with immersive spatial audio. Your existing library of photos and videos looks incredible at remarkable scale, and panoramas wrap around you, making you feel like you're standing right where you took them. It's powered by the combined might of an Apple M2 chip, a new chip called the R1. The M2 chip handles the general processing tasks, while the R1 chip handles the spatial computing tasks, such as tracking your head movement, eye gaze, hand gestures, and voice commands. The R1 chip also enables foveated rendering, which is a technique that renders the area where you're looking at in high resolution while reducing the resolution of the peripheral areas, which eventually saves battery life and improves performance. Vision Pro will be available early next year in the US for 3000 doll $499, with more countries coming later next year. It will be compatible with iPhone 12 or later models running iOS 17 or later. So essentially, it's an pretty awesome device that promises to revolutionize how we interact with technology and each other. But what makes it even more interesting is how much AI is involved in making it work. And that's what I'm going to talk about next. If you watched Apple's event where they announced Vision Pro, you may have noticed something interesting. They barely mentioned AI at all. They talked about spatial computing, machine learning, neural engine, but not a AI. Why is that? Well, it's because Apple has a different approach to AI than other tech giants like Google or Microsoft. Apple doesn't like to use the term AI too much because they think it's overused and misunderstood by many people. They prefer to focus on user experience rather than tech buzzwords because they want to make AI invisible and seamless for their users so they don't have to think about it or worry about it. Apple has a history of naming their products and features in a way that reflects their user centric philosophy. For example, they didn't call their smart speaker HomePod AI. They called it HomePod. They didn't call their face recognition system Face AI. They called it Face ID, and they didn't call their mixed reality headset AI Vision or something. They called it Vision Pro. But just because Apple doesn't use the term AI too much doesn't mean they don't use AI at all. In fact, Apple uses AI in many of their products and services without explicitly saying so. For example, did you know that Your iPhone uses AI to autocomplete your texts, transcribe your voicemails, recognize your handwriting, and suggest apps and actions based on your context? Did you know that your Mac uses AI to organize your photos, optimize your battery life and protect your privacy? Or that your Apple watch uses AI to detect falls? All of these features are powered by AI, but Apple doesn't make a big deal out of it. They just make it work for you in the background without you having to do anything. That's the invisible power of AI that Apple is using in its products, particularly Vision Pro. But how exactly does Apple use AI in Vision Pro? And why do they avoid mentioning AI in their product discussions? Well, Apple has a unique strategy when it comes to AI. Unlike other tech giants that like to boast about their AI capabilities and achievements, Apple likes to keep a low profile and under promise and over deliver. Apple's CEO Tim Cook has shared his thoughts on AI and how Apple approaches it. He believes that AI should enhance human intelligence rather than replace it, and that AI should only be used for things that benefit humanity. These statements reflect Apple's core values and explain why they don't talk much about AI in their product discussions. They don't want to create unnecessary excitement or fear around AI, nor confuse customers with vague terms. Apple wants to demonstrate the benefits of AI, and instead of just talking about them, they aim to improve their products and services using AI, focusing on machine learning, a subset of AI that allows systems to learn from data and improve without explicit programming. And essentially, machine learning is the secret sauce behind many of Apple's features and innovations. It enables Apple to create products that are smart, adaptive, personalized and secure, but also products that are simple, natural and seamless. Apple has been using machine learning in its products, particularly with the Neural Engine. This dedicated hardware component accelerates machine learning tasks on Apple devices powering the iPhone, iPad, HomePod, AirPods Pro, AirPods Max and Mac. The Neural Engine enables faster and more efficient machine learning tasks. It allows face ID to recognize faces quickly, even with glasses or masks. It helps Siri understand voice commands in noisy environments and and organizes photos in the Photos app. The Neural Engine is also part of the R1 chip in vision Pro. This chip handles tasks like tracking head movement, eye gaze, hand gestures and voice commands. It uses machine learning for real time, accurate performance such as adjusting focus based on eye gaze and recognizing hand gestures. One impressive feature of the R1 chip is Persona. It creates a personalized avatar in the digital world using machine learning techniques like face detection, alignment recognition, reconstruction, animation and synthesis. Persona allows customization and communication with other Vision Pro users, mimicking expressions, body language and speech patterns. Persona showcases Apple's use of machine learning for personalized experiences without explicitly labeling it as AI. It is marketed as a creative way to express oneself digitally. When I say that Apple's approach to AI is very different from other tech giants like Google or Microsoft, I mean that these companies are more vocal and transparent about their AI capabilities and achievements. They often showcase their AI products and features in their events and announcements and publish their AI research papers and open source their AI tools and frameworks. Google and Microsoft are more focused on advancing the state of the art of AI and pushing the boundaries of what AI can do. They are more interested in solving hard AI problems and creating general AI systems that can perform any task across any domain, which makes them more ambitious and visionary in their AI goals and aspirations. Apple focus on creating their own hardware, devices and chips, giving them an advantage in integration and optimization. They prioritize on device AI over cloud based AI, believing it's faster, more efficient, secure and respects user choice. Apple has invested in on device AI capabilities like the Neural Engine and Core ML framework. However, they face challenges with limited data access, collaboration and diversity in their AI endeavors. Apple has also been experimenting and researching new conversational AI techniques and methods for years, such as neural networks, generative adversarial networks, reinforcement learning, deep learning, natural language understanding and more. Apple CEO Tim Cook recently shared that he uses ChatGPT and finds its possibilities fascinating. During an interview on Good Morning America, Cook mentioned that he finds it helpful. However, it's interesting to note that Apple has now banned its employees from using ChatGPT following a similar move by Samsung. This decision was made because Apple is concerned about privacy. When asked if he personally uses OpenAI's ChatGPT, Cook enthusiastically confirmed, oh, of course I use it.

Yeah, I'm excited about it. I think there's some unique applications for it and you can bet that it's something that we're looking at closely.

According to Cook, AI tools like ChatGPT and Google Bard show a lot of promise, but they also carry risks such as bias, misinformation and potentially even worse outcomes in certain cases. He stressed the need for regulating AI and implementing safeguards. Cook recognizes the power of AI and acknowledges its rapid advancement. So while Apple downplays the use of the term AI, it obviously plays a vital role in their products, including Vision Pro. And while Apple's commitment to user centric design is commendable, a more open and collaborative approach to AI could foster further innovation and advancements. Balancing privacy, ethics and the potential of AI remains a challenge for Apple and the industry as a whole. I hope you enjoyed this video and if you did, please give it a thumbs up and subscribe to my channel for more AI and technique news and reviews. Thank you so much for watching and I'll see you in the next one.