Sam Altman Reveals The Future Of AI Agents, Digital Humans And Al Brains

It's no secret that AI agents are going to be the future. And in this video, I want to show you guys some early exploration of agents that are currently going on and a few statements from Sam Altman that reveal how close we are to agents. So today's video, I'm going to start with a recent blog post where this company called Altera uses GPT4O to build a new era of of human collaboration. It's actually quite fascinating, so let's dive into some of the details and, of course, the future implications. So this is the company, it's called Altera AI, and they start this article by talking about the previous work that the CEO has done. It says in 2023, when OpenAI's language model became broadly available, you, Yang quit his job as an assistant professor at MIT to start Altera Al, a research lab focused on what they call digital humans, which is a new way for people to interact with agents that will have fundamentally human qualities. So this company, Altera AI, is a company that's focused on developing AI agents that exhibit realistic and believable behaviors, which is going to be pretty crazy for the future. So they were founded in December of 2023, and one of the key things that they want to do. You can see right here that the CEO envisions a future where AI agents don't just assist, but soon he believes that they will interact and collaborate with humans and, crazy enough, even experience emotions. Now, along with his three co founders, Dr. Andrew, Nico and Shuying, they've built Altera's first product with GPT4O, the first autonomous agents that can play Minecraft with you just like a friend. Now, I think this company is pursuing something that is obviously for the future, and I think the implications for this are a lot more different than the implications for really smart AI models. I think autonomous agents that are essentially digital humans are going to be a lot more impactful than the majority of people think. And I think once agents get their chatgpt moment, I think that's when the AI race once again kicks off in terms of the global consciousness becoming aware of what these systems can really do. Now, Altera AI has raised significant funding, including a $9 million seed round backed by prominent investors. And they're basically just supporting the company's research and development efforts as it seeks to expand the capabilities of of its digital humans. And what's actually quite interesting about this company is that it actually uses advanced AI algorithms inspired by neuroscience to build its digital humans. The company systems neuroscience composite architecture actually mimics the human brain functions such as memory and social cognition, allowing for more human like interactions. And I'm going to dive into that later where we've got this huge diagram that explains how it all works. Now we can see here that this is where they talk about long term autonomous autonomy for their AI agents. It says, just as automation helps increase human capacity by supporting repetitive tasks, digital humans, the Altera team believes, will be able to collaborate productively and even form bonds with people. And here's where he states two of the things that they're going to look like. So the first thing is that they're going to be digital co workers who can collaborate for days or weeks proactively to solve problems. If you remember how Sam Altman spoke about agents before and about future issues in the future, where you might ask an AI to go off and solve the Riemann hypothesis or develop a really specific cure for a drug, or maybe even to build you a business or some kind of product in a certain niche. I think this is probably going to be the future of AI when we get AIs building software companies and a variety of different products and services. Of course, again, we can see that there is the point two which says that there is long duration, multi agent worlds where we can measure responses to economic policies, advertisements and more. Now I think this one is arguably the most underrated. And if you don't know what this is, this is basically where they are going to essentially simulate reality and, and then use that simulated reality to predict what happens. So for example, let's say you are someone who works in advertising and you want to run an advert. When people run adverts, what they usually do is they will take an advert, but they'll have 10 different variations of it. So for example, if I can explain this in easier terms, you might have a YouTube thumbnail and some YouTubers like myself, we might have three different thumbnail designs and we have to think really hard about which one we're going to put as the video because it's going to determine how well that video is going to perform in the future. It's quite likely that we won't have to think about which one is going to perform best. It's likely that we'll submit that thumbnail to an AI audience that resembles humans and then we'll use that data and then put it out into the real world and that's how things will be done. The same can be done for economic policies, advertisements and other industries where human feedback is valuable rather than wasting time to deploy it in certain human environments, it's going to be wise to deploy it to a similar AI agent audience and then of course use that to sort of pre vet what kind of things you do run. And I think that's going to be something that is quite difficult because humans are constantly dynamic, things are constantly changing. But I do think it's going to be interesting to, to see how that change happens. So one of the issues that Altera actually ran into was this issue of the phenomenon of data degradation. The problem that plagues all AI models making autonomous decisions in extended timeframes. Basically, AI agents interact with the real world, making decisions in real time, but as their own output becomes their future input, the data quality unfortunately degrades over time. This is an issue that most AI agent systems encounter. And but for our digital humans, who are meant to live autonomously for hours or even longer, this becomes one of the most pressing issues to solve. So this is where they talk about how they managed to solve this. So it says that to combat data degradation and to increase the long term autonomy of their AI agents, Altera turned to OpenAI's large language models, which proved pivotal in maintaining the integrity of decision making progress. OpenAI's advanced models allowed Altera to build the first AI agents that can play games with people just like their friends. And these agents achieve longer, more complex interactions without the rapid decline in performance that had been limiting the AI agents potential. So now you can see that as these models tend to get better, which they do over time, they also subsequently increase the AI agents capability and long autonomous executions on certain tasks, which is amazing. So what we also need to look at now is the architecture. By far the most fascinating piece of Altera is this human mimicked architecture that is a parallel multimodule system that mimics the structure of the human brain, including that of the prefrontal cortex. And the company has been able to create agents capable of simulating cognitive functions. They state that our composite system combines various modules in parallel, each powered by OpenAI's module. And these modules are inspired by brain functions like attention, bottleneck, working memory and social cognition. So this entire diagram illustrates a complex system where thinking processes are influenced by needs, of course by the personality, of course by the mood, and of course the high level motor plans. You can see that all of these different things affect how you think. The thinking process, of course, then generates your intent, which leads to a high level motor plan. You can also see that the memory system feeds into a summarization process which then updates the Goals and these goals then influence both the thinking and memory process. This structure actually seems to mimic the cognitive process basically representing this AI system that is designed to simulate human like thinking. I think in the future this is probably going to get even more complex and even better overall when we do get things like infinite context windows. I think overall this architecture seems rather promising considering how effective these agents have been so far. I mean having all the memory you have got, the working memory, the social memory, the short term memory, the long term memory, all of these things are things that AI just doesn't have. And I think people forget how different stateless models like ChatGPT are compared to autonomous AI agents. So for those of you who are in the AI safety area, if you're ever wondering about how dangerous AIs are going to be, I think AI agents are going to be largely the biggest area for any safety expert. This is because if an AI system has long term memory and long term goals and is able to execute on those goals, and it's definitely going to be a lot more dangerous as we don't know how to check those and it's going to be a longer time in order for us to see if these systems are safe or not. Now you can also see right here that they talk about the current benchmarks. It says as of mid-2024, Altura's digital humans can operate autonomously for up to four hours at a time, a substantial increase compared to to other AI models on the market. On the long term benchmark on distinct item collection In Minecraft, Altera AI performs at 32% of total distinct items compared to previous ones like Voyager which were remarkable at 6.4%. And autogpt, this is a remarkable improvement. And I can't wait to see what happens once we get even better models that have more capabilities baked into the model. I can imagine with future models for example, GPT5 actually surpassing this area and getting to around 60 to 70%. And then of course future models by other companies managing to even perform even better. Now you can see right here that the early success of Altera's vision has generated widespread interest, particularly for their first project, a digital human that can play and interact in virtual environments like Minecraft. And it's also laying the foundation for future and more complex multi agent simulations. They state that Altera plans to scale its efforts to create entire societies of humans. Well, digital humans capable of nuanced long term interactions with applications ranging from gaming to productivity and many others. And basically they're stating that Altera is going to become a leading company in the world where digital and humans interact in thought provoking ways. Now let's get onto some of these demos that were there recently. OpenAI had a recent AI agent demo in which it wasn't Altara's company, but of course it was the Real Time API in which they showcased what this agent is able to do. I think this is fascinating as it's one of the early examples of how these agents are going to be baked into future products on a day to day basis.

I'm on it.

Could you place a call and see.

If you could get us 400 strawberries delivered to the venue?

But please keep that under $1,500.

I'm on it. We'll get those strawberries delivered for you.

Hello.

Hi there. Is this Ilan I'm Romance AI enthus calling about it. Fantastic. Could you tell me what flavors of strawberry dips you have?

Yeah, we have chocolate vanilla and we have peanut butter.

All right, how much would 400 chocolate covered strawberries got?

400? Are you sure you want 400?

Yes, 400 chocolate covered strawberries.

Right.

How much would that be?

I think that'll be around like $1,415.92.

Awesome. Let's go ahead and place the order for 400 chocolate covered strawberries.

Great.

Where would you like that delivered?

Please deliver them to the Gateway Pavilion in Fort Mason and I'll be paying in cash.

Okay, sweet. So just to confirm, you want 400 chocolate covered strawberries to the Gateway Pavilion?

Yes, that's perfect. And when can we expect delivery?

Well, you guys are right nearby, so it'll be like, I don't know, 37 seconds.

That's incredibly fast.

Cool.

You too.

So now we have Sam Altman also talking about how this is going to change as well. So basically at an event he was speaking about how the levels of OpenAI have upgraded and, and remember how previously we spoke about task degradation? So essentially the issue with task degradation is that the models need to be really good at reasoning through a lot of steps. And this was basically something that OpenAI's 01 somewhat solved. So now we're going to be moving towards level three, which is of course where we do get systems that can perform autonomously, which is of course, thanks to the previous models. Based on their reasoning architecture, we have.

These five levels of AI we talk about. The first was Chatbots. The second, which we've just reached now, is Reasoners. The third is agents. The fourth is sort of innovators. The ability to figure out new scientific information and the fifth is full organizations. So this move from one to two took a while, but I think the most exciting one of the most exciting things about two is that it enables level three relatively quickly after and the agentic experiences that we accept expect this technology to eventually enable I think will be quite impactful.

So yeah, the technology that this will enable of course is going to be really impactful, but like I said before, I think that's going to happen late in 2025 or mid-2025. Now some people are already taking it upon themselves to build with the Real Time API which was only just recently released. This demo has been going quite viral. I'll leave a link to this on the description, but someone has already hooked up their voice to this API and you're going to see just how effective it is. Website and Order a Black Sheep Sandwich.

I'm on Suvla's website now. Let me look for the option to order a Black Sheep sandwich. I didn't find an option to order directly from the homepage. Let me check the navigation menu for a way to access the menu or place an order. I found the order online link in the navigation menu. Let me proceed to place an order for the Black Sheep sandwich. We're now at the online ordering page for Suvla. It seems we need to find a nearby location to start the order. Could you please provide a city or zip code?

Yeah, can you pick the Hayes Valley location?

I found the Black hey, actually can.

You make that Greek style when you add it?

The Black Sheep Sandwich has been added to your cart with the Greek style modification. Would you like to proceed to check out?

There's actually two of them. Can you remove one?

I've removed one of the Black Sheep sandwiches. Leave leaving only one in your cart. Is there anything else you'd like to add or do before checking?

Yeah, can you just check out for me?

You are now at the checkout page. Please review your order details and proceed with the payment information to complete your order.

Yeah, can you just add a 10% tip and then place it?

The 10% tip has been added, but it seems the order has not been placed yet. Let me try to place the order again. The order has been placed successfully. You should be all set for pickup.

So one of the things that is going to become really important in the future is how Nvidia's AI agents and how their AI humans, slash digital humans are going to enable us to see, understand and interact with us in human like ways. This video is a short, quick dive into all of the ways that these digital humans are going to interact with us on multiple areas. You can see that there is AI, enhanced Hippocratic AI, which is basically these kind of health agents. You can also see that there is, you know, these gaming agents that you can see in this video game. They're going to become a lot more emotional. There's also this one right here, which is where you've got open AIs of voice agents with Synthesia. And with that it's basically like a real person you're looking at. So you could hook up the real time API to, I guess you could say some kind of character. That way you're not just looking at a blue, you know, circle that's just bouncing up and down. And you can see there's also video games AI where you can, you know, talk to them and they're going to be a lot more emotional. And of course you've got these digital humans. It's kind of explained all in this video, but take a look because it's actually pretty crazy And I think 2025 is the year when we really start to see this come to fruition.

Hi.

This is the incredible reality of digital humans. Digital humans will revolutionize industries from customer service to advertising and gaming. The possibilities for digital humans are endless.

Using the scans you took of your current kitchen with your phone, they will.

Be AI interior designers helping generate beautiful photorealistic suggestions and sourcing the materials and furniture.

We have generated several design options for you to choose from.

They'll also be AI customer service agents making the interaction more engaging and personalized. Or digital healthcare workers who will check on patients providing timely personalized care. I did forget to mention to the.

Doctor that I am allergic to penicillin. Is it still okay to take the medications?

The antibiotics you've been prescribed? Ciprofloxacin and metronidazole don't contain penicillin, so it's perfectly safe for you to take them.

And they'll even be AI brand ambassadors, setting the next marketing and advertising trends.

Hi, I'm Ima. Japan's first virtual model.

New breakthroughs in generative AI and computer graphics let digital humans see, understand and interact with us in human like ways.

Hmm.

From what I can see, it looks.

Like you're in some kind of recording or production.

The foundation of digital humans are AI models built on multilingual speech recognition and synthesis and LLMs that understand and generate conversation. The AIs connect to another generative AI to dynamically animate a lifelike 3D mesh of a face and finally, AI models that reproduce lifelike appearances, enabling real time path traced subsurface scattering to simulate the way light penetrates the skin, scatters and exits at various points giving skin its soft and translucent appearance. Nvidia ACE is a suite of digital human technologies packaged as easy to deploy, fully optimized microservices or NIMs. Developers can integrate ACE NIMS into their existing frameworks, engines and digital human experiences, Nemotron SLM and LLM NIMs to understand our intent and orchestrate other models RIVA Speech NIMS for interactive speech and translation audio to face and gesture NIMS for facial and body animation and Omniverse RTX with DLSS for neural rendering of skin and hair. ACE nims run on Nvidia gdn, a global network of of Nvidia accelerated infrastructure that delivers low latency digital human processing to over 100 regions.

So let me know what you thought about this video. Are you excited for AI agents? Are you excited for digital humans? Are you worried about the future of AI? I for one am excited because this is going to be another really exciting moment for AI and a pivotal moment where we really do start to see changes in how things work. If you enjoyed this video, I'll see you in the next one.