Googles New Robot Just SHOCKED The Entire INDUSTRY (MOBILE ALOHA)

So we actually have something that is absolutely incredible. Some student researchers from Google DeepMind and Stanford University have built something amazing. It's called Mobile Aloha and it's a really incredible robot that will show us what's to come in the near future. So you can see right here that this is the robot that they created. This is some of the demos that they've showcased on their page. And you can see that this robot here is being used to make a three course meal. This. Now I do want to say that there are many, many things that will be discussed later on in this video because this is a very, very fascinating project that you definitely want to pay attention to because it highlights the dexterity and the way in which these mobots move, which is much more accurate than many people had assumed. So this initial demo was posted on Twitter and it did take the Internet by storm for very obvious reasons. One of the main reasons that this took the Internet by storm was, was due to the fact that robots currently haven't really been able to do many tasks, especially at this scale in terms of the cheapness where you can actually do a lot of tasks which people think only humans can do. And trust me when I say that a lot of the demos later on in the video only get more intense and more amazing. So let's actually take a look at the team and this entire project to see exactly how things work. So this is the team here. We have Zipeng Fu, a Stanford and AI Robotics PhD at Stanford Lab and a student researcher at Google DeepMind. We have Tony Zhao, a PhD student at Stanford and a student researcher at Google DeepMind. And we have Chelsea Finn, CS Faculty at Stanford and a research scientist at Google DeepMind. So these are the people behind this. Now you're going to want to read the abstract and the abstract is essentially just the introduction for the research paper in which they document how they manage to do everything. So so essentially it starts out by saying that imitation learning where robots by watching humans work really well for tasks done on a table like picking up objects. But this isn't enough for tasks that need moving around and using the whole body. In this project we made a system called Mobile Aloha which is like an upgraded version of an old system Aloha. This new system can move around and is designed to control its whole body makes making it more versatile. We use Mobile Aloha to gather data by having it mimic human actions. Then we teach it to perform tasks by itself through a process called supervised behavior cloning. And then we improve Its learning by using a method called co training, which I'll dive into later on in the paper, where it learns from both new and mobile tasks from the older aloha data, which is more static. And this approach is super effective. By watching and learning from about 50 examples of each task, the robot gets really good at doing complex things like cooking shrimp, opening cabinets, using an elevator and washing dishes. In fact, this method, the robot's ability to complete these tasks correctly can go up to 90%. So let's take a look at some of the tasks that they were able to do autonomously because by far this is absolutely incredible. So you can see here that this is a demonstration of some of these tasks. Now, what's even crazier is that this is one 100% autonomous. Later on in the video, I will tell you guys about teleop mode and how it actually works. But here you can see that this robot is completely going about its day, doing the daily tasks on its own. Now, we did see some other examples of this from other robots, like the Palm E demonstration in which there was a robot there. But I think this one is far more intriguing because of the way that it's able to do certain tasks, which we really only thought were possible with the just hands. And it was able to do many of these tasks and some of them, like this one with the chairs that weren't even in the original data set. So by far this is definitely fascinating stuff because I also remember seeing a video here where there was actually even some interference with the task, meaning that I think someone was throwing some stuff at the robot and throwing some stuff into it and it still was able to complete the task. So a robot being able to cook shrimp at this level is pretty crazy if you ask me. So some things that you do need to know about this robot, one of the first things is that the movement, this robot can actually move as fast as a person walking. And the paper, they talk that it can move pretty decently. In addition to that, it's steady even when handling heavy things like spots. So the stability of this robot is very effective. So it's not going to just be tipped over. In addition, you can control its arms and the base. It moves on and at the same time. And what's crazy about this is that it is self powered, so it has its own battery and computer. And for the moving part, they chose a mobile base called Tracer, which is used in warehouses. It's fast, can carry heavy loads and can move over small obstacles and slopes. And it's also cheaper than similar products. And they designed the robot so a person can control it by being connected to it. This way, the person can move the robot around while. While using its arms, which is definitely helpful for tasks like opening cabinets. And it also does have cameras on its arms and one in front for seeing. And it can feel and record its movements. Now, like I said before, this is essentially version two of the original aloha here. And you can see exactly how accurate this robot is at doing certain things. I mean, it's able to put a ram stick and a robot, then it's able to get this box cutter knife, able to open it up, and then, you know, do some very, very precise cuts to be able to open that package. Then it's able to get that out and apply that so that thing can move. I'm not exactly sure what I'm looking at. I know that those are bike gears. And yeah, so this is very, very impressive stuff because as you can see even here, this is in the teleop mode, and essentially tele mode, which I'll talk about more in just a second, is where a human essentially controls the robot via another arm. So essentially the human is giving me input, and then that is able to do that. Now, the thing about teleop is because it's essentially allowing you to control the robot in a very precise way. And why it's good is because we can then see exactly how good these robots could do if they were given the right data set or if we were able to program them in the correct way, which shows us that how effective these robots are. So with VR teleop, this is something, you know, autonomous versus VR telep or just standard teleop, because there are many different ways that you can do it. You can do it in VR, you can do it in standard. So, for example, if you don't know the difference, this right here is essentially VR teleop from Pollen robotics. And this is essentially where you have someone in VR and they are able to control that robot. So teleoperation, also known as remote operation, just refers to the operation of a system or a machine from a distance. And it's mostly common associated with robotics and mobile robots, but it can be applied to a wide range of systems. And the term is often used in research, academia, on technology context, and of course, in the context of robotics. Teleoperation involves a human operator controlling a robot from a distance. And this control can be achieved through various communication channels, such as radio links, satellite connections, or cellular networks. And the distance involved can be as little as across a room and, and across continents. And it's particularly useful in situations where it's unsafe or impractical for humans to be physically present. For example, teleoperated robots are used in search and rescue operations, nuclear and pharmaceutical industries, maintenance and research sectors where on site operation would put people at risk or where a clean environment needs to be maintained. And other common applications include space exploration, underwater vehicles, forestry, mining, agriculture, surveillance, rescue and surgery. This one here is of course a VR teleup, which is definitely some fascinating stuff. But in the actual paper we did get some of the physical teleop which is right there. So you can see he's able to clean the restroom with this robot right here. And some people might be asking what is the point of him even doing this if he could do it. But like I just stated there, there are many different applications for this and, and of course you also get to see what the robot is actually capable of. And once you train it autonomously, you can then get that robot to do similar tasks. So I do think that it does show us how crazy robot dexterity is getting and how effective these robots might be in the future at certain tasks. Now here was a demonstration of where they had the robot at home. And you can see here once again with the teleop system. And remember, they do have two kinds of ways to control the robot. They have the teleops systems and the autonomous systems. You can see here that it is able to really do pretty much any tasks that I can honestly think of. Like a lot of this, you know, stuff that you look and you see and you think, ah, a robot doesn't have hands, it won't be able to do that. It won't be able to, I guess you could say, you know, for example, cook me a three course meal. As we did see in the start. It won't be able to control its hands to be able to do that. It won't be fast enough, it won't be quick enough. But of course you, you know, there are some things that you should know, some of these clips, I'm not sure exactly which ones, but it will say in the left hand corner or the right hand, you know, side, whichever one it is. But it will say how sped up it is because sometimes, and I will show you the real time speeds of these robots in around, you know, two minutes once we finish analyzing this clip. Because it also is very effective at, you know, just honestly controlling a lot of stuff. And one of the things that I do find really interesting Was that this robot isn't actually shaped like in human hands, it's shaped like two claws, kind of like a crab, I guess you could say. And it just goes to show how much, you know, how many different ways there are to interact with the environment that are still very effective, that aren't 100% completely human. So that is something that I did also find pretty fascinating because some people would argue that human hands are just simply the best, they have the most dexterity, the grip strength is unmatched, and I guess you could argue that. But you can see that with these kind of robots, you are definitely accomplishing quite a lot here. So this is definitely something that is quite fascinating. That and even that right there, you know, being able to zip that up is, that's that's pretty, pretty hard stuff. Even, even for some humans, I would argue that that is something that is quite hard. So, yeah, what we do need to take a look at as well is of course the autonomous nature of this product. Because I do believe that this project, if scaled correctly, could definitely be something that could be applied to many different things. So let's take a look at. So let's take a look at the rest of this clip and then I'm going to show you guys the actual real autonomous stuff and it in real time, Foreign. Let's take a look at some of the real time autonomous tasks. So this right here is of course the real time autonomous task of this robot cooking shrimp. So you can see, of course it isn't as fast, but it still isn't as painstakingly slow as some of the previous tasks that we have seen from other robots and earlier generations. Now, some people could argue that a human is way faster, but like I've always said, this is always development. What we are seeing is the continuous evolution and progression of these robots. They might not be lightningly fast right now, but remember, year on year, month after month, day after day, as researchers and as many different students delve into the industry, I'm sure there will be more discoveries, more efficient ways of making them faster than that we will eventually discover which will lead to robots that are probably going to be too fast for our liking. So there are many different tasks that this robot was able to do autonomously. And it will be most certainly fascinating to see what other robots or what other tasks this is able to successfully accomplish. And I cannot be the only one surprised by how effective this robot is at doing certain tasks. So here we have the push chairs task. And this is of course completely autonomous. And this is in real time, so you can see how fast it actually is. And I did want to show this because I wanted to be transparent in not just, you know, saying that, oh, my God, this robot is so amazing. It definitely, absolutely is amazing. But it's important to know where we currently are, because even at this speed, I still do think that it is pretty impressive, because although some humans could do it faster, this is still something that looks to be very effective at not only completing tasks effectively and not only controlling robots with a very precise grip. And then what we have as well about this was that it was only trained on the first three chairs, and the last two chairs are out of data extrapolation. So then, of course, we have this one here. And like I said, there was an interesting example on Twitter, but I unfortunately wasn't able to find it. But in that video, it did actually show some adversarial disturbance in which someone did put some things in the pot, which. Or they did move the pot multiple times. But this goes to show that, you know, you might come home from one day and then, you know, your robot has managed to clean up everything. Not just like a standard room, a Roomba robot, which just, you know, glides around and picks up dusk, but a real, real actionable robot that, you know, moves around your kitchen or wherever and is able to, you know, fold your clothes, able to move your stuff. I think this kind of technology is something that people should be paying attention to, not only because it's fascinating, but, you know, I think there the elephant in the room. You know, one of the things that many people have been concerned about recently when looking at these autonomous robots, and, for example, you can see the robot here being able to rinse out this pan in real time is whether or not these robots are going to be taking people's jobs. Because, of course, as you know, any task on a computer in the nearby future could of course, eventually be automated by AI agents. But the robotics debate was something that many people have said for a while now is something that is going to be fairly slower than the software advancements that we have with software like ChatGPT, but with videos like this and with projects like this from Stanford and Google DeepMind, it's showing us that robots won't be that far behind as long as there is this level of research taking place. And then, of course, here we have completely autonomous, and in real time, we can see the robot coming over to here, and, of course, being able to wipe this down. Now, there definitely has been an influx of robotics Research being looked at just across the globe, based on what I've seen. And I do think that we are going to be getting a stellar level of research and development in 2024, not just from students, but not just from companies, but just honestly across the globe because of the many things that are now going on. For example, they actually did open source this project. And one thing that they did also talk about that was absolutely incredible was the price of this. This entire rig you could get for only $32,000. You might be thinking, well, you know, for the average person, $32,000 is a lot of money. And you'd be correcting saying that. But remember, a lot of the other humanoid robots that are able to do, you know, pretty basic stuff, are a lot of money. I'm talking six figures, you know, mid six figures, around $200,000, you know, which is definitely the price of like, you know, a supercar or a medium sized family home, depending on where you are. And that's definitely not something that people can afford. But if we have something that's open source and people can figure out how to make it cheaper and cheaper and cheaper, just like as technology has done, I mean, if you look back, you can remember how giant our TVs were and how giant our computers were and how expensive and slow they were. I think the trajectory is definitely, definitely clear. Now, something that I also do want to talk about is the co training. So co training is like teaching someone to cook using two different cookbooks. So one cookbook has recipes for basic dishes, and the other recipe has more complex meals that involve moving around the kitchen to use various appliances. Now, in the mobile Aloha project, co training and means the robot learns from two sets of examples or cookbooks. The first set shows the robots how to do the tasks that require both using the hands at the same time. Like cracking eggs with one hand while stirring a bowl with the other. But these tasks are done in one place without moving around. Now, the second set of examples teaches the robot how to move around and do tasks in a different part of a room. Like going to the fridge to get ingredients and, and then moving to the stove to cook. By learning from both sets, the robot gets really good at tasks that need both hand coordination and moving around. Like cooking a complete meal where it needs to fetch ingredients, prepare them and cook them on the stove. This way, the robot becomes more skilled and can do more complex tasks than if it had learned from just one set of examples. And we can see time and time again co training outperforms the no code training. This is also a tweet from Chelsea Finn, one of the people on the team who did this awesome project where they're talking about the hardware that, you know, only cost $32,000 including all peripherals, and they were actually surprised by how capable it was. And of course, they left a link in order for you to build it yourself. Now, something that is also important to talk about is the conclusions, limitations, and the future directions. So essentially, I read the conclusions in the paper and in simple terms, the project is about making a robot that can move around and use both hands to do complex tasks like a human would in the kitchen. And the robot system, of course, as we know, was built on top of the existing Aloha system and it was able to do many of these tasks very well with just 20 to 50 demonstrations and also cost less than $32,000. And they're available freely for any, anyone to use. However, there are some things that they do want to improve. For example, the robot is a bit large in some spaces and its arms can't reach high or low enough for tasks like using an oven. Now, of course, in certain houses we do have different spaces and different sizes. So ideally for any large scale robot that's going to be for a home usage, we do want it to have minimal, as you know, we do want it to be essentially as small as possible, and we want it to have as many degrees of freedom as possible in order to be able to work in the widest range of places and to be able to grab things that most humans can't grab and to be able to walk in places that most humans can't walk. Now, they also plan to make this robot smaller, which is of course what we need, and give its arms greater range of movements, which is something that we do expect. Now, on the software side, the robot is currently only learning from tasks shown by expert users. And the team wants to improve this so that the robot can learn from a wider range of examples, including less perfect ones. And in the research paper, they actually show that an average user can actually become an expert within only around, I think it was five examples or five training runs. I'm not sure exactly what the indicator was, but it definitely didn't take that long. So I think this project, this entire thing to start off 2024, is definitely a very good look into the future of robotics and what is going to be done at scale. I mean, think about it like this. If three students, and of course to note, definitely three very, very smart and well capable students can produce this. What could a team of 300 at a large corporation do? I mean, it definitely goes to show that if people can do this on such a smaller scale, what is going to be available on a bigger scale and the fact that it is open source means that this does once again raise the bar for the base level of what we're able to see with the robotics. So let me know if you found this to be intriguing. Are you going to be taking a further look at this? All links will be in the description. And are you excited for robots, even if some of them can do these autonomous tasks, which may actually take some of our jobs, but at the same time should make some of our lives easier? Sam sa. Sam it.