#358 How AI Agents Will Work While You Sleep | Ruslan Salakhutdinov, Professor at Carnegie Mellon

Again, a lot of these things I think in the future will just be automated. And we're seeing success rates of existing models hitting, you know, maybe like 45, 50%, which is very impressive.

One of my favorite questions on the show is asking where you still need a human in the loop. So far, no guest has been brave enough to say, let's just let the AI agents run wild. But there's a definite trend towards autonomy. Today I want to know how we're progressing towards this goal of autonomy through better reasoning, running tasks for longer, using more tools.

From 80%, people went to 90%. It was very hard, and going beyond 90% just became impossible. It's hard for me to see that it's going to, you know, hit 100% because there's a fundamental limitation of these systems.

Our guest is Russ Salikhoudinov, a professor at Carnegie Mellon University. Russ had a prestigious start to his AI research career as one of Geoffrey Hinton's postdocs. He's also spent time as an executive at Apple and Meta before returning to his academic roots.

Somebody gave full access to some of these agentic systems and the agent just deleted the entire database in 8 seconds.

Let's learn about the latest AI agent research. Hi, Russ. Welcome to the show.

Well, thank you for having me.

Yeah, great to have you here. I'd like to talk about agents to begin with. So first of all, what is the most exciting use case of AI agents that you've seen?

So I think over the last couple of years, we've seen really big improvement of agentic systems in coding. I think that's one extremely important use case. And we're seeing this right now, just even over the last few months. Genetic systems from, from Anthropic Code Claude, we're using it here at CMU quite extensively. It's been, it's been remarkable. I'm also seeing some of the agentic systems becoming more and more useful, what we call computer use agents. So these agentic systems that can help you with computer tasks, for example, finding some information online or filling forms for you, or, you know, because these agentic systems can probably do it better than humans can. That's the area that we also at CMU are looking at quite extensively. In general, sort of any system that can be automated or can be autonomous in solving tasks, I think, you know, you'd consider to be a good agentic system.

Absolutely. I mean, so agents seem to be the hottest story of 2026. I feel like I've been talking about them for months now. But yeah, having that increased level of autonomy is amazing. It's like how we get things done faster. So I'm curious as to where the limits are, like how far can you push this? And what can we not do yet?

You know, one of the things that, again, in coding, in setups like coding, for example, the interesting thing there is that you can have what's called verifiable rewards. Basically meaning that when you write the code, you can pass it through unit tests. And if the model passes through unit tests, you can kind of like say, okay, I've sort of set things correctly. And then there are other systems like computer user agents or what we call web agents, agents that can go online, shop for you or find information and do sort of like routine tasks. There it becomes a little bit more difficult because, you know, a lot of times these are long horizon tasks, tasks that can take you hours, for example, to accomplish, but also where rewards might not be kind of what we call verifiable. Many different ways you can solve the same task. And that's where the challenge comes in. These are sort of more open-ended systems. And this is where a lot of research is happening right now. Exactly how we define rewards, how we train these models, how can we make them be more robust. And the existing systems are getting better and better. We were surprised by doing research here at CMU, how much progress we've seen over the last year. In terms of these open-ended sort of agentic systems that can go online, find information, and actually go through fairly difficult planning process. And it's quite amazing. It's not there yet, but ultimately I do see these systems will— a lot of tasks, a lot of routine tasks can be automated.

Absolutely. I'm looking forward to it. And so you talked about the idea of giving rewards to the models in order to encourage them to do good things. This sounds like reinforcement learning here. So maybe we'll get into that in more depth later. But for now, you also mentioned long horizon tasks. So like, how long a task can you complete with an agent at the moment?

You know, we've done the research right now. It's in use. The paper is going to be coming out with a couple of students here. We're finding that, you know, tasks that require on the order of a few hours right now we would consider being long horizon tasks. Obviously, tasks that can go into days would even be better. So for example, you know, we've been looking at tasks, what we called hard tasks, you know, tasks that even humans will have a hard time completing. You know, some examples, and we actually tried getting some of these tasks from actual users, like people who actually do use computers or do go online. And again, tasks that we're looking at right now will take on the order of 2 to 3 hours to accomplish. And that's pretty long. And that requires us to, you know, do proper planning for the models to do proper planning, for the models to execute, you know, do trial and fail. There's something that's called Monte Carlo Tree Search, which is like agentic systems that can go and try to, you know, finish a task. If they can't, they can backtrack and try to finish the task in a different way. Again, I think longer term, we'll have systems that can function on the order of days to accomplish tasks. And again, a lot of these tasks, you know, is going to be even hard for humans to accomplish.

Absolutely. I mean, that's really interesting. I think like a year ago, we were talking about agents running for minutes at a time. And the fact that we're now talking about hours, days is really kind of impressive. Well, maybe days is in the future. But certainly, I can see there's a lot of value in, I set my agents to run when I leave work, and then they run for 16 hours or something. And then the next morning, when I get in, there's been some work done and I can go and process that myself during the day. What do we need to do to get there? I mean, I think you're closer to the research than I am. Like, what are the sort of cutting-edge things here?

A lot of that is being done in the frontier labs, labs like OpenAI, Anthropic, Google. One of the key sort of missing pieces right now that we're seeing is that when we go to these long-horizon tasks, again, as I mentioned before, it's very difficult to kind of define these verifiable rewards, like has the task been successfully accomplished or not. And so existing systems today, using reinforcement learning, are set up in such a way that you get your agent to do the task, and after the agent completes the task, you basically say, well, was it correct or was it not correct? So you only have almost like one bit of information. This is how, for example, existing reasoning systems are working today. Called the math reasoning. I give you the math problem, the model sort of tries to do the reasoning, try to figure out different solutions and comes up with a solution. And then we say, yes, that's a correct solution. If that's a correct solution, we reinforce, we make whatever the agent did more probable. We basically tell the agent, well, do the same thing. And if it's incorrect, we basically tell the agent, don't do the same thing. The issue with long horizon tasks is that if it takes you on the order of a few hours to accomplish, you need to define sort of some form of what we call intermediate rewards or partial rewards or something where we tell the system, yes, you're on the right track of accomplishing this, not just sort of, you know, when you're done, we basically say, nope, that wasn't quite right, you know, redo the whole thing. And so that's one interesting area of research. There's a lot of work on defining what's called rubric-based judges. So rubric-based is basically create a rubric and basically saying, well, I've looked at what you've done and, you know, Over here you did it correctly, over here you didn't do it correctly. And so you try to give sort of these extra signals, extra learning signals to the agentic systems. And that sort of right now sits at the cutting edge of research, exactly how you define, you know, what are these rubric-based approaches that you define? Are they consistent with human judgment? And, you know, how do you define them? And it sort of points to this bigger problem of what's called credit assignment problem, right? When I do a lot of different things, I want to assign the credit and say, yes, this was, you were on the right track, this was correct, and this is where you were incorrect. And that's generally much more difficult to do when you have these genetic systems doing, you know, solving tasks for a very, very long time. And hence sort of the challenges with long horizon tasks.

Okay, so it sounds like these are similar problems to teachers grading homework then. When you're in math class, then did he get the right answer? That's not all the credit, you know, you want to show you're working as well and did you get the intermediate steps right? So are you following a good process or not as well?

That's exactly right. I think that's exactly how we're beginning to train these systems and that's exactly, I think, one of the bigger problems is again the credit assignment problem and of course it's going to be, first of all, it's harder to come up with these long horizon tasks and second of all is defining, like you said, the signals, intermediate signals to to teach the system that, you know, what it's doing is correct. But it's exactly right, like kind of like teaches, you know, teaching kids, you know, it's not just the final answer, but were your steps correct? This part was correct, maybe this part wasn't correct. So the final answer was incorrect, but this part was correct. And you want to provide this to AI systems. So, you know, it learns to use parts that it's correct and sort of, you know, update the parts that are not correct.

Okay. And does this work beyond this sort of, actually, the hard science of like programming, mathematics, where you have very kind of strict, like, is this right or not? Does it work in a broader sense?

I think it does. I think, but the most, again, right now, most successful sort of systems we've seen are the systems that we basically, you know, given particular task, I can tell you exactly, did you solve it or did you not solve it? Like math, for example, and physics. And that's why we see a lot of progress in those domains, because again, these are verifiable rewards. I don't have to come up with intermediate sort of signals. Either you solve the problem, you didn't, and I can precisely tell you, like, for this math problem, that's the answer. Was the answer correct or not? And this is where the reinforcement learning algorithms come in. This is a lot of sort of reasoning systems today that we have. A lot of them are basically based on that. But as I mentioned before, now we see more and more frontier labs and more and more research shifts towards this partial credit assignment problem, because that's the only way for us to actually, you know, build systems that can operate on the order of days.

Okay, so it seems like the research is progressing. It's a case of like finding what's a good credit system, and then, and I guess training better models based on this. All right, so I think the other problem, as well as getting agents to work for longer and stay on task for longer, this long horizon problem, the other is, can you make them smarter? Can you do better reasoning? So it seems like reasoning models have taken off a lot in the last year or so. What progress is being made there?

Yeah, that's a good question. I think that, you know, in terms of some of these systems, I do see right now there is a shift from sort of these monolithic systems. So because when you're solving this more complex task, whether it's a task that you're doing on the web, whether it's a task controlling your computer, whether it's a reasoning task, solving a math problem, especially for these agentic systems and coding problems as well. For these agentic systems, a lot of times the agentic systems would come up with a plan with a high-level plan of how you would execute, and it would go and proceed in executing that in steps. And whenever you're failing in executing certain steps, you sort of roll back and try to fix mistakes. And so the system's becoming, you know, the way the reasoning and intelligence comes in is this notion of like creating the plan of what you need to accomplish. Right now, and I'm also seeing the evolution from these monolithic systems where a single model does the plan and sort of executes, you're now beginning to see multi-agent systems where, perhaps a bigger model, a more expensive model, frontier model, creates a plan and how the tasks need to be solved. And then you have these local agents or small agents that would go and solve, or specialized agents that will go and solve specific subtasks and communicate back to the manager or to the overall system. So I am seeing sort of the next sort of wave of these systems where instead of having a single model, you have these multi-agent systems. And we'll see, again, we're doing some of the research here at CMU on, we're seeing some fairly positive results, especially in the settings where a lot of tasks can be paralyzed. So you have this swarm of local agents that will go and solve different pieces, then come up and then you have a larger model that can integrate this information, do the reasoning and sort of, you know, reason about what to do next.

Okay, yeah, so rather than having like, I guess, like a one super worker, it's like a lot of—

Exactly.

Junior staff do it, doing the task together.

And that's, I think that's, I'm seeing over the next year, couple of years, we'll probably see more and more of these multi-agentic systems sort of, you know, operating together.

Yeah, so on one level, it seems smart to kind of break things down into smaller problems. I guess the challenge then is about orchestrating them, making sure that the whole the system as a whole works. Is that the main thrust of research at the moment, is making sure everything works together?

Exactly. How do you orchestrate these systems? What's the communication protocol between sub-agents that go and, you know, do the tasks and then as well as the main agents? And the idea there is that, you know, the sub-agents can be smaller models. They can be potentially much cheaper agentic models that can, you know, do just, like in the case of computer use, for example, we've been playing with systems where we have Claude Opus as a system that orchestrates, you know, and plans what it needs to do. It's a frontier model, but then we have some of the smaller open-source models running locally on your device and executing, you know, the tasks. How you orchestrate the entire system is a very interesting area of research.

Okay, so lots of exciting stuff coming soon. So hopefully we get test, well, models that can run for longer and they can solve harder problems. I'm curious, what kind of things is that going to unlock for workers?

That's a very good question. Existing systems are still not 100% there where you can, you know, sort of like fully autonomously rely on sort of automating some of your workflows. I do see a lot more automation happening in the coding domain, so coding agents. I think it will unlock a lot of potential in that domain. I'm quite sure, which is, you know, one of the big areas, you know, if you can replace like one of the things that fascinated me, for example, is how my students started using coding agents, which is, you know, we run some experiments experiments overnight. And if some of the experiments fail for whatever the bugs are, you invoke the agent. The agent analyzes the log as to what happened. Is it a memory issue? Is it just, you know, some segmentation fault? Tries to fix the issue and then restart the experiment. So whenever I'm sleeping, you know, I can— my agent can just fix the problem and continue running experiments. And sort of like, it's extremely simple but extremely kind of useful use case, right? Because I'm not wasting 6 hours of compute. If something happens to my run and it dies at midnight, right, the agent can fix it and start running it again so that in the morning I can come in and look at my results as opposed to coming in and say, ah, it was a silly error. And so I'm sort of wasted a few, how many hours of not being productive. But I do sort of see a lot of, especially in more general cases like computer use cases, any sort of tasks, ultimately any tasks that you're doing on your computer can potentially be automated. A lot of times we see examples where you actually ask users like, what do you use computers for? What do you use web for? A lot of times people are searching for jobs, or a lot of times people are searching for specific health physicians. And, you know, it's a, it's a lot very laborious work. A lot of, you know, we had one example where we have a PhD student who was looking for faculty positions. And just as an example, looking for faculty position is like laborious process, you know, it's information gathering. I have to go through all the schools, I have to see which schools are hiring, I have to see which department is hiring, I have to see, you know, what areas they hiring, are they hiring machine learning or they're hiring in systems, you know, what is the deadline, what is, uh, what is the requirement, uh, and so does it fit with my research or not. Imagine an agentic system that can just go and do it for you, and it will probably do a much better job than you can, right? And so imagine you can ask this question to the agentic system, it will go on the web, you know, and after a few hours it will sort of generate you a spreadsheet of precisely which schools are hiring, what area they're hiring, does this fit with your research, what is the requirement, all the information, and all information is verified sort of with easy access for you, for you to go and verify it yourself, and tells you exactly what you need to submit, by what time, and everything, right? I mean, again, a lot of these things I think in the future will just be automated, which will be, which will be exciting. We still try to figure out exactly what the use cases will be, but any use cases where you know, these are sort of like menial tasks, these are sort of routine tasks that you want to automate, you can, and you can run these systems every day. And, you know, they'll do things that you want them to do.

I mean, I do love the idea of automating menial tasks and things that you don't want to do yourself. You mentioned the idea of checking problems of things that are running over. And I do love the idea of just being able to take a nap and having work done for me automatically.

That actually was an extremely, extremely kind of like, it's a simple I mean, you can think of it as a simple task, but at the same time, it's such a useful task because a lot of times, you know, we run things overnight, things can fail, and then you come back at 8 o'clock in the morning, something failed at midnight, and you just basically say, well, it's, you know, and so that can be automated.

Absolutely. So I particularly like the example of the job search as well because finding a job is surprisingly hard. Like, by the time you're looking through lots of different job sites, and then you actually have to read the job description because job title doesn't tell you anything. It's a tricky task.

It's a— that's exactly right. And we've actually tested some of the existing systems and, you know, on some of these tasks that we'd consider like a hard task, you know, where we give it a task that, you know, a human would be able to do, but it's a very laborious task. It's well specified. And we're seeing success rate rates of existing models hitting, you know, maybe like 45, 50%, which is very impressive, but not at like 99.9%. So there's still lots of room for improvement.

Absolutely. So that's one of the challenges is agents tend to work quite a lot of the time, but not all the time. So you need some human checking their work. Talk me through like what you think a good workflow is like, when do you want to have humans doing things? When do you want agents doing things?

I think that that's a very good question. I think a lot of existing systems, you know, I think there's a bunch of companies and startups are trying to work on, you know, this notion of human in the loop for these systems. Ideally, a smarter system would be able to take your request, try to do as much work as possible autonomously, but when it's uncertain about certain aspects of the task, it would come back to you and sort of ask you either a clarification question or give you options or kind of, you know, almost like be a copilot for you. Because, you know, like when I'm looking for jobs, obviously it's great that the agent can go and give me some information, but I would never trust it 100%, right? I would actually go and start verifying everything because I know that it cannot be, you know, it cannot do 100% for me. But if you have a system that sort of understands where it's making mistakes, where it's missing and would come back to me and say, look, I found these pieces, but I think I'm uncertain about these other pieces and I don't know what I should be doing here. Give me, work with me to figure out like, what should I do? And that requires for existing systems to have very good uncertainty estimation and sort of this nice interface back to the user and not just basically saying, here's what I found. That's it. But basically giving you, here's what I found, here's where I don't know, here's what I'm not sure about, here's possible options, which ones would you want me to, you know, as an example, I would never, I would never use any of the agentic systems to book me a flight. Even it's a simple task, you know, I'm, you know, next week I'm going from, from Pittsburgh to San Francisco, you know, I can ask any sort of agentic systems to go, you know, find a flights from Pittsburgh to San Francisco, you know, next Tuesday and book a flight for me. I would never do that. Even if it's 80% correct, there is at least this 20% gap where it will go and do something crazy, right? And I cannot, that cannot happen, right? So even though these systems are sort of impressive, the fact of the matter is they would have to be almost like 99.99% accurate for me to fully trust them, right?

Yeah, you want them to be at least as trustworthy as like an executive assistant, like a real human one.

Exactly.

For that sort of situation.

Exactly right. And right now, a lot of products that I'm seeing, a lot of frontier models, they do provide that feedback, but it's mostly they sort of give me the feedback and I have to go and verify and sort of try to— again, in domains where things are verifiable, it's easier, like in coding, like when I, when I give it a task and I say, you know, like, here's the task, you know, I have a bug in this code, you know, go figure this out. And here's the unit test that you can run. To make sure that, you know, everything is good. If the model finds the bug and does all the unit tests, then it makes me confident that it solved the task, right? But for these more open, like a job search, or like, you know, it's very hard to, you know, for the system to come back and say, like, I'm 100% sure that I've done it correctly.

Absolutely. So you mentioned that it'd be really nice if the agents would give you context and say, I wasn't sure on this thing. And they give you feedback that they weren't guaranteed to be correct. So I think there's a kind of a trait of large language models in general is that they will be confidently incorrect. So they don't have that level of, there's no self-awareness about when things aren't quite right. Is this like a fundamental problem or is this something that can be solved and you can get this background feedback?

It's a fundamental problem. I do think a lot of times we get the feedback from from these agentic systems based on large language models, they sometimes incorrectly confident that they're incorrect. And that's a problem. I think that there's ways of mitigating this. A lot of times, again, I've mentioned things like Monte Carlo tree search, there is sort of some way you can do ensembling where you can run multiple agents in parallel. They try to solve the task and then you look at the agreement. So there's certain ways of mitigating this, but It's not clear right now whether you can get there, you know, at 100% at this point. So maybe we need some new breakthroughs or some new solutions. I think what will happen in the shorter term is that these systems are going to be useful. And if people can figure out, or like, you know, Frontier Labs can figure out the right interface for me. So I'm in charge, I'm actually doing it, but you have a system that goes and finds the information and kind of like does these pieces for me. And then, you know, but ultimately a user would have to verify. Like, again, I would never trust— I mean, there was just a post this morning where somebody gave full access to some of these Agentic systems and the agent just deleted the entire database in 8 seconds, right? Because it, you know, found some bugs and, you know, these sort of nuances that, again, obviously a lot of things like deleting some something, or, you know, charging your credit card, you know, things you can put the safeguards around it. So you can prevent these systems from doing sort of these harmful actions, what we call destructible actions or non-reversible actions. Because, you know, if I book something online, it's not like I can go to my GenAI system, no, no, no, reverse, go back to the previous state. You know, like you just, you just can't do that. And that comes to the area of research on the safety and alignment. But yeah, I think what you've mentioned is a fundamental, you know, issue where the agent has the ability to do something incorrectly and be confident about it. Okay.

So, um, I mean, you mentioned the idea of like deleting a database, which could potentially be ruinous to, uh, to a business. Uh, and there are lots of things that can go wrong and agents can do things wrong much faster than humans can. So talk me through like what kind of safety mechanisms could you put in place to make sure you don't destroy your business with agents?

Obviously, there are sort of guardrails that you would put around, like making sure that the agent can never delete things or certain sort of destructive actions you can kind of handwire into existing models. I think actually the safety aspect of it, the alignment aspect, is a very big area of research right now. What's happening today is People do use what's called reinforcement learning from human feedback or from model feedback. And the idea is that if I see the agent doing some destructive actions, I can train it to say, for this task, this was incorrect. And so don't do that. But it's still very hard to build systems that are— when the model does something incorrectly, when the agent does something incorrectly, it's very difficult to train it in such a way that you kind of like prevent it, you know, 100% not happening. It's like one of these things, somebody was giving me this example when the airplane crashes, you know, you investigate, you find the fault, why it crashed, and you put the system in such a way that that will never happen again, right? That's sort of from engineering standpoint. Existing LLMs, uh, agentic systems, you cannot, it's very hard to do that. Right? Like if you found a mistake, it's very hard to post-train the model and say, that should never happen again. Right? So it's sort of like we do adapt these models to prevent them from doing harmful things, but it's done sort of in a soft way. You know, we just train the model to say not to do it, but there's always ways of breaking the model so that it does something that it's not supposed to do. So this research on alignment or putting guardrails into these systems is still out there. I mean, a lot of models can still hallucinate, you know, even today's, even from the frontier labs, there's always a way where they can tell you something that's incorrect or can hallucinate. And, you know, so kind of like have to verify it. They're getting better. I think there's what people are doing today is that whenever the model gives the output. You have another model that looks at the output and tries to verify, is this correct? Is it factually correct? Or is this action kind of could be destructive? Or, you know, is this a good action to take on or off? So we'll probably just see like more and more orchestration of multiple agents verifying the outputs of each other and sort of, you know, that can potentially improve the systems. It's hard for me to see that it's going to, you know, hit 100% because there's a fundamental augmentation of these systems.

It sounds like you need lots of layers of safeguards then. So you need some kind of guardrails built into the model itself, and then you need orchestrations. You have other models checking the work of the existing model or agent. And then presumably you also need some like deterministic security control to limit, to sandbox the agent's capabilities, say you are not allowed to delete these things, you don't have access to these specific things. And then probably some human processes as well. So is that the kind of gist of it?

Absolutely. Absolutely. I think it's, and it's sort of like also creates like an interesting problem. Again, there was this work done at CMU where, you know, somebody gave a very simple instruction to the coding agent, you know, and the instruction was, you know, go open this file and add my name, you know, to the file. Very simple task. The agent would go try to open the file. The file is password protected. The agent would go online and search for 10 most commonly used passwords, right? Pick up those passwords, try, you know, a bunch of the passwords. Password number 5 works, opens the file, adds the name to the file, closes the file, comes back to the user and says, you know, I've accomplished the task. Would you consider this sort of like a correct execution? You gave it the task, it executed the task. You know, any sort of reasonable person would basically say like, you know, if, if, if, you know, if it was your assistant, you're opening the file password protected, you go to the user and say, look, this password is password protected, you know, if you want me to, you know, do it, then you have to give me permissions and not sort of hack the file to, you know, to accomplish the task, right? And so these are sort of nuances that people are, you know, thinking about, or, you know, another example would be, you know, you know, I can tell my agent, go and download me the latest Taylor Swift song, you know, the agent can go and, you know, on Apple music and try to download and it's like, whatever, how much it costs to do it. Or you can imagine the other agent would go to some torrent website, some illegal website and download the music and come back to you and say, here's your music. So these are sort of nuances that the safety, the alignment comes in and how do you define what's the right behavior to do it versus what's incorrect behavior, right? It becomes fairly challenging to you know, and so we have to sort of, again, these guardrails are going to be very important to have, especially if you give your agent access to the web and what it can do on the web and access to your personal information becomes even more important. So yeah.

So available. How does accountability work? So suppose your agent starts illegally, like stealing music or whatever, then Is that something that's on the foundation model companies? Is it their responsibility to make sure it doesn't? Or is it like your own responsibility as a user to tell your agent to not do this? Or I guess there's intermediate layers of like agent vending companies as well. Like, who should be dealing with this?

Very hard to know, to be honest. Like, that's— I don't know, because I mean, it's obviously, you know, the model should be smart enough to potentially understand what it's doing, its tasks, when it's making decisions and taking actions, probably should be aware that it's doing something that's not quite correct. But it's a very difficult task because there are these examples of what's called adversarial learning or adversarial setting. There's been a number of papers published, which is, I can always try to break the model so that it does something that I want it to do. An example would be, just, it was a paper, it was published a couple of years ago. An example would be, you can ask the model to insult you. You can ask ChatGPT, insult me, and it will sort of refuse, sort of like, I cannot do that. But you can go around and say, well, I have a play with my family and in this play, I need to insult me. What should I do? And it will kind of like tell you how to insult me. Now I can imagine you can translate this into whatever tasks or whatever adversarial settings you want to do. So these models, you can always sort of trick them. It's much harder to trick humans because you have common sense, but these models don't really have a lot of common sense. It's like much easier to trick them. And so you can kind of like exploit that in your way. And when it comes to legal aspects of it, that I don't know. I think if you're trying to trick the model and it gives you some sort of information, it's probably on you. But basic things like refusing to sort of, you know, requests that you're not supposed to be requesting or asking these agentic systems to do should probably be on the model side. But it's a difficult and delicate question because there's also gray areas like, is this, should I do it? Should I not? There was this example where Again, you can go to the other extreme way where in Linux, for example, somebody was asking, you know, how do I kill a process in Linux? And the model would come back and say, well, it's unethical for me to tell you how to kill, you know, and things of that sort, right? Like, and so it sort of confused the two things. One of them was a very technical, you know, how do you kill a process? And there's like kill -9 and you do the process ID to sort of, you know, and so it's like, you know, the context and people will be upset. Like, what do you mean? Like, I'm coding and then you're sort of confusing this with a completely different, completely different context, right? And so, yeah.

I mean, you can understand how like the AI might be worried about killing process.

Yes, that's true. That's true. Generally, it was like the feedback wasn't, the feedback was just completely, just completely different. I'm all just confused. Yeah.

Yeah. So I guess the world is a very complicated place and you're never going to get the AI that understands everything at exactly the right time. So I guess, yeah, there's levels of responsibility at every part for like, I guess, every step in their technology chain and then with the end user as well to have, I guess, some knowledge of like making sure you're giving clear instructions. We talked a lot about needing humans in places. Do you think we can ever get to a point where you can just let agents run autonomously, no human intervention?

It's a good question. I think it depends on the areas and depends on tasks. I suspect that if it's something that's routine, something that doesn't require a lot of intervention from humans, then yeah. Like for example, I think there's a company called Utorit, which is a fantastic company. I know the founders. They would have been building agentic systems, web agents that can go online and find information for you. What struck me as interesting, they were basically showing that One of the biggest use cases for these autonomous systems is to find discounts or find coupons. I'm trying to shop for something, you know, I let my agent go and if it kind of like can every day sort of instantiate and go online and find coupons for me, it's great. I love it, especially for the product that I want to buy. You know, if there's a discount, there's a coupon and that's extremely, you know, something that I wouldn't think about, but, you know, people people did find it to be useful. I think that again, for tasks that are routine that people wouldn't want to do and that doesn't have a lot of uncertainty in execution, those things would be automated.

Okay, so once you got that level of routine, then there's less variation, it's easier to test, is it getting the right answer? And if it gets the right answer a few times, because there's not much variation, it's not going to get it wrong in the future.

I think so. So sort of like, you know, a lot of sort of like routine works. Like, for example, filling your taxes. If you don't have sort of any sort of complex structure to it, it's a routine thing, like you should be able to do that, right? And there's a lot of tasks that don't have a lot of uncertainty, ambiguity. I think a lot of these tasks will be automated.

Okay, I guess the other side of things is what are the consequences of getting things wrong as well? So I guess you mentioned finding vouchers. If it doesn't find a voucher, then there's no, like, there's no real problem there. Maybe just waste some time typing in a voucher code that doesn't really exist, but it's not a terrible outcome. Whereas obviously there are many, many worse things that can go wrong with AI.

For sure. For sure. You're absolutely right. Right now, I do think in the next year, couple of years, you're absolutely right. People will be using AI systems, agentic systems, especially in these sort of non-verifiable domains other than coding, right? Things that you're looking online, things that are controlling your computer and such. If tasks are not very, not very sort of critical in the sense that you miss something, that's okay. That will see a big adoption of agentic systems. And of course, for critical tasks, I think what's going to happen is that people will still use AI. Still use agentic systems, but they would have to go through human to verify the outcome and verify the final. So the final decision is going to be kind of made by, it's going to be made by the human. And I do see a lot of, you know, there's been a lot of surge recently with Open Claw, which is an agentic system that sort of allows you to, you know, communicate through your WhatsApp or through your messenger. And, you know, you can ask it to, you know, people are doing interesting things like hooking up AI to your speakers and setting up alarm clocks and doing all kinds of things, sort of like simple routine things that are very useful to you as a person, but don't have huge consequences, like critical consequences. Yeah.

Okay. I mean, so since you mentioned connecting AI to physical objects, I know some of your research is around physical AI. So So maybe we'll spend a few moments talking about that. Yeah, talk me through what are the use cases for physical AI?

Yeah, so we basically, again, we've talked about digital AI systems that can control, you know, your computers, systems that can go online, search systems that can code, and, you know, and then there is sort of a parallel thread on physical AI. And physical AI, you know, I think of it as systems that can potentially reason about physical world. You know, obvious, the obvious, one obvious instantiation of that is robotic systems, you know, actually robot interacting in the real world. And so you have to have a good understanding of what's called spatial intelligence, understanding objects around you, understanding the physical world around you. And in that domain, I think, you know, we've looked at, the community have looked at something that's called embodied AI. So these are systems that can navigate in physical worlds, avoid obstacles, and this is called navigation, which is almost solved at this point to the point where like our visual models, recognizing objects and has become so good. There is also locomotion, like you see a lot of physical robots moving around, going up and down the stairs. I think that field is also evolving rapidly. And then there is a final frontier for physical intelligence, and the final frontier is manipulation, physical robots manipulating objects. And that's sort of the frontier that is a lot of work, but extremely difficult to do, right? So for example, like, I don't know, we have systems that can can win, you know, math competitions and International Math Olympiads, but getting me a robot that can reliably unload my dishwasher or load my dishwasher, it's just extremely hard. You know, people do it in like very constrained setting, like here is, you know, but just being able to do it across many households and, you know, just it's extremely difficult.

It surprised me. I was— the idea of having a robot to do your housework task for you. I mean, that's been like a staple of, I think, well, The Jetsons cartoon was the 1950s, 1960s. There was a robot maid. We're still apparently not making much progress toward this. Why is it so difficult to get robots to manipulate physical objects?

I think that one of the things is that, again, we have robots that can very reliably navigate around your house, recognize objects, you know, go up and down the stairs. You have robotic systems that can sort of, you know, getting there. So that progress is there, but manipulating objects turns out to be such a difficult thing because especially dexterous manipulations, like when you have two hands, the ability to grab this cup and move it around and not just this cup, I can train the model to do it, this cup, but being able to do it for any cup, it turns out to be extremely difficult. And part of it is, you know, being able to develop hardware, you know, systems that can can, you know, have touch sensors, can manipulate, and the ability to train models that can fairly precisely figure out I can grab the object this way, I can grab the object that way, it turns out to be extremely difficult. I think the progress is happening right now, and whoever cracks this particular problem, manipulation problem, I think it'll be, you know, the next trillion-dollar company. And not just, you know, we see a lot of examples where people manipulate objects, but these are very specific objects that are trained on. I can manipulate this specific cup, I can manipulate this specific object. But, you know, putting a robot into my home that can deal with all the messiness in the home and the ability to manipulate any object in my home is just extremely difficult. So much diversity of objects. And, you know, ultimately, to me, If you want to have a useful physical AI, it has to do something. And again, we see a lot of examples of specialized robots in factories and Amazon has these amazing robots that can move things around. So it's been like sort of vertical integration of these robotic systems in specific environments, specific factories, but building general physical AI, Again, one sort of successful use case right now that I see of physical AI is self-driving cars. I think that's happening. It's very impressive, the progress that has been made over the last 5 years, like Waymo, for example. It's like they can drive completely autonomously in San Francisco and other cities, but in home sort of home robotics, we're still very far.

Yeah. So self-driving cars has been an interesting thing because there was a lot of hype, maybe around like 2010 to sort of maybe 2015 sort of thing. And then it turned out to be a lot harder than we were expecting and it sort of died off. And then just in the last year or two, it seems to have picked up again. So talk me through like what's the progress then? Yeah.

So with self-driving cars, I think that one of the things what happened in 2010, 2015 is that we went very quickly with deep learning. We went very quickly from zero to like 80%. And then there's been lots of startups, you put the cameras and you can kind of like control the steering wheel. And so people, I think, got excited, you know, that we can, you know, by the same amount of time we went from 0 to 80%, from 80% we can go to 100%. But what happened is that from 80% people went to 90%, was very hard and going beyond 90% just became impossible. Right, and you start seeing all of these nuances, with self-driving cars. Like there was this example, this funny example where, you know, you drive with a car and in the spring the vegetation would come in and, you know, at some point you have this like big sort of vegetation for, you know, floating onto the road and the model would completely get confused. Like, what is this? And sort of would completely act abnormally, right? Or there is, you know, these like sort of corner cases that started hitting, and this is when the progress basically stalled. This is where Apple, for example, they had Project Titan at a time, and then eventually they shut it down. Uber had a lot of ambitions to, you know, here they've built the Uber Labs at CMU at a time, had a lot of ambition to sort of have it, shut it down. So a lot of startups working in that space got shut down. But now finally, we're actually seeing progress. The progress is much slower than what people believed in. And the question is, is the same thing going to happen to physical AI, where there's a lot of, you know, progress? We have— we see a lot of robots, and you see a lot of actual robotic systems coming out of China. You know, they can dance, they can do backflips and everything. But I think the real test case is going to be, can they be useful? Like for tasks, you know, as opposed to just sort of, you know, a robot that can walk around and, you know.

Yeah, backflipping robots, very cool, very TikTok friendly, but not necessarily useful for most people.

Yeah, can it go and clean my dishes? Like, can it go clean my kitchen? Like things that I would want it to do when I go to sleep, right? Like, I mean, those things, the question is like, when do we get to that point. Are we sort of, you know, a year away, 2 years away? Are we 10 years away? That's not clear. I think that there's probably going to be some robotic systems in very sort of specific environments that they can be successful, but I think that putting robots in your homes is going to take a while.

Okay. So are there any things that you think better, more general robots are going to unlock for like either for work or for the general public? You said like robots in the house is going to be like a long way away. Are there some work use cases?

I think that one of the interesting use cases, I was talking to one of the startups, is if you can get robotic systems to even, you know, in my view, even go like open the fridge, get like, you know, one of those frozen dinners, put it in the microwave, heat it up and bring it to me. Even these use cases would be extremely useful. Especially for people with limited mobility, right? Like for people who live in homes and it's hard for them to, you know, do some of these tasks. If, if, if, you know, the robot doesn't need to kind of like do everything for you, but even some of these tasks, if it can be, if it can do reliably well, it's going to unlock extremely a lot of, you know, use cases and a lot of sort of, there's a lot of potential for that.

Yeah, certainly nursing care is incredibly expensive in most parts of the world. It's hard. So if you can get robots that can help people with disabilities, that seems a good sort of social good.

That's a social good. And it's like one of the things that I think potentially could be extremely useful for society. You know, there's a lot of, you know, maybe not in the United States, but in Japan, there's a lot of aging population, right? And so there's a lot of cases when, you know, these systems can actually be be useful, right? Again, for people with limited mobility and elderly and sort of even my dad, my dad lives in Toronto and I thinking in the future, I would love to have a system that can kind of, you know, do some of these tasks for him. And of course, I mean, there's a lot of use cases. Like I think one of the immediate use cases for robotic systems is factories. That's why, you know, Tesla bought and there are a bunch of other companies like Figure, they're trying to do partnerships with companies like Tesla Bot is going to be in Tesla factories and trying to, you know, do some useful works in some of these factories because it's a much more well-defined environment. It's not like you get into my home and it's a complete mess. You go to somebody else's home, it's like completely different, complete different mess, right? So whereas in factories, and this is one of the first use cases that that these systems are going to be deployed at.

Yeah, absolutely. I mean, certainly factories have waves of automation, I guess, going back from like the Ford motor cars, like a century ago, with the invention of the production line. And then, yeah, we keep getting waves of like slightly better and better robots. And yeah, hopefully, as they get more general, they can maybe accomplish more tasks. Okay, so one thing you mentioned earlier, was that with self-driving cars, we got to they work 80% of the time and that was fine. And then progress got very, very, very slow. And it's like once you get it past working more than 90% of the time, it gets very difficult. The lessons in that for agents, because it seems like that's where at a similar point where it's like getting to that 80% success rate is fine and then getting to 100% is impossible.

I think that this is where kind of like you will see people trying to adopt these systems, but the adoption is going to be, I think it's going to be a gradual process. And again, I think that with agentic systems, digital agentic systems, if you can define problems where 90% is actually good enough, like finding coupons, that's already useful, right? In self-driving cars, 90% is not enough, right? I still have to have a driver in the seat. So if it's 90%, it's useful because it can drive for me most of the time, but I still have to have control versus like a completely car without a wheel where I can fully trust it. You just sit and it drives you wherever you need to drive. So getting to that point with agentic systems, again, is going to be probably more difficult, especially these more general agentic systems, like as I mentioned, like job searches and such. More constrained environments like coding, the adoption will happen much faster, primarily because you can verify the solution. If I can reliably verify that I solved the task, that's the correct solution, then the adoption is going to be much faster because, look, if I fix the bug and it passed my unit tests, I know that it's done it correctly. Of course, I do something else incorrectly, but that's another story. But at least I know that it's it's like I can trust it that it's, you know.

Absolutely. So it sounds like there are some sort of lessons learned, I guess, for picking what projects you're going to use AI for. So you want to think carefully upfront about what is the likely success rate and what success rate do I need in order for this to be viable? And I guess also, how much can I constrain this in order to increase the success rate as well?

For sure. Yep. I think that's correct. And again, I do think that With evolution of these systems, they'll get better and better. We'll eventually solve the partial credit assignment problem, rubric-based systems. We will do reinforcement learning and these systems will get better and better and better and smarter. I'm not sure we can get to the point, we'll get to the point where it's like fully 100% autonomous, but it's going to be extremely useful. Again, for critical applications, I don't think we'll get there soon. But for tasks where it's just useful for you, I think we kind of like getting there. And it's a lot of use cases, actually probably tasks that people don't want to do. And then, you know, even if you get 90% of it done by machine, that's already extremely useful.

Absolutely.

Yeah.

So not every problem is like saving the world in a general way. It's like a lot of the stuff you do is kind of smaller tasks. And more routine and sometimes not that exciting and you want to automate them because yeah, better to not do this. Now we've talked for a while before we wrap up, I'd like to talk a little bit about your career trajectory because of course you were an executive, you were at Apple and Meta and now you've moved to academia at Carnegie Mellon. So first of all, like what prompted the switch and how is AI research different in academia versus industry?

Yeah, it's a very good question. I actually started in academia. And then I built a startup with a couple of my students, and that's how I ended up at Apple, because we sold the company to Apple. And then again, I came back to CMU. At CMU, with students here in another faculty, we've built one of the first agentic systems. And then Microsoft got interested, Meta got interested, and we eventually went and started building inside Meta. And now I'm back at CMU, and I'm I'm also launching a new startup. But one of the big differences that I see between academia and industry is that industry is much more well-equipped in its engineering efforts and its scaling efforts. So the ability to, each sort of major lab has a lot of GPUs and ability to really scale. I think that's one of the biggest advantages of being in industry. However, a lot of breakthroughs, early breakthroughs, a lot of kind of breakthroughs that are happening in AI are also happening in academia. And so, for example, if we look at the transformer architecture, just an example where a lot of technology is being built on, the transformer architecture was developed by Google. It was the first sort of paper, But the pieces of that architecture, like attention mechanisms, they already developed in academia, early days of academia, because industry, at least right now, they tend to be less exploratory, but more exploit kind of, because they have to build these systems. So they, you know, a lot of frontier labs, they know what works and they sort of continue to scale, clean the data, do the engineering effort, whereas in academia, people do explore new ideas. One of the big assets of OpenAI in the early days was that they were very good at taking critical ideas done in academia and executing them and scaling them very well.

Okay, I like that. It's very complementary there. So yeah, you still need universities for the fundamental research to come up with those wildcard new research ideas. And yeah, maybe even better than some of the industry research labs, I think.

It's fantastic. I mean, I'm sort of like in this position, interesting position, because I can be in academia and then, you know, when I do have time, I can spend some time in research. And I do see a lot of my colleagues, a lot of my friends, they do sort of being in machine learning. I think being in academia is amazing because you can explore these things, but also seeing what's happening in industry in terms of the frontier modeling and engineering part of it is also fascinating because it's it's a massive, massive effort, engineering effort to build these models.

Absolutely. It just requires like, I think, yeah, billions, like tens of billions, maybe hundreds of billions of dollars to like get these frontier models going now. So yeah, it's certainly not cheaper at this point. All right, wonderful. So finally, I always want more people to learn from. So Can you tell me whose work are you interested in right now?

Oh, that's a tough question. I think there is a lot of, you know, I'm interested in sort of agentic systems. There's a lot of very good work coming out of Stanford, some of the agentic systems. There's amazing work coming out of FAIR, Phase Fundamentally AI Research at Meta. I was part of the team. There's a a lot of interesting work coming out of that lab. I would say that I think that academia right now is much more open in terms of what's happening. And unfortunately, some of the research that's happening in places like OpenAI, Anthropic, it's sort of closed at this point. And so we see the glimpses because they have the compute so they can discover certain things that we probably don't know about. But unfortunately, they're not sharing. Not sharing the research broadly. I think that right now the research is very distributed. And so it's very hard for me to name one particular, but I do think that if you, for example, for audiences who want to learn about that domain, I would recommend looking at conferences like NeurIPS, ICML, ICLR. These are major machine learning conferences and looking at the work that that's being published there in a specific area. I think those are probably the best way to learn what's happening in our field. Absolutely.

I mean, it's good that you can't pick a single person because there's just so much going on around the world.

So I think there is a lot of, because right now, you know, after 2022 with ChatGPT, there's a lot of shift towards studying these large language models, studying the open source models, that it's no longer just, you know, 5 or 6 people kind of working in that domain. It's actually, you know, a lot of people are working. There's a lot of good work coming out, not just out of the top-tier US schools. There's a lot of good work coming out of other schools. You know, there's a lot of— Hong Kong is a university in Hong Kong that published a lot of very good work on agentic systems. There is a lot of good work happening in Europe. Europe as well as the US. And what's interesting, sometimes, you know, these frontier labs, they do publish extended papers on how they've built their systems, what went into those systems. Those are extremely, you know, interesting reports to look at because it's, you know, it's the science but also engineering part of how these systems are built. So I would recommend, there was just a recent report coming out of DeepSeq. There was also another one coming out of Qimi. These are sort of frontier labs out of China that publish a full report of how the engineering is done, science plus engineering. But from scientific breakthroughs, I think that again, looking at some of these conferences is where a lot of really exciting work is happening.

Absolutely. Yeah. Certainly, even if you can't visit them, then just have a look at who's speaking and what are they speaking about. That's good for giving an overview.

I think that's an excellent way to kind of like just kind of like understand what, what way the frontier right now is.

Oh man, you can even build an agent to go and scrape conference websites.

You can just kind of build an agent, scrape the conference website to give you summaries. I mean, that's actually how a lot of times even I myself, like a lot of times that are reading the full paper, you give it to the agent and summarize, kind of like gives you the right, you know, information. And then you can decide where you want to go and actually dive into more details, more technical details.

Wonderful. Well, it's been a pleasure speaking to you. Thank you so much for your time, Russ.

Thank you. Thank you so much for having me.