A Foundation Model for Food, with Chef Robotics CEO Rajat Bhageria

Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week we'll explore their revolutionary ideas, and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host Eric Thornberg. Hello and welcome back to The Cognitive Revolution. Today, my guest is Rajat Bagaria, founder and CEO of Chef Robotics, a pioneering company that's taking on one of the biggest challenges and commercial opportunities in AI today. How to incorporate the latest advances into embodied systems so that they can reliably handle the diversity of real-world contexts and deliver practical economic value. Chef focuses on food assembly, which I was surprised to learn constitutes an estimated 70% of labor costs in food manufacturing facilities, more than ingredient preparation and cooking combined. And Chef robots have now assembled some 20 million meals. In this conversation, Rajat shares a detailed account of Chef's technology, including the form factor of their robots, their use of imitation learning, the tremendous efforts they've made to ensure consistency in dynamic environments, their multi-layered strategy for resolving production issues, the data flywheel effects they're beginning to see, and how they are leveraging that data to create a new foundation model for food manipulation. Looking ahead, Rajat sees Chef's robots moving into commercial kitchens of all kinds and believes we will ultimately see a reorganization of the food industry with a proliferation of cheaper and more customizable meal options produced in robotic ghost kitchens and delivered by small autonomous drones. Perhaps surprisingly, in contrast to the more familiar Jetsons-style vision of domestic robots that cook for us in our homes, Rajat envisions a world in which the cost of high-quality food approaches the cost of ingredients and thus home kitchens become an optional add-on for hobbyist home chefs. As someone who is always looking for realistic glimpses of daily life in the AI-enabled future, I found this vision quite thought-provoking, and it's inspired me to think about what other second-order effects we might ought to expect. As always, we appreciate it when listeners share the show, and we'd love to hear your feedback. You can contact us via our website, cognitiverevolution.ai, or you can DM me on your favorite social network. And briefly, if you are an AI engineer or an AI advisor who's excited about helping diverse real economy businesses apply AI technology, or if you're interested in sponsoring the Cognitive Revolution, I would be excited to hear from you. Now, I hope you enjoy my conversation with Rajat Bagaria, founder and CEO of Chef Robotics.

Rajat Bagaria, founder and CEO of Chef Robotics. Welcome to the Cognitive Revolution.

Thank you, Nathan. I'm super excited to be here.

I am too. This is cool. We've done obviously a ton of episodes on a ton of different things, and including a few on robotics research, but I think this is the first time that we'll have an opportunity to talk about a product that is deployed at some non-trivial scale in the robotics wild. So I'm excited to get into that with you. I guess for starters, you wanna just give us the very high-level overview of what is Chef Robotics, and then I'll dig into all the technical and socio-technical nooks and crannies from there.

Yeah, that sounds good. So yeah, Chef Robotics, we make AI-enabled robots for the food industry. We're starting with actually food manufacturing, and that's because it's actually a very kind of relatively good place to collect training data, add ROI to customers.

And we can talk more about the nuances of that, but it's a really good starting point where we can kind of relatively quickly ship robots.

It's like L2 autonomy, you could say, in the AV parlance.

And we can generate ROI for our customers. And of course, the more robots we deploy in this area, the better that our chain dataset gets and our AI gets, which then allows us to have a more dexterous system, which then allows us to do hopefully more complex applications, right? Such as ghost kitchens, such as fast casuals, et cetera. The ultimate goal, of course, is to deploy an AI-enabled robot in really every commercial kitchen around the country and the world.

So, that's broadly what we do.

And, of course, we lead from there. Cool.

So, when you say food manufacturing, What should I conjure up in my imagination there? Is this a packaged meal that I would get from a Costco-type outlet that is getting created?

Yeah, more or less.

Yes. When most people think about food manufacturing, they think that's already automated, basically. And actually, like, there are parts of it that are automated.

So if you think about, if I'm Kraft Heinz making ketchup or something, it's already automated. Right.

On the other end, that's called low-mix manufacturing, where essentially you don't have that many SKUs. So let's say you have 10 SKUs. What you're going to do is get kind of 10 dedicated lines for each of the SKUs, and you would run them all day long. Then there's this idea of high-mix manufacturing. High-mix manufacturing is if you have hundreds of SKUs, for example, meals.

So, you know, prepared sandwiches, salads, party trays, meals you see at Costco or Trader Joe's, frozen meals, things like that. Meals you might even find at a hospital, for example. Patient trays, schools. They have a lot of variety, right?

Some people like vegetarian meals, some people like meat lovers, some people are vegan, some people are gluten-free, et cetera, et cetera.

And each of these has 20 different SKUs within it. So there's just a lot of SKUs. So the way that these meals are actually made in status quo is actually a very manual process, a lot more manual than people might think. You essentially have these long conveyor lines where you have people scooping food from big tubs into individual containers. And that's the kind of the chad, at least, that we're kind of going after to start.

Interesting. So what is the form factor? I've checked out and seen a little bit of the video footage on the website. It appears to be an arm that is mounted to a table.

Is that right?

We have these modules. These modules are actually on casters, so you can move them around easily.

Wheels, basically, right? You can move them around easily.

And it's kind of like a mobile manipulator, I guess you could say. But basically, like, our customers aren't moving these robots around all the time.

It's like maybe once a day, twice a day type thing.

So you actually don't need to be autonomously mobile. We have a 6-dof arm on it, it's a collaborative system.

And then there's a bunch of different sensors on it, like RGBD cameras, force torque sensors, scales, a bunch of different sensors that kind of help us make sense of the world, make sense of the environment.

And that's kind of it. It's a very simple system.

And then you might have different kind of end effectors, different utensils based on what you're manipulating.

On the hardware side, we really try to make it an extremely simple system that We can kind of mass manufacture.

It's not like we're trying to make custom hardware per customer. It's really, let's have a module and this module is the same footprint as a person. In other words, it can slide onto an existing line without retrofitting. Let's mass manufacture it. Let's use mostly off-the-shelf hardware. And then it's really put our blood, sweat, and tears into the autonomy and make it as flexible as possible. So you can do as many tasks as you possibly could. Interesting.

Okay. So I'm imagining now. I have a household of 7, so we do not infrequently buy the Costco prepared meals. And I'm imagining the sort of before is like people standing at workstations and doing an assembly line type process, except managed where the individual tasks are done by different people. So is it like today a person is sitting there chopping onions all day that go into the stir fry or whatever. And then strategically, people can get one of your robots and say, okay, we now don't have to have a human chop the onions. We'll have a robot chop the onions. They put in, they can roll this thing around and put it into different positions and then do— it's not a mounted arm, but it is arms, right? It's like primarily, is it one arm?

Is it 2 arms?

Do the arms work together? Tell me a little bit more about the arms.

Yeah. The way you can think about the process right now is in basically every commercial kitchen, there's 3 processes.

There's prepping, there's cooking, and there's assembly or plating.

Plating is what you might see at Chipotle, right?

Which is a human kind of scooping ingredients out of a hotel pan, a tub, and they're scooping that into a burrito or tray or a bowl, whatever it might be in that instance. What you find is actually that the most labor-intensive process is actually the assembly or plating.

And the reason for this, and we could talk about this more, but the cooking and the prep actually scales relatively sublinearly when it comes to at least non-fine dining.

For fine dining, cooking is the most expensive.

But for everything else, so like food trucks, fast casuals, prisons, hotels, universities, K through 12s, manufacturing, everything else, airline catering, all of that, the prepping and cooking scales sublinearly.

In other words, if you need more volume or if you need more throughput, you don't necessarily need more humans.

And then on the other hand, the assembly scales more linearly.

In other words, if you need more throughput or volume, you usually need linearly more humans.

So we really focus on food assembly.

So your kind of picture is correct though, right? So when you come to a production room, you'd find a long conveyor, you'd find 20 people standing on either side of it. Each person would have a big tub of food.

They basically—

they have a utensil, or maybe they're using their hands, scooping ingredients out of these big tubs into trays or wraps or burritos or sandwiches and move down the line. And they might be doing this in a 34-degree room, which is actually the usual status quo. And so that's the status quo.

And so now what we do is we make this robot module.

It's the same footprint as a person. It can just slide onto an existing line. It takes the same footprint. There doesn't need to be any retrofitting. And then basically we do the same thing that a human might be doing, right? The idea, of course, is that this is a very kind of hard job, right? It's extremely monotonous. It's 34 degrees. Oftentimes, like I said, you're actually touching frozen ingredients with your hands. And you can't even feel your fingers after 5 minutes. So there's an extremely high turnover rate and it's really hard to hire for this job.

So I think our idea, of course, at least for this go-to-market, is that we can help these customers really have human equivalent that can then allow them to offset that existing labor to a different part of the plant.

And that existing set of folks can hopefully help with doing kind of higher-value tasks, things that humans really should be doing. Continuous improvement. Let's improve this process, make it more efficient, material handling. Things like that, right?

So that's how the process might work. Now, to your other question about number of arms, right now we actually use one arm. And that's at least for this task, the food assembly task, which is of course like, just to be clear, 70% of the labor in food companies. So it's a gigantic market just if we just do food assembly, right?

So we're not too worried about things more than that at the moment.

But of course the idea is that we can do more down the line. Right now the hardware very much is one robot arm per module.

But of course we can have multiple end effectors, multiple utensils per robot arm to do various things.

Okay. That's really interesting. The fact that 70% of the labor goes into plating is definitely not something I would've guessed. Calls to mind a— we did an episode with the, one of the founders of Codium and kind of somewhat similar stat when it comes to how much time software developers actually spend coding is only maybe a third of their time as well. So that's surprising, but interesting.

Okay.

How do people set these things up if they get a robot from you? And now they, I don't know if this comes in a box and they assemble it or go onsite and assemble it for them and interested in kind of some of those delivery details. But once they've got the thing on wheels and they can roll it around and it's okay, Mr. Robot, this is your station.

Yeah.

What does the process look like of dialing in its behavior to this particular meal and the serving size and the place it's supposed to go, et cetera?

Yes.

And then of course, that's the hard part, right? I think there's 2 different parts to this. So on the day of production, let's say, so I guess maybe instead of taking a step back, like we ship the robots in crates, they're like wooden crates, they'll arrive.

The deployment process itself is actually quite simple. You literally take the robot out of the crate. already on wheels. You slide it onto the line. Usually it's a food facility, so they'll wash it down and do sanitation, and they'll take like a swab, like a bacteria swab, to just make sure there's no like pathogens we're introducing.

But anyways, like basically slide down to the line. We require 2 inputs from the facility, which all kind of facilities have.

One is just 110 AC power, and then 2 is like compressed air. And that's just to actually manipulate the end effector, right? It's a pneumatic end effector.

So you plug those 2 things in, and that's like And then part 1 now for actually using it is by somebody who's actually on the floor. So on the floor, we try to make it as simple as you possibly can imagine. And the reason for this is like our audience is not technical folks.

It's not often even English-speaking folks. It's often people who haven't— don't have a 4-year college degree, et cetera, right? So it's— we try to make it as simple to use as maybe they would use an oven at their house.

Which is to say, you go up to the robot, there's this nice touchscreen. And you can use your gloves to say, hey, this is the meal I want to run today.

So this line today is doing the pad thai meal. Just made that up, but whatever meal it's doing.

So I'm doing the pad thai meal.

And robot number 1 is going to do the noodles. Okay, great. So now you have that.

And then the system will say, okay, great. Now you need to attach this particular utensil. We'll recommend a utensil to attach.

The user will attach this utensil. And at that point, what will happen is the user will load the tray of food. right into the module.

And basically the system will fine-tune itself, which is to say it'll pick and dump ingredients from one pan to the other and just like really fine-tune with this particular batch of ingredients. And the reason, by the way, this is important is that the thing with food is it's, first of all, it's an organic compound, but also it's cut and cooked by a human.

And every human, of course, is different. And every single day it's different. No batch of food is ever the same. Even if you'd like it to be the same, it's not.

So, You have to fine-tune for that particular batch, which is to say today the cook added a little bit, they cooked it a little bit longer or less long. It's more sticky than we would've expected, or it's more oily, or maybe it's a little more dense, or maybe there's different parts of them. Like if it's a curry, maybe it's different parts of the curry are less dense or more dense.

There's different properties that change based on the material.

And at that point, basically it's fine-tuned.

You press play and it'll start to pick and place onto the line.

So that's kind of the day of. Again, that's an extremely simple process.

This process is in really any language you want. So you can do it in Spanish, you can do it in whatever language you want as a user. So that's the production day-of process, right? Now there is another process, which is, okay, how do you actually onboard a new ingredient? Now, when we're first initially deploying the robots to a customer site for a new customer, our solutions engineering team would do this for the customer so that the customer kind of is ready to go day-of, right? On an ongoing basis, what we do is we create a lot of software to allow our customers to self-onboard new ingredients. So the way this might work, let's say I'm a production manager now, or I'm a planning manager, and I have a new meal that I'm running next week, or maybe even tomorrow.

The way this kind of works is that each of our customers have these, like, what we call, like, meal cards. They're kind of like PDFs. And this meal card might say, okay, well, the first ingredient's going to be the noodles, and it's going to have this image, and it's going to be this portion size, and I want to place it into Compartment 2. And, and you maybe have like 7 or 8 ingredients on this meal card.

So what we do is we have a web app that we give to customers and we essentially have them upload those meal cards and then we'll use a VLM to kind of parse that.

And we'll take that unstructured data about all these metadata about this meal into our database.

That's step 1.

Now, actually, that is sufficient to actually have the robot run. You can imagine that just with that information, which is here's the ingredient name, here's the portion size. You can run. But of course, the consistency might not be very good. It might be spilling. It might not be placing as precisely as you need it to place.

In other words, the performance might not be there right then and there with just that initial kind of ingestion.

So you need to do some stuff, which is to say you need to kind of have a labeling process to actually make this run with high performance at runtime. So now what we do is we try to get as much information as we possibly can. So the starting policy of how the robot manipulates this Novel ingredient is a good one, right?

So now we kind of go ingredient by ingredient.

So let's take the noodles.

Now we have the image, we have the string name, we might have the the portion size, maybe the compartment.

So we take all this information and we say okay. So we use LLM to kind of give the users a few different image. We ask the user a few different questions, and for each question we give them different images so they can have a slider about okay where does this ingredient stand from an attributes perspective. For the noodles, we might. And we, of course, ask intelligent questions. We might ask the question, okay, how long are the particle sizes? And on the very left side, it might be like an edamame piece.

On the right side, it might be like a very long piece of noodle.

And again, our goal is just to get as much understanding from the user as we possibly can about this ingredient. Another question might be, okay, how sticky is it?

We might show an ingredient that's very sticky, like maybe it's cream cheese. And then on the other side, we might show an ingredient that's very non-sticky, like water. And of course, there's Generated images in between here.

So we generate those questions, we generate those images, and now basically the user has told us—

and this takes all of a minute, by the way—

the user has told us like, here's generally some attributes about this new ingredient. So now we have those attributes, we have the image itself, we have the string name, we have a portion, we have a compartment. All of this kind of comes together and we— and we have a model that actually throws— spits out the starting policy. Of the ingredient, of course, the starting policy is based off of existing production data. So at this point, Chef has made 18 million servings in production.

We have a ton of production data.

I've done thousands of different ingredients, right? This the starting policy for this novel ingredient is probably based on something we've done in the past. So now we have a starting policy, and then you can deploy that starting policy.

The user, the production management, deploy that starting policy on the web app to the system itself.

And now we're back to that initial story.

So now when the system, when maybe the line manager now, or even the line worker actually says, hey, I want to run the pad thai and I'm running the noodles, they'll slide the noodles pan into the system, it'll pick and dump, and it'll fine-tune that starting policy to a more fine-tuned policy for that particular batch of noodles and for that particular customer's batch of noodles, which is of course different than every other customer's, how, how they cut and prep their noodles. That's broadly how the onboarding process works for us right now. Hey, we'll continue our interview in a moment after a word from our sponsors.

That first process sounds like a foundation model for this form factor where it can take in any of a super wide range of variables. And then are you actually fine-tuning that same underlying model with, it sounds like a self-fine-tuning process where actual weights are getting changed based on a few iterations of whether the noodles hit the spot or not?

Yes, at least for the former, I think like Shav's been in production for a couple years before foundational models were really a thing, I would say. So there are a bunch of parts that are foundational models, and there's some parts that are more like deep learning, right?

Like deep neural networks, CNNs, things like that. So now what we're doing is actually today, as of right now, basically we're making that, like you said, like more generalized food manipulation model for that first kind of process. And then on the system itself, like you said, yeah, there's like fine-tuning that's happening and the system's doing that itself.

And, and the way this of course works is that we have a lot of metrics, right? So we, we can close the loop. Um, and, and there's a few metrics that matter for, for Shep, right? The most obvious metric is the pick weight, which is to say, if you wanted X ounces, how close are you to X ounces? And of course we look at both the kind of bias, which is like, where's the average pick weight away from the target? Mm-hmm.

How far is it from the mean as a whole, the whole distribution? But we also look at standard deviation, things like that.

So that's the pick weight aspects. And then we can also look at other aspects.

So we do placement QA.

So we can get a sense of, okay, when you place, is the food spreading the way it should? Is it spilling into the other compartment? And we have different scores that we assign for the placement based on this. That's another metric we have. Another metric we, of course, have is reliability. And there's a lot of proxies for reliability. Right.

So one proxy, for example, might be like, one of the things, for example, we do is we have force torque sensors in the robot. And so we can say, okay, let's take a path that's going to minimize the incidence of like things, protective stops, for example, to make sure that the robot doesn't protect itself. So the idea of protective stop, by the way, is that collaborative robots are kind of pressure and force limited to 150 Newtons for safety, physical human safety.

That's great.

But sometimes, of course, it'll protective stop when you don't want it to protective stop, which is to say when you're manipulating food. We try to prevent that, for example, from a reliability perspective, because of course that means that the line will be down for whatever, 30 seconds, which is not the end of the world, but it's not great either. And then we also look at things like, okay, we can measure the pressure and the flow of this, of the, in the end effector itself. So we can make sure that we're not crushing the material and things like that. So there's a few different kinds of things we do to really make sure that this model, as it's picking and dumping, it's actually converging to a policy that's going to maximize Some of these outcomes we want, mainly pick weight. That's the most important for the customers.

How much variation is there on that in, let's just start with maybe like a human-powered setting. I think it's always instructive actually to get these baselines and in so many instances they're just not there. People have the sort of sense of the human is doing a good job. What is a good job and how much variation is there in what is currently passing as a good job? So I'd be interested to know what the accuracy and consistency of a pick weight is with humans and how you guys compare to that with the robot.

Yes.

Yeah, no, it's a really good question. And actually it's a really important question. And the reason for this, by the way, is like when our customers, and basically this is true for humans, I think, like they have this perspective that the robot should be like perfect. You see this in autonomous vehicles too, right?

Like people expect an AV to be perfect. People certainly expect our robots to be perfect. Like I've had conversations where like customers like, oh, you do this like cool AI thing, right? If you use AI, then obviously it's gotta be perfect.

It's the whole point.

Why am I getting robots that aren't perfect?

So like, you know, from a sales perspective and a customer success perspective, that of course leads to like a lot of like mental churn for both parties where it's like, well, the robot's never gonna be perfect because the, the input, the, like the food itself, it's very highly variable. Actually, yeah.

So part of our kind of deployment process, our solutions engineering process is that when we first deploy with a customer.

We'll actually go and take baselines.

And these baselines are basically like, okay, let's actually collect human data in a kind of as blind as we can get manner. So we try to do that to like really understand where the humans are.

And what's interesting is that they, you know, when we actually present this data to the customer, they are like often not in belief of that.

They don't believe how bad their humans are basically because they have these like very tight bounds oftentimes. And, you know, we show them this human collected data and it's, oh wow, this is actually not that great.

But by the way, what that does then is that sets our bar. Like, that's like we have to meet or beat that.

And one of the things that's interesting with the food industry is there's these governing bodies like FDA and USDA and, and of course the customer itself, like you and I as customers, right? We—

and people really care about if the meal itself is like X ounces, like 5 ounces. If you're below 5, then it's really bad, right?

So what that means is that like the The customers, like the leadership at any of our customers, which is food companies, they are very highly incentivized to make sure they're never under because there's going to be fines and maybe their customers are going to churn. So, what they're telling their line managers and their production managers is to always overpick.

So, what you usually find is there's this very high bias, is what we call it. Basically, they're always picking well above the target weight.

So, if the target weight is 1 ounce, they might be picking on average at 1.2 ounces. Right? That, by the way, is very expensive because in a food company, 40 to 50% of COGS is the material itself.

So if they're just like giving away—

it's called giveaway— if they're giving away food, that really hurts their bottom line.

So one of the big things that we find is that we can really help them dramatically decrease that giveaway, which is a really good ROI, by the way, for our customers. And it's nice because we can also— we have closed the loop, so we know that we're better. We have data to show and prove that we're better.

There's also some interesting, more macro perspectives. Right now, we don't have enough robots to make a really big dent of this. But you can imagine if we have millions of robots in production, not only in manufacturing but also ghost kitchens, fast casuals, prisons, hotels, etc., all commercial kitchens, then that can actually have a pretty big impact on food waste and things like that, which is cool too. Can you—

is that 1 ounce to 1.2 a typical number? I, I would love to get just a little bit more concrete, like how much is the overshoot and, and how much do you improve on it?

It really depends.

On it, there's so many variables. It depends a lot on the customer.

Some customers are really good at this.

Some customers are not as good at it. And it really depends on the ingredient.

And I'll give you some exact numbers, but just for a piece of context, like imagine broccoli. The thing with broccoli is that the particle sizes are quite big. And oftentimes there's 4 pieces of broccoli, like 5 pieces. So in other words, if you scoop it and you got 1 broccoli too much, then you're super high. Or if you got 1 broccoli too low, you're super low. It's not— On the other hand, let's think about like a grain of rice.

You get a few grains of rice too much, no big deal.

A few grains of rice too less, too little, no big deal. So there's certain ingredients that are just really hard for humans. So for broccoli, humans might have something like 30%, 25% bias. Even for us, we have to bias up because we really want to prevent the negative case we underpick.

So we might have something like 5% bias for a piece of broccoli. Generally though, for most items, let's say imagine something like a potato mash or a sauce or diced vegetables or cooked diced vegetables, whatever it might be, humans might have something like 4%, 5% bias, which is overpicking above the mean. And then Chef might have something like more like 1% bias. We can actually basically guarantee that we'll have less bias. We can guarantee we'll have this because we have like a control loop, right? That's constantly going about like how much pick weight we had.

So that's on the bias side.

And then on the standard deviation side, what we do is we have this metric called CV, coefficient of variation, which is basically like standard deviation divided by the target weight.

And what this basically allows us to do is standardize across pick weight.

Because of course, if you have bigger portions, of course you're going to have more standard deviation.

So from that perspective, humans might have something like 16, 17, 18% CV. And again, it just differs so much by customer.

But yeah, like humans might have something like that. Chef might have something like, our best is probably like 5 or 6% CV. But I think on average, probably more like higher, 7, 8% CV. But that's how we think about that. Like CV and bias are the 2 metrics that we really try to Optimized for at least the pick weight part of the equation.

So you said these things are like drop-in replacement. I think you've used the term like human equivalent a couple of times.

Yeah.

If I'm imagining myself doing this, I think maybe I can do 10 scoops a minute, maybe 12 scoops a minute. Is that kind of the same rate at which these things are?

Yeah. You also see a lot of variation there.

So for this particular problem, there really wasn't any There's just no automation that really existed for this problem before Chef. It's been a, it's been an industry where you have tens of thousands, hundreds of thousands of humans who do this, right? The reason I bring this up is because they've really optimized the humans very well. There's a lot of, you can imagine a lot of like process optimization. And that's really, that's like ergonomics.

Like how do you sit relative to the line? And yeah, it's, are you sitting versus standing?

Like raising your left hand versus right hand? There's a lot of ergonomics to think about. But anyways, all of that to say that, like, I think most times we go to a customer site, they're running anywhere from like 15 to 20 trays per minute type of thing. Sometimes they'll do this thing where they'll pick up a bunch of material and they'll do something like a place, place, place, like one pick, multiple deposits. So you amortize the transport time over multiple deposits. So you get slightly higher throughput. But Chef is around 20 trips per minute. And then if a customer needs more than that, and sometimes customers need more than that, we also have this like multi-pick system, which allows you to have multiple grippers, multiple utensils, I should say, per system. And that allows you to have much faster throughput. The main constraint for us, of course, I think like the obvious question many people think about is, okay, why don't you just put like an industrial robot here, right?

The thing is there's a lot of safety constraints with that.

The whole point of Chef, from a more philosophical perspective, is like we try to partially automate a process. And the reason to partially automate a process is if you have a full-line automation, the thing is if you cannot do one of the SKUs, one of the ingredients, or let's say I'm FedEx and like a robot can't do one of my parcels. Then I basically haven't automated the process because I still need a human to do it. So you find a lot that if robotics companies try to fully automate the process, they have a ton, a very long sales cycle because it's very risky, very long deployment cycles. And the technology bar is extremely high because your autonomy needs to be able to handle literally every single SKU. Whereas for Chef, we can partially automate a line. So, you know, if there's a particular ingredient we can't do, that's okay. For example, we couldn't do Let's say we can't do shrimp.

That's okay. Human can do the shrimp.

We can do everything else.

So, you know, what's nice about this then is that like the deployment process is quick, the sales cycle is quick, the technology, the way of flywheel, the more robots we deploy, the autonomy gets better. So all these things are very nice. But then what it means from a safety perspective is that you are constantly around humans. There's a human to your left, there's a human to your right, there's a human refilling the robot.

So from a safety perspective, collaborative robots are a necessary requirement. And there's of course laws around the max force that you can impart. And there's laws around the max kind of joint velocities you can have with—

not necessarily joint velocities, but basically there needs to be a strong safety case around this.

And that's where we found that collaborative robots plus the right tooling, end effector tooling, allows us to get the throughputs that our customers need.

So if it's 3 or 4 seconds per trip, That comes out to pushing 1,000 per hour. And then you've got 20 million pushing— I like to do, you know, round numbers. So pushing 20 million servings, 1,000 an hour would be 20,000 hours. So it sounds like you've got kind of 10 man-years worth of work completed so far.

Yeah.

What does that look like in terms of the portfolio of customers? Like how many robots actually is that? And how do you see that scaling into the future?

Yeah. Right now we have robots in, mostly in North America right now. So we have robots in 6 cities.

I think it's 5 states and US and Canada right now. So we went into production in mid-2022. That's when we kind of deployed our very first systems. And since then, it's from '22 to '23, we scaled 4x in terms of like revenue and And this year expecting something similar, something more like 3x this year.

So I think what's really nice about this market is that it's huge.

It's not something that many people think about, but like production is just gigantic. And so I think right now our goal very much is to scale within our current customers and of course scale to net new customers. What's nice about our current customers, we actually have a few whales. And generally the thing with production is that If you open up a plant, you're not going to have 5 people doing this. You're going to have 50, 100, 20, whatever the number is, but it's going to be a significant number. So, and each of our customers often have multiple plants. So the point is that to get to, let's say the bar is $100 million in ARR, that doesn't require 50 or 100 or 1,000 customers. Like we can get there with 20 to 30 customers. And of course what that requires is like a ton of focus on customer success and really making our customers very happy. So we spent a lot of time on customer success to really make that happen.

And I think the goal is, yeah, just continue growing 3x or so per year. Just, and over time, there's gonna be a lot more other products we sell, other products, other markets, but at least for this market, we think we have found product-market fit.

It's kind of just like scale now.

Hey, we'll continue our interview in a moment after a word from our sponsors.

I'm gonna come back to the business model, maybe a little bit more Toward the end. But let's talk a little bit more about the tech stack. I mean, there's obviously been an insane— and I think this is a challenge that many companies have faced, especially on the software side, more so than probably on the robotics side, but maybe there too, where you set out to build an AI company 5 years ago, and just as maybe you were starting to get things to work, all of a sudden a whole new wave of enabling technologies have come online. And I think for a lot of companies, those have presented some really hard dilemmas where they're like, man, we just got this rough edge sanded down, and maybe the new foundation model paradigm doesn't quite do that in the same way, but it has all these other things. So tough choices. So how has your kind of core AI strategy evolved? And you could cash that out into different kinds of models or different training regimes or whatever.

No, it's a really good question.

I really do think actually that Foundational models have really uncapped a lot of robotics potential, I would say.

The issue with robotics has always been the world is very highly dimensional.

Like, if I can pick up a wine cup, that doesn't mean I can pick up a teacup, and that doesn't mean I can pick up like a water glass.

It's just a very highly dimensional space.

And in our world, every customer cuts food differently, cooks food differently.

Every individual does that differently.

Every site does it differently. Different countries do it differently.

And there's just so much variety in the world. And that's always been the The issue with robotics?

The thing with AI is, and especially like the modern kind of, the more like recent advancements in it, right, with VLMs and LLMs, things like that, they're really good for like high-level task planning, right?

Which is to say, you know, what's in the scene, but they're trained on internet data, right? So there's a lot of like image and language pairs, but there's no like understanding about like how to actually move the arm.

There's no motion data, if you will, right? Unlike something like a GPT or a large language model, we at Shep can't just download the internet, if you will, as training data.

That's not possible because there's no trained dataset with food manipulation. So what do you do then? There's a few options. Option 1 is you do sim. And sim is interesting for some companies, I would say.

So for example, if I'm a self-driving car company, a lot of the training does happen in sim. It's not perfect, but you can get some of the way there when it comes to modeling things. If I'm doing rigid body manipulation, let's say I'm doing like Palatizing, depalatizing, case packing, things like that, then I can use sim.

With food though, of course, it's wet, sticky, deformable, malleable things, often sometimes non-Newtonian things. The physics simulators out there just aren't good at that. So sim doesn't really work.

You can do testing things from a reliability perspective for like your high-level motion, but you can't do that for grasping. And the hard thing with Chef is the grasping. That's the hard thing. And that's the thing you can't actually do in sim.

So what we had realized is that really the best way to actually get this training data is like in production. And that's why this 20 million serving food done is so important. So that's kind of like status quo. Now, where does this new stuff, like I said, why does it help us?

I think the part that really helps us is like leveraging transformers to do things like learning from demonstration and imitation learning. So specifically, like one of the big problems we face is kind of like transferring knowledge across ingredients, right?

So like if we see a sticky rice at customer 1, can we then leverage that to do another sticky rice at a different customer? Because of course they're cooked very differently.

And transformers actually really help here, right? We can kind of encode important characteristics about one sticky rice into the embedding space of the transformer and then use that to determine ingredient similarity for a different ingredient.

So I think imitation learning and by extension reinforcement learning are things that we're really heavily investing in.

We've already now done a lot of.

to actually basically reduce the time it takes to onboard a new ingredient. So initially we talked about the process to onboard a new ingredient with this web app that leverages a lot of like more traditional VLMs and LLMs.

And then of course we've fine-tuned them for our own use cases. But I think a lot of the novel AI stuff we're doing leveraging transformers and GenAI is really IL and learning from demonstration, I would say.

Yeah, that's quite interesting to say the least.

Did you see the project? I'm sure you did.

It was spearheaded by folks at Google, but it involved groups across a tonne of different, mostly academic labs where they collected, I think, all for robot arm form factors. They collected a bunch of different datasets and then created essentially the foundation model combining all those different datasets and then returned that back to the community.

Yes.

Is something like that useful to you?

I think it's useful, like something like that.

We haven't tried it, so I don't want to say yes or no just yet. I think something like that could be very useful when it comes to that first part, the onboarding part on the web app for us.

The key again is that, at least from what I understand, that training— the training data is really key here.

So if you train on rigid body items, you're not going to be able to manipulate malleable items. Even if you think about the food demonstrations that people do, You're not always doing the primary handling of it, I guess you could say, right? We actually have to like handle the raw material and it's dripping everywhere and it's sticky and it's wet and it's all these things, right? And it'll compress and then you can't uncompress it. So I think our approach right now is how do we leverage imitation learning, reinforcement learning, and then also our production data. And I would argue that the most important part of this is the real-world production data to then train our own kind of in-house kind of generalized food manipulation model. And that's what we're trying to do, which is like this end-to-end system that, that we've trained to manipulate food. And we're starting with food assembly because that's the majority of the labor issue. But over time, you can imagine that this does other stuff. And of course, with IL, you can very quickly teach it new tasks, right? It's like, there's a lot of operations you see in a food facility that you might not imagine. Taking sheets of baker's trays and putting them into a rack. I mean, that's not like directly engaging with the food, but there's a lot of like random tasks, right? Like this. Right now we focus on assembly and plating, but over time we can do some of these other tasks, all powered by this kind of generalized food manipulation model. Right now I would say that we're very focused on IL and RL more than that second part, because that's like the most useful for our customers and us. Because it basically allows the time to onboard a new ingredient, just like a new class of ingredients, I guess you could say, much quicker.

So for reinforcement learning purposes, you clearly have some objective scoring capability or opportunity around Did you get the right weight, for example? I wonder if you see a use for things like the Eureka project that came out of Nvidia. This is one that I keep going back to over and over again, because what I'm trying to do broadly is understand what is the state of the art, what's possible now.

Yeah.

It's obviously moving so fast that it's a full-time job just to keep up with. But one of the things I used to say was, when I was summarising AI, state of AI for other people, I would say no eureka moments. Like, the AIs are getting really good at routine tasks, but they're not coming up with insights on their own that certainly don't surpass and seldom even rival the sort of insights that people can have. Now, one exception to that, I would say, has been this Eureka project where they had GPT-4 write the reward function that they used to train this robot hand to do various manipulations. And I always think about that as, for one thing, because I used to say no eureka moments and they called the project Eureka. So that's uncanny unto itself. But the idea that it is better at writing these reward functions than the people that they had participating was quite interesting. So I guess you could take that question very specifically at the Eureka project level, or more broadly, is there sort of a bootstrapping dynamic that we're starting to see where Models can feed into this cycle of learning new tasks?

Yeah, I mean, it's a very good question. So we talk a lot about this, right?

So it's like a parameterized policy with a non-parameterized policy. What I mean by this is the 4 foundational models and IL, like IL has been around for a while, of course, but I don't think it really started to work until transformers. And I think transformers have actually been really useful for that, like diffusion policies, basically. So let's think back to like, I'm in 2019 and I'm starting a robotics company.

I think that the self-driving car industry, the AV industry, set a really good precedent of, okay, you have a rule-based system.

Of course, it's using a lot of deep learning and CNNs, but it's a rules-based system.

What we would do is, back in that time, we would have a policy per ingredient, and that policy would be bootstrapped, like you said, and parameterized.

So we'd have 200 parameters basically.

And this would be everything from a bunch of physical attributes about how do you actually—

the pressure and the flow and the dwell time of the food and how many shakes you do and what's the amplitude of the shakes and like a bunch of different parameters about how do you manipulate this ingredient, right? And then we built a lot of like infrastructure on how do you like onboard new ingredients? Leveraging those existing policies, that was kind of how we did it back then. Now, and that's like parameterized.

There's these different parameters, but those parameters are set by a human.

It's set by a human engineer or like an applications engineer.

But it's not necessarily true that those are the right parameters.

We don't really know, right? We think they work well, and like we said, we have this like control loop that allows us to actually know the result we're looking for, which is things like pick weight and how much spillage there is and placement quality and all these things reliability. But we don't really know if it's actually like a local maximum or a global maximum. What imitation learning allows us to do, and learning from demonstration, is what we can do is we can teleop using— let's say you have 2 robots side by side. You can have one robot literally mimic a human scooping it.

And it's, by the way, it's embodied.

It's grounded, if you will, because we have the exact embodiment that we're teleopping doing that. And the exact— it's the same robot doing where the human's doing the teleop, the demonstrations, and the robot that's actually doing the manipulation. So there's no bounding problem, I guess you could say.

And now basically it's like there's, it's like not, it's not like there's human written parameters.

It's like fully end-to-end.

And I think that's been pretty cool actually, because we haven't had like an insane, whoa, like how did it ever think of that?

We haven't had a moment like that, but it's like cool because it's really an end-to-end system and it's quite simple. There's not like a bajillion lines of code and it's like quite simple and it actually works quite well.

So. I think that's interesting because the time it takes to onboard a new ingredient is one of the most important metrics for us. And the reason is that the crux of what we are doing is providing a flexible automation solution to our customers. And in other words, the more ingredients we can do reliably, consistently, the more useful this thing is. In other words, they're gonna use it more. Like if they could go, if we could do 1,000 ingredients versus more applicable, maybe like if they can do 100% of their ingredients versus 60% of the ingredients, they're just gonna run the robots more.

If they run the robots more, they're going to buy more robots, which of course is great for our business, right?

So the number of ingredients we can do is really critical. And if these new technologies allow us to compress the time it takes to onboard ingredients, then we can have higher utilization of our robots, which means that our customers are going to buy more robots, which means we have even more training data, which allows us to accelerate this data flywheel.

And on top of that, there's another flywheel, which is the more robots we ship, of course, there's other flywheels. We have case studies, and these case studies allow us to get even more customers. We have More volume of robots are manufacturing, which allows us to improve our bill of materials and supply chain benefits. We have operational benefits because we're scaling to more plants and more sites, which means we have more people all over the country supporting these robots, which then allows us to support other similar customers in the nearby area.

So there's all these kind of flywheels, but I think the crux of it always comes back to the number of ingredients we can do.

And I think the reason transformers, at least I would say, and diffusion policies are really powerful is because Of that idea. It accelerates the flywheel for us, which is the crux of the business, ingredient coverage.

Those sort of macro outer circles are really interesting. What about the inner loops that kind of comprise the system? Actually, I was thinking back, we did another episode with Skydio, the drone maker, and they, he walked us through, they're pushing up to the point where you can give it like a verbal command, hey, go survey that cell tower and come back with a a full inspection of it. So that's like the highest level outermost loop. And then the innermost loop, I think, was running at 10,000 hertz or something. And it's just like the very low level voltage control. I imagine you have something similar going on there, although if it's 3 seconds, like, I could imagine you could maybe generate the whole 3 seconds and let it run. But what are the sort of loops? How many corrections? Are there moments in that 3-second cycle where you get to say, hey, I'm off course? What's that look like going down to the lowest level?

Yeah.

Well, so I think that to start, there's 2 computes per robot cell, per module.

One is the actual physical robot controller.

And so the robot controller itself, that has its own kind of joint controller, just, okay, on a more, more high level, relatively speaking, level, what do I do? And then there's like a motor controller. What does this particular motor do? And exact kind of, Voltages that I need to send to actually do something. So that's on the actual physical kind of hardware controller for the actual robot arm.

And then we have the Chef compute.

And by the way, the reason to get to separate these is like the robot one is safety rated and we can talk more about that, but that's really important for like safety and safeguarding. It's a completely safety rated system. There's like certifications we've gotten from a third party around safety.

And then on our side, like we have our own compute. And that is, I guess, at least 2 layers, more like 3. But there's like more like high-level task planning loop.

That's high level. And then we have like trajectory placement loop, which is, okay, like what are the strategies I need to take to actually execute that? And then there's like a control loop actually controlling the robot controller itself.

So that's how we think about that whole thing, right?

Like how controlled does the environment have to be? We've seen some of these demonstrations. Which are certainly not production ready, but Google has put out a couple of these things where they give the robot a task, like in the SayCan vein of research, right? Where they say, go get me a bag of chips off the counter. And then somebody comes and like snags the chips out of its hand and puts it back on the counter. And because there's these nested loops and the robot can realize like, oh, I don't have it anymore. Better go back and get it again.

Yeah.

So do you have that kind of Correction step between the sort of generation of the policy? If somebody moves the tray a little bit, does it adjust to that on the fly?

Yes, yes.

Yeah, exactly. By the way, that's super critical. I think there's multiple reasons why Chef is hard. One of them is just changing material properties, but the other one is what you said. We react to and adapt, I guess you could say, to a few different parts of the environment. One is the food itself.

Again, batches, every Hotel pan, if you have a run of 6 hours, you might be going through hundreds of hotel pans. So like every pan is different because it's cooked by a human and humans of course aren't perfect.

So we adapt to that. And then within each pan, so if you have like a pulled meat, yeah, yeah, mostly meats, right? Different parts of the animal have different densities and different material properties. So there's actually variations within the pan itself. So we'll adapt to that. And again, we have sensors like force torque as well.

So we can get some of those readings as well.

So that's one layer. Then on the placement side, of course there's humans who are usually tray denesting, putting the trays onto the line. These trays are various orientations. They're sometimes farther, they're sometimes closer.

So you have to adapt to that.

Usually customers have multiple kinds of conveyors. Some of them are different colors, different widths, different heights.

So you have to adapt to that.

That's all dynamic.

That's not like hardcoded or anything. It's like dynamically figuring out where to detect and track the containers and place onto the containers. There's also a lot of like small little nuances that come with like just production and deployment.

So for example, food facilities are always sloped for water drainage. Conveyors are always sloped. And you don't really know because the—

interestingly, the slopes are kind of like sinusoidal. In some regards, obviously it's a longer plane, but— or horizon. But yeah, basically you don't know like where the conveyor is going to be or the slopes. You have to detect that.

And that changes because the conveyor is shaking and the robot's shaking.

So that changes and you have to dynamically detect that.

Sometimes like humans will bump into it and, you know, conveyor is skewed now.

So you have to place farther and higher. For example, there's some line manager who's like trying to get their humans to work more or less, maybe so they'll drive up the velocity of the conveyor. There's also humans, again, like I said, to the left and right usually of the robots. So those humans are kind of interacting with the system.

They're pulling bowls underneath it.

They're taking— putting bowls in.

They're changing the scene a lot.

So look, like, Chef is not a self-driving car where it's like extraordinarily highly dimensional scene, but it's a fairly kind of hectic scene and there's a lot of varying, like, changing. And like, those are the kind of issues that I think you find in production that are hard, right?

You can scoop food in a lab.

It's not hard.

And you can even make it consistent. You can use a bunch of these advancements in AI to make it consistent, which is already hard. And then you can even get it to be doing a lot of ingredients. But then like when you get to production, you see all these weird edge cases.

You have no idea because there's a lot of humans interacting with the system and they do things that you don't predict. And that's where I think a lot of work has gone into get these 20 million servings. Like, how do you actually make this thing reliable? But when you have dozens of different systems around the country to actually deal with these kind of changes in the scene.

Yeah, the world is a messy place. So scanning down the list of kind of technology questions I had, I think we covered most of them. Oh, energy was one I was interested in. And compute. Is the compute hosted on the device, or are there like cloud calls that you're making? What does the energy usage of one of these robots look like?

Yeah, so like we can divide it between training and inference.

All the training is in the cloud. So like we use things like A100 to kind of do training for the IL, imitation learning, and things like that. And then the inference is all of course on the robot itself.

So the robot is of course connected to the internet, but all the inference is done locally on an edge compute.

And that's really important, by the way, because our customers usually ask Bodywide, We can't depend on Wi-Fi, even LTE or 5G. Like, you can imagine a hotspot, that doesn't really work in our customers too because it's a facility and then there's like often obfuscation.

Like, LTE basically doesn't work great. So yeah, so that's how we think about that.

And then from an energy perspective, the ARM I think is like 200 watts or so, and then the CPU is like 35 watts. It's relatively low. And then the GPU is like 50 watts.

Like, the entire system is actually relatively not that power hungry.

I'm like, one of the questions that some of our early customers asked is like, hey, look, yeah, I'm not paying for labor, but is my electricity bill gonna go up the— go out the roof here?

And we did the math and it wasn't actually that great.

It's $5 or so per day.

I think even less than that.

Even if you're running it like 24 hours a day type of thing. So.

Yeah. In my market in Detroit where I live, it is like just under 20 cents per kilowatt hour. And so if you are at 300 watts, then obviously it would take you, let's say, 3 hours to get to a kilowatt hour. So that means you're at just a couple bucks to run it 24 hours. I think that the, and most of that is going to the actual operation of the robot. The robot arm.

Yes.

I have a distinct thesis that I'm working on packaging up that's like, I think the energy concerns associated with AI are significantly overstated. And what the fact that the leaders are talking more about energy concerns implies is that they see an unbelievable amount of adoption around the corner. So I think about those kind of orders of magnitude quite a bit. Does that internet limitation also mean that teleoperation has to be done locally? Or I was understanding previously that you maybe provide that as a sort of remote service, but— I'm not sure now.

But yeah, we don't actually do teleop in the—

like, most people think about teleop, right?

Yeah, I think the crux of this is that we're working with like production customers.

They have very low tolerances for failure.

Like, basically the thing needs to work or else pull the robot. And specifically what I mean by that is like, one of the things that's nice about Chef is like, it's not going to like— it's like relatively safe broadly, right? It's not going to hurt anybody. And like the worst it's going to do is it's going to not be consistent or Maybe if there's a bug, it's not going to pick at all, or it's going to spill too much. But it's not like the end of the world.

What will happen if there's a failure is that the operator can pull the robot off the line and put a human in type of thing, or put another robot.

Usually it's not a hardware failure, by the way. So putting another robot in won't necessarily do it.

But let's just say it's a hardware failure. Pull the robot out, put another one in.

But generally, yeah, like the teleop, we don't do that in the colloquial sense. The way we think about this is how do we address the reliability problem? Problem is, first of all, we try to build the system to be as autonomous as it possibly can be. So it needs to adapt to changing material properties, it needs to adapt to different tray types, rotations, distances from the robot, it needs to adapt to all these kind of different scenes, like the thing we talked about, right? And then, I mean, that should hopefully solve most edge cases. Like a human, for example, the system detects a tray, it starts tracking the tray, and then a human randomly Pulls out the tray, and now the system's confused, and it's like, "Where'd my track go?" And maybe it places on the conveyor.

That's a bug we might have seen. But now, of course, we've built in logic so that okay, that it'll be a little more smarter about this. That's just one example.

But that's part one. Part two is that okay, well, there's still of course from time to time going to be issues. One of the things that's nice about Chef is there's always a line manager.

Even if it's a fully automated Chef line, there's six robots of Chef on the line. There's always a line manager.

Let's say the line manager finds that oh. Something's wrong. Maybe there's a tiny compartment and it's not placing right in the tiny compartment, spilling a tiny bit into the wrong compartment. And so what we do then is we try to have a really nice way for the user to visualize what's happening. We give them a really nice 3D view.

Maybe you've seen a WAME similar to that, right?

We give a really nice 3D representation of what the system is understanding and what the scene is. And at least that allows the user to figure out what's happening. For some examples, That might solve. One example might be that, like I said, in the morning they'll come in, they'll like crank up the conveyor velocity and it might be running super fast. And they're like, ah, it's missing bowls. What's going on? Well, if we have a really nice 3D view where it shows like it's outside of the ODD of the system, it's just, it's too fast. Then, then we can very visually, right, not using words, but visually tell the user about that. Or maybe we find that the human denester is putting trays way too far from the Robot, it can't reach that far. So again, we can detect those containers and we can tell the user, oh, you need to place closer to the robot. There's a lot of operational stuff like this we see. So that's like part 2. If that doesn't work, then there might actually be like an issue, like something that the human needs to fix. So at that point, we give the operator very simple and coarse ability to fine-tune the system.

Yeah.

So they can fine-tune the system, whether that's in a picking perspective or placing perspective.

And of course we don't give them too many knobs Because they can potentially make it worse, but we try to give them some knobs so they can like fine-tune. And by the way, this fine-tuning is also useful when we onboard ingredients, by the way. One of our people can fine-tune themselves to onboard initially too.

And finally, if that also doesn't work, then what we do is we have a— we built a lot of data infrastructure. So we're collecting all this telemetry, right?

That's going to the cloud. And we built a lot of like cloud infra that kind of self-annotates these Events.

So let's say an event might be like a very high-weight event, like basically super overpicked, or super low-weight event, a fault, a missed goal, a double deposit, whatever these events might be, like a pick hesitation, right? There's these different events we see, and we'll annotate that.

It'll do it itself.

And then basically it'll collect—

like, we use ROS, so it'll collect the ROS bag of here's all the data around this event, maybe 20 seconds before, 20 seconds after. It's this like bag file of all the data that happened and why this decision was made and why the robot reacted this way.

That'll get uploaded and then our customer support team will get an alert. And if it's the daytime, it'll be on Slack.

If it's at night, they'll get a PagerDuty. And obviously there's different severities and we try to assign a severity to this too. Okay.

And then that customer support person can actually like figure out what's happening and then fix it.

And if it doesn't, if they can't fix it, then they'll elevate it to like the applications team or even the engineering team. And that's how this works from a support perspective.

And I think that's been fairly good for us, I would say.

So when you mentioned earlier imitation learning of one robot to another, is that something that is all happening like centrally as part of the sort of pre-deployment process?

Gotcha.

Exactly. Exactly.

Yeah. So we would do that at our office. We don't have customers do that.

You can imagine customers do it too down the line, but right now that's something that our team would do.

The value of that technology for us is onboarding net new ingredients. And by the way, if there's already like an ingredient that we can do that, maybe we'll use it too to make this generalized food manipulation model, but we're really focusing IL on like net new ingredients, new stuff that's just hard.

So what is hard right now? I guess you've got the flywheels going. It sounds like 20 million servings is a lot. What are the sort of current frontier challenges, and which of those do you think you have to solve on your own, and which can you wait and let the world do for you and just take advantage of?

I think the hard thing is what it's always been, which is like food manipulation is just really hard. And the thing is, there's really nobody else in the world that has a really good training dataset for food manipulation.

It's just, they don't.

And like a lot of the existing technology Sim doesn't work. So there's a lot of stuff we have to invent, I would say. So I think the hard thing is still food manipulation and being consistent. In other words, our robot can— you give it literally more or less any ingredient. We can do—

you give it any ingredient, we can manipulate it.

But that doesn't mean it's going to be consistent. That doesn't mean it's going to—

if you take a very soft ingredient off the shelf, it might damage the ingredient.

It might spill the ingredient. It might not be able to— the ingredient might stick to the end effector. We might not be able to place in the right compartment without getting into the other compartments. There's all these things we might not be able to do. So it's this very hard problem, which is like, how do you have a system that can basically do any ingredient fast? Because of course throughput is really important. Reliably. And of course in engineering, anytime you have more flexibility, reliability usually suffers.

And so you need both here, reliability and flexibility.

While of course being consistent and not having giveaway and Not spilling and not damaging and placing the right compartments and working with any conveyor and working with any container. Like that whole thing becomes a really big, highly dimensional problem. And even forget the placement part, just the picking part, just the breadth of ingredients is hard. So I think our big focus honestly is like, we really do think that these new technologies are like really going to uncap the potential. We've already seen that.

We've already seen that.

Okay, let's take the combination of like production data, Which we have, we think we have the most production data in food now.

Let's take that plus demonstrations and let's try to build this generalized food manipulation model. So we have one kind of system to manipulate food and hopefully really have the best manipulation.

That's really the continued, that's what, that was the focus day one and that's still the focus right now because that's the hard problem here.

If we solve that problem in like the fully formed way, like this is a gigantic company is the way we think about it.

So when you look at these kind of new wave of humanoid robots, whether from Tesla or Figure or whatever, it sounds like you basically think, good luck to them. They may even be successful, but food is going to be one of the last things they're going to come for. And it's just so far down their checklist that you don't expect to have any meaningful competition anytime soon.

Yeah, I think that's definitely right. I think that's definitely right. And there's a few other kind of like ways I think about this. One thought that always comes to mind is like, when it comes to like food, let's just talk about like production or even like fast casuals, ghost kitchens. Having a system walk around is not terribly useful. It actually does not actually add much value, right? So like in a manufacturing setting, it's very obvious. You have a conveyor line, You have a station, the robot's on the station, that's it. Like I said, our robots are on wheels.

You can move it around, but they move it around like once a day, twice a day.

And if I have a humanoid, I'm spending like half of my compute just not trying to fall down. And then on top of that, I have all these safety concerns. Like it just really doesn't add much value. In a fast casual setting, a ghost kitchen setting, you literally rip out the prep table, you put our prep table in, slide it in, our module in, and that's it. You don't need to like move around. So anyways, All that to say, I think there's going to be certain applications where a humanoid is quite useful, right? The colloquial tote picking in facilities you often see is a good one. Maybe hospitals, right?

Like moving samples around a hospital.

My opinion, and I think I've talked to a few like the humanoid companies, is if you look back, if you think about, okay, what does the world look like in 2050? I think if you've ever watched WALL-E, I think the world's going to look like a little bit like WALL-E, which is to say you're going to have Tens of millions, hopefully hundreds of millions of robots that are very good at certain things.

Like our food manipulation robot doesn't need to walk around. It doesn't really add value.

And great, so let's do that. Just like your refrigerator doesn't need to do your laundry, so to say, right? Like let's just let it do that. And there'll be millions of robots, again, like tens of millions, hundreds of millions doing specific things. And then there's millions of robots that are more general purpose. And of course, They might not be doing that particular— I think it's just, it's a volume question. Our robot is doing this thing for 16 hours a day. In a fast casual, it's doing it for 8 hours a day. So it's just doing the thing all day long. Now, if I'm in a home, or if I am in a hospital, or if I'm in a kind of like a more— a different environment where I have 50 different tasks and those tasks are radically different, then maybe something like a humanoid makes sense. But I think—

Yeah.

That the market is actually smaller for that than—

Like the market for all these different applications, you can have specific things, I would say.

Like, that's one thought. There's another thought, which is it's a time horizon question. As of today, it's like Uber versus Uber ATG.

Uber is a great business to build. Maybe over time Uber ATG is a better business, but Uber ATG is always 10 years out. It didn't really happen yet. It still doesn't, hasn't happened. Maybe it does happen, of course, in full form. But, um, I think it's a time horizon question. Humanoids might be a thing in 20 years at full But AV has also taken like 20, 30 years to get there. It's still not totally there.

I think humanoids are kind of like AV.

It's just going to take a lot of time, a lot more people than, in my opinion, people think. And yeah, so it's a time horizon question.

Like today, this is what makes sense. Over time, let's build software that is hardware and embodiment agnostic. Like we built our codebase to work with any hardware. So let's say I'm wrong there, right? And 10 years down the line, humanoids are the thing in every application.

is done by humanoids, which I still don't think makes sense. I don't think it makes sense to put car doors onto cars with a humanoid, but let's just say that's true.

Then we built our software to be hardware agnostic. Okay, great.

Let's leverage those hardware systems to do human manipulation.

So I think those are a few thoughts at least I have.

We'll see what kind of how it plays out, but I think we're safe right now. There's a giant market that we can just like rinse and repeat and execute on.

So you mentioned the sort of Chipotle line. And I guess getting, circling back to your business model and the societal integration of this sort of technology, I guess for starters, I don't know if you share this publicly, but do you have like list pricing? And are there details available on your business model? Do you sell these things? Do you lease them? What does that look like?

Yeah. So our business model is robotics as a service.

We'll basically charge a yearly recurring fee to our customers, and a yearly recurring fee is usually less It is always less than the cost of their status quo workers.

There's a lot of benefits to this, I think. And there's a lot of good reasons why we do it.

The benefits, like, for the customer is that, like, they don't have to have a half a million dollar outlay upfront, which is quite expensive, or whatever the number might be.

Instead, they can pay us this yearly recurring fee. They're taking it from, like, the payroll bucket, if you will, of their books, as opposed to, like, the CapEx budget, which is nice. Like, they don't need a ton of huge approval.

They're not really, like, spending more money than they make right now.

So that's really nice. From our perspective, of course, we have recurring costs on our side, right?

There's compute costs and cloud costs and everything else. And then of course, we were constantly upgrading the hardware and the software if there's new features we have. And that's great for the customer, but it also requires a recurring model on our side.

So that's how we think about that, like robotics as a service.

We don't really do CapEx sales. That's not our business model. Yeah.

So I guess I wonder, like, how do you see this changing the way we live? You know, we've got like the Jetsons future is like one vision. And maybe I have a future specialist system that is in my kitchen that can maybe wheel over to the refrigerator to get something and can take it out of the refrigerator and maybe can fry it and then can plate it for me. But it's like a specialist thing that lives in my kitchen or spends the night in the closet. I can imagine one at the Chipotle counter. And this seems like maybe the most likely, as I think about this right now, maybe you're taking the world toward more food manufacturing, maybe the extreme of which would be like kitchenless apartments start to become a thing. So what is your kind of big vision for how life changes and maybe some surprising knock-on effects of your big—

Yeah, of course. Think about this a lot. And I think there's multiple layers. So layer 1 is just food manufacturing. So within food manufacturing, I think the goal would be And that's just, by the way, that's just kind of go-to-market.

That's our starting point.

I think the goal there would be a few-fold.

Really helping this industry, which hasn't really seen a lot of innovation in a while, help overcome the labor shortage, increase production volume, and really allow them to meet demand. They have a ton of demand, but they don't have enough folks to do it.

So I think that's like part one.

And by the way, hopefully help reduce food wastage too, because food production, of course, is a really big part of that, right? That's part one.

But I do think, I agree with you that the biggest impact for Chef, I actually think will be in the day-to-day world. And when I first started the company, I had this kind of interesting thought experiment I did with friends, which was like, let's say you have 2 options.

Option 1 is like a robot in your house, the Jetsons way, if you will, and it's going to do whatever you want it to do. That's option A.

Option B, and food-wise, I was just thinking about food at the time. Option B is imagine a world which has a few characteristics. One is that your food's still made by a robot. And so, by the way, the cost should be cheaper of your food, and it should probably be more consistent, right? In other words, you're not going to—

The person who's pulling it is not going to forget the avocado or the guacamole or whatever protein you ordered.

So it's quite consistent and it's also safe and nobody sneezed on it. So you feel good about it. And by the way, it's like a lot more customizable because it's a high-mix robot.

So it can do whatever you want it to do.

So it's cheap, it's safe, it's consistent. And then let's say it's made in a ghost kitchen, cloud kitchen. So The price is also lower because real estate is cheaper.

1/3 of the cost of food is usually real estate or delivered food, like fast casual food or restaurant food. It's usually real estate.

So if your real estate's cheaper, then now you have like double benefits.

Labor's cheaper and the real estate's cheaper.

The cost of raw materials at scale is relatively cheap, I would say. So you really brought down the cost of labor and real estate. And then also imagine a future where you have delivered food. So now you have much faster delivery and also cheaper than human drivers. So which one would you choose? And basically everyone said they would choose option B.

And this was very interesting to me because like at the time I was actually really interested in an Atom robot, like something beautiful, like an Apple product, beautifully industrially designed.

And this was a very interesting kind of realization.

I talked to these friends about why they picked option B, the second option, and they said like things like, I like the brands I like. I like Sweetgreen or Mixed or whatever brand I like. I like that brand. I don't want that to go away.

They're like, I don't want to have to maintain a robot, right?

I don't want to have it in my house. I don't want to have to maintain it. It's going to break. I don't want that. I don't want to buy it. I'd rather rent the food, if you will, than buy it. It's like the Airbnb idea. Or I don't want to have a car. I'd rather rent the car. They said that I don't have to deal with like refilling and groceries and stuff like that. I'd rather just have something that I can press an app on my DoorDash and it just shows up and maybe I can customize it perfectly.

This was interesting to me because it led me to this idea of atom robots. It might not be as big of a thing as people think. There's certain things that you need to keep in your house, like cleaning. You can't outsource cleaning, if you will. But if you think about it, there's a lot of history about people doing things in their house and then over time outsourcing that. So textile production, people used to make textiles in their house.

And then over time, they of course learned about factories and The cottage industry went away and you can make textiles in a factory.

I would argue that same thing is going to happen with food. It's not going to be in a food factory.

It's going to happen in a ghost kitchen.

But what's going to— I believe the way the future is going to look is, yeah, you're going to have an app, whatever the interface might be, the human-computer interface might be at the time, where you can perfectly customize a meal.

And you can say, I want this much salt or this much, whatever you want, right? And then it's going to be made by a robot in a ghost kitchen and then delivered by a robot.

Yeah.

And that price is going to be at parity with cooking at your house. And yes, I do believe we're going to see a lot more kitchenless houses, especially when you— especially at least in America, where if you look at, there's a lot of studies that show like 80 to 90% of Americans do not like to cook.

They just hate it. They would love to not have to cook.

So I think there's a lot of like tailwinds around this.

And I do think you're going to see a lot more delivery using robots.

So. I think that's quite exciting. So that's like probably layer 2.

I think there's another layer, which is even bigger than that, which is like, I think my opinion here is that the biggest impact of AI is going to be on the physical world.

I think there's a lot of innovation in the digital world.

Software is of course really awesome. But if you look at the entire software industry relative to global GDP, it's very small, honestly.

The biggest industry is the labor market.

It's $50 trillion of global GDP. So I I think the biggest impact for AI is going to be in the physical world, which is like manual labor and things like that.

And I think the food industry is a really big part of that, actually.

By market size, the food industry is number 3 by number of humans in the US who do that work.

But the first one's retail salespeople and the second one's nursing and nursing aides.

Those are relatively intractable, I would say. They're very human today.

I think the food industry is going to see—

I mean, there's like tens of millions of people who do that work, and I think That's going to see a lot of changes. And more macro, if we can be the robotics company that succeeds on a massive scale, having deployed tens of thousands, hundreds of thousands of robots, then hopefully we can inspire other engineers, founders, engineers, investors, operators to do AI plus robots as opposed to just pure AI. And I think that I would love if that is the legacy. That, of course, is very grand vision. That's very hard.

But if we can actually like deploy, if we can be the success story, if you will, If we can be like Facebook in the era of Friendster and MySpace, or we can be like Intel in the era of like Tropi Semiconductor and Fairchild Semiconductor, then we can hopefully inspire the next generation of people to do AI and robotics.

That, I hope, is the real impact of SHAFT.

I think that might be a perfect note to end on. Is there anything that we didn't get to that you wanted to make sure to touch on?

No, this is a great conversation.

I would just say, I would hopefully encourage everyone to like really think about embodied AI.

I really think that's quite powerful. And I think there's so much great research and innovation happening. And hopefully, like, the more people we can have doing embodied AI, the more we can really create that Jetsons future we all want.

That's like exciting, I think. So I'm very excited for that.

Yeah, it's cool. I think your vision is really interesting and it's a great reminder of how things may be reorganized around new enabling technologies. And that the ultimate equilibrium could in fact be quite different from the sort of naive first guess. I don't think many people think of kitchenless apartments as one aspect of the AI-enabled future, but after hearing everything that you have built and are continuing to build, it definitely sounds like something that we should be watching out for. So I appreciate you spending the time to educate us on all this today. Rajat Figueiredo, Chef Robotics, thank you for being part of the Cognitive Revolution.

Thank you, Nathan. Appreciate you having me on again. This is fun.

It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co, or you can DM me on the social media platform of your choice.