Large Action Models (LAMs) & Rabbits 🐇

Foreign.

Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI related tech is changing the world, this is the show for you. Thank you to our partners at Fly IO, the home of changelog.com Fly transforms containers into micro VMs that run on their hardware in 30 plus regions on six continents so you can launch your app near your users. Learn more@fly IO.

Well, welcome to another fully connected episode of the Practical AI Podcast. In these fully connected episodes, we try to keep you up to date with everything that's happening in the AI and machine learning world and and try to give you a few learning resources to level up your AI game. This is Daniel Wytnack. I'm the founder and CEO of Prediction Guard and I'm joined as always by my co host Chris Benson, who's a tech strategist at Lockheed Martin. How you doing Chris?

Doing very well, Daniel. It's enjoying the day. And by the way, since you've traveled to the Atlanta area tonight, we haven't gotten together, but you're just a few minutes away actually. So welcome to Atlanta.

Just got in. Yeah, we're within. Not maybe a short drive, depending on your view of what a short drive is.

Anything under three hours is short in Atlanta and I think you're like 45 minutes away from me right now.

Yeah. So hopefully we'll get a chance to catch up tomorrow, which will be awesome because we rarely get to see each other in person. It's been an interesting couple weeks for me. So for those that are listening from abroad, maybe we had some major ice and snow type storms recently and my great and embarrassing moment was I was walking back from the office and the like freezing rain and I slipped and fell and my laptop bag with laptop in it broke my fall, which is maybe good, but that also broke the laptop. So actually the laptop works, it's just the screen doesn't work. So maybe I'll be able to resolve that.

But it's like a mini portable server there, isn't it? You know?

Yeah, exactly. Yeah. You know, you have enough monitors around, it's not that much of an issue. But yeah, I had to put Ubuntu on a burner laptop for the trip. So yeah, it's always a fun time. Speaking of personal devices, there's been a lot of interesting news and releases not of, well, I guess of models, but also of interesting actual hardware devices related to AI recently. One of those is the rabbit R1, which was announced and sort of launched to pre orders with a lot of acclaim. Another one that I saw was the AI Pen P I N which is like a little, I don't know, my grandma would call it a brooch. Maybe like a large pen that you put on your jacket or something like that. I am wondering, Chris, as you see these devices, and I want to dig a lot more into some of the interesting research and models and data behind some of these things like Rabbit, but just generally, what are your thoughts on this sort of trend of AI driven personal devices to help you with all of your personal things and plugged into all of your personal data and sort of AI attached to everything in your life?

Well, I think it's coming, maybe it's here, but I know that I am definitely torn. I mean, I love the idea of all this help along the way. There's so many, like, I forget everything. I'm terrible if I don't write something down and then follow up on the list. I am not a naturally organized person, so my wife is. And my wife is always reminding me that I really struggle in this area and usually she's. She's not being very nice in the way that she does it. So it's all love, I'm sure. But yes.

Yeah.

So part of me is like, wow, this is the way I could actually, you know, be all there, get all the things done. But the idea of just giving up all my data and just being. It's like so many others, it's not. That aspect is not appealing. So I guess I'm not leaping.

How much different do you think this sort of thing is than everything we already give over with our smartphones?

It's a good point you're making.

I mean, we've had computing devices with us in our pocket or on our person 24, seven for what, at least the past 10 years, you know, for at least for those that adopted the iPhone or whatever when it came out. But yeah, so in terms of location, certainly account access and certain automations, what do you think makes it. Because obviously this is something on the mind of the makers of these devices. Because I think both the AI pen and the rabbit R1 make some sort of explicit statements in their launch and in their website about privacy is really important to us. This is how we're doing things because we really care about this. So obviously they anticipated some kind of additional reaction. But we, we all already have smartphones. I think most of us, if we are willing to admit it, we know that we're being tracked everywhere and all of our data goes everywhere. So I don't know, what is it about this AI element that you think either makes a, an actual difference in terms of the substance of what's happening with the data, or is it just a perception thing?

It's probably a perception thing with me. I mean, because everything that you said I agree with. You're dead on. And we've been giving this data for years and we've gotten comfortable with it, and that's just something that we all kind of don't like about it, but we've been accepting it for years. And I guess it's the expectation that with these AI assistants that, that we've been hearing about for so long coming, and we're starting to see things like the Rabbit come into market and such, that there's probably a whole new level of kind of analysis of us and all the things. And in a sense, knowing you better than you do, that is uncomfortable and probably will not be as uncomfortable in the years to come because we'll grow used to that as well. But I have to admit right now, it, it, it's an emotional reaction. It makes me a little bit leery.

Yeah, maybe it's. Prior to these sorts of devices, there was sort of the perception at least, that, yes, my data is going somewhere, maybe there's a nefarious person behind this, but there's sort of a person behind this, like, the data is going all to Facebook or Meta, and they're like, maybe they're even listening in on me and putting ads for mattresses in my feed or whatever. The thing is, right, that perception has been around for quite some time, regardless of whether Facebook is actually listening in or whatever, or it's another party, like, you know, the NSA and the government listening in. But I think all of those perceptions really relied on this idea that even if there's something bad happening that I don't want happening with my data, there's sort of a group of people back there doing something with it. And now there's this sort of idea of this agentic entity behind the scenes that's doing something with my data without human oversight. I think maybe that's. If there's anything sort of fundamentally different here, I think it's the level of automation and the sort of agentic nature of this which does provide some sort of difference. Although there's always like, you know, if you're processing voice or something, there's voice analytics and you can put that to text. And then there were always NLP models in the background doing various things or whatever. So there's some level of automation. That's already been there.

But I agree and I think, but you mentioned perception up front and I think that makes a big difference. I guess with like you mentioned, NSA intelligence agencies, I think we all just assume that they're all listening to all the things all the time now. And that's one of those things that's completely beyond your control. And so there's almost no reason to worry about it, I suppose, unless you happen to be one of the people that an intelligence agency would care about, which I don't particularly think I am. So it just goes someplace and you just kind of shrug it off. There's a certain amount of what we've done these years with mobile where it's. You're opting in, I think it's leveling up to. We're saying with some of these AI agents coming out, we know how much data about ourselves is going to be there. And so it's just escalating the opt in up to a whole new level. So hopefully we'll see what happens. Hope it works out.

Well, we haven't really, for the listeners maybe that are just listening to this and haven't actually. Maybe you're in parallel doing the search and looking at these devices. But in case you're on your run or in your car, we can describe a little bit. So I described the AI pen thing a little bit. The Rabbit I thought was really, really cool design. I don't know if there's any nerds out there that love this sort of synthesizer, analog sequencer, teenage engineering stuff that's out there. But actually the sort of hardware design, teenage engineering was involved in that in some way. So it's like a little square thing. The rabbit R1, it's got like one button you can push and speak a command. It's got a little hard actual hardware wheel that you can spend to scroll and the screen is kind of just. They show it as black most of the time but it pops up with, you know, the song you're playing on Spotify or some of the things you would expect to be happening on a touch screen or that sort of thing. But the primary interface is thought to be in my understanding, speech. Not that you would be pulling up a keyboard on the thing and typing in a lot. That's kind of not the point. The point would be this sort of speech driven conversational and they even call it an operating system, conversational operating system to do certain actions or tasks which we'll talk a lot more about the kind of research behind that. But that's kind of what the device is and looks like.

It's interesting that, you know, going with the device route, you know, and the fact that they're selling the actual unit itself. And, you know, over the years we've been, you know, we started on, on our computer or. Well, we started on desktops and then went to laptops and then went to our phones. And the phones have evolved over time and we've been talking about, you know, wearables and things like that over the years as they've evolved. But I think there's a little bit of a gamble in actually having it as a physical device because that's something else that they're presuming you're going to put at the center of your life. That versus being kind of the traditional phone app approach where you're using the thing that they are that your, your customer already has in their hands. What are your thoughts about the physicalness of this offering?

I think it's interesting. One of the points, if you watch the release or launch or promotion video for the rabbit R1, he talks about sort of the app driven nature of smartphones and there's an app for everything. And there's so many apps now that navigating apps is kind of a task in and of itself. And the Silicon Valley meme, no one ever deletes an app, right? So you just accumulate more and more and more apps and they kind of build up on your phone and now you have to organize them into little groupings or whatever. So I think the point being that it's nice that there's an app for everything, but the navigation and orchestration of those various apps is sometimes not seamless and burdensome. I'm even thinking about myself and kind of checking over here. You know, I got in the Uber, oh, I forgot to switch over my payment on my Uber app. So now I've got to open my bank app, right? And then grab my virtual card number and copy that over. But then I've got to go to my password management app to like copy my password or like there's all these sorts of interactions between various things that aren't seamless as you might think they would be. But it's easy for me to say in words conversationally, hey, I want to update the payment on my current Uber ride or whatever, right? So the thought that that would be an easy thing to express conversationally is interesting. And then have that be accomplished in the background if it actually works is also quite interesting.

I agree with that. And I can't help but wonder, if you look back at the advent of the phone and the smartphone and the iPhone comes out, and it really isn't really so much a phone anymore, but a little computer. And so we kind of the idea of the phone being the base device in your life has been something that's been with us now for over 15 years. And so one of the things I wonder is, could there be a trend where maybe the phone doesn't become, if you think about it, you're texting, but a lot of your texting isn't really texting. It's messaging and apps. Maybe the phone is no longer the central device in your life going forward, and maybe you're actually having your primary thing. And so that would obviously play into Rabbit's approach where they're giving you another device. It packages everything together in that AI OS that they're talking about or conversationally. It runs your life if you expose your life to it the way you are across many apps on the phone. But it's an opportunity potentially to take a left turn with the way we think about devices. And maybe the phone is no longer in the future, in the not so distant future, maybe the phone is no longer the centerpiece.

If you're listening, you know that artificial intelligence is revolutionizing the way we produce information, changing society, culture, politics, the economy. But it's also created a world of AI generated content, including deepfakes. So how can we tell what's real online? Read, Write Own. Building the Next Era of the Internet, a new book from entrepreneur and investor Chris Dixon, explores one possible solution to the Internet's authenticity problem. Blockchains. From AI that tracks its source material to generative programs that compensate rather than cannibalize creators. Read, Write Own is a call to action for a more open, transparent and democratic Internet. One that opens the black box of AI, tracks the origins we see online, and much more. This is our chance to reimagine world changing technologies to build the Internet we want, not the one we inherited. Order your copy of Read, Write Own today or go to readwriteown.com to learn more.

All right, Chris. Well, there's a few things interacting in the background here in terms of the technology behind the Rabbit device and I'm sure other similar types of devices that have come out. Actually, there's some of this sort of technology that we've talked a little bit about on the podcast before. I don't know if you remember, we had the episode with Ask ui, which they had this sort of multimodal model that I think a lot of their focus over time was on testing. You know, a lot of people might test web applications or websites using something like Selenium or something like that that automates desktop activity or interactions with web applications and actually automates that for testing purposes or other purposes. And Ask UI had some of this technology a while back to kind of perform certain actions. You using AI on a user interface without sort of hard coding, like click on 100 pixels this way and 20 pixels down this way. Right. So that I think has been going on for some time. This adds a sort of different element to it in that there's the voice interaction, but then they really emphasizing the flexibility of this and the updating of it. So actually they emphasize, like, I think some of the examples they gave is, you know, I have a certain configuration on my laptop or on my screen that I'm using with a browser with certain plugins that make it look a certain way and everything sort of looks different for everybody and it's all configured in their own sort of way. Even app wise apps kind of are very personalized now, right? Which makes it a challenge to say click on this button at this place, it might not be at the same place for everybody all the time. And of course apps update and that sort of thing. So the solution that Rabbit has come out with to deal with this is what they're calling a large action model. And specifically they're talking about this large action model being a neurosymbolic model. And I want to talk through a little bit of that, but before I do, I think we sort of have to back up and talk a little bit about AI models, large language models. ChatGPT has been interacting with external things for some time now. I think there's confusion at least about how that happens and you know, what the model is doing. So it might be good just to kind of set the stage for this in terms of how these models are interacting with external things. So the way that this looks, at least in the Rabbit case is you click the button and you say, oh, I want to change the payment card on my Uber unclick and like stuff happens in the background and somehow the large action model interacts with Uber and maybe my bank app or whatever and actually makes the update. Right. So the question is how this happens. Have you used any of the plugins or anything in ChatGPT or the kind of search the web type of plugin to a chat interface or anything like that?

Absolutely. I mean, that's what makes the. I mean, I think People tend to focus on the model itself. You know, I mean, that's where all the glory is. And people say this model versus that. But so much of the power comes in the plugins themselves and or other ways in which they interact with the world. And so as we're trying to kind of pave our way into the future and figure out how we're going to use these and how they're going to impact our lives, whether it be the rabbit way or Whether you're talking ChatGPT with its plugins, that's the key. It's all those interactions, it's the touch points with the different things that you care about, which makes it worthwhile. So yes, absolutely. And I'm looking forward to doing it some more here.

Yeah, so there's a couple things maybe that we can talk about and actually some of them are even highlighted in recent things that happen that we may want to highlight also. One of those is if you think about a large language model like that used in ChatGPT or, you know, Neural Chat Llama 2, whatever it is, you put text in and you get text out. We've talked about that a lot on the show. And so you put your prompt in and you get a completion. It's like fancy auto complete and you get this completion out. Right. Not that interesting. We've talked a little bit about RAG on the show, which means I am programming some logic around my prompt such that when I get my user input, I'm searching some of my own data or some external data that I've stored in a vector database or in a set of embeddings to retrieve text that's semantically similar to my query and just pushing that into the prompt as a sort of grounding mechanism to sort of ground the answer in that external data. So you got sort of basic autocomplete, you've got retrieval to insert external data via a vector database. You've got some multimodal inputs. And by multimodal models, I'm meaning things like lava. And actually this week there was a great. Published on January 24, I saw it in the daily papers on hugging face. Mm. LLMs recent advances in multimodal large language models. So if you're wanting to know sort of the state of the art and what's going on in multimodal large language models, like I just mentioned, that's probably a much deeper dive that you can go into. So check out that and we'll link it in our show notes. But these are models that would not only take A text prompt, but might take a text prompt paired with an image, right? So you could put an image in and you say, also have a text prompt that says, is there a raccoon in this image? Right. And then, you know, hopefully the reasoning happens and it says yes or no. If there's a.

Isn't there always a raccoon in the image?

There's always a raccoon everywhere. That's one element of this as a. That would be a specialized model that allows you to integrate multiple modes of data and there's similar ones out there for audio and text and other things. So again, summary, you've got text to text autocomplete. You've got this retrieval mechanism to pull in some external text data into your text prompt. You've got specialized models that allow you to bring in an image and text. All of that's super interesting and I think it's connected to what Rabbit is doing. But there's actually more to what's going on with, let's say, when people perform actions on external systems or integrate external systems with these sorts of AI models. And this is what, in the sort of LangChain world, if you've interacted with LangChain at all, they would call this maybe tools. And you even saw things in the past, like tool former and other models, where the idea was, well, okay, I have. Maybe it's the Google search API, right? Or SERP API, or one of these search APIs, right? I know that I can take a JSON object, send it off to that API and get a search result, right? Okay. So now if I want to call that Search API with an AI model, what I need to do is get the AI model to generate the right JSON structured output that I can then just programmatically, not with any sort of fancy AI logic, but programmatically take that JSON object and send it off to the API, get the response, and either plug that in in the sort of retrieval way that we talked about before, or just give it back to the user as the response that they wanted, right? So this has been happening for quite a while. This is kind of like we saw one of these cool AI demos every week, right? Where, oh, the AI is integrated with Kayak now to get me a rental car. And the AI is integrated with, you know, this external system. And all really cool. But at the heart of that was the idea that I would generate structured output that I could use in a regular computer programming way to call an API and then get a result back, which I would then use in my system. So that's, that's kind of this tool idea, which is still not quite what Rabbit is doing. But I think that's something that people don't realize is happening behind the scenes in these tools.

I think that's really popular in the enterprise, you know, and you know, in the enterprise with air quotes there. Because that approach is, you know, in large organizations they're going to other, you know, the cloud providers with their APIs. You know, Microsoft is, you know, has their relationship with OpenAI and they're wrapping that, you know, Google has their APIs and they're using RAG in that same way to try to integrate with systems instead of actually creating the models on their own. I would say that's a very, very popular approach right now in enterprise environments that are still more software driven and still trying to figure out how to use APIs for AI models.

Yeah, and I can give you a concrete example of something we did with a customer at Prediction Guard, which is the Shopify API, Right. So e commerce customer, the Shopify API has this sort of Shopify, I think it's called Shopify QL query language. It's structured, right. And you can call the regular API via GraphQL. Right. And so it's a very structured sort of way. You can call this API to get sales information or order information or do certain tasks. Right. And so you can create a natural language query and say, okay, well don't try to give me natural language out, but give me Shopify Ql or give me something that I can plug into a GraphQL query. And then I'm going to go off and query the Shopify API and either perform some interaction or get some data. Right? So this is very popular. This is how you sort of get AI on top of tools. What's interesting, I think that Rabbit observes in what they're saying and others have observed as well. I think, you know, you take the case like Ask UI like we talked about before and the observation is that not everything has this sort of nice structured way you can interact with it with an API. So think about, pull out your phone, you've got all of these apps on your phone. Some of them will have a nice API that's well defined. Some of them will have an API that me as a user, I know nothing about, right? There's maybe an API that exists there, but it's hard to use or not that well documented or maybe I don't have the right account to use it or something. There's all these interactions that I want to do on my accounts, with my web apps, with my apps that have no defined structured API to execute all of those things. So then the question comes, and that's why I wanted to lead up to this, is because even if you can retrieve data to get grounded answers, even if you can integrate images, even if you can interact with APIs, all of that gets you pretty far, as we've seen. But ultimately not everything is going to have a nice structured API or it's not going to have an API that's updated or has all the features that you want or does all the things you want. Right. So the question, I think the fundamental question that the Rabbit research team is thinking about is how do we then reformulate the problem in a flexible way to allow a user to trigger an AI system to perform arbitrary actions across an arbitrary number of applications or an application without knowing beforehand the structure of that application or its API? So I think that's the really interesting question.

I agree with you completely. And you know, there's so much complexity, they refer to it as human intentions expressed through actions on a computer. And that sounds really, really simple, you know, when you say it like that. But they're so that's quite a challenge to make that work in an unstructured world. So I'm really curious. I haven't. They have their research page, but I don't guess they've, they've put out any papers that describe some of the research they've done yet, have they?

Just in general terms. And that's where we get to the, the exciting world of large action models.

Somehow that makes me think of like Arnold Schwarzenegger.

Large, large action heroes.

There you go.

Exactly.

You know.

Foreign.

You know, when we started Podcasting back in 2009, an online store was just the furthest thing from our minds. Now we have merch.changelog.com and you can go there right now and order some T shirts. And that's all powered by Shopify. It's so easy. All because Shopify is amazing. Shopify is the global commerce platform that helps you sell at every stage of your business. From the launch your online shop stage, to the first real life store stage, all the way to the. Did we just hit a million dollar stage? Shopify is there to help you grow. Whether you're selling security systems or marketing memory modules, Shopify helps you sell everywhere. From their all in one eco ecommerce platform to their in person POS system. Wherever and whatever you're selling, Shopify has got you covered. Shopify helps you turn browsers into buyers with the Internet's best converting checkout up to 36% better compared to other leading commerce platforms and sell more with less effort thanks to Shopify Magic, your AI powered All Star. You know, nothing gets me and Jared more excited than when our guests get that coupon code in their email when their show ships. Or to everyone out there who loves Changelow podcasts and can go to merch.changelog.com and get your favorite threads to support our podcasts. It is just the best thing ever. From stickers to threads. All that is@merch.changelog.com and did you know that Shopify powers 10% of all e commerce in the US and Shopify is the global force behind Allbirds, Rothy's and Brooklinen and millions of other entrepreneurs of every size across 175 countries. Plus Shopify's extensive help resources are there to support you and your success every step of the way because businesses that grow grow with Shopify. Sign up for a $1 per month trial period at shopify.compracticalai all lowercase go to shopify.compracticalai now to grow your business no matter what stage you're in. Again, shopify.com PracticalAI.

Yeah Chris, so coming from Arnold Schwarzenegger and large action heroes to large act models, I was wondering if this was a term that Rabbit came up with. I think it has existed for some amount of time. I at least saw it at least as far as back as June of last year, 2023. I saw Silvio Savaris article on Salesforce AI research blog about lams from large language models to large action models. I think the focus of that article was very much on the sort of agentic stuff that we talked about before in terms of interacting with different systems, but in a very automated way. The term large action model, as far as Rabbit refers to it, it's this new architecture that they are saying that they've come up with and I'm sure they have because seems like the device works. We don't know, I think all of the details about it. At least I haven't seen all of the details or it is sort of not transparent in the way that maybe a model release would be on hugging face with code associated with it in a long research paper. Maybe I'm missing that somewhere or listeners can tell me if they found it, but I couldn't find that. They do have a research page though which gives us a few clues as to what's going on and some Explanation in kind of general terms. And what they've described is that their goal is to observe human interactions with a ui. And there seems to be some sort of multimodal model that is detecting what things are where in the ui and they're mapping that onto some kind of flexible, symbolic, synthesized representation of a program. So the user is doing this thing, right? So I'm changing the payment on my Uber app and that's represented or synthesized behind the scenes in some sort of structured way and kind of updated over time as it sees demonstrations, human demonstrations of this going on. And so the words that they. I'll just kind of read this. So people, if they're not looking at the article, they say we design the technical stack from the ground up from the data collection platform to the new network architecture. And here's the sort of very dense loaded wording that probably has a lot packed into it. They say that utilizes both transformer style attention and graph based message passing combined with program synthesizers that are demonstration and example guided. So that's a lot in that statement. And of course they mention a few in more description in other places. But it seems like my sort of interpretation of this is that the requested action comes in to the system, to the network architecture, right? And there's a neural layer, so this is a neural symbolic model. So there's a neural layer that somehow interprets that user action into a set of symbols or representations that it's learned about the AI, like the ui, I mean the Shopify UI or the Uber UI or whatever. And then they use some sort of symbolic logic processing of this sort of synthesized program to actually execute a series of actions within the app and perform an action that it's learned through demonstration. So this is sort of what they mean I think when they're talking about neuro symbolic. So there's a neural network portion of this kind of like when you put something into ChatGPT or a transformer based large language model and you get something out. In the case of we were talking about getting JSON structured out when we're interacting with an external tool, but here it seems like you're getting some sort of thing out, whatever that is, a set of symbols or some sort of structured thing that's then passed through symbolic processing layers that are essentially symbolic and rule based ways to execute a learned program over this application. And by program here I think they mean they reference a couple papers. And my best interpretation is that they mean not a computer program in the sense of Python code, but a logical program that represents an action like here is the logical program to update the payment on the Uber app. You go here and then you click this and then you enter that and then you blah, blah, you do those things right? Except here, those programs, those synthesized programs are learned by looking at human intentions and what they do in an application. And that's how those programs are synthesized. So that was a long. I don't know how well that held together, but that was my best at this point, without seeing anything else from a single single sort of blog post.

When you can keep me quiet for a couple of minutes there, it means you're doing a pretty good job. I have a question I want to throw out. And you won't. I don't know that you'll. You'd be able to answer it, obviously, but it. Just to speculate. While we were talking about that and thinking about multimodal, I'm wondering. The device itself comes with many of the same, you know, sensors that you're going to find in a cell phone these days. But I'm wondering if that feeds in more than just the, the speech and that obviously it has the camera on it. It comes with a Magnum meter, I can't say the word gps, accelerometer and gyroscope and obviously so you know, it's detecting motion, it knows location, all the things, has the camera, has the mic. How much of that do you think is relevant to the lams, to the large action model in terms of inputs? Do you think that there is potentially relevance in the non speech and non camera concerns on it? Do you think the way people move could have some play in there? And I know we're being purely speculative. I'm just. It caught my imagination.

Yeah, I'm. I'm not sure. I mean it could be that that's used in, in ways similar to how those sensors are used on smartphones these days. Right. Like if I'm asking Rabbit to book me an Uber to here or something like that right now, it could infer the location maybe of where I am based on where I'm wanting to go or ask me where I am. But likely the easiest thing would be to use a GPS sensor, right, to know my location and just like put that as the pin in the Uber app. And now it knows. So I think there's some level of interaction between these things. I'm not sure how much, but it seems like at least in terms of location, I could definitely see that coming into play. I'm not sure on the other ones.

Well, it looks a lot Like a physically, it looks like a lot like a smartphone without the phone.

Yeah, yeah, a smartphone. Different sort of aspect ratio, but still kind of touchscreen. I think you can still pull up a keyboard and that sort of thing and you know, you see things when you prompt it. So yeah, I imagine that that's maybe an evolution of this over time is sensory input of various things. Like I could imagine that being very interesting in running or fitness type of scenarios. Right. If I've got my Rabbit with me and I instruct Rabbit to post a celebratory social media post every time I keep my mileage or my time per mile at a certain level or something and it's using some sort of sensors on the device to do that. I think there's probably ways that that will work out. I'm not sure about now.

It'll be interesting that if this approach sticks and I might make an analogy to things like the Oura ring, you know, for health wearing that and then competitors started coming out and then, you know, Amazon has their own version of a, of a health ring that's coming out. Along those lines, you have all these incumbent players in the AI space that are, you know, for the most part very large, well funded, you know, cloud companies and in at least one case a retail company blended in there. And so if this might be an alternative in some ways to the smartphone being the dominant device and it has all the same capabilities plus more, and they have the lam behind it to drive that functionality, how long does it take for an Amazon or a Google or a Microsoft to come along after this and start producing their own variant? Because they already have the infrastructure that they need to produce the back end and they're going to be able to produce, you know, Google and Amazon certainly produce front end stuff quite a lot as well. So it'll be interesting to see if this is the beginning of a new marketplace opening up in the AI space as an entrant.

So there's already really great hardware out there for smartphones and I wonder if something like this is kind of a shock to the market. But in some ways, you know, just as phones with external key buttons sort of morphed into smartphones with touchscreens. Otherwise I could see smartphones that are primarily app driven in the way that we interact with them now being pushed in a certain direction because of these interfaces. And so smartphones won't look the same in two years as they do now, and they won't follow that same sort of app driven trajectory like they are now probably because of things that are rethought. And it might not be that we all have rabbits in our pocket, but maybe smartphones become more like rabbits over time. I'm not sure. I think that that's very likely a thing that happened. It's also interesting to me. It's a little bit hard to parse out for me what's happening, what's the workload like between what's happening on the device and what's happening in the cloud and what sort of connectivity is actually needed for full functionality with the device? Maybe that's something. If you want to share your own findings on that in our slack community@changelog.com Community, we'd love to hear about it. We'll My understanding is there is at least a good portion of the LAM and the LAM powered routines that are operating in a centralized sort of platform and hardware. So there's not this kind of huge large model running on a very low power device that might suck away all the all the energy. But I think that's also an interesting direction is how far could we get, especially with local models getting so good recently with fine tuned local optimized quantized models doing action related things on edge devices in our pockets that aren't relying on stable and high speed Internet connections. Which also of course helps with the privacy related issues as well. Agree.

By the way, I'm going to make a prediction. I'm predicting that a large cloud computing service provider will purchase rabbit.

All right, you heard it here first. I don't know what sort of odds Chris is giving or I'm not going to bet against him, that's for sure. But yeah, I think that's interesting. There's definitely a lot of. I think there will be a lot of action models of some type, whether those be tool using LLMs or lams or SLMs or whatever, whatever we've got coming up.

And they should have named it. They could have named it a lamb instead of a rabbit is. You know, I just want to point out they're getting their animals mixed up, man.

I actually. Yeah, that's a really good point. I don't know if they came up with rabbit before lamb, but maybe they just had the lack of the B there. But I think they probably could have figured out something.

Yeah. And the only thing that could have been just been the lamb is raccoon of course. But that's beside the point. You have to come around full circle there.

Of course. Of course we'll leave that device up to you as well. Yeah. All Right. Well, this has been fun, Chris. I do recommend, in terms of if people want to learn more, there's a really good research page on Rabbit Tech. Rabbit Tech Research. And down at the bottom of the page there's a list of references that they share throughout that people might find interesting as they explore the technology. I would also recommend that people look at LangChain's documentation on tools and also maybe just check out a couple of these tools. They're not that complicated. Like I say, there's sort of they expect JSON input and then they run a software function and do a thing. That's sort of what's happening there. So maybe check out some of those and the array of tools that people have built for LangChain and try using them. So, yeah, this has been fun, Chris, thanks.

It was great. Thanks for bringing the Rabbit to our attention.

Yeah. Hopefully see you in person soon.

That's right.

And yeah, we'll include some links in our show notes, so everyone take a look at them. Talk to you soon, Chris.

Have a good one. Foreign.

Thank you for listening to Practical AI. Your next step is to subscribe now, if you haven't already, and if you're a longtime listener of the show. Help us reach more people by sharing Practical AI with your friends and colleagues. Thanks once again to Fastly and Fly for partnering with us to bring you all Changelog podcasts. Check out what they're up to@fastly.com and fly IO. And to our Beat Freakin residents, Breakmaster Cylinder, for continuously cranking out the best beats in the biz. That's all for now. We'll talk to you again next time.