Serving deep learning models with RedisAI

Bandwidth for Changelog is provided by Fastly. Learn more@fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode Cloud Servers. Head to linode.com changelog this episode is brought to you by DigitalOcean, the simplest cloud platform out there and we're excited to share. They now offer dedicated virtual droplets and unlike standard droplets which use shared virtual CPU threads, their two performance plans general purp CPU optimized They have dedicated virtual CPU threads. This translates to higher performance and increased consistency during CPU intensive processes. So if you have build boxes, cicd, video encoding, machine learning, ad serving, game servers, databases, batch processing, data mining, application servers or active front end web servers that need to be full duty CPU all day every day, then check out Digital Ocean's dedicated virtual CPU droplets. Pricing is very Competitive starting at 40 bucks a month. Learn more and get started for free with a 100 of credit at.co changelog again.co changelog. Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community. Follow us on Twitter. We're @practicalai FM. And now onto the show.

Welcome to Practical AI. This is Daniel Whitenack. I am a data scientist with SIL International and I'm here with my co host Chris Benson, who's a Chief AI strategist at Lockheed Martin. How you doing Chris?

Doing great, Daniel. How's it going today?

It's going great, you know, finally got a break in the heat wave that's been all over the US here. I know you're a little bit further south than me, but I stepped out yesterday and I almost thought I needed a jacket because the, the difference in temperature was so drastic.

I haven't, I haven't gotten the benefit of the cold weather down here. I'm in Atlanta and so it's still quite toasty and humid here. You know, last week I was up in Boston the whole week and I was really counting on nice, cool Boston weather and it just didn't happen. It was, it was only about 5 degrees, you know, cooler up there than it was down here. So it was sad.

Yeah, well, I have, I have a bit of story. So when I, when I first joined or my first job as a data scientist was with this startup and you know, at the time everybody was into microservices and all of this stuff. And you know, we started talking about microservices. I learned what those were, but they kept saying things like Redis this and Redis that and we're going to use Redis for this and REDIS for that. And eventually I learned that, that Redis is this cool data thing that can be used for like message passing and data caching and data store and all this stuff. And recently I saw that Redis released this project called Redis AI. So that's going to be the topic of today's conversation. We're joined by Peter Kayo, who is a senior product manager at Redis Labs. Welcome, Peter.

Hi guys.

Yeah, so Peter, could you just give us a little bit of a background about how you got into software and how you ended up at Redis?

Oh, that's. Well, it's quite a long story, but I'll try to keep it short. So effectively, when I was as a kid I was into astronomy and I wanted to always kind of like motorize my telescope so it could effectively follow the turning of the sky. So to have long exposures of photos.

Right.

That sounds awesome.

Yeah. That brought me into electromechanical engineering and then afterwards computer science. My first job was at TomTom. Well, sometimes in the US they say TomTom so depends a bit. But I was working on the, to the portable navigation devices, right. I was working on the map backend in Belgium, in Ghent. Map quality assurance is a difficult problem, right? So TomTom tries to differentiate and how will the quality of these maps and they have many rules and one of those rules is a very expensive rule. Is the map still connected? Can you still travel from everywhere to everywhere? You could imagine, for example, you have a continent, you have an island, there's a bridge in between. And the map editor says this bridge is now one way traffic. So in the morning you go to the island, in the evening you want to go back, but your device says no, sorry, can't do that would be a problem. That would be a problem. So for that effectively we were using a Graal database, we were effectively using Neo4j and that rolled. Well, I rolled into the NoSQL database world because I joined Neo4j, worked there for three, three and a half years. And afterwards I joined Redis first as a solutions architect and then afterwards they said hey, we've got one of these modules here, graph related. And I joined effectively the product management team to initially first support the Redis graph module. But now I have seven of these modules under my what I'll call it supervision or umbrella.

Oh, wow, that's great. Yeah. How many in total? Like, what is the ecosystem of modules around redis? I imagine there's quite a few.

There is quite a few. Right. So I don't know exactly, but I think we have 20, 25 open source modules. Maybe it's a bit good for the audience that I introduce what these modules are and effectively introduce what REDIS is, because that will make the story nicer.

Yeah, that's no problem.

So Redis is a key value store. It's an in memory database. It's a NoSQL database. The most typical in memory database that most people know is memcache. And in memcache the value is always a string. In Redis, however, you can have different data types. It can be either a string, but it can also be a numeric value, a list, a sorted set, a hash. So you have many data types. You can effectively see this memory as a service. So if you're coding, you could say, okay, I have this data structure that is residing within a distributed database.

So could you just give an idea? So you've kind of called out REDIS as this key value store and a NoSQL database. And you also mentioned like graph databases. Could you just kind of generally describe how the no SQL databases are separate from what people might have in mind that they execute like select statements on Sometimes confusing.

Right. So NoSQL stands for not only SQL. So it was the movement that. Well, we're living in some buzzwords, right? So there is big data, there is much more data.

We have a podcast about AI, so we totally get the buzzwords.

Nothing but buzzwords.

Nothing but. Yeah. So database had to consume or to ingest high volumes of data. And there were some performance issues. So it's not the entire picture, but what people kind of like did was they create some databases and they threw away everything that was kind of like reducing the performance or slowing down the performance of your database. So NoSQL databases typically have a. Well, they also have a different paradigm, but they also have like less strict authentication. Quite frequently they're schema less so you can put lots of heterogeneous data inside it. And in that NoSQL section of database you have quite a lot of database models. So there's a key value store, but there's also graph. You could see search also as a database model, there's a time series which is effectively ramping up lately, which is a lower database model for which we also, by the way, have a very Interesting module. But yeah, I don't know if you want me to explain further or Talk more about NoSQL databases or how they came to life.

That was really helpful for me. I know that if we kind of zero down in on the key value stores. So you mentioned one distinguishing feature of Redis is that the values don't have to just be strings. So you gave some other examples of things that they could be. I forget what those were.

There is, for example, there is a list. So it's a simple. It's a list data type, right? So you can effectively. It's interesting, sometimes when I give a talk, I ask people. So for example, let's do the exercise, right? So you have two sets of items and inside Redis they will reside as a set. So for example, imagine you have two products and they have a set of tags. Now what could you do with these two sets? Which questions could you ask the database with two sets? Maybe I should hint.

Overlapping entries or something like that.

Correct. So you could do an intersection, you could do union. This is kind of like what REDIS is, right? So your data structure effectively. All the methods you could do in your programming language, they kind of like have commands, well, commands that you could execute against Redis.

Okay, so you could have a bunch of values as lists and execute some command or in some REDIS client or something to get the intersection of those values correct.

So in a list you could get the first element, you could get the last element, you could query your range or get the exact element. You could also pop an element. But in a set you could ask, for example, well, is this in a very efficient way, does this element belong or present in your set, etc.

So I was going to ask, could you kind of like, what are some of the most typical ways that Redis users are using Redis? What's a couple of really common use cases that people are most likely to do?

It's very broad, right? It's a general purpose database, you could use it for anything. But the most common thing that people are using Redis for currently is for session caching. So your shopping baskets, et cetera. So everything that you keep track of in sessions, also for caching. But there are also many things you could use as a queue, right? There is next to the. Well, originally there was the list data typer. You could effectively distribute messages with blocking on this list on retrieving the first element, the last element. But nowadays we also have a streams data type, which is, we'll call it Kafkaesque. So it has similar capabilities. You can use it as a stream or as a topic. Sorry, as a topic or as a queue. So in a queue you can have for example, a consumer group and each message will only be consumed by a single consumer. But if you use it as a stream, you could for example, say I want to start reading the stream from here or I just want to catch up from the end of the stream, et cetera. So also with that specific data type, you could effectively do message broker or event sourcing use cases.

Gotcha.

But there are many more. There are many more people use it sometimes to do this reader looking.

Yeah. So you've kind of also alluded to the fact that at least some of these modules of Redis are open source, like Redis as a company. I know that there's some things that are open or open. Ish. How does the company operate And Redis itself, can you find it on GitHub and what's the model around Redis?

That's a good question. Redis itself is entirely open source. So it's a BSD license. So you can take it, you can fork it, you can resell it, you can do whatever you want with it. Then there is a piece where we have these modules. These modules have a rather source available license. So the idea is that you can effectively do anything with these modules apart from creating a database as a service with these modules so you can embed it inside your products. And effectively it's a bit of a strategic way to compete to cloud providers, just simply taking all the open source products and then effectively offering them as a service. The last part is our Redis enterprise part, which is a closed source part. It doesn't have any data or engineering specific capabilities. However, it has lots of enterprise features that your company might be interested in or is most likely interested in. So there's easy to scale your database. There's actually an entire database management system. The way I sometimes position this is that do you want to become an expert in building your application or do you want to become an expert in deploying Redis? So if you deploy. Yeah, that makes sense. If you don't want to do the second one, then we've got an enterprise solution for that.

So could you give us some examples of some of the modules that you have available? Curious about what the scope of those are?

Well, the story effectively is, so we have all these data types and the original core contributor or creator of Redis, Salvatore San Felipe, he was receiving lots of requests and said like, oh, you don't have this data type. How about this data type? So he created an API so you could effectively create your own modules and your own data structures and your own commands against those data structures. So the commands were, for example, do the intersection in between two sets and the sets was effectively the data structure. And that way effectively could say, like, well, I'm receiving lots of requests now. You can build your own data structures, and by creating those, you effectively inherit all the goodies from the mothership. Right? So Redis has many things inside. So it has memory management, but it also allows you to cluster your database. And one of the misconceptions that many people have, by the way, is that Redis is purely a cache, it's effectively a database. However, data lives in memory first and then we have tunable durability. So now if you implement your data structure inside a module, you can also benefit from that durability and also have an entire spectrum of client libraries that can effectively connect to your database and reach out to those data structures in those modules. Now, to answer your question, right, so there are quite a lot of them, but the ones that we effectively think that are, well, that we are effectively contributing the most to, there are seven of those. So the most mature one is most likely going to be Redis Search. So it's an inverted index. It's kind of like Apache Lucene and Apache Lucene. For those who don't know, Apache Lucene is effectively the core building blocks of elasticsearch and solar technologies. But we've built our own inverted index. It's written in C. We serve the entire data or the index from within memory, and also the index is instantly reflected. So we do have some advantages there. We don't have feature parity, of course, to these products that have been around for decades. I think, however, we get a very, very high performance in that search. And there is Redis Graph, which adds graph capabilities. Well, I know all these modules, right? So I can talk for hours about them. So Redis Graph itself is a very interesting technology. So I always try to differentiate graph native technologies and graph technologies. So graphics, you can put a graph API on top of a relational database. However, yeah, it will be quite slow. You make one table which is nodes, another table which is relationships. But when you have to do a traversal or search, for example, a Dexter algorithm, which is a search algorithm between two nodes, you will have to do a join in between those tables continuously. So a graph native database.

I'm so glad you're calling this out because I actually got burned by this very issue on a project where we are trying to implement graph database and I mistakenly chose like a graph layer on top of MongoDB, which is another database. And it was so slow. And basically the graph logic was really interesting and good for the project, but we got burned because of this issue that you're talking about. So thanks for explaining that.

Yeah. Yes. So the difference is that you have graph native, you have an O1 time complexity to go from one node to another node, Right? So instead of an olocn in a relational database, and typically you try to keep the data as close as possible, residing within the same key effectively. In Redis, we've got Redis Bloom, which is a set of probabilistic data structures like a Bloom filter, a cocoa filter. We also added top K. We've got Redis JSON, which is going to take your JSON document and is going to split it up inside a tree so that in atomic operation you can. In a large JSON document you could, for example, append some data to an array, or you could increment a numeric value inside your JSON documents without having to fetch it and putting it back. Then there is Redis time series, which is the last one that went ga, which effectively adds time series capabilities to Redis. It allows you to do, for example, in the role of IoT. It's very interesting that you've got lots of raw data, but you also would like to downsample it. You could do that inside Redis already, right? There are many ways to do that. However, you had to write lots of client side codes and Redis time series comes with a toolbox to do aggregations over lots of samples or to downsample them so you don't keep all your raw data the further you go into history. One more interesting module before we dig into, potentially into Redis AI is Redis Gears. And it's interesting to mention because Redis Gears will be kind of like the serverless of Redis. So you can do two things. You can do like MapReduce type of operations on top of your entire data set, because, well, Redis allows you to key value stores so you can shard it, you can partition it. So if you want to run a query across all these shards, there was no clean solution for that. So Redis Gears will give that to you. The second part is that it can also execute certain scripts based upon certain triggers or certain events. So it could be, for example, you add something to a stream or you change a certain key, but it could Also be time based and then you execute certain codes. And this kind of like makes the entire multi model story very beautiful. And I know in our company I'm the first one to say if somebody says, hey, multimodule, multi model, please be cautious about that. It's not because we have multiple modules in a database that we're a multimodal database, but Redis Gears kind of like allows you to do that. So you have for example, stream of data, you get some name change. For example, you could say, hey, the script is going to react upon that new piece of data is going to say, hey Radis search, please update the name of this person. But also, hey Redis graph. Make sure that graph person or the graph node is updated so it can effectively talk to all these modules. And then it becomes very, very interesting.

So you just mentioned kind of this idea of multimodules or multimodule and the ability to run scripts, the ability to put things together. You also mentioned Redis AI, which is really what we want to get into in the rest of this discussion. But I was wondering, just kind of in terms of background, what is the background on Redis AI in terms of why the team was motivated to create it? Was it something that users were asking for? They were like, oh, we love Redis, but we want to integrate this somehow with AI models Or what's the backstory on that? And kind of how did it come about?

It came effectively from three ways. The first one is that Salvatore, when he created this module API, the first module he created was Neural Redis. So that was the first kind of like indication that there was potentially a need for that. Then there was also some engineers inside Redis Labs who created a module called RedsML. So they told that they could, for example, have very specific data structures, for example for random forestry, for some regression, some specific data structures for these models or these AI models and effectively host them inside Redis. And then there was also somebody who was creating Redis DL, another module which is focused on deep learning. And we kind of like combined these efforts and we created one module which is called Redis AI. Now that's how it came to life, right? But the need or explaining the need was also quite important. So there are two things we believe there, right? So there is data locality and then there's effectively the DevOps part of publishing your model. The publishing of your model. It's always straightforward, it's always convenient. So you can, for example, have your model, you wrap it inside the flask application, but then you have to scale it yourself. You have to add all these things manually. So with Redis we effectively have all these things already. We can scale it, we can cluster your database so we have all these clients already can connect to it. We already have high availability. We have effectively also durability. So if your data, if your flask application goes down and you didn't, for example, well, if there's no high availability, then your mole serving would no longer be available. So all the goodies that Redis brings as a database, you suddenly also get for free. If you would add Redis AI to or these models to Redis. The second part is that we believe is a data locality for that maybe it's interesting to give a model or to tell a story. For example, you have a chatbot application, and this chatbot application has a certain model. So it takes in two tensors or takes in two input parameters. The first one is your latest message that the chatbot received. But also the second one is the intermediate state. So there's an intermediate state of the past or the history of that conversation. And combined with that, it produces an output, and the output is the response. But it's also a new intermediate state that is a new history of your conversation. Now you want to keep that as close as possible to that model. You don't want to go and fetch it from another database, then bring it to your application or to your own flask wrapper where you've written, where you try to deploy your model, adds another data fetching. For sure that data you want to fetch becomes large, there will be some latency. And well, what we believe in Redis is that everything should be extremely fast because there's a high throughput. Well, there's a demand for high throughput databases or high throughput requests with very low latency. Everything needs to be more and more snappy. By actually creating these data structures or by creating Redis AI, we could now run your inference where your data lives, because people sometimes also have, for example, some extra input data. Get a user profile and feed it to your model to do a better inference or classification so that now that data doesn't need to be fetched from another host or from another database, it's already there. It can live within Redis next to the model.

So as you talk about the chat bot example, where is the tensor serving, where you're executing deep learning models, how does that fit into this? Could you kind of give it to us, give the context to that so that we can kind of map it together?

There are three data structures inside Redis AI that we add. So there's a tensor, there's a script and there's a model. So you can effectively now the key value store, the value now can be a tensor. So you can host your tensors inside Redis. Is that answering your question?

I think so. I was just wanting to kind of like break it down in the sense of with the chatbot kind of walking through how you're using the module, how, how Redis AI is fitting into that example. If so if you were coming into it for the first time as a user that was wanting to use this, just kind of taking that as a, as a contextual example about how I might do that, like if I sit down after a conversation and try to do it, can you just kind of describe it end to end in terms of how that happens?

Right. So your training part is still going to be going to be the same. Right. But once you've trained your model and you're sure that it's the right fit for it, you can effectively import it inside Redis AI. So we're building quite some tooling around it to make it more easy so that you can do it for example, directly from some Scikit learning. There is already a connector for that. We also have connector, for example from Spark. So you take your model and you can put it inside Redis. The next thing is if you want to build an application, you will have some client library that is going to connect to Redis and from the client library you could post effectively your tensor inside Redis. You could say, hey, here is my new message. There is also the output tensor. Well, there is also the input tensor of the intermediate state and then you can effectively run that model. It's another command that you can run from the client side or the client library application side to do your inference.

Yeah, I'm going to kind of. So I have a theory going in my mind and I want to kind of check it. So you mentioned the three data structures. You got the tensor, the graph or the model and then the script. So would it be true that you could kind of have tensors sort in Redis and these might be like test examples or maybe these are like knowledge base articles. If you're answering questions out of a knowledge base or maybe it's like some images or something and you could execute like certain of the scripts of the script database or data structures on those tensors to maybe do like pre processing or something and like combine them in interesting ways before they would be passed off. To maybe the graph data structure to actually perform an inference with a model. And then maybe it would go back to the scripts for post processing and then stored the results stored in tensor data structure again. Is that kind of on the right track?

That is exactly it. That is exactly it. That's how we want it to be. So you can have these three data structures and you can combine them and mix and match them as you go. So you can also use that if you would like to run several models and combine the results, you could also use a script and run several graphs underneath. One more thing is that, well that is coming up is effectively you will have a command which is a direct acyclic graph so you can effectively execute those commands combined. So you could for example, do you set a tensor, but then the output is passed to a script, the script's output is then passed to a model, and then the model output is once again passed through a script. And you can then the output of that script could be a tensor that is persistent inside Redis. So you can kind of like pipeline those commands in a single call towards Redis. But your explanation was spot on.

Awesome. Yeah, so I'm just part of the reason why I have that. Excuse me, I have that in mind is like I have a direct use case I'm thinking of in the problems I'm solving now. So we have like a bunch of knowledge base articles that we're doing reading comprehension on, but we're also doing like a full text search of those based on TF idf. And so my thought is, oh well, what if I could. So I don't know, check me on this to see if this would be a proper way to use REDIS AI. But I could store the knowledge base article text as certain, maybe string elements, string data structures in Redis. And then I could use maybe a script in combination with the graph that would store like my vectorizer, my TF IDF vectorizer to transform those into like actual tensors in the tensor data structure. And then when I would do like a document search on those, maybe I could use some of the like overlapping or score generation script or something to actually do the do the matching. I don't know. What do you think?

I think that makes perfect sense to me that this is actually what we want to do and it can even be more beautiful. You mentioned, for example, you would fetch your knowledge graph as a tensor. If your knowledge graph is changing over time, you could effectively host that knowledge graph inside Redis graph. So that is there. It's living As a database, you can effectively let it evolve over time. It might be for example, that you want to add scoring or you could say, well, on given searches, you want to bumper it certain nodes or certain relevancies. Or you might want to make custom queries you want to use as an input for your graph. Right. If you want to say, well, if you want to make something very tailor made and then effectively gears could effectively be also be gluing all these things together. So your script could already. Well, not at this moment in time, but it's something we plan to do. Your script could effectively then consult Redis Graph and fetch that data for you.

Okay. Yeah, so there could be like. So there could be routing logic that routes between all of these different scripts and the graphs and the scripts could call the graph so the scripts could get data from the tensors and all of that. Is that right?

Yes, correct. It's a bit confusing here because we're saying graph we can say, well, that's why we renamed it to model inside Redis AI.

Okay.

Because we already have Redis Graph. That's why we.

Yeah, I know, that makes sense. I'm looking at one of these figures from the, from the Redis AI docs and there's a graph blob section. So that, that makes sense. That would, that would help with the, with the confusing terminology, which is I guess always the hardest part of software engineering. Right.

I'll take that as a note to myself to update the documentation.

Yep, this is, this is truly practical AI. So you know, solving, solving problems on the fly.

Hey, guess what? Brain Science is officially launched. Episode number one is on the feed right now, so head to changelow.com brain science to listen to subscribe and to join us on this journey of exploring the human mind once again. Changelog.com brainscience or search for Brain Science in your favorite podcast app.

It Looks like Redis AI is currently supporting PyTorch via LiveTorch and TensorFlow via Live TensorFlow and also Onnx runtime as backends. That's pretty awesome. How big of a role did onnx play in your strategy around modules and stuff? And can you describe it a bit to our users on even defining what ONNX is actually?

Onnix runtime is effectively is the Microsoft backend or the Microsoft variant. The reason why we added it was because we wanted to combine well, we used to have redsml, right. So that was purely machine learning. And onnx runtime effectively allows you to also serve machine learning models compared to PyTorch for example, and TensorFlow.

So OnNX provides like intermediate between a bunch of different frameworks, right? Yes, it does, yeah. Am I right that you have like PyTorch and TensorFlow sort of the native libraries and then you kind of use onnx for everything else. Is that the kind of thought process?

That is the thought process, exactly right. So we can export, we can use onnx runtime as an intermediate format. Effectively we're using it in certain ways. Where we, for example, want to fetch some models from Spark, we can effectively then transform them in ONNX runtime and then upload them into our backends. It's interesting to talk about these different backends because the model commands on how you execute or you effectively want to do your regression classification agnostic from the backend that it's running for. So when you would set your model, when you say, hey Redis AI, I want you to host or serve my model, you would have to add some backend specific stuff. But once you want to run, doesn't know anymore which backend is effectively underneath. Obviously Redis AI knows, but your client does need to specify, hey, run me this with TensorFlow. So which makes that your client library or your application developers that want to work with your model or your application as a data science trained model, they get a fixed API. And as you, as a data scientist, you decide, well, over time we think that our model in Pytorch is better than our model in TensorFlow. You can just update that by setting a new model with the same key inside Redis. Effectively Redis AI and all your client libraries will still keep on working.

Yeah, that's pretty cool. So I'm just trying to think. So what you just said makes sense. I'm trying to connect it to like, you know, practically. If like, let's say we take the example that I, that I had before. So I've got like my vectorizer that I've created, let's say in scikit Learn, which I can convert to Onyx format and I want to. And I create a bunch of knowledge graph or knowledge base article entries in Redis. What would be kind of the steps I would go through to actually get something running to where I could take all of those articles in and then vectorize them with my scikit Learn TF IDF vectorizer and then save them back into Redis. What would be required for me to do? Would that be like writing some, some Python? Would that be like some sort of custom client? What all would be involved there?

So it's a good question. We do have a couple of client libraries. So there's a Python client for Redis AI. There's also a conversion toolkit that we kind of like created to help you convert in between these different models, if that would be necessary. So via Onx runtime, which we can probably share later on or at the end of the podcast. So from within your Python code, you could effectively just publish your model inside Rest AI. You could push it directly. You could connect, hey, here's my database, I want to push this model. Obviously, there might be other ways that you still want to want to do this, right? You might have some intermediate, some versioning or exporting your model as a binary, so you could actually version it and then you could ship it to DevOps. So they can effectively. Well, clearly there is a component missing there to make this easy to version your model and to effectively say, hey, this is now the new release we want to publish inside Redis AI. You could do it directly, you could say, hey, here's my client, and I directly push it to Redis. And then afterwards, if you want to do that, you want to run that model, you can once again use the same client. However, that client can also be wrapped in any other application that wants to consume that model.

So I'm just kind of curious. We've kind of talked a little bit about some of these use cases before we move on. Are there any other really typical use cases that you're seeing Redis AI being used for that we haven't already covered?

One of the things we think is transaction classification. The reason why we think that's a very good fit is because it's. It's a high volume, high number of requests you want to do, and you only have limited time to do so. So you don't want to waste time to find all your metadata or to fetch all your metadata from user profiles that live in potentially in different data source. So I think transaction classifications, some fraud detection, are the ones we think that will really benefit from that data locality and the scale we can offer with. With Redis.

I have a kind of, I guess, more philosophical question to ask. So I would say that probably, at least from my experience, there's probably a lot of, maybe even most software engineers these days, they've at least heard of Redis or they may even be familiar or worked with it, but maybe data scientists and AI people are less familiar with Redis. So do you envision. I guess my question is there's a lot of people already using Redis for certain things like you've already mentioned, like message brokers and other things where they have data already flowing through Redis. Do you think that it's likely that kind of the software engineers already using Redis will kind of be able to kind of bolt on AI sort of functionality into those existing applications? Like maybe they want to introduce like fraud detection in terms of the traffic going across their network or something like that? Or are you kind of hoping to kind of position Redis AI more as like a general purpose sort of model serving framework where, you know, AI or data science people, when they go out looking for the best way to serve their model, kind of Redis AI is up there with like, you know, TensorFlow serving or whatever else it is. Do you think that there's like, I don't know, maybe it's unclear right now in terms of which direction it would go, but I was curious to hear your thoughts.

I think we would love it to be both effectively, obviously. So I do think that we still have to build some tooling around this, right? So to make the full fledged mobile serving engine, to make it very practical. By the way, one thing I also want to mention here is that we're effectively not all development of Redis AI is done by Redis Labs. It's also done by TensorWork, so we're partnering with them. And TensorWork is a company that is building kind of like several tools for AI purposes. So one of the things they're building might be interesting for you guys to also look into is a versioning system for data. So you could effectively, well, like Git, you could effectively branch and make PRs on top of your data, the data you want to serve to. Well, it's called Hangar and I also recommend you to look into it. But over time, effectively, we think we will get a very nice toolkit and Redis AI will be one of those pieces into it. I think combined with tensorwork we would like to make the story where we can say, hey, we've got a complete solution towards you for serving your model. On the other hand, on the other end of the spectrum, in that case, well, I don't know who the Personas would be for sure, it's a data scientist. But Also for example, DevOps people that want to have a good solution for serving those models. On the other hand, there is potentially also data engineers and software engineers who do already have Redis as a tool in their toolbox. And with these modules they can make all these beautiful new applications. So ways you could use Redis AI combining them, for example, with other modules. So if you have a search application and you want to say, well, hey, I have these top hundred results here, or have 200 results, why don't I run them again through some model to update or to fine tune the scoring in between? Those results might have a big impact on the user experience. So we hope in that way. This is just an example. Right. We hope in that way that we will enable that many new or interesting applications will appear.

So we're kind of already talking about what you're thinking of going forward, so I'll just go ahead and ask, what does your development roadmap look like? What future directions are you interested in taking? And I know that we've. Obviously there are features that you're going to have in mind, but it sounds like we've really covered a lot of tooling. Are there any as part of that, Are there any kind of higher level applications that might sit on top of Redis as part of that development roadmap?

Yes, we should definitely look into that. We're open for lots of tips and recommendations on top of that. For that, we do have a kind of like a roadmap for Redis AI itself. We do have a couple of features that we want to add. So we would like this Redis Gears module to talk neatly to Redis AI. It might be interesting to. Well, yeah, you have the script, but you could also fetch data more easily or maybe even connect to other data sources outside Redis. There is also some batching that we want to have so that we can effectively. You can run several models in one go. So there will be also one other thing that we want to add is some tooling to get some performance statistics about how well your model is running or behaving. So what is the average time and the beauty about. Well, the beauty that we hope to achieve is that, well, there's already an entire ecosystem around Redis. So some guy somehow will have written, for example, some Prometheus connector. Right. So if you can connect that up, then you immediately get some Grafana dashboard out of the box that could show how well your model is behaving. That is one thing. I'm sure we will have many more. There's some versioning potentially of models that we also would like to do. So we can have or can keep track of several models or versions. So you can switch in between them or do AB testing in between them. That could also be an interesting thing to look into in the future.

Awesome. So as we kind of wrap up Here I was wondering if you could just share kind of maybe where. I think on one front, like where people that are interested in experimenting with Redis AI, where they should go to kind of get up and running, I guess. And then second, I'm sure that it's. I mean, there's a lot of work to do, there's a lot of tooling that's needed, integrations, all of that stuff. So maybe there's people out there that have used Redis or not, but that would be willing to contribute. So where should they go to kind of get plugged in? So on one front, people that want to be potentially users and then maybe people that want to contribute as well, what's the best way for those groups to get involved and find out more.

So if you just want to get started with it, there's a docker container, effectively all the info is on redisai IO to quick start. There is, for example, there's a docker container. You can just pull it. It comes preloaded with all the backends, so it comes with TensorFlow, PyTorch and X Runtime backend. You can just get started with that. If you would like to contribute, there is a Google Groups where you can effectively, well, all Redis AI. Right. So if you would search for the Google group on Redis AI where you can effectively reach out to all the developers that are working on Redis AI and contribute there. Right. It's a GitHub project. You feel free to open issues. We would love that. Right. All the feedback you guys can give us would be highly appreciated. I think we actually react quite quickly to those two. Right. We would love to help or to make Redis AI a success.

Awesome. Yeah, I think that's great. And of course we'll post all of those links that were just mentioned on our show notes. And of course, if you, if you're having trouble finding things or, or want to kind of follow up with questions, find us on our slack team@changelog.com community or our LinkedIn page. We're happy to make the connections for you forward on links, but I, I thoroughly enjoyed today's conversation. I was able to like, nerd out a bit and think about my own, my own problems a little bit. So. So I appreciate you being, being patient with me, Peter, and sharing so much great information.

You're welcome. No, I really enjoyed it too. So it was an honor for me to be here. So thank you for asking me.

All right, thank you for tuning into this episode of Practical AI. If you enjoyed this show. Do us a favor. Go on itunes give us a rating. Go in your podcast app and favorite it if you are on Twitter or social network share a link with a friend. Whatever you got to do. Share the show with a friend friend if you enjoyed it and Bandwidth for Changelog is provided by Fastly. Learn more@fastly.com and we catch our errors before our users do here at Changelog because of Rollbar. Check them out@rollbar.com changelog and we're hosted on Linode cloud servers. Head to linode.com changelog check them out. Support this show this episode is hosted by Daniel Whitenack and Chris Benson. The music is by Breakmaster Cylinder and you can find more shows just like this@change law.com when you go there. Pop in your email address. Get our weekly email keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.