AI/ML, MongoDB in DevOps and More with Richmond Alake

Get ready to discover some of the most actionable DevOps techniques and tooling, including performance and reliability for some of the world's smartest engineers. Hey, I'm Joe Colantonio, host of the DevOps Toolchain podcast and my goal is to help you create DevOps toolchain awesomeness. Hey, do you want to learn more about AI machine learning and MongoDB and all the things while you're in for a treat? Because we have a specialist with us today. We have Richmond, who is a developer advocate at MongoDB and an AI machine learning practitioner with an academic background in computer vision, robotics and machine learning. Really cool stuff. He's worked in software development and machine learning roles since 2016 and he's taught many, I think a few online AIML data courses, written over 200 technical articles with over 1 million views. Really knows his stuff. He's really passionate. I've checked out some of his oldest stuff, so really you're in for a treat. You don't want to miss it. Check it out. Hey, before we get into this episode, I want to quickly talk about the silent killer of most DevOps efforts, that is poor user experience. If your app is slow, it's worse than your typical bug. It's frustrating and in my experience and many others I talk to on this podcast, frustrated users don't last long. But since slow performance is as sudden, it's hard for standard error monitoring tools to catch. And that's why I really dig Smart Bears INS Hub. It's an all in one observability solution that offers front end performance monitoring and distributed tracing. Your developers can easily detect, fix and prevent performance bottlenecks before it affects your users. Sounds cool, right? Don't rely anymore on frustrated user feedback. But I always say try it for yourself. Go to smartbear.com or use our special link down below and try it for free. No credit card required. Hey Richmond, welcome to the Guild.

Hi there Joe. Thank you for having me. I'm looking forward to talking about MongoDB, talking about AI with you and your audience.

Yeah, love it, love it. Really excited about this because there's been a lot of buzz around AI machine learning. But as I mentioned in your bio, you've been here before. I think Gen really took off. So maybe a little bit like how did you see the vision of being AI machine learning? Cause it seems like you really got into the game before it became more of a buzzword or really a hot, hot, hot in the industry.

Yeah, so very quickly, before I was got into AI I used to be a web developer, so I used to.

Do a lot of.

I was a full stack web developer for a few years and in my perspective I reached a ceiling in terms of how I can develop. So the next, what seemed like a logical step was to explore intelligent systems and machine learning. And I bought a book. It's a very hefty book, right. And I thought, okay, let me actually learn machine learning from this book. And I opened the first page and there were so many equations it scared me back to uni. So that's when I decided to go to uni and do a master's in what we call AI today with a bit of robotics as well. So really I wish it was a calculated foresight. It was just curious. I was just curious and I was a bit bored being a web developer. So looking what I was looking for the next challenge.

Nice. So were you surprised by when Gen AI really took off? Probably like what was it, 2020, 2022? I mean in November, probably when Gen AI would really seem to take off. Were you surprised by that?

I was surprised at the uptake from the mainstream folks. Right. ChatGPT really became a very, very prominent consumer AI product. There have been AI products in the past as in generative AI is not new. Natural language processing is not new. There have been a few. When I was in uni, we did a bunch of image text generation from image with recurrent neural networks and whatnot, trying to caption Instagram images automatically. But ChatGPT came out time when it was needed because you know, with building software and startup ecosystem is all about timing, right? Timing and execution and it came out at the right time. We're in November right now. So ChatGPT came out two years ago. Two years ago. Which is amazing, right? Because it feels like age. It's been out for ages and in two years the space has just developed so much. But long story short, no, I wasn't surprised. I wasn't surprised, but I wasn't skeptical as well. I wasn't just putting it aside like with other stuff we've added a past such as like Internet of Things or maybe big data in some respect.

Absolutely. So, you know, kind of off topic. We're about to head into 2025. Gen AI has been a thing. Do you see other forms of AI or machine learning being neglected that you see maybe is about to pop off as well? Maybe in 2025?

I think really the whole space is moving towards a convergence in terms of the capabilities or modalities of AI. So you're going to have generative AI is mainly focused on text and at the moment with text generation, but you're seeing image generation as well with a lot of diffusion models. But really there's going to be a convergence of functionalities that comes out from this model. So I don't think any capability or modality is going to get left behind.

Nice. So I know one thing this year seems to be a trend is multimodal AI. So before it was more like text, now it could work with images and things. I think your background is also in computer vision. Is that the same thing, multimodal AI? Cause I've seen like Claude just came out with something where I think it could automate a lot of business tasks. It's not very successful right now. But is that the same thing as computer vision?

Well, it's using some computer vision elements, I'm guessing. I don't know how the team over Anthropic or what they're doing behind the scene, but there have been similar tools that have come out in the past where we're taking screenshots of the actual computer or the system and passing that into a model for understanding or interpretation of next steps. Right. So they're doing this obviously in real time at very, very high speeds, high latency, low latency. I mean, but one thing is what Anthropic put out with. I guess you're referring to computer use.

Yes, yes.

Yeah, it's pretty much the promise of AI, right. Which is automation and freedom from manual labor and what we're seeing there. And again, what they put out is not new because Microsoft have done, they did something similar maybe a few years ago they had a bit of a backlash in terms of privacy and sending up your data from your computer over to a big tech. But one thing I would say is we're going to see more systems like that and we can categorize them as a gentic system where you have this systems are able to act on your behalf and actually use the components of your system infrastructure to actually achieve an objective. And over at MongoDB we're seeing a lot of this conversation popping up and we're getting involved.

So what do you mean by you're getting involved? Then I guess let's bring in MongoDB into the equation here. MongoDB, what does it and how does AI machine learning play into it and how are you building it out to meet these needs like you just mentioned?

Yeah. So I'll talk about what we're looking at, what we've seen in the past two years, Right, the form factor. So just to give a very quick background, we saw ChatGPT came out around two years ago and what we could do is just query and send in the text and we got some response back. It blew our minds because this little model was a world compressor and had most of the Internet inside of it so we can interact with it. The nurse form factor, we brought in our own data, supplemented it with the user query. Now we're getting really good responses that are relevant to us, that are personalized. Now we're seeing more agentic system where the systems can actually act on our behalf when we give them objective. So MongoDB is we've been at the game of storing data and retrieving data for over a decade. So one thing that we know is how to retrieve relevant data and supplement this data over to this agentic system or to this LLM application. Mostly rag pipelines and AI agents and agentic system. So where MongoDB really plays in is. And just to give a bit of background when we're talking about supplementing data, one of the key things that has happened in the two years is I guess the proliferation of vector embeddings. So you can imagine a data object. So let's take like a piece of text, we could actually parse it into this embedding models and out comes a numerical representation of this data object. But this numerical representation has captured the semantics and meaning of the data object. And what that allows us to do is we can actually start to calculate the similarity, the vector similarity between various data objects in various vector embeddings in a high dimensional space. Now, one thing to say is these things are not new. Vector embeddings have been in the world of computer vision for years and NLP for years. But now they're more accessible. The models are a lot more capable. They have several billions parameters, which means that there is more performance that we can get out of them. Now what MongoDB allows you to do as a developer when you're building this application is actually store this vector embeddings in a database right alongside all your usual operational data. And then we allow developers and our customers to use this data to build LLM application in whatever form factor they take. So when I'm talking to customers building RAG, I tell them MongoDB is your operational database and your vector database. But when I'm talking to developers building a Gentex system, I get a bit more higher level and I tell them MongoDB is the memory Provider for your agentix system. Right. So if you think about agents and you think about humans, humans need memory to actually perform really well, right? We need different forms of memory. MongoDB is the memory provider for your agentic system. I'll put a pause there. I've said a lot. I bet you have some questions.

Yeah, so you know, I'm a little bit older, I assume a lot older than you. And so MongoDB is, it's been around for a bit, like you mentioned, but so when I think of. And also I'm a novice with AI, so I never thought about where AI, the LLM is storing the data. So is it always storing in a database or are you saying MongoDB is, is. In order to have an LLM, it needs to store information in a database and MongoDB has been created to be optimized to deal with all kinds of LLMs for that.

Yes, exactly that. But one thing I need to mention is a caveat, right? LLMs, they have something called a context window where they could hold data temporarily for maybe for the conversation, right? And so we have this context window. You, you put your input, you could put a bunch of data there, but if you have another conversation or another session, that data is lost, right? So we need to store that data, we need to persist the data and retrieve it whenever we're engaging in or picking up from a session. So essentially you have, you use MongoDB as a conversational store. But also with MongoDB you can actually start to build something that we call retrieval augmented generation, which means that you're using domain specific data as input to the LLM to get more better output. So you might have some data on your laptop about yourself. And now you can have like a local LLM where you can just have this LLM, get access to your data. But what MongoDB can do is store information about the data, such as the vector embedding, and enable you to retrieve data through semantic search and supplement that to any query to an LLM. This is what we call RAG retrieval augmented generation.

Nice. So give an example of a real world application that would benefit from this.

Yeah, So a very simple one is a chatbot. We all know chatbots. We've used them for, like you said, you're much older than me. I don't know when one chatverse came about, but I feel like for me, chatbots have been around since I could use the Internet.

Ouch.

You got in your age up. I did it.

But really chatbots have been about right. So for several years. But one thing that chatbots have been lacking is a bit of personalization, which is the first thing. The second thing is the ability for this chatbot to retrieve information behind the scene that is relevant to the current conversation, be it the customer or. Or the problem domain.

So what we have today is a.

System or mechanism of retrieving information by semantics and meaning. And that is also supplemented by the incredible power of LLM to generate response that are just that sound. So natural language like. And with this, we, you can bring this capability into chatbots and start to create customer service chatbots that are that sound human like, that can use all the data within your organization to help troubleshoot problems and could help people get to a resolution much faster. So this is within the customer service industry. And I'm using that example because it's common. Right. Most companies have chatbots or have a customer call center have a customer service function to help solve problems. So that is one of the common form factors and use cases we see.

Gotcha. Awesome. All right, so once again, I'm gonna show my age. I know back in the day, having a database just with stored procedures, sometimes it caused a lot of performance issues. So just stuffing this with all this other data now, are there performance issues? Has this been a solved thing that I missed out on? Like, how does this handle? It sounds like you're using large data sets. So how, how does it impact performance at all?

Yeah. So MongoDB has been at the game of storing data and retrieving data for over a decade. And what we actually do for our customer is bring up all of our learnings on how we handle data, unstructured data, and the way that we actually structure data within MongoDB is so familiar to developers. So we bring all of this learning directly to our customers. So what that means is we know what it takes to optimize data retrieval and we know the applications that our customers are building and the use cases. So MongoDB is very well built for performance and actually low latency for AI application, especially if you have your application running in real time. So we have a bunch of customers that are building conversational AI that can operate in real time, or chatbot that can actually solve problem and retrieve problem, retrieve information in real time. And then we have more features that allows performance to be balanced across different data types, which is another sort of another key learning. So imagine you have a chatbot that uses vector data more heavily than the operational data. We have features that allow you to actually balance the performance or the Allocation of compute to a certain operation or certain node. This is one of a very key feature that we learned across the years and we're bringing to our customers that are building the future essentially.

Right. And so good point. There is once again we used to have like a database admin. He owned the database, he knew everything about it. You come from a background where you start off as a web developer. So nowadays is it more like everyone is a full stack, so they need to be a database expert as well. And MongoDB also makes it easier to stay up to speed on what they need to know.

Yeah. So I would tell you, I work in MongoDB now, but I've been a fan of MongoDB for years, since I was in university. And that's because MongoDB for developers did this thing. It unified the application stack when I was a web developer. So I didn't start off as a full stack web developer. I started off as a front end developer. But out came MongoDB and Node JS and it used similar paradigms that I was familiar with as a front end developer within the data layer and with Node JS and a bunch of orms like Mongoose were able to connect the entire stack. And then I could transition literally because mongodb I could transition into being a full stack developer. We're doing the same for AI. We have an AI stack and what MongoDB does is it applies the same paradigm, the same framework of operating with LLMs or other components of the stack. So what I mean is LLMs understand JSON structured data. In fact, for to enable two use in LLMs, the schema you provide to the LLMs is in JSON and we've had some customers where we actually enable scalability by storing these JSON schemas in MongoDB. It feels like such a natural fit and you see that with the amount of code you have to with the amount of code you have to write, which is quite short. So to basically answer the question, that's what we're doing, we're just shrinking the space. And it's not one thing I find is most developers don't want to become database admins. Right. So we want to do the cool stuff, which is build cool application. So I don't want to spend a lot of time thinking about optimizing my database. And so what MongoDB does is it's a very developer friendly database. We have a developer friendly platform where you can start to see performance insights on the interface, query optimization on the interface and it tells you key information we make developers develop as opposed to maintain database systems.

Love it. Yeah, so that's a big selling point. So also you are a developer advocate. How much education do you need to do in order to let them know? Yeah, MongoDB has been around for a while, but it has these AI ML capabilities that's going to enhance your future development as well.

Yeah.

So because MongoDB is done its job really well, we actually quite fortunate in the fact that within the developer space we are known as the NoSQL database.

Right.

So again, right from when I was university, I was introduced to MongoDB up until now. So we're very popular within the space of application development now. AI today presents an opportunity for developers to create new intuitive tools and applications that they couldn't before. MongoDB has always been with the developers, so we're almost like a natural fit.

For developers are willing to build innovative products and do it very quickly. So we really focus on developer productivity and we take that to heart because we partner with most of the application, most of the key technology provider on the application stack and enable a developer to spin up a database and create an application, an LLM application within minutes and even a more complicated application within a day or so. And that's because we focus on making our developers productive.

Nice. So you made the point that developers don't want to be DB admins. Also with LLM AI, the thing lately has been security. So I'm sure they don't want to be security people either. Is there anything built into MongoDB to make that aspect a little less daunting for developers?

Yes, security is nothing new when it comes to application development. And because we've been in the game of storing Data securely, safely, MongoDB has trusted, is a trusted platform for holding your data securely. So we have different features, various level of securing and storing your data. So we have role based access control, we have credible encryption, we have field level encryption, document level reduction as well. So there are different features and tool sets at various level for developers to actually enforce enterprise level security and governance within their application. And the one thing we do is we don't make you a database admin. Right. So all of these features are very easy to understand, very easy to use and just gets a developer more productive and doing what they do best, which is developed.

All right, so developers listen to this, like, oh, I used MongoDB with the IML. I'm going to create something really quick. What do they need to know? So you've started as a web developer and then you taught yourself a ML. A lot of people always ask me, what do I need to know in order to be successful as a developer with AI ML? Any first steps or things you think are critical?

The first thing is don't do what I did, which is buy a master textbook and scare yourself to getting a masters. I've seen a lot of YouTube videos that probably covered a few of the courses I did at university, and I paid a hefty price for a university degree. So I would say the need to go to university to become really knowledgeable in what we define AI as today. And that's very key. What we define AI as today is reduced. We have the Internet, there's YouTube, Stanford lecture, videos are available online and top university videos are available online. Deep learning AI has some very good resources, including a resource showing how you can use MongoDB for AI applications. So if you're a developer today and you want to get started with AI and machine learning, I would say always start with what languages do you know, what programming language do you know and you don't know? Right. Python is pretty much the most dominant language when it comes to AI application development or data science. So if you need to learn a new language or you don't know Python, you need to learn Python or even JavaScript, maybe that is the gap you need to fill. You could start looking at your environment and start to think, what can I solve with AI today? Right. So that's how I started. I looked for a problem and I tried to solve it with AI, and that sent me down a path of exploring different tools, different form factors, and you start to build up knowledge through curiosity, which is how I got into AI. Anyway, I was just curious. So we have loads of resources over at MongoDB that help developers with backgrounds in Python, in JavaScript, in Java, get started with what we define as AI today. Right. Rag and agentic system. So in the show notes below, there's going to be the links to actually get you into our GitHub repo. So if you are very hardcore developer, you just want show me the code, right? So there'll be a GitHub repo that actually gives you a bunch of resources, a bunch of Google Collab notebooks to show you how you can build AI applications.

Love it. All right, so you know, in my industry with software testing, a lot of testers are saying we're going to be replaced by AI machine learning. Our role is going to change. So as a developer, AI machine learning expert, where do you see it because I was speaking with someone that came up with me at the same time, he's a developer now, he's a, a cto, and he said he was using Cursor to create an app. And he said it was incredible, like he didn't have to do a lot of things to get a working application. Do you see development being replaced by AI machine learning? And what skills would people need in order to keep up with maybe the advancements that you see coming down the road?

So I'll be lying to you if I say I don't see some development being replaced by AI tools today. But to say, to go to the extent of saying developers will be replaced, I think that's, it's too early to call, right? But one thing that there is some development that will be replaced by AI, and that's because when you're building a software and you're building tools, there's a lot of boilerplate code that is just repetition, right? So those things AI can probably help you with. Oh, create a button in this image, okay. I want a button to maybe send a HTTP rest call to the backend. Okay, we create a backend. And all of this are all sort of like paradigms that are standard. But there becomes a point, there always comes a point where you actually need actual knowledge of building applications and system design and debugging. And AI can't help you. AI might help you get to the 80% and which is very useful for becoming productive, but there is 20% which is just experience and expertise. And again, that 20%, I don't bet on language models or the on AI ever. So that 20% will eventually become maybe 5% in a year, in maybe two or five years time. But the job and the function of a developer will only evolve along with it, right? So this is not the first time humans have gone through a revolution, right? So we went through the agricultural revolution, industrial. So there are different things that happen within mankind history that forces us to rethink the nature of work. AI is a forcing function and it's an accelerating function, which means we have to rethink what we define as work. And start to rethink what, how we start to maybe define our purpose and find fulfillment in the things we do. But it's not going to replace. If there was an AI that could replace me, I would use it in a heartbeat, really. I've got a lot of educational code materials to be writing, so if I have an AI like an AI version of Richmond just coding away, I would Love it. While I'm doing the podcast, there's just an AI version of me just coding away. That'll be lovely.

Nice. So you also have a background in robotics. Where do you see robotics going? This is off the wall. But you know, Tesla released something and it was forgot the name of the robots are, but someone said was it that were kind of being remote controlled, so really isn't where people think it is. Do you see robotics take being the next level from AI?

So very quickly, my background in robotics was actually in space robotic. So we have rovers that do space exploration and we were just calculating the freehand motion in. In space, which was just inverse kinematics and kinematics, which is basically mechanics. Right. We did a bunch of path planning and working with SLAM and all of the good stuff, which is all fun. But where do I see AI in terms of robotics? Right. Going today with the stuff we're seeing from Tesla, Optimus and.

Yes.

And Boston Dynamics.

Yes.

Very quickly, we're not going to see IROBOTS anytime soon. So I don't know if you've watched the movie iRobot, but yes. Yeah, that's not going to happen next year. But again, I don't bet on the innovation, on the speed of innovation we have in AI.

Right.

And also the determination of us as a collective when we set our eyes and solving problem, especially when there is huge capital involved. So maybe in 10 years, I would say in 10 years we might be having household humanoid robots. And that is not even far fetched because we have toy robots that kids can buy and play with. But again, the definition of useful robots or humanoid robots is going to be a spectrum. Do you have one in your house that can just maybe lift his arm? Or you have one that can just actually do all your manual labor and all your laundry and whatnot. So I think we're going to get there. That's the objective. How quickly is anyone's guess. But there is a lot of very determined and educated people working on this problem.

Love it. Yeah. And as populations get older, I'm looking forward to a robotic caregiver for sure. So hopefully 15 years, it'll be there.

You're always bringing up your age and you don't even look old, Joe. You look very young. So.

You'Ll be there to see our robot overlords?

Yeah, I hope so. Okay, Richmond, before we go, is there one piece of actual advice you can give to someone to help them with their MongoDB AI machine learning efforts and what's the best way to find or contact you.

So one piece of advice is there is a lot of noise in the space. You need to figure out how to block out the noise. And one way to do that is to follow people such as Joel listening to podcasts such as this to get some ideas of what, at least at a higher level, what people that are in the space are thinking. The second thing you should do is look for learning resources. Learning is not as expensive as it used to be, except for university education. The prices are going up here in the uk, but learning is. There's so many learning resources online and amongst that in MongoDB, we in the art of structuring and creating educational resources for developers that makes them productive as quickly as possible. Not just for using MongoDB, but for actually building and understanding RAG systems, Agentic systems and LLM application. And what that means is we have a portal called the Developer center that has tons of tutorials that could teach you how we build these systems using MongoDB. We have a GitHub resource that could give you the code that can show you how to develop the system and implement them. But even higher level, we have several articles, our thought leadership pieces that actually show leaders and developers what to think, what questions should they be asking when they're thinking about rag, the AI stack or Agentix systems. So we are really, we care about the developers, we care about the people building the systems, and we provide educational content for various audiences. So all of where we do this are going to be in the show.

Notes and you can find it down below. So definitely check it out. All right, Richard, I'm just thinking of people that are listening to this like, well, MongoDB sounds awesome. I assume all of them are using MongoDB, but for those that aren't, they're working on a more legacy application. They want to move to MongoDB. How developer friendly is it? What does it take to migrate from one to the other? Is that something that's common?

Yeah. So we actually built internally a product that helps people move away from relational to MongoDB databases. So I'll get a link below to our relational migration. And we have a bunch of programs that help modernize legacy systems in enterprise organization. So we have the app modernization program and we also have the relational migration tool that helps people migrate. And one thing we also see people doing is in, when they're building this AI application is using maybe specialized databases, right? They go into this space thinking, okay, I need to use vector data because I've been told I need to search by meaning so okay, now I need a vector databases. But what most people then find out is I need to store more than just vector data. I need to store my metadata. So they're pretty much invested in a database solution. Then they plug in another database solution to host their metadata operational data. Now you have two database solutions, right? You have to hire two developers or maybe more. You have to deal with two development systems and you have to sync data between them. And I already said developers don't want to become database administrators and definitely not for two database systems. So what we do in MongoDB is unify the functionalities that we need today to build AI application. So within MongoDB database you can store your vector data and your operational data. And this makes MongoDB one of the most intuitive developer friendly databases for most folks are building out there. So that's my 2 cents on that.

All right, before we wrap it up, remember frustrated users quit apps. Don't rely on bad app store reviews. Use SmartBear's Insight Hub to catch, fix and prevent performance bottlenecks and crashes from affecting your users. Go to smartbear.com or use the link down below and try for free for 14 days. No credit card required. And for links of everything of value we covered in this DevOps Toolchain show, head on over to test guild.com forward/p176 so that's it for this episode of the DevOps Toolchain Show. I'm Joe. My mission is to help you succeed creating end to end full stack DevOps toolchain awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation, testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges. We're excited to collaborate. Visit testguild.info to explore how we can create transformative experiences. Together, let's push the boundaries of what we can achieve. O the Tescill Automation Testing Podcast with lutes and lyres the bards began their song A tune of knowledge, a melody of code through the air it spread like wildfire through the land Guiding tester showing us secrets to behold.