How Databases Work in AI Applications with Andi Gutmans, VP and GM of Databases at Google

Databases are incredibly important for anyone working in data and AI. Same is true of generative AI, but I'm wondering how they fit together. So can you just tell me how our database is used by AI applications?

Yeah, no, great question. And you know, I think first of all, any AI application is foremost an application, right? And we know there's really, there are really no applications without a database underlying them. So that's, I would say, the starting point. Now the interesting thing is, as we think about Gen AI, we have these really powerful foundation models like our Gemini model. But those powerful foundation models, they do hallucinate. And really the way to get the real benefit from these AI models is to take your operational data and really merge that together with the foundation models so you can get very accurate, contextually relevant, enterprise grade experiences. So the way I really think about this is you can't be successful with gen without really understanding where your operational data sits and having the tools and capabilities to bring that operational data to your foundation models. Okay.

So it's really about hooking up. Well, we've got all this data that just make sure that the AI is given the correct answer and it's finding ways to sort of hook that up to your application.

Yeah, absolutely. Like just think about a chatbot like, hey, when's my flight? Today? Right. The flight is actually recorded somewhere in the database. So we've got to make sure that that foundation model ability to interpret your question ultimately gets translated to a database lookup that actually looks up your flight. So that's where the connection comes in.

Yeah, that makes a lot of sense. So because there are just dozens, maybe hundreds of different kinds of database out there, what sort of databases do you need for your AI for your AI applications?

I would say what we've really been seeing over the past year or plus is that pretty much all databases have started to add the capabilities needed to be able to be integrated into generative AI applications. So, you know, we also have to purpose build vector databases, right? That kind of emerge with Gen. But what you're actually seeing is a trend of, you know, relational databases. Whether that are, you know, databases like Oracle, like Postgres, with PGvector, you know, they're all adding vector capabilities, they're all adding support for some tooling ecosystem like LangChain. And so the way, you know, what I tell customers is yes, they're going to be, some databases are going to be somewhat better than others on things like vector lookups and so on. But really, if you already have your data in a specific database the first thing I would think about is, hey, does that database actually have the base capabilities to help me exploit that data with Gen AI?

Okay, that's interesting that it's not just about vector databases, which is still, I guess, quite a lot of the limelight with respect to generative AI, but you actually need relational databases as well. Are there any particular properties of databases that you think are important here?

Yeah, absolutely. So one is, you know, vector support. So being able to do efficient vector indexing, vector search. So for those who don't know, vector embeddings is a way to represent semantic data. So in the good old days of full text search, right, we would basically do lookups with a piece of text and usually use an inverted index to go and look at the documents that actually match that. In the era of Genai and vector embeddings, we can actually have a semantic representation. It's usually a vector of numbers that we can index and then do a similarity search and actually find things that are very similar. So this way you can actually find things like King being very similar to Queen as an example, you can find similarities in data. And so that's very powerful both on its own and then also in the context of retrieval, augmented generation with Genai workload. So one is basic vector indexing and search super critical for Genai, but then also capabilities like natural language interfaces to the database, having the ecosystem integration with tools like LangChain and Llama index, being able to transparently call vector embeddings APIs in services like Vertex AI. So we're seeing just a bunch of capabilities starting to be built into the database to just make it so much easier and more efficient for customers to build these Gen AI apps.