#255 Not Only Vector Databases: Putting Databases at the Heart of AI, with Andi Gutmans, VP and GM of Databases at Google

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com the technology landscape is moving so fast. The business is moving so fast. You've got to transform your business. Don't waste your time integrating the different pieces and managing your own like infrastructure. Right? I think one thing I'm hearing loud and clear from customers is we need more bandwidth to actually make business impact. So I think, you know, this is where I really advise, like, where it makes sense, which I think is most of the time, like use managed services, enable us to do the undifferentiated heavy lifting for you so you can really focus your time on innovating and delivering business outcomes versus dealing with the plumbing.

Welcome to Data Framed. This is Richie. Databases are central to every business, and since they've been around for decades, there's now a database for every application or purpose you can think of. I get confused about which database is needed for which use case. The rise of generative AI has made things even more complicated because now you need to understand a whole new class of databases, namely vector databases. Actually, it's not that easy because although vector databases have become synonymous with Genais, it turns out that graph databases are also useful for AI applications. If that sounds confusing, well, you better listen carefully for the next 45 minutes. Our guest is Andy Gutmans, the VP and General Manager of Databases at Google. That is, he's in charge of product engineering and user experience for all the databases on Google Cloud platform. He's got over 20 years of experience as a tech leader skill, starting as a founder of the Zend Web development platform and including a stint as VP of Analytics at Amazon Web Services. If anyone can help us figure out which databases we need for our AI applications, it's Andy. Let's hear what he has to say. Hi Andy, welcome to the show.

Hi, thanks for having me.

Just to begin with, databases are incredibly important for anyone working with data and AI. Same is true of generative AI, but I'm wondering how they fit together, so can you just tell me how our database is used by AI applications.

You know, I think first of all, any AI application is foremost an application and we know there's really, there are really no applications without a database underlying them. That's, I would say, the starting point. Now the interesting thing is as we think about gender, we have these really powerful foundation models, right? Like, like our Gemini model. But those powerful foundation models, they do hallucinate and really the way to get the real benefits from these AI models is to take your operational data and really merge that together with the foundation model so you can get very accurate, contextually relevant, enterprise grade experiences. So the way I really think about this is you can't be successful with Genai without really understanding where your operational data sits and having the tools and capabilities to bring that operational data to your foundation models.

Okay. So it's really about hooking up. Well, we've got all this data that just make sure that the AI is given the correct answer and it's finding way to sort of hook that up to your application.

Yeah, absolutely. Like just think about a chatbot like hey, when's my flight? Today? Right. The flight is actually recorded somewhere in the database. So we've got to make sure that that foundation model ability to interpret your question ultimately gets translated to a database lookup that actually looks up your flight. So that's where the connection comes in.

Yeah, that makes a lot of sense. So because there are just maybe hundreds of different kinds of database out there, what sort of databases do you need for your AI applications?

I would say what we've really been seeing over the past year or plus is that pretty much all databases have started to add the capabilities needed to be able to be integrated into generative AI to applications. So we also have the purpose built vector databases, right. That kind of emerge with gen AI. But what you're actually seeing is a trend of relational databases. Whether that are, you know, databases like Oracle, like Postgres, with PGvector, they're all adding vector capabilities, they're all adding support for some tooling ecosystem like LangChain. And what I tell customers is yes, they're going to be, some databases are going to be somewhat better than others on things like vector lookups and so on. But really if you already have your data in a specific database, the first thing I would think about is, hey, does that database actually have the base capabilities to help me exploit that data with gen AI?

Okay, that's interesting that it's not just about vector databases, which is still, I guess, quite a lot of the limelight with respect to generative AI, but you actually need relational databases as well. Are there any particular properties of databases that you think are important here?

Yeah, absolutely. So one is, you know, vector support. So being able to do efficient vector indexing, vector support, search. So for those who don't know, vector embeddings is a way to represent semantic data. In the good old days of full text search, we will basically do lookups with a piece of text and usually use an inverted index to go and look at the documents that actually match that. In the era of gen and vector embeddings, we can actually have a semantic representation. It's usually a vector of numbers that we can index and then do a similarity search and actually find things that are very similar. So this way you can actually find things like, you know, King being very similar to Queen as an example. You can find similarities in data. And so that's very powerful both on its own and then also in the context of retrieval, augmented generation with gen AI workloads. So one is basic vector indexing and search super critical for gen AI, but then also capabilities like natural language interfaces to the database, having the ecosystem integration with tools like LangChain and llama index, being able to transparently call vector embeddings APIs in services like Vertex AI. So we're seeing just a bunch of capabilities starting to be built into the database to just make it so much easier and more efficient for customers to build these gen AI apps.

Maybe let's talk about this in the context of Google and Google, Google Cloud platform in particular. So there are hundreds and hundreds of services on there now I have to say I tend to get lost and not sure which ones I might need. So can you just give me an overview of like what the different database services are and which ones apply to AI?

Yeah, absolutely. And you know, I run the operational databases for Google, so I'll talk mostly about operational databases, but we also have analytical engines like BigQuery. But on operational database side, I kind of think about it as three categories. The first one is the category of managing third party databases. So these are databases as is like MySQL, Postgres, Oracle SQL Server, Redis, Valkey, and those are managed services that just enable our customers to focus their time on innovating as opposed to integrating and managing the systems. Then the second category is what I call the modernization category where we primarily position AlloyDB as a great migration target for customers coming off legacy databases and really wanted to modernize on open Source and open APIs and by the way that engine also plays a very important role in Genai because Postgres has become very popular to gen developers with PGvector and in AlloyDB we've actually created a lot of additional differentiation around building gen apps. And then the third category is what I call the cloud first databases. These are databases we first of all start to build to run Google's billion user businesses. Those are engines like Spanner, BigTable. We also built Firestore, a document database on top of Spanner and those are the very transformative databases. Now the way I look at this is all databases need to be gen enabled. So we've been building in vector capabilities across all these databases. We've been building them into the ecosystem like LangChain and Vertex AI. And really our belief is everywhere where customers have data needs to basically be easily exploitable for Genai applications.

So you've got the sort of, whatever, the standard database, you're probably already using your Postgres or MySQL or whatever and you've got some new things like AlloyDB and then you've got stuff for, I presume, very big data that's well, things like Firestore or Spanner. Is that right?

Yeah.

Okay, so you mentioned AlloyDB is like the sort of the new thing that is like a modernization target. What does that mean? Like how is it different from some of the existing databases?

Think about AlloyDB as being 100% Postgres compatible, but really Postgres on steroids. We've taken the Postgres engine and we solved some very fundamental challenges that customers had when they were coming from very high end databases and wanting to modernize on open source. So we solved issues like performance, scalability, manageability. So for example, it is four times faster than stock postgres. We have an analytics accelerator that allows it to go 100 times faster. We've solved the linear scaling issue so you can actually literally scale up on cores, which you can't really do with open source postgres. And by the way, we've made this available everywhere. So it's not only on gcp. We have a version of it called Alloy DB Omni that customers can actually download. They can run it on premises to modernize, they can even run it in other clouds like AWS and Azure. So that I would say is AlloyDB. And we've just seen tremendous success both in customers coming from legacy databases or just customers who have very high end postgres workloads and just need that additional scalability and performance. Now the other piece that is related I would say to Gen AI is we've also been innovating really quickly on Gen AI within AlloyDB. So for example, the vector capabilities are. It's actually built on 12 years of research we've done inside of Google that runs already internally some of our Google applications. We've brought some of that innovation into Alloy DB on vector processing. We've also been working on natural language interfaces to AlloyDB. So a lot of innovation also around Genai happening in this engine.

Okay, so the idea of something that's a bit like postgres but faster and more scalable, that does sound quite appealing.

Exactly. And our commitment is 100% compatibility, so no vendor lock in use alloy DB. And you know, if at any point in time you want to just use open source postgres, you can of course you'll then forego some of those benefits I mentioned.

So you mentioned that your focus is mostly operational databases rather than analytical databases. Do you want to talk through like what's the difference? And whether you would care about one or the other for using with AI applications?

Yeah, it's a great question and you know, the lines are blurring and I would say that definitely as a vendor, because of how we've built our infrastructure, we're probably best positioned to actually make this a much easier experience for customers across both transactional analytical engines. But just to give folks a bit of an idea, transactional engines tend to be very much focused on short latencies, transactional workloads very much on the serving side of the house. Analytical engines tend to be in a columnar format as opposed to a row format. Of course there's always it depends answer. Because some systems are a bit hybrid and because they're in the columnar format, they can be much more efficient at processing very large amounts of data and are usually built to be more of a data warehouse data lake capability. Now BigQuery is a market leading analytics engine. Alloy DB and spanner are market leading transactional engines. What's really interesting about some of the work that we're doing at Google is because we have built our infrastructure with compute and storage completely disaggregated, our ability to start blending these worlds is actually quite high. So for example, BigQuery can actually query Spanner or Bigtable in production with full workload isolation at any scale. So customers can now benefit from having both the transactional analytical capabilities without having to move data. Another thing we did in AlloyDB, we added a columnar engine which automatically will translate some of the key columns of your data to A columnar format and then give you that hundred times fast transyl on the transactional engine. So a lot of work happening not just on making sure both our analytical and operational databases are market leading, but also the fact that our customers have an easier and easier time to efficiently use whatever they need on the same piece of data.

So in theory, if you've got storage, data storage in one place and then separate databases, you can have different people within the organization querying the data in different ways. So I'm trying to think of a good use case for this. So you can have your data teams querying using an analytical database and then maybe what commercial teams making use of their transactional capabilities.

Yeah, I mean, let me give you an example. So for take Spanner as an example. Spanner runs some of the most mission critical workloads for our customers. Spanner is underlying systems like Uber and others, you know, that people are very familiar with. But you often have folks in the back office like data analysts who want to do kind of real time operational analytics to make sure they can optimize the business or alert on issues on the business in the traditional way. You have to actually move data from the transactional system to analytical system to go and analyze it. But given that we built this data data boost capability that allows customers to do real time querying from BigQuery to Spanner with full workload isolation, meaning it's not going to have any negative impact on the transactional workload. Like that is very unique in the market and it's because of how we built our infrastructure where any piece of compute on our site can access data in microsecond latencies. In traditional architectures that we see with other cloud providers, they use Blob storage. It's usually tens of milliseconds of latencies. And so these systems become very complex, they use caching, they become very brittle. We don't have that issue at Google because of how we built our system.

For organizations who are like, okay, we want to perform a database transformation program, we want to update things like where do you get started?

I think there's always, and it depends, it really depends on your use case. But I would say what we've definitely seen become probably the most common. The most popular engine in enterprise is Postgres. You know, we're seeing about 50% professional developers use it. And we've actually made sure that three of our engines have a Postgres interface, that is Cloud SQL that basically has managed Postgres. AlloyDB which we talked about, which is Postgres on steroids. And then even with Spanner, we added a Postgres interface so developers could use a familiar interface. So I would say I would probably use that as a starting point. But then depending on the use case, you probably want to broaden your aperture. So for example, if you want caching, you may want to look at Valkey, which is the new Redis fork. If you have a data fabric you're building, you need very high ingestion rates on key value. BigTable can be a really good solution. So we do help our customers make that determination of the right engine based on their workload and beyond the tooling.

I suppose the skills you need to work for these things are probably going to change. So I guess maybe first of all, who is typically involved in working with these cloud databases?

First of all, the benefit of these fully managed services is the customers can really focus their time on innovating and not integrating. And so we're seeing as developers have a lot more flexibility now to choose the database they want because we're doing the management for them. And so they don't necessarily have to have as much in house skill set of really operating these databases as they used to have. So the developer side of the house is definitely critical. The second team we tend to work with is the platform engineering team that are the teams that really think about the resiliency of the application architecture as a whole. They think about security, governance, data protection, and so they tend to support the development teams on the operational side. Again, they don't have to be as deep on the database because we manage it for them, but they do have to make sure that the application architecture as a whole is sound and that they're managing that application in a secure and available manner.

Okay, so in this case it's mostly software engineering teams, data engineering teams, infrastructure teams. How about the rest of the organization? Does it change the way, for example, analyst teams need to work or other people who are working with data? Like once you move to a cloud database, is that going to change people's workflows or what they're doing?

I think what we're seeing is that a lot of the data analysts Persona are still using kind of our traditional data warehouses. But these data warehouses are becoming way more powerful because now they can also reach into the data lake, they can reach into the operational databases that have AI capabilities. So what we've basically done is we build BigQuery as kind of that single entry point for the more of the analyst practitioner. And then from there they can Both use their existing SQL skills, but some of the new skills they're going to learn is how to maybe do more federated queries, how to use embedded AI in their queries. So there are some new skills they're picking up, but we're definitely giving them an ecosystem, an interface that they're quite familiar with.

Okay, so that's kind of cool that most of your standard SQL skills is the same, but then you've got a few extra things. So you mentioned federated queries and embedded AI queries. Do you want to talk me through what those things are in a bit more depth?

Yeah, absolutely. And actually, you know, even before I talk about that, just one more interesting anecdote is I still think SQL is still the language that most folks feel comfortable with. So even in a service like BigTable, which was really the service that launched a NoSQL movement, we just announced SQL support, no SQL databases. Most of them actually have added SQL support over the years. So that is still a good interface to your question though, on both the federation and also the AI embedding. So federation means you can actually quite easily query across different data sources. So even if you're in BigQuery and a lot of your data is ingested in this part of BigQuery Native story, if you have some logs that have been indexed in your data lake on Parquet, or using a file for an ingestion format like Delta Lake or Iceberg, you will actually be able to join Both data in BigQuery and data in the data lake. It also integrates with services like AlloyDB, like Cloud SQL, like Spanner. So basically, in a single query, a user could actually query across these systems and join them. So they can always make sure that they're actually querying the freshest data. It doesn't change very much in their skill set because it's still the same SQL they love it's external table. So it's very natural to use. So that is one element, I would say on the AI side, we have AI functions that make it very easy both to start, both to train and fine tune models based on the data customers have or to do inferencing, meaning, let's say you have a column in BigQuery that has text and you want to calculate a vector embedding. We can make it very easy for them from within SQL to basically generate another column with the vector embedding based on that text column. So that is just one example. There are obviously a lot of different inferencing APIs, but the nice thing is the practitioner doesn't have to leave their tool, they're still using SQL, so we've really made it super easy for them.

Okay, that just sound interesting. So I think perhaps a lot of times when you have text data, you really just want to turn that into like something a bit more tangible, kind of almost feature engineering. And so being able to do that in SQL, rather than having to like go and break out Python or something, and then you're messing about with going backwards and forwards between the database and maybe local analytics to seem like it makes a lot of sense. And are there any other skills that you think are becoming more important with the move to cloud databases?

I do think understanding how to use gen AI is becoming more and more important. So the good news is you don't have to be an AI expert anymore, you don't know how to do training, model training and so on. So the pendulum has really swung from the data scientists and model trainers to the developers and what I call kind of the inferencers. And so some of the new skills that folks need to learn is how to do prompt engineering, how to do retrieval, augmented generation, really understanding the notion of vector search, vector indexing and vector search, how it works, there's always these trade offs between performance and recall. So I definitely think that the skill step folks need to pick up is how to leverage these foundation models. And also a lot of the folks who are actually learning those skills are the developers because they're the ones who are actually building these applications. So I think that's been really exciting. I also think that because this data now has to be low latency, has to be live, what you're seeing is, you know, it's not just about analyzing data in the data warehouse. We're seeing a lot of reverse ETL happening, meaning once we have insights, we actually move it back into the database, into the serving side, so that when a request comes in, we can make that data very quickly accessible to the foundation model. So I think the other piece, what we're seeing is that folks need to have a broader understanding of the data landscape they're in and actually learn how to use more than one system.

Yeah, it sounds like generative AI. On the one hand it's making things easier because you need to know slightly less machine learning, but on the other hand it's having big impacts on data engineering stuff. So you need new tools or new workflows like reverse etl. And then also you need to use more tools than you did before. Okay, so actually do you want to talk me through what reverse ETL is.

Yeah, yeah, absolutely. So, you know, in many cases you will bring a lot of disparate system data into your Data Warehouse, like BigQuery or into your data lake. Right. We Support both with BigQuery. And then you're going to run more batches jobs to really derive like the more deeper insights into your data. And that could be things like product recommendations, it could be a bunch of other capabilities. Once you get those insights, very often you want to actually move those insights back into the serving side to like a spanner or a bigtable for fast, low latency serving. And so as you think about ETL was how do you bring your operational data to the data warehouse? Reverse ETL is how do you bring those insights from the data warehouse back into the serving site so you can actually connect it to the application? And we're seeing a lot of that happening. And so we're also making investments to make that reverse ETL workflow much, much easier for customers.

Okay, that's interesting that you're then generating a feedback loop. So you brought everything into the data warehouse, you then crunch it and you get some insights and then you're like, okay, those insights are then going to feed back to do something else. All right, I'd like to go back to something we talked about in the start, actually. So you mentioned how vector databases are pretty much essential for a lot of generative AI applications. Can you stumble me through some specific use cases like where vector databases come in handy?

Yeah, absolutely. And you know, a lot of folks talk about retrieval, augmented generation, which is how do we assemble a prompt for a foundation model that kind of has the data that that foundation model needs? Because we can't just feed it the whole database. It's going to be too expensive, too slow. And most models then don't actually support that one method. And it's not the only one is to actually use vector indexing and then vector search to retrieve the documents or the rows or the pieces of data that are most relevant to the question. You'll often see that in kind of the text like interfaces or even in images, because doing that similarity search on vector embeddings is very efficient in that kind of model. So instead of bringing in a thousand data points into the model prompt, you can say, hey, what are the 10 closest ones, the 10 most relevant ones? And actually feed it. That is one, I would say, most popular way. But the other thing that we're seeing is just if you just think about search, just regular application search you're going to an online store, you're looking for a cup, right. And maybe you're looking for a red cup. And how do you find the most relevant? In the past this was mostly done with full text search. Now you could also use vector embeddings and vector search to also find, you know, similarities that are more semantic based versus just full text search based. What we're seeing is a lot of customers are actually blending those two. They're doing their traditional search query often with full text search, they're doing a vector embedding space search and then they're merging those two data sets and then doing the traditional ranking on what to show to customers. So this is another really interesting use case. Folks often don't talk about it too much because we're all focused on rag, but the reality is we're seeing a lot of customers in retail and other industries actually just using vector embeddings for much, much better and more accurate search results.

Okay, yeah. And I suppose at Google you have a bit of an interest in good search. So yeah, that's interesting. And certainly there are other shopping websites where you think, I'm sure I've searched for exactly what I want. This retrieve like useless results even more fun.

Not just what you want, like take a picture of it and search. Right. Like, I mean this is what vector embeddings do is like you can actually now just upload a picture and we can show you stuff that is similar. That is really what's so powerful with these vector embeddings. It's not only text based, it can also be image based or video based.

Okay, so I take a photo of like myself wearing a shirt and I'm like, okay, find me other shirts that you sell that are a bit like this.

And by the way, we've actually had that capability in Google search for quite a few years and I've known about it for a while. I've used it once in a while, not that often. I actually use it all the time now. Like I've gotten so used to searching with images as opposed to text. And by the way, you can also combine, you can like snap the image and add some text. Like it's worth trying out because those are the kind of experiences you can now deliver to your customers through your applications.

Okay, so that seems like it could affect like a lot of at least retail websites. Does it have applications beyond this? Like think about any other sort of corporate website. How can the search be improved there?

Yeah, you know, I think, you know, the Sky's the limit on what you could do, but it could also be used in like inventory management. And you know, I mean there's just a bunch of interesting use cases where using images as opposed to just text or other unstructured data can really help you find the information that you need. Think about just, you know, service, right? Like actually, I'll give you, I'll give you another example I got from a retailer which is inventory management on the floor, right? Actually being able to go and snap a picture off the shelf and it immediately tells you if you have any more in the inventory or not. Or you can actually place an order right there to actually replenish the inventory. So a lot of I'd say service level roles that are in the field that are not like in an office, right. Can really benefit from being able to take images and video and very quickly translate that to action.

Okay, yeah, that's certainly going to be faster than like typing in some like product key or something and trying to look, do a, look a bit, write a SQL query while you're in the field.

I mean the most amazing, the most amazing thing with this is you don't have to be like really a sophisticated developer to do this now. Like anyone can do it, right? That's what's so amazing with this technology is any developer pretty much can use a foundation model, give it an image in the prompt, get an embedding, do a vector search on their postgres database and they get some matches.

Okay, that's pretty cool that the workflow is like fairly simple. I mean, I guess it's not something you'll learn in five minutes, but like that, that's like an hour or two to build, I guess. So we got changes to websites, we've got changes to like operation stuff, like on the floor stuff. Are there any other areas where you think this is taking over?

I think pretty much every industry right now. When I think about our customers, whether it's in financial services or manufacturing or retail, you name it, transportation, I think we're seeing all of our customers really rethink what the future of their business looks like and how they can really leverage AI to be a lot more efficient to deliver better customer experiences. So the sky's the limit and I think really consumers are really going to see the benefit from that. Businesses are going to see the benefit and frankly, employees are going to see the benefit too. So far what we've seen is it's really enabling employees and also reduces some of the undifferentiated, heavy Lifting employees have often had to do and really enables folks to focus on the higher value activities. So I'm quite optimistic on how this has been supporting both employees and customers.

Okay, so this is pretty cool that like just because the cost of development's gone down, it's going a lot more accessible. These features are just easier to build, I guess than they were before and they become more economically viable.

Absolutely.

All right, so there's one other type of database which I'd like to talk about which we've not mentioned so far. This is the Graph database. Now I know Spanner has a graph database built in, so I guess to begin with, just talk us through what is a graph database.

Yeah, and you know, I'm super excited about what we've done in Spanner and I'm really excited about Graph. So think about graph databases as a database that really has a focus on not just capturing the data itself, but also how the data is interconnected. You can almost think about it from a relational perspective on like, you know, you have foreign keys and so on between data. But think about this makes it really easy for you to basically make the connection between friends between things people have purchased, connecting images or interactions that people have. So it's all about capturing that kind of interconnected and context that becomes more and more important as you think about Genai. Because Genai very often you want to exploit the connectivity between information and things like knowledge graphs. So you use graph and use cases like knowledge graphs, recommendation engines. So I can easily say like hey, what products did my friends buy that I haven't bought yet? That's a very easy query in a graph database since it can kind of very immediately do a recommendation use cases like anti money laundering where you're looking for rings connectivity within a graph to kind of look for bad actors it topology management when you're trying to kind of see like hey, what are the, what are all the instances and networks that have in my topology? And if one goes down, what's going to be the negative impact of that Security use cases. So Graph is being used in a lot of use cases. I think with Genai it's become even more important both because of things like knowledge graphs. By the way, Graph Rag has proven that additional context you can get from graphs actually improves the accuracy significantly in these RAG based applications.

Okay, actually tell me more about this because it sounds like combining two very exciting ideas at once. Talk me through Graph Rag.

Yeah, so you know, as we talked about before, Rag is really a way to feed the foundation model Right with the right prompt and the right data. What we're seeing with Graph Rag is because we now also have a better understanding of the connectivity of data, the kind of decisions we can make, and what data points are really interesting to customers goes up a notch. So it's really those relationships that help. Also, some customers build graph embeddings, meaning they actually kind of build more of these semantic relationships between their data points. And so that also helps them rank the data better. So just think about it as it's another tool in the toolbox to get to a higher level of context and a higher level of connectedness. Now, one thing we did really special in Spanner is it's not just about Graph. We brought Graph together with relational, with vector search and very advanced full text search. And so now customers have in a single database, a multimodal interoperable database, all the ingredients to truly create a lot of context. For example, you can do a vector search or a full text search, find the nodes in the graph that are most relevant, for example, what are the people that match a certain name or, you know, and then from there you can start traversing the graph to get more of that interconnected insight and get more of the relevant data. So it becomes a very interesting connect connection between kind of search and graph to really get to the highest level of insights and most relevant data points.

Okay, so this answer is going to be very useful if you have maybe complex questions to us. So I presume it's going to be fairly useful in like support chatbots and things, because it's got a lot more context around, like what the actual problem means.

Yeah, yeah, exactly. So the problem is very basic question and answering. If the query doesn't completely understand what you're asking, you can go wrong pretty quickly, I'd say in a graph you have a higher chance of getting to the right entry point in the graph and then traversing it and being able to infer what the user was actually trying to ask. Those are some of the second approximation benefits you can have. And what we've done in Spanner Graph is we've made it really easy to overlay this on your existing relational data. And we've integrated this with an industry standard called GQL into SQL. So you can really use your existing data sets that are relational, your existing skill set, which is SQL, to start getting the benefit of Graph without having to kind of adopt a whole new database, a whole new model. So we've really made it easier for customers to start getting this benefit.

Okay, Actually, I must confess I've been sort of interested in thinking I should probably learn how to do like all these sort of graph languages, like there's a graphql and all these things, but the syntax is just slightly weird enough that I've never got around to doing it. So what's the easiest way to go about learning how to work with graphs?

You know, without this sounding too self serving. In the Spanner Graph documentation, we have a quick start that kind of gives you copy and paste. Like here's how you create the schema, here are queries you can run. I actually did that myself because I always like before we launch a product, I really like to play around with it. And that was super easy. And then I was also able to change the queries a bit and just get a feeling for how traversals work and so on. I mean I had that skill from different graphics databases I've used in the past, but not from Spanner Graph specifically. So that's probably an easy way to do it, is just use it. Look at our quick start and it really gives you a copy and paste so you can just play around with it in the console. Super easy. And you'll get a feel for how these SQL embedded graph traversals work.

Learning by doing that, that's quite dear to our hearts at Data Gap. So I like that approach. Okay. And it does seem like maybe AI ought to be helpful with this because you can write your queries in natural language and hopefully it's going to be able to create graphs for you.

Absolutely. So, you know, there's a lot of investment we've made on natural language to SQL. That's not an easy problem to solve. I would say even the state of the art today is probably in the 55, 60% accuracy, but we're raising the bar consistently and we think we can probably be the best at that. And then while we haven't offered that natural language to SQL yet, in the Spanner Graph context, that's also something that's obviously top of mind for us to make it even easier for users. That is, I would say one area of innovation. The other one is natural language to query, which is, I do imagine that over time more users are just going to want to use natural language as a way to create a database, not as a way to generate SQL that they can then review and edit. That has its own problems because you need a much higher degree of accuracy. But that's also a really interesting space to watch in an area that we're innovating.

In So I guess rather than just generating SQL, it's then it's just executing the query and giving you results and maybe you inspect this, the SQL, maybe you don't.

Okay, very hard problem to solve because English language or any language is actually quite ambiguous. So it's not just about being able to do it, it's also about well, what if someone asks an ambiguous question like how can we then disambiguate that? Maybe ask a follow up question. Right. And so on and so forth. So it's a pretty hard problem but it's a really exciting problem to be working on.

So yeah, I guess what's the solution here? You just make sure that there's some kind of conversation going until both the user and the AI agree. Or how do you approach this?

There's lots of, I mean it's a combination of many approaches. Definitely one of them is, you know, also feedback loop and conversation but is also around like how do you organize the data, how constrained do you have it? You know, there's a lot of elements that go into this but it is definitely a difficult problem but one that we believe is worth solving. So we continue to kind of work in that direction.

So have you seen any success stories around like the great use of data in AI?

Absolutely. I mean all the time we're working with customers. We have a few thousand customers right now using things like PGvector on Postgres building AI applications. Actually some of the use cases I mentioned we're seeing chatbots. So for example, Rignology in the regulatory compliance space built a chatbot both for their customers and their employees to be able to ask regulatory questions and that then queries their data. We have retailers who've been using it for things like search, as I mentioned, like storefront search. So we're seeing a lot of really interesting and exciting use cases on how our customers are using gen AI with operational databases to deliver better outcomes both for their customers or internally for employee productivity.

Are there any specific customer stories you're asked to talk about?

Well, rignology is one of them. In the regular regulatory compliance space, you know, we have other customers like Bear who are in the, you know, in the pharmaceutical space. We're also using AlloyDB, you know, in their context for AI applications. I'd have to think about the public references we have. I can't talk about customers that I'm, I don't 100% remember if they're public. But we definitely have customers across a variety of industries who are using kind of vector capabilities, whether that's in AlloyDB, in MySQL, in lots of our engines.

And I guess the flip side is, are there any common mistakes that you think organizations make when they're trying to make better use of data with their AI applications?

One thing that I've seen is in the past because a lot of the AI was kind of driven from the training side and data science side. You saw a lot of that accumulate like in the data science side of the house. And you even have folks are kind of the data and AI leaders in these organizations who look at analytics and AI as a way to deliver AI to the organization. I think what we're seeing though is because the pendulum is swung so strongly from, you know, having these off the shelf foundation models where developers are really in the driver's seat to exploit those. Some organizations haven't quite figured out that shift yet where that's happening. And so I do see kind of this interesting situation where you have these two different Personas and roles in the enterprise and they're not necessarily talking that much even though they both have a shared interest in making AI successful. So I do think there is an organizational element here that customers are discovering quite quickly, which is how do they really think more strategically about both their data state and then how every all the Personas in the enterprise can take advantage of that. Not just the traditional analysts, but actually the developers. That's where we have quite a strong advantage because of how we've integrated both operational analytical systems into a single data cloud that is very easy to exploit with AI. That's really an area that we've focused on, on help break down those silos within our customer base.

All right, so the big problem is then that normally it's just like it's one or two teams that really sort of get to take advantage of these things and actually want everyone in the organization to be able to take advantage.

Exactly. And you know the other thing that of course everyone struggles with, which is what's the balance between accuracy and cost and latency? People often ask me like, hey, what's the best model? I'm like, well, it depends that it depends as hard. It really depends on your use case. It depends on kind of the latency tolerance your use case has, the cost elements, the accuracy needs. So I think that's another difficult problem for organizations. We're of course focused on really helping organizations figure that out, but that is a non trivial problem for everyone.

Do you have any examples of when you might care about accuracy or when you might care about cost or when you might care about latency the most.

Yeah. So for example, you know, if you're looking to really process an important document and generate insights from the document, you may actually want to use a more sophisticated, bigger model that may be a bit higher cost and a bit higher latency because it's not in the serving path. So that's an example where probably that additional cost would help. But if you're for example, doing a chatbot and inferring what someone needs doesn't have to be as sophisticated, then probably a smaller model that can respond much faster and be cheaper is more effective. Right. You don't want the customer to have to wait for eight seconds to get a response. You'd really want them to get like a response within a second or two. So those are maybe two examples, like document understanding. You would use a much bigger model, probably more expensive, higher latency model, chatbot interface. You want to optimize for latency and cost.

Yeah, I can imagine a chatbot where you're just waiting a very long time for every response. That's going to be a terrible user experience. Like you want the AI to be faster than a human can type, I suppose.

And then of course also vector embeddings, which we mentioned before. You know, generating vector embeddings, let's say you have a terabyte of data, could take an awful long time. So even there you want to make sure that you know, your embeddings model is relatively low cost and low latency as long as it gets the job done. So that's always the. But it's got to be good enough.

Okay. Yeah, make sure that's good enough. And I guess focus on the speed after that.

Yeah.

Okay, so before we wrap up, what are you most excited about in the world of databases?

You know, this is one area that I'm really excited about. I think just operational data is so key to really getting the benefits of AI and foundation models that I think that how we can innovate to make this super easy for customers I think is really, really exciting. So I think that is one element that I'm just super excited about. I think the other one is how we start bringing these modalities together. So we, for example, talk about spanner, like how do we start bringing kind of relational and graph and vector search together to make it really easy for customers to build those kind of very high value context. I do think graph is at a bit of a tipping point. You know, it's been one of those technologies that like never really took off. I think the need of this additional context has become so profound that we're really going to see this explode. And so I think that area, even though you people will say, well, we've had Graph for many years and we've had some multimodal databases for many years, I think now is the time. And I think we've taken a very unique approach to really try and make a big impact on behalf of customers.

That's interesting that maybe Graph's going to be the next big thing. And that's kind of a cool thought. So finally, do you have any advice for organizations wanting to up their database game?

One thing I tell folks is the technology landscape is moving so fast. The business is moving so fast. You've got to transform your business. Don't waste your time integrating the different pieces and managing your own, like, infrastructure. Right. I think one thing I'm hearing loud and clear from customers is we need more bandwidth to actually make business impact. So this is where I really advise, like where it makes sense, which I think is most of the time, like, use managed services, enable us to do the undifferentiated heavy lifting for you so you can really focus your time on innovating and delivering business outcomes versus dealing with the plumbing. I think that is really one piece. The second one is really thinking about gen AI and the possibilities it opens up. It shouldn't be one team's problem. It should be in the DNA across the organization. And by the way, it's not always the right fit. Right. Some applications don't need genai, but I think every application team should be empowered to take advantage of AI in the cases where it can make a difference. So kind of really making sure that that access is democratized, the costs are clear, developers know how they can engage and use. It is important to make sure transformation happens across the organization and not just in a single central organization that doesn't necessarily really have the influence on all the different application areas within the organization. All right.

I do like the idea of just making sure that every single team gets to take advantage of these things, like gets to take advantage of the data. All right, super. Thank you so much for your time, Andy.

Yeah, well, thank you so much for having me. This was really fun and really appreciate it.