#263 The Data to AI Journey with Gerrit Kazmaier, VP and GM of Data Analytics at Google Cloud

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

I think the key to being successful with data is understanding the essence, the very essence of data driven value generation, which is very simply put, the more data signals you can bring together, the more you find hidden or latent unintuitive patterns in your data. The more you can build services or models using that data, the more customer value or customer services you can create which generate more data for you, for which you gather more data signals, allowing you to build better services and models generating more data. That's the flywheel of data, right?

Welcome to Data Framed. This is Ritchie Perhaps the two most important tools for data analysts are the database and the BI platform. I want to find out how both of these things are evolving. No surprise, the big trend right now is shoving generative AI into as many places as it will fit. But the more interesting part is figuring out the details. So, for example, how will AI benefit analysts? Do the tools work well yet and how can you take advantage of them? There's also a bigger question looming. If AI chat interfaces are getting good enough to do data analysis, does this mean the death of the dashboard? I have a guest perfectly placed to answer these questions. Garrett Kazmier is the Vice President and General Manager for Data analytics at Google Cloud. He leads the development and design of Looker, BigQuery plus a host of other databases, data warehouses and analytics tools on Google Cloud platform. He's got extensive experience managing analytics and data engineering products and he's got a mission to build a unified data platform for all types of data processing. And previously he spent over a decade at SAP where he worked his way up from developer to Executive Vice president. So let's find out how AI is changing databases and BI tooling. Hi Garrett, thank you for joining me on the show.

Hey Richie, great to be here.

Excellent. So, just to begin with, I want to talk about why you might want to use AI in data tools. So why might you want any AI features inside your data tools?

There's a long list of reasons actually. You know, one is as much as AI has helps in many other functions, it's a real productivity gain. Code generation encoding is very helpful for writing the many lines of boilerplate code, remembering all the syntax or design patterns that you might not have handy right now. And you know, in the data space that equally applies. You know, there are data professionals writing a ton of code in SQL and Python, you name it in what framework, and having the ability to get high quality generation of larger pieces of code is immensely useful. And I think that's one big category, right? And we can translate that actually to many tasks in the data space. It's always the same idea though, right? You make people more productive, slash, help them do their jobs better. And that's kind of the idea of technology all the way along, right? Technology is meant to facilitate a certain problem solving a certain task. And AI is, if you will, one way of doing that. Right. If that's data governance, data exploration, whatever it may be. On the data side though, there is also another angle to it which I think is the more interesting one and that's, as it turns out, humans are really bad in reasoning about data. So on the one side there is great books like thinking fast and thinking slow about all of the biases that each of us carry. That kind of makes us want to look for the data we want to find rather for looking across all of the patterns. That's one piece, but the other piece, piece is also a. There's a genuine limitation in our ability to understand complex pattern and reasoning about that. If you doubt me, right, you know, then I don't know, you know, why is Chess Hub, right? Why is it hard to multiply four digit numbers? Right? Because our brains is not made for that. And this is where AI technology is massively helpful because it kind of takes all of the best practices of good data analysis, right? How do I reason about data, how do I look for data, how do I present that and makes that broadly accessible. So you know, it's on the one side for professionals, a great productivity gain, great. But on the other side, it genuinely helps us working better with data and using it for all sorts of processes, decisions, products, ideas to make them truly data driven versus data validated.

Okay, I'll add that. So it's a mix of doing technical things like getting the AI to help you write code, but also the maybe slightly less technical stuff and more on the interpretation side of things, making sure that you're not having any psychological biases in your analyses. So Are these tools, are they purely for data teams or does it go beyond that? Is there anyone else that's going to benefit from AI and data tools?

Yeah, and it needs to go beyond that. And a tool always needs to make sense for the person using it, right? So that's, I think that's really important to keep in mind. And the tools for data professionals, they are different for the like tools for layman, right? Just take a Formula one steering wheel. Well, I couldn't use it, but I can drive my car every day. So tools need to fit in the context, in the mental model, in the skill level of the person using it. And it means that with AI we're also going to have a spectrum, right? We're going to have on the one side table, technical tools for technical oriented people. If you think about the whole idea of code generation is really built on the idea that someone understands the output. By definition, this means that this family of tools is made for a highly skilled technical expert. And on the other side of the spectrum we have people who don't know code, who don't know SQL, who don't know data structures. And in today's world, basically they have two real options, right? One is they ask someone who is, and that's the usual cycle of people in a business function asking a data skilled person to do something for them and they explain the problem and that person, you tries to understand the problem and then tries to do the analysis and then gives them something back. That's one way of doing it and the other way of doing it is not doing it at all. The interesting piece though is that a very good way of thinking about generative AI is actually how can it be used to scale scarce resources? And data professionals are scarce, they're not in abundance. And what it allows us is what I just described, that when you have a question that instead of calling one of those few data professionals, you can actually talk to a data agent who is doing the same steps, understanding what you're looking for, doing the analysis and giving it back to you in an understandable form. And this is only built on the idea, right, that you are not leaving your space of competency, right? You're not required to switch to a technical domain, you're not required to see or write code. You're interacting with the AI system like you would interact with an analyst. And so that means that you will have a broad spectrum of AI and data depending on what the person is, what their context is and what their level of expertise is.

I like the two Options. And I think for many people the idea of just not working with data at all, it may be appealing, but it's not going to have a great benefit. So back to that first idea of maybe commercial team members interacting with data teams or it's like a pain because it's slow and there's never enough data people around to answer all your questions. You're suggesting that data people can be replaced by AI. I think this is maybe going to terrify a lot of our audience. The idea that your data scientists are going to be taken over by AI, is that what you're intending?

No, I don't think so. And I actually think this is not about replacement because that idea is wholly, I think, built on the idea that basically today we have one way of doing it and now it's a better way of doing it. So we're going to replace the one way with the other. I actually think where Gove, you and I started from is the reality that most people don't have access to data and insights. So there is a lot of if you feel unmet demand. And what AI creates now for data analysts and data specialists is a whole new way of how they can enable that demand. Because in the past organizations have tried so many ways, built up centers of excellence, build so called BI people and business teams, but it always was basically built on that idea, build more skill. And now what data teams can do, they can actually say, I'm going to build an AI agent for this, right? Because someone basically still needs to define, and also in an enterprise context basically stand for the correctness of that, right? So defining what data to be used, defining what questions should and could be asked, defining, you know, the semantics of that. Basically all of that context setting that's going to be done by data teams. You know, we cannot think of that like full autopilot, you know, that is way too reductionist. This is, you know, like a kid would imagine it, right? Suddenly we have this robot coming in and doing all of these things. Actually the reality is much more sophisticated, right? We're going to have data professionals which today are cranking out dashboards which today are having like this long request line of people wanting more basically being able to create these agent based analysis basis which then people can freely interact in and do a whole bunch of things. They would have originally come back to them with to ask them that and have the possibility to focus then on this other category of data problems that you and I talked about, right? The ones that cannot be described in a system like this. That require data professional and in the big picture actually give way more people access. You know, there is way more demand than what we are fulfilling today. I think it's an amplification.

Okay, I like that. So it's basically churning out the stuff that the data scientists would never quite get round to in the priority list or maybe reducing the scope of work for them so they can do things more effectively. Now you mentioned that a lot of data scientists spend way too much time cranking out dashboards. So I'd love to get into how the world of dashboarding is changing. So yeah, how does like powerful generative AI change what you do with the dashboard?

Dashboards are great and at the same time they're horrible. So for one thing, dashboards used to be the best way, one of the best ways to surface data information, right? And there is the whole, there's like a whole, I think science around how to visualize data, right? Even Hugh, if I remember correctly, and so many others who worked in that space. And how do we make data human, comprehensible and we have tools that then are smart in how they can compose widgets and all of that. But you know, reality is again, if you then look into, into real enterprises, they have thousands if not tens of thousands of dashboards which are unused or hardly ever used. Many of them are one time explorations, one time questions that just got stale. When you look into real dashboards, they are quite complex. It becomes a complex system in itself and then, you know, you are largely contingent on the quality of the data person of the day to catch and to act on interesting signals or not. So that's already, I think telling us how problematic or how much improvement potential there is still to have. Right? Because what you would rather want is like I said earlier, right, you should not create a dashboard for one time exploration. There should be a way of how you could do this in a modality that actually people work in. Earlier I said tools need to fit to the workflow someone is in. Even my job is not checking dashboards, right? If you're on email, if you're on chat, if you're on any application, information should find its way there. Rather requiring you to kicking out to a different tool and then trying to translate it back into your context. So that's one limitation that dashboards have today that will be changed with bi, we will see fewer dashboards which have more standardized views, if you will, and all of this free from discovery. One time exploration will be driven through new modalities, you know, like conversational interfaces and it will be deeply embedded into the work environment that people are actually in. And the other part of dashboards, which incredibly is limiting, and dashboards try to be smart about it because they allow you to do drill downs and filtering and all of that sort of things. But the problem is you only see what you see. So that has two inherent challenges, right? What if the interesting stuff is hidden in big aggregates in other dimensions, in something which is not visible to you? Right. So what do you want to do? Do you want to create like a hundred charts? That's impractical. No one can consume that, no one can hold that state. So actually you do need an assistant which is able to look behind the charts, actually tell you which are the interesting ones to look at, which are the micro signals that you should be acting on. And maybe thirdly, I do think it's really interesting as well, is that dashboards are really made for humans to comprehend data. Right? That's kind of the idea, right? Visual data discovery. Big question though, is, is this really the end of data driven insights? Because if you are thinking about a system which is doing something in real time, basically adjusting as data signal changes, acting on data streams, triggering an action in a real business interaction. For instance, Right? For instance, when you call a call center. Right. And did you talk about the service issues that you are experiencing? Right. For the service agent, basically surfacing all of the information about your history, the product, possible ways of resolving that. Right. If you want to do something like that, dashboards are not the way to go. Right. You need to have something which is acting on way smaller signals at a way higher frequency and then basically becoming a proactive provider of insights rather than a reactive display of data. Those three areas tell us that dashboards are great, they got us this far. But I think, you know, the true promise of data is still way more way bigger and will be driven from other modalities.

This is fascinating. It seems like all three of those things you mentioned. So the first one was about having integration with the existing tools you're working with. The second one I think was just having insights pulled out automatically rather than you having to go and point and click and filter things or whatever. And the third one is really around like having real time insights. So they're all about just you not having to look for data, but having it, the insights pushed to you. So I'd love to talk about this further. I guess let's take the one about integration with existing tools. How do you envisage a dashboard being integrated With I guess what, like email or Slack or whatever your office tools are, how is that going to work?

Yeah, your question is already interesting, right? Because the question is how do you integrate a dashboard? And probably the answer is, well, you won't, right? Why would you? The answer to that question is what if the data agent that I spoke about earlier, right, what if they have an API so they can be reached by a Slack or Google Chat and that you can basically in the context of a group chat that you're having, or in the email chain that you're having, and then you are basically either looking for data, wondering about data, or like we said, basically getting supported with additional data through this agent in the medium that you're working in. Right. Why wouldn't it be just a chat message? It just take my professional work experience on operating services product performance. It's much more natural for me actually to be in the meeting that you're having, in the chat group that you're having and saying, hey, what did this actually look like last month? Or how is this actually trending then actually have then a specific factual answer for the given circumstance versus okay, here is a link to a dashboard. Click the link, okay. Find all of the right filters. Okay. Setting all of the filters. That takes already way too long for me to actually do this unless it's really, really important. Right. And it makes it worth all of that effort. And so when I say meeting where the meeting the people that they are is bringing data and insights through new interfaces, new APIs. These APIs that are able to take questions, hold context, generate visualizations and provide them back in the work medium that you are actually using.

Okay, that sounds pretty cool. I like the idea. It's like, well, I suppose of course the answer to how do you integrate stuff is going to be APIs. But that's interesting that you're almost doing away with the dashboard. Then it's just like these are specific insights that you asked for and then that's going to be put in the place where you're working.

Yeah. And even some standardized dashboards that have like information which is required to be frequently looked at. There is a whole space of regulation and compliance. There is a whole space of enterprise standardization. Right. In businesses, like in operating a service, you want it. Everyone is up to date on a certain set of KPIs every day. But the majority of dashboards today out there, they're not like that. These are these one time discoveries, few time discoveries project. I had a question and that creates that proliferation of that like thousands, tens of thousands of node DI dashboards that are sitting stale. And I think the big opportunity space is actually transforming them into something which is actually a part of someone's workflow and also doing away with dashboard as the vehicle of delivering it.

I can certainly see how dashboards is something useful. If something like, hey, this is what our current company cash flow looks like. Management is going to want to look at that, like most days, I think. But then if it's more for exploratory data analysis then, or for a simple one off analysis, then dashboards are going to disappear for that. You think?

Yeah. The interesting part, Richie, is even when you see information, what does your mind usually do? Why? Why is that? Right? And why is it like this and why is it not like that? And then you have these questions, you know, why, why, why? Or you know, what if, or how would it look like in that configuration? Like the cash flow, right? Why is it trending like that? What has changed? And the opportunity is that you actually literally, you just type, why is it like that? What has changed? And you get an answer. This is what's different, this is what has changed. It sounds so natural, right? You know, this is why we have Skjano. We watched Knight Rider and you know, all of these shows or Star Trek right there, people always had that intuition. It would be so great if I could just talk to a computer. It would understand what I want. It has all the right context and it just can help me with progressing with my task. Right. Because they're not an end in itself, they're means to an end. And the best way of how we could facilitate that so far was dashboards and long change requests with analysts. And now AI is giving us this whole new capability set right. Of we actually can know people. You just write in chat, you know, why, why, why, why? And you're gonna get all of these insights that are lurking in your data that were unearthed before.

Okay, yeah. And I mean, Knight Rider was around in the 80s. Like, why don't we have Kit yet?

Maybe we will, you know, Maybe.

Yeah, maybe we will.

Okay.

So I'm curious as to these changes having an effect on I think. Well, I mean, obviously at Google you have looker. So is looker changing to accommodate sort of new ways of thinking about dashboards?

Yes, that's the main area of our work actually. So basically our work in Looker exactly falls into the same two categories where we started. On the one side, we're introducing Genai and Looker to help the data professional working In Looker, we help them with creating content, writing LookML code, helping to reasoning about the system, helping them to generate slides and all of the typical tasks or jobs that data professionals have on a daily basis. And we think about ways, how can we use now AI as another way of making that task better? No different from how technology develops. It's all about making it simpler, making it faster, making it more accessible so people can get their jobs done. And on the other side, on the looker side, which is the part that I was speaking about last, is how can we actually make it so that Looker allows you to basically talk to your enterprise data? A good test question is in any audience, how many of them use Google? And usually everyone does. So everyone knows how to use it, how to use a search engine. That's what I mean. It's so natural that from children to grandparents, no issue in Google, every person I know in a professional context knows how to use a search engine. When you ask the same question about a BI tool, you get a very different answer, right? This is very hard. And oh my God, I have to understand data. And I remember distinctly early in my career we did a survey about why people don't use more data. And one of the biggest points was they're genuinely insecure. It seems so technical. This is hard stuff. Do I get it wrong? There are so many hurdles in the way. And so what if we can make the eye as easy as searching a search engine? Why wouldn't it be right? Why shouldn't it be? And the interesting piece is this is technically very, very hard. So this is, it sounds so simple, but it's incredibly tough problem to solve for. And our approach, Google's approach, is that we take a set of steps to solve that problem. Because the key with BI is once you give someone that tool, it needs to be 100% correct. There is no space for, hey, I might get it right 80% of the time, because how could you trust in that to take any sort of decision? And the whole point of data is that you help people reasoning and come to conclusions and come to decisions. So there is a whole space of building specialized AI models who are specializing at the task. There's a whole space of understanding a data system and how it works and basically building a knowledge graph of data to ground it in the factuality of an enterprise context. And then there is, because you asked for Looker specifically, there is like the last element is giving data strong semantics way richer than the semantics that you would have, like in a database schema it's way more sophisticated than that. And it's actually semantics for an AI model to reason with and then exposing them this semantic model with the right set of APIs. So multiple agents can now start and to operate on that. And those are the two threads we have in Looker, right? So one is help the data professionals in their jobs and giving them this new way of exposing data to virtually anyone in their companies to freely explore it by using their own language, their native tools and stay in their broad flows.

Okay, so that's interesting with the two different audiences then because like maybe you're a data analyst, you just want to build your dashboards faster, at least for the near future. And then for everyone else in the organization, you want to be able to understand the data and sort of consume more of it more easily. So I guess the second one is supposed to the bigger audience then. And you mentioned the idea that a lot of people just feel insecure about working with data, about understanding with it. Can you have like a purely technical solution that to that or there are other things that organizations need to do to help people feel more confident with.

Data, as always, right? The cultural change always is bigger than the technology change. There's one thing that helps us. Like we joked about Kid and Star Trek, right? I think there is a human intuition that this is how it should work. So everyone knows that idea and it comes natural. So I think in general people already am prepared for it to interact with a system like this. And the second piece is that I think it's always the same, right? Technology develops from the primitive over the complex to the simple. And I think that's like Antoine de Sant Dixupuri, I think, who said that originally. I think this is exactly what we see today, right? Data used to be primitive, it used to be expert systems handled by a few people and very limited. We are now in the complexity stage where you have like a million tools, dashboards. That was the big contradiction of self service bi everyone should self serve. But that meant everyone should be really becoming a data expert here, which of course is not viable. Now with generative AI we approach the simple stage where actually people can experience it and through the experience with it, they will see the benefits and use it. And Apple develop that fast adoption cycle that we have seen also with Genai all along that basically through experience and through recognizing the value chain generates there is a big problem that people want to solve. And if basically they can experience the solution. I don't think there is any other barrier than just if you will the ability for organizations now to shift their focus on how they think bi, how they think data and the role of generative AI in it.

It's a very sort of compelling vision. But as you mentioned, the idea of self service analytics self service BI has been around for a long time. We still not quite get there. So do you have any sense of like how you would go about trying to get closer to self service BI across your organization?

Yes. In looker we are providing these new agentic experiences and APIs. And their whole point is actually that now a data professional, they can build a data agent to basically in their step handle a task for a set of problems. Even my experience, right. I'm fortunate. Right. We have an analyst team that I can ask questions. But I ask a lot of questions because I'm very curious. Right. I always want to know more. So what they have done is basically they built that agent for me and now their part was is making sure it does the job. It answers the question what they want me or wanted to answer. They gave it all of the guardrails and now I can interact with the system. And the great thing is I can do it all day long. I can ask a million times, why not Ashamed I didn't get that can like that maybe can you show it to me like that? And that's such a big unlock. Like you said, the job of getting insights is as old as the first decision systems. Right. Maybe as old as business or maybe as old as humankind. I don't know. The desire to reason better, to understand more that persists. And I think now the opportunity is that actually now many people can that beforehand didn't have an analyst or didn't want to send them like a million but you know, like 10 questions, you know, rapid fire one after the other. And organizations who also you know had to basically not the ability to invest in a data function like that before and just a few analysts, right. Which really want to solve complex problems are basically drowning in business requests to support dashboard. And this is the demand that I think is a exists that is unfulfilled by self serve bi. And we now have a technical solution that actually can meet the job that people actually want to do in a much better way.

Okay. Yeah. So just cutting out that sort of middleman of the data for at least some of the questions, maybe the easier questions and then you can gradually build out that sort of broader BI self service capability.

Okay.

As well as talking about dashboards and bi, I'd also like to talk about the more technical side of things. So particularly with databases, because that's obviously just a huge part of any data function within any organization. And SQL code generation is becoming popular in a lot of databases. So can you tell me how this works within the Google databases? And I know you've got a lot of databases.

SQL code generation is extraordinarily hard. And one reason is that we're not just generating code, you're also mapping to a schema, right? You know, so there is like two pieces to the puzzle, if you will. And schemas are usually proprietary, they don't exist in the training data of large language models. So there are some really interesting challenges in the generation of SQL code itself. Also, data systems tend to be very big, right? So when we talk about context, it's much more complex than the BI problem, if you will. But on the other hand, there's also a big opportunity and this is once you're talking in the database context, you're actually working with a data professional who understands code. Basically, Persona does change in the BI part. The problem challenge really is finding the right modality for a business user in the space that we are now in. We have a technical savvy person who can understand the output of CO generation, for instance. And the challenge much more becomes in how do I actually make this productive and useful in a, like in a bigquery, tens of thousands of tables in a single instance or single instance, single customer, how do I actually make it work in that context? And the answer to this is there is a whole new category of how we build data systems. For instance, one part that we all know what makes generative AI models good is having the right context. So one word I mentioned earlier was it in the knowledge graph of data, right? You actually have to understand what does the data landscape looks like, what is a person looking at, what is their permission, what are other people with their permission looking at, what has been looked at by people like them in the same period of time, what has been used recently, what has changed recently, right? So basically you suddenly need to think about your metadata system much more differently and you basically consider it all. Your metadata becomes one huge search index that you're basically clearing for context all the time. When a data professional is generating code that's actually there are a whole lot of the art actually resides in. It's also, you know, why when you look at all of these big benchmarks like BERT SQL or many public benchmarks of generating SQL code, you basically see it tapping out like 60, 70% of accuracy something like that, right? And the key is on a big part of the key is actually setting the right context without someone explicitly specifying the context just by the use and the metadata and the knowledge of the system that you have. So that's one big bucket. And the other big bucket is, I said earlier, it's in a reductionist to think everything is on autopilot. It's also reductionistic to think there is just one model instance doing all of the jobs. Right? So you know what we are working on on the Google side is actually building task specific models who understand domains really well. And within that domain you actually know, right, what is the relevant context that you're working with. The model, for instance, for building data pipelines, wrangling data is different from analyzing data. And the model for basically doing drill downs and finding for interesting subqueries is yet another model again, right? So there's also a whole part of actually engineering the system. So you have a multitude of models who are then plugging into the data scientist or data engineer or secret analyst tool chain. And thirdly, it's also about changing the user experience. So what today most data systems look like is one SQL console, right? So that's probably what all people have grown up with. What I grew up with is left side some sort of data set explorer or schema explorer or right side one bright or black window for writing SQL code or maybe an Emacs or vi. But same idea that is inherently difficult because on the one side all of the information about context and state is in a different pane. And then where we write our code, we have basically nothing that now a Genai model can work with or reason about than just the code we write or generate. So one of the big changes that we are introducing on our site is actually thinking through how would a user experience look like if it would be designed AI first. And one of the experiences we, I think we've just created just last week actually is data canvas. And Data Canvas is now basically saying that, well, if you have an AI agent, you would actually start the flow and the creation process different, right? You would more work in data frames and you would actually not only work in data frames, you would work in association of data frames. And if you do that, the model understands the structure and the relationship of your train of thought. It's not just reliant on the schema information that you have broadly available in the database catalog. And you would not interface with it just by a jet side panel. You would basically have it continuously working with you Basically in every single data frame that you build and start to give you more recommendations, more generations based on the graph that you're building up yourself, essentially, right? So in summary, those three things, right? One is understanding a knowledge graph of data and building it. The second one is that understanding of having a family of models that are highly specialized to basically plug into the data professional tasks. And thirdly, rethinking also the user interfaces and the experiences the data professionals work in to make them design for high quality AI generation. Because you can help AI. We all know this, right? You know, it's so funny, right? Everything is in the prompt, right? So like in the good old days, the secret is asking the right question. You're a podcaster, you know that. And it turns out that we can of course make it the user's problem of writing the perfect prompt, right? We use this table and then do that and then, you know, put it from here and you know, basically make the prompt like an essay. Or we actually build experiences that build up the prompt continuously and retrieve the right context continuously as you're working and basically make it part of the tool and that makes the data professional AI experience really work at the end.

Okay, I have to say that last point sounds especially useful because certainly, at least when these sort of chat tools first came out, it was like, okay, you generate me some SQL and you're basically writing almost exactly what the SQL should be like, well, take these particular columns from this table. And it's like, well, you've not actually done much, I might as well just write the SQL at that point. So, yeah, having the automation done so you can really just stick to simpler language and not have to write an essay in your prompt does sound incredibly useful. I'm curious because you mentioned context is the important thing to getting this right. Do organizations need to provide some of this context? Like what do you need to do so the AI understands about your data?

You're cutting, I think, to the core of, maybe not used to the core of the opportunity rather. And I actually think you can't think of your AI system and your data system differently because what you have just described, Richie, I think your experience with using SQL code generation basically shows the key constraint of a large language model is it's probabilistic, there is no factual knowledge. It basically operates on token by token probabilities and then it converges, it regresses, it's autoregressive towards a final output. So if it doesn't know your schema, right, you know, if there is no Probability of using the right columns, because it didn't exist in the training set, it won't know if you're using syntax or stored procedure functions or anything like that, right? Or libraries, not in SQL, but in Python. It won't know that. So a big part is actually understanding that when we build these AI capabilities, they are closely intertwined with the data system. For instance, they need to have an ability to fact check themselves against your data schema. And that's more than a prompt, right? Because the prompt basically says, I would like you to do this. Then the model basically probabilistically does something. And then at the end, actually you do have to go through and saying, you know, are these the right columns? If not, which are the ones that actually would be the right ones? Is this the right function? Is there maybe a function that you know would do this better? So basically you start thinking in what I called earlier, this ensemble of agents, and they all need to work together to actually get to high quality SQL generation. Because at the end, like you said, you wanted to just get it, get the right data, get the right functions, get the right syntax. So it actually does help you. So that is a huge part of when I say context, it's actually not just generating the right prompt, but thinking of AI as a system that closely interfaces with your data system. And I think that's one of the key issues actually, that right now everyone tries to aify everything, obviously, and you see that people are trying to stitch together, right? I take my AI model from here and I take my data system from there, and then I just build a prompt and it's all going to work. And the answer is no, it won't, because the reality is much more complex than that. So a big part is understanding AI and data as part of a platform working together. And the more you can specialize this together, the higher quality the output will be. Now, the problem is, of course, if you are a vendor selling data warehouses and you are not having any roots in AI, you see playing out in market, right? You acquire, you hire, you try to build that gap. But you know, it's going to take you, what, years, five years, seven years? It's just, you know, data thinking, right? So you want to choose a platform which is data and AI first.

Okay, yeah, that's really interesting because I think a lot of people think, well, AI is getting smarter, we're going to have like one super powerful AI that does everything. But actually the reality is you end up with having lots of smaller AIs working together. You mentioned the term ensemble. So it's lots of different smaller AIs playing together. So the SQL example is like one thing sort of figuring out like how do I write the SQL? And other ones like checking, does this match the schema? And things like that. So it's more narrowly focused.

Tricky territory because when you look at any of the large benchmarks, large models beat small models, even specialized ones. So the professor model, if you will. Right. Gemini, the large models, they tend to do well across all sorts of tasks. Even better in tasks than smaller specialized models. So when I say a sample of models, I don't necessarily mean not using big models and many small models. Instead, what I mean is when we think about a model acting like an agent, basically we want to give them focus on a specific task. And this is the craft of AI, if you will. For instance, SQL generation works better if you ask the model twice. If you ask it once, what is the logical plan? Think about a logical plan. Once you have the logical plan, translate it into SQL. Basically we ask the same model to play two roles. One is do the abstract reasoning and then do the CO translation. And I think it's natural divide and conquer. That principle in computer science. Decompose a problem into smaller problems. And that's what we are seeing here as well. We are basically building an AI system. There are multiple agents collaborate with each other in this ensemble. But I think still it requires. It's an ensemble of large models. Right. We are not at a state where we have any evidence that super small, highly specialized models can outperform large models.

Okay, so it's more task deconstruction than necessarily like picking lots of different model, lots of different types of AI things.

And maybe fine tune them specifically to a task and align them to a specific task. But yeah, I certainly don't believe that you can just distill a bunch of small models and can expect the same performance as you get from large powerful models like the Gemini model family. And we are specializing in using a Gemini model family specifically and basically building it ensemble with it.

So you mentioned that you shouldn't think of what you're doing with your data platform and your AI platform separately. Everything needs to be integrated. But I think the reality is many businesses at this point they're using dozens, maybe hundreds of different like data and AI tools. And there's more and more tools come out all the time. Is there a secret to getting all the different tools you're working with play nicely together?

I actually think having many tools is fine. A practical example is in BigQuery, people use a large set of tools against it in one customer even, right? And it works because they're all sharing the same data. The tool is more of a form factor of how does someone want to consume it. Doesn't change the meaning, semantics, any of that, right? It doesn't, if you will. Basically, that proliferation is basically making it more consumable. It's not really a problem by itself. It may be a cost concern and people would like to just buy one license eventually, but it is not. I think the big issue of standardization. I think the standardization issue comes in when you use multiple platforms. I think there is a big distinction between platform and tool. The same basically applies with models, right? Is there an issue with using many models? No, definitely not. Does it mean you should use multiple platforms? Probably not, right? Not a great idea. One of the reasons why in Vertex AI we have that model garden which gives you access to over 100 model plus hugging faces. Because depending on the use case, depending on the scenario, you will need to choose, right? And you have different cost considerations, latency considerations, whatever it may be, right. But the platform aspects of governing, building lifecycle, deploying models, this is basically what a platform guarantees, the data space. It would be security, correctness, semantics, lineage, all of these aspects to also be part of one governed platform in the data space. There was this terrific phase when we talked about data meshes and you know, everyone loved that idea of data mesh, right? Because again, it sounds so promising. And what though was a bit unfortunate. Many people then basically took all of the issues which they had, all of their many data platforms which duplicated data, inconsistent governance and so forth, and just said it's great, we just call it a mesh. We are done here, right? We just modernized our data architecture. But Data Mesh specifically was specific about one point. That shared infrastructure component on which all of these data products are then being built. And what I say platform is that shared data foundation. And I think the key to being successful with data is understanding the essence, the very essence of data driven value generation, which is very simply put, the more data signals you can bring together, the more you find hidden or latent unintuitive patterns in your data. The more you can build services or models using that data, the more customer value or customer services you can create, which generate more data for you, for which you gather more data signals, allowing you to build better services and models. Generating more data, that's a flywheel of data, right? That's the basic idea and that is built on the premise that you can build these not big, but very wide data records bringing many data signals together. Because this is actually where the interesting patterns that you want to leverage in data are being exposed. They're not in the data slices in silos that you have in an application, for instance in business system, right? You want to know your HR data and your service performance coming together, right? Or you want that your supply chain and your front order deliveries coming together, right? You want all of this together and that tells you that by definition you need to have the shared data foundation. This is the very foundation on which data driven value generation is being built on at scale. And the key what history of Google was. Google basically is a data driven business to the purest definition. Maybe in the core of the definition was looking at the data technology market and saying, hey, we want to build this data system that basically gives us this property of building these wide data records that allow us to harness all of the data signal and couldn't find any solution in a technology market. So Google built its own. And as it so happened, now we call it BigQuery, but BigQuery beforehand tremle and that idea of this query engine and the storage model behind it that can be used to make these immensely big data sets and analyze them is basically what is designed to build that common data foundation. And my point would be, if you build that common data foundation, don't spread the tools as much. And it's maybe something you want to optimize, but it's not the big leverage. The big leverage truly is having that wide data record foundation that allows you to capture all data signals and bring them in a common platform where you can associate them with each other. So you do see these latent patterns, meaning the hidden patterns that your data already has, but you only will see right, if you bring it together. This is why I also think the big data age is finally coming to an end. Because it's not about big data, it's about wide data. I think it's a way better way of thinking about it.

A lot covered there, but it sounds like particularly for the lower level, the data infrastructure, you want that to be common across your whole organization because you don't want different teams to have to worry about data security or things like that. It should just be standard throughout the company. But then I guess as you get closer to the business use cases, that's where you want a bit more tailoring so you can really focus on, well, this is how we add business value with our models and data products and things like that.

Yeah. And you want, you want this data foundation to grow, right? If someone brings in your data, you don't want this to be an island of its own. Right. You want to associate with all of your other data when you have the ability, right? You know, your customer data, your sales data, your service data, the social media data, all of that stuff. And you see that wide data record that is the true definition of a customer360. Right. It's not doing a CRM query and knowing what they have ordered. It's basically seeing all of that in conjunction. And this allows the creation of new and exciting services, for instance, or making processes much more efficient because you find now these relationships that you couldn't see before and that emergent record of making it ever growing and bigger. I called it a flywheel, right. Of data that only works if you basically build a common foundation. If basically everyone builds their own. You have many small flywheels, but you don't get that momentum and compound effect of what your enterprise data actually has. And organizations need to bring it together to unlock it.

Absolutely. I'm sure in every single organization, basically no one knows the extent of what all the data is there, I mean, vast amounts of data at DataCamp, I know quite a lot of it, but I'm sure there are huge pockets of data I have no idea about. And so, yeah, having access to those is going to give you ideas for things you can build that you wouldn't understand otherwise. So I think it feels like this goes beyond tooling though, right? Because you can buy these shiny new tools, but getting to business success, there's another step there. Do you want to talk me through what that looks like?

Yeah. Business success. I think there are three very concrete ways. And as I said, right. Data is no end in itself. Right. So you want to use data to make existing processes more efficient. And that is something that businesses are really good at. Measuring how one space is in your efficiency side. Right. How much better am I in managing business metrics like service requests in slo day sales outstanding. And data is a huge accelerant to that. Right. Giving either people the right information to complete their tasks or creating new self service offerings. Right. That's a huge space in how many service center calls can a company prevent by for instance, giving them a chat agent that is able to understand what they're trying to do and help them all along. So that's deficiency space is one big one. I think a more interesting one is in the, let's call it in the new feature, new capability space. How can you use your data to augment your existing products? For instance, I'm wearing a garment watch here, no advertising. The reason why I'm wearing this watch is because of all of the value at the data signals are giving me. And Garmin basically convinced me from saying hey, this is not a great looking watch. I'm not sure if I want to wear that to saying that I want to wear this every day because it basically tells me so much about my well being, my sleep, how it's trending, if I should sleep more, if I should work out. And these are all new services. So when you think about your favorite streaming platform, right. The quality of the recommendations, for instance, or the ability to take usage data and create new content, this is a way of using data to build new products and that should result in engagement and revenue. These are really hard intangible business metrics and I think there is even a third space of what you can call new companies. How many companies are built on their ability to understand and match data? So many come to mind from ride sharing to hospitality, so many great examples. That's really a data platform that basically creates the product in itself. So I think there are some real hard business metrics that any company can associate back to their data initiatives. And then there's the fourth one. It's a bit of a harder to quantify one because how do you quantify the ability of people to make fast and confident decisions, non trivial to measure, but ultimately. Right. What Google search is doing is helping, I don't know, billions of people getting to the answers which they want and it's so quick. Then just imagine not having it would create a huge problem for me if I looking for something to buy or looking for something to read. And the same applies, you know, if we do bi well to how people work every day. Right. And it's not as easily captured as in this one business metric because it will make workflow faster, it will make customers more happy and employees more engaged and feel more productive. But that's much more of an ambient quality and I think is an equally important one though harder to measure. Absolutely.

Again, just perhaps that last one is the biggest one. Just making people make decisions fast, that's probably going to add vast amounts of value in there for any organization and just all the individuals within it. So we talked about dashboards, we talked about databases, we talked about getting value from your data. Are there any other data trends that you're most excited about at the moment?

Oh you know Richie, it's really the most exciting time in data right now with the opportunity space that generative AI creates. There is one sub trend, if you will, that I am mostly excited about. It's interesting. We have two big gaps. The one gap we talked about, right, people have not the access to information that they could have. And we talked about how this is going to change. The other side is there is so much data which is not being used. How much knowledge and wisdom is locked away in all the documents, video, audio recordings that exist. Unstructured data is by far the largest data type that exists, and it's the one that's hardly ever used in any enterprise data landscape. And the reason is it's just too darn hard. It's very specialized. If you build an unstructured data processing pipeline, it's very difficult. It's not like a table which you can query this way and that way and join it, but it should be, why can't it be? And generative AI gives us, for the first time, a way to reason about unstructured data in a fairly flexible way. And when I think about the opportunities that we have by just harnessing that data, how much richer that will make any company in their ability to capitalize on data. I do think it's going to be maybe the biggest step change in all of this because the data is there, everyone has it. And now basically we got the key, the key to the save, which is called generative AI. And I think it will unlock a whole set of new use cases and improvements that basically everyone can tap into. Now that really excites me.

That's a very good point. That there is a vast amount of just really useful information just stored away in documents and videos and things that no one really has access to. I'm sure there are many Google Docs in our organization that it's like. I'm sure it says something useful, but I don't know where it is or what it says. So, yeah, being able to access that and get some value out of it sounds incredibly important. All right, super. Thank you so much, Garrett, for your time.

Thank you, Richie. It was great being here today.

SA.