Knowledge Graphs with Juan Sequeda - Episode #49

Welcome to the Agile Data Podcast, where we talk about the merging of agile and data ways of working in a simply magical way. Welcome to the Edge of Data podcast. I'm Shane Gibson.

Hi Shane, I'm Juan Sequeda. I'm the principal scientist and the head of the AI lab at data.world. It's a pleasure to be here.

Hey Juan, today we're going to discuss the pattern that is knowledge graphs, and to be fair, I think I know what a knowledge graph is, but then I don't. I'm really looking forward to this deep dive, picking your big brain of knowledge graphiness to figure out what it is, why we use them, what all these terms mean. But before we do that, why don't you give us a bit of background about yourself and how you got into the wonderful world of knowledge graphs and data?

Yeah. Okay. So if I step back, I've been in this kind of space of not getting involved in knowledge graphs before that, I was actually a. Thing probably, oh gosh, almost now, almost 20 years ago, probably 18. So my background, I'm a computer scientist. I did my undergraduate and my PhD in computer science at the University of Texas at Austin. But early on, I got involved into this whole area called the Semantic Web. And it was the early vision of how the web was, is evolving. And for me, one of the most interesting aspects that I learned was, hey, what does it mean, the semantics part, right? So if you think about the example was on search, So I mean, think about 2005, how the web was at that time, and search was the main kind of thing that we were doing. We see Google, and the example was if you're searching for Paris Hilton, what do you mean? Do you mean the person Paris Hilton, or you mean the Hotel Hilton in the city Paris? And which city Paris? So that for me was like my aha moment of like, oh, the syntax semantics in search, and then this is data coming from multiple places, and then fall into the whole area of data integration. And so that's how I got into that as an undergraduate. I found it super interesting at a previous company at that time, and just doing data stuff, and I realized, oh, semantics, the meaning of things, this is a big gap. That ended up being my research where I decided to go do my PhD, was really on figuring out how to integrate all these different technologies. You have your relational databases, which have existed since the '70s, and then you have all these— at this time, the technologies that were being standardized by the W3C, which were these semantic web standards, specifically RDF Graph, which is a graph data model, OWL, which is an ontology language for to define semantics, SPARQL, which is a graph query language. And also note that these are just modern manifestations of these technologies. Being able to implement defined semantics knowledge representation has also been something that in computer science we've been doing for decades and decades. So this isn't new. So I was just trying to understand like, what is the relationship between these modern technologies? You have relational databases and these semantic web technologies. And that was my PhD. And from there we did a lot of theory and systems work. And basically, one of the deliverables of my PhD was being able to define a semantic layer and ontology, being able to create mappings from your sources to your target, right? Mappings, transformations, rules, and then be able to do virtualization to take a query in terms of the target. And in this case, it was a graph model and be able to translate that over your source model, which was SQL. And so we did all that and all this semantic virtualization federation work. And that was a company I started back around 2014 out of my PhD, a company called Capcenta. And we did that kind of early on before the modern data stack, things before dbt or even things before Airflow and stuff. Long story short, the folks at DatatWorld, I've known them for a long time and it was just a match in heaven at the right time, right vision, right timing. And I sold my previous company, Cap Santa, to DatatWorld over 4 years ago. We're on that mission to be able to help organize enterprises' data. In the structured world, you have all these SQL databases and people, so it's all about first bringing in your metadata and cataloging that. And eventually what I call the crawl, walk, and run, right? Is understand what you have and then from there understand what these things mean and then be able to go access the data in terms of that meaning, which end users in the organization are the ones that talk about. Anyways, that was a very long answer. I can go longer on it, but—

No, good answer. So good journey. And I think, yeah, I don't think we can talk about knowledge graphs without talking about semantics and semantic layers, but maybe we'll get to semantics later because that's quite a deep discussion at the moment, especially in the data world, because it's all hot. But let's go back to knowledge graphs. So if you pretend you're ChatGPT, explain to an 8-year-old what is a knowledge graph. And I won't get you to do it in the style of a pirate. You can do it in any style you want.

All right. Look, the way I like to keep this is that a knowledge graph represents a collection of real-world concepts and the relationships between those real-world concepts. And they happen to be in the way of a graph. So the way to think about this is real-world concepts. Let's think about a domain. E-commerce domain. So you have orders, you have customers, you have products, and so forth. You have addresses, billing. So then those are the real-world concepts, and you start making these relationships between them. Oh, and a customer places an order. An order consists of a set of lines. A line has products. So effectively, this is, this is your kind of your enterprise data model, your ER schema and stuff like that. So that's one thing, is that you start defining these, the, these concepts. And this is where the, the knowledge part comes, the meaning, right? These concepts and relationships between the concepts are first-class citizens. This actually encodes what that knowledge is, and we can get much more expressive, right? A customer can place multiple orders, right? An order is shipped to exactly one address, and so forth. And then the data part, the graph part, is really— now you start bringing in the, the structure of the data that is also in a graph. If you think about it, in your relational world, things are defined in a way to model an application. And here we are modeling data so it represents actually that domain that end users think about it, the real-world concepts in your domain. Bottom line, when you start thinking about a domain, something you want to go model, you end up going to the whiteboard and you draw bubbles and lines. It's already a graph. We naturally think about it this way. So we're just making this as a first-class citizen. It's really a way to integrate data and knowledge at scale. And why it's a graph is because it's built on the whole premise of the web, because the web itself is a graph. Everything is linked.

Okay, so let's unpick that a bit. When I'm coaching, uh, data and analytics teams, one of the exercises I get them to do is, is one based on a a great TED Talk called How to Make Toast. And the way it goes is you give them a blank piece of paper and you go, draw how you make toast. And what they end up doing is they end up drawing a bunch of pictures, because no words are allowed. So they draw a bunch of pictures and a bunch of lines, and some of them have lots of steps and some of them have a few steps, and some of them are funny and some of them are well drawn. And it all goes back to this How to Make Toast TED Talk and this idea of nodes and links. And so what we do is we then take that process and we say, okay, how does this work in your day job, right? What's the value stream you do? What do you do with data? Who does what? What's a node link. And so for me, a node's a dot, it's a circle, it's a thing that's done, and a link is a relationship to the next step. And when I think about the way you describe graph and knowledge graphs, what you're saying is we have some core business concepts, customer, product, employee, those kinds of things in their circles, their nodes, and then we're linking them to something else to say there's a relationship. And the key thing being the line itself is a first-class citizen, where in the data world, typically we go, well, I've got a table and a table, and the line actually is just a blob of code. It's got many first-class citizens embedded in that code. I'm conforming customer records, I'm cleansing addresses, I'm doing a whole lot of nasty shit to that data, and it's all encapsulated in this blobby of crappy code in the middle of the line, and it has no first-class citizen. It's a bunch of things. It's a Lego bucket of stuff.

And to add to that is a lot of the semantics, a lot of the knowledge is embedded in the application code, so it's not connected to the data. So the software that uses that data, in that software, a human being took knowledge, took meaning of something, and they coded it in Java and Python or whatever. And that is disconnected from the data that's being stored. So these things are disconnected. So if I show you the data by itself, you don't know what it means. If I show you the code, you're like, okay, I think I know what it means, but I don't see the data. So I need to put these two things together. You want these two things that should ideally come together. And this is a mindset shift because we don't think about it that way. We've just traditionally been grown up to say, yeah, you have a database where you store your data and then you write code and the code implements all these requirements. But all these requirements have clear semantics, meaning. So what happens is that the next application comes along and then what do you do? You reinvent, you add again the semantics and you do that again over and over again. And are you interpreting the same way? I don't know. So the idea is that you connect all this knowledge and data together such that Knowledge and data together are first-class citizens, and it is independent of the application. Applications will come and go for the rest of our lives. The data, the meaning of it, how your company is organized, like, that stuff should be first-class citizens, and they should live together, not dependent on a particular application.

Let me give you an example then and see how this plays out. In our product, what we do is our lines, our code is effectively a rules engine, and we use a pattern from Gherkin, from the testing language, which goes, given this data, and, and then do something. So let's say somebody's done that horrible bloody thing in an application where they've built a model of thing is a thing of a thing. So I have a table of things and I have another table of thing types and I've gotta connect thing table to thing type table to figure out whether it's a customer, it's an employee, it's a supplier. So table of everybody in the world and then a typing table, which is really efficient for an operational system, but incredibly shitty to use from an analytics point of view. So what we do is we'd say, grab that table called thing. So that's the given, this table called thing and these IDs, then filter out where type equals customer, right? And then populate your customer entity, your customer node. So what I've got now is I've got two circles and a line. Circle one is table thing. Circle two is table customer. Middle, my blobby bit of code that says filter on, on thing type. Equals customer, but that logic is just a bunch of code. And so what you're saying is we bring out this first-class citizen of customer, and we probably bring out this thing of type, and rather than being embedded in the code, it's stored as an object with a relationship. And then ideally the code's talking to that semantic model, that graph model, and saying, let's use that as config or data rather than encapsulating it in the code itself. So now I have something that's been described, I'm calling that that thing in its context that it has and using it rather than just embedding it in the code with no context, no semantics, no understanding. Apart from the person who wrote it.

Is that right? Exactly. This is exactly it. And this is what these languages, these W3C standards like OWL, which is the Web Ontology Language, are made to do. You have ways to express all these types of semantics, meaning, hey, this is a class. This is a property. The property has a domain. It has a range. The domain can be a class. It could be a union of classes. It can be intersection of things, right? You can get very expressive. Around this stuff. And you may not need all this expressiveness, but you have that power there. So, at the end, all the semantics, the meaning, just become codified. And at the end of the day, this is all metadata.

So, that changes the way we work, though, because what we're effectively doing is going and populating some form of, let's call it a dictionary, a list of words and a list of contexts and a list of relationships, and then the code's using that, ideally without us, And that's quite different between deterministic and probabilistic. I always get those things wrong. But what we are saying is, rather than me going in and being very deterministic in terms of, this is the answer, what I'm doing is I'm saying, here's a bunch of things, you tell me what the answer is. And that's a completely different way of working. And I have seen some tools attempt that over the years, to be that transformation code engine that's based on that way of working. But what I find is that complexity, that cognition as a human, to be able to understand how to define these objects and then just call the relationships to do our transformation of data. It's quite a large cognition level compared to just writing code, to writing a set of steps and being able to test those steps. So, is that what you've seen, is that the complexity of this way of working in the data space means, actually, it's one of the reasons that it hasn't been well adopted?

Things haven't been as adopted as I wish. It's really a human incentive. After a decade looking at this stuff, I'm like, here is the ideal way of doing this. Why don't you do this? It's because it's incentives. First of all, change is hard. And I think one of my favorite quotes from Ludwig Wittgenstein, the limits of my language are the limits of my world. So if all— I already know how to do things in SQL and Java, this thing, I'm just— don't change it. And if you show me something else, I'm going to compare it to that, right? If I have a hammer, everything's a nail. So you're used to these things. Now what happens is that because you're used to these things, you are efficient in them. If I have a task, I mean, task, I'm being tasked to do something, I know how to go do it. And we've been doing all these things for decades and decades. So it's not that it doesn't work. It does work. But the issue is the amount of continuous work that you have to do it again. Oh, here's a change. Here's a new application. You do all this work over and over again, but we're just used to it. And this is why all the silos get created. You have one system and then we move it to another system, but we're just focusing on our present, on being efficient to do this today because I am being incentivized to get the job done as fast as possible right now. And I have the tools, so don't bother me. Let me get it done. We're not being incentivized to think, to see what are the implications about doing it this way. So I call this always the balance between being efficient and being resilient. We are focused and we are being incentivized to be efficient. We are focused and being incentivized to— we're focused on the use cases of today. We are not being incentivized to be resilient. We are not being incentivized to say, "I'm going to do this, probably a little bit more work right now, but it's going to help me scale for later," because I'm not incentivized to think about resiliency. I'm not incentivized to think about the unknown use cases of tomorrow. And then let's go change. Why not? Because my job is to get this done in 1 or 2 years. I'm not being paid to go do this in 5 years. For 4 to 5 years because guess what? I'm probably gonna leave anyways, right? My bonus is tied for the next 1 or 2 years. I don't care about all these things. So we're not incentivized towards these things. So that's one aspect that I think the reason why the thinking about semantics and knowledge graphs hasn't really picked up as much as I would like to, number 1. Number 2, the second one, it's because semantics, understanding, meaning, this means that you have to talk to humans. And ooh, that's scary. And as a technologist, I have a career in technology because I just want to code and do things, and I don't want to talk to people. And this is, I think we've hit the wall. Any new technology that's going to think about semantics and stuff where you're not talking to people, all you're doing is reinventing the wheel and coming up with a different syntax, right? ELT, ETL is the same stuff. Batch and streaming, you're just moving data. Databases, data lakes, data warehouses, lakehouses, they're all different types of storage and compute engines. Creating a new thing is not going to solve this problem. What we really need to go focus on is on methodologies, understanding the roles of knowledge engineers, of data product managers. So it's more of the people and process side. And I think that I'm now starting to see hope that is changing. And I would argue that this is really one of the— in the last couple years, the whole data mesh movement really helps us focus on the data product management and getting requirements. I think that's a huge step in that direction. So that's the second thing is it's a people thing, but I'm very extremely hopeful because one, people are afraid of the people thing because it's manual. Guess what? LLMs help us to automate a bunch of stuff. And I'm not saying just automate everything. It's like it's helping us be much more productive. So that's going to help. And second, I think all large big tech companies use knowledge graphs. Google Search, everything is in a knowledge graph. Amazon, Netflix, all these things use knowledge graph in the backend. So the big companies have already realized it. Because LLMs are gonna help us accelerate this, we're gonna start seeing the barriers to come down and then realizing, oh yeah, the incentives may not change, but at least things are gonna be faster. People are thinking about, hey, I, I can do a much better job now faster because I have these LLMs that can help me do it. Okay. That was a lot.

That was a lot. Let, let me unpack some of that. So we'll, we'll roll back and we'll go through it. So I agree around speed to market versus resilience, that, that sugar rush versus the consequences of that sugar rush. And we've seen it before. As humans, we love to repeat the same cycle and wonder why bad things happen, or we wear the consequences. We've seen BI tools come out that were self-service. Then we saw something like Business Objects come out with their universe, their former semantic layer. So a single pattern of a semantic layer rather than the broad brush. But what we saw then was all the reports going and talking to one thing to get that semantic meaning before it was used. And then we saw Tableau come out. The rise of self-service BI and visualization with no universe, no semantic layer. So, what we saw is massive enablement, which is a good thing. We want more people to be able to do the work that we used to do. But then we see chaos because with self-service comes chaos. So, we saw thousands of Tableau dashboards. And then again, we start seeing some centralization again. And then we see DBT come out. Yeah, again, good thing. It's helped people who can write SQL to do the data work and they don't have to use these enterprise data transformation tools that were always hard to use. Self-service great democratization, but again, chaos. Now I've got 5,000 DBT models, and don't get me started on a rant about the use of that term as a model. There should be semantic crimes where you go and do that, you use a word in a completely different way, you should just get put in prison. Orange suit, you sit in the naughty data prison, and whoever defined DBT code block, a blob of code as a model, goes to naughty orange, uh, data prison in my head. But anyway, um, So 5,000 blobs of code. And so again, we see that with self-service comes chaos. That's one of the problems. The second one is LLMs are going to make it worse. And here's a question for you. I constantly get people asking me to use LLMs in our products to auto-create the description of a column. And when I talk about AI, I have my standard joke. It used to be data mining back in the day. 80% of data mining is a group by. 5% is a regression, then maybe we might get onto some K-means and some neurals and that kind of stuff. But the way I talk about it now is I talk about Ask AI, assisted AI, and automated AI. And I say, Ask AI is where a person's having a conversation with the machine, with an LLM pretty much, right? I have a question, it's gonna give me an answer. I've got another question. I'm gonna keep doing that until I've got what I think I need. I'm gonna go and then do the action. So I'm using my cognition and I'm doing my action. Assisted AI is where I'm doing something and the machine's watching, and then it's coming back and making a recommendation, right? Saying, you're working with that data, and based on that data, I think this is the business key. I don't know for sure, but I'm just hinting. Oh, I'm looking at this data that you're working with, and I think that's a bunch of people's names. You may want to mask that or secure it. So it's the machine's looking and it's assisting and recommending me for some stuff that I may or may not need to do. Known to do. And the third one's automated AI, which is the machine doesn't, the human doesn't do anything. So the machine just completely runs it. Something where the machine is actually just doing the work for us and the human's never involved. There's no feedback loop in that kind of— when I think about LLMs on column descriptions, what worries me is I run the LLM, it looks at the data in the column, it creates a description for that column, and I do that for 70,000 SAP columns. Now our consumers are gonna come in and see that description and they're gonna see there's a description there and it will be well-worded cuz that's what LLMs do. They give you a sentence that looks like a bloody good sentence. So they're gonna trust that actually is the description of that field. But no human's been involved. There's been no review or feedback loop. And so my view is maybe we can use the LLMs as a recommendation engine. Where the human can look at it and say, yeah, that's good. I'm happy with that. Approve. But then we fall into human behavior, which is tick, tick, tick, speed over resilience. So that's going to look at them and go, yeah, yeah, job done. Because there's no consequences to them of that semantic meaning being wrong. So I think that standard pattern of LLMs giving us descriptions for data columns is an incredibly dangerous pattern. If we think about resilience, what's your thought on that?

So it always, the pendulum swings on all this stuff. The way you've described things is from a very technical perspective. Active. So the question here is, why do we need descriptions? And then, okay, why do we need descriptions on the 70,000 columns from SAP? How is this going to provide value and to whom? Explain that to me. So this goes back into the people side. As technologists, we don't ask these questions. We should really be pushing back. If somebody tells me we need to have descriptions of 70,000, I should be saying, who the fuck is asking for this? And why? How is this going to be useful for anybody? So that is the problem. And I think this is the problem of being focused so technology. And this is my annoyance. We have to be data literate. No, screw that. We need to have business literacy. We need to have the tech people being able to ask those questions. So that's a big change. So then you go off and you say, here is the problem that we're trying to go solve. And you understand how that's valuable to the company. Then you say, okay, what do we need to be able to go solve that problem? And then you scope it down and bring it. And then you say, I don't need 70,000. I probably need 100 or whatever. Then you are incentivized to figure out what those things are. So I think The technology is there. We just got to use it and use it for our productivity means. But we are the ones, the humans, are going to need to talk to other humans to understand what the heck we're doing and why. And I think this is what I would call the business literacy. So that's my answer, which is an indirect answer to what you're saying. This is the problem, looking at the problem from a technology point of view.

One of the other problems that I see in the space around knowledge graphs is actually a semantic problem.

Problem.

So when you say graph, in your head, I'm assuming you're thinking nodes and links. You're thinking those visual graph models that always look like boobies to me. You stand in the middle where the nipple is and you move the outside around and it looks like a wobbly jelly. But to me, that's how you think. Whereas when you say graph to me, I typically think bar chart, line chart, and not a pie chart. So again, that semantic language of graph actually has two different meanings, two different strong meanings, and probably five other meanings that I don't know about. Is that what you find? Is it when you say graph, do you find that some people start going down visualizations, graph models?

Honestly, no. I think if you're talking to somebody who's a— if they do think about it for that, then you're like, it's very evident if we're miscommunicating, and then you would clarify it. I think that— so, it's not a thing. One thing that does happen a lot is that people think about knowledge graphs in the sense of nodes and edges, and then they want to go see it. I'm like, "Okay, you wanna go see the graph. Why?" Really, it's because they think it's cool. But then I'm like, "What problem are you solving by looking at the graph?" I'm gonna shoot, right? If I show you one node by itself, it's like showing you one row of data or 10 rows of data. By looking at 10 rows of data, what is the problem you're gonna solve? You're actually gonna go query it, do an analysis, do a groupby, and be able to do that, right? This is a big pet peeve I have with people. "I wanna go see the graph." It's okay, I'll show it to you, but so what? You wanna see the whole thing? Here's a hairball. What problem did that solve? That's a pet peeve that I push back so much on it. But eventually people just, it's like some eye candy, you have to go give it to them.

In the data transformation space, I call it mad person's knitting, where we see those lines all over the place and things going left, right.

Yeah. People wanna see the lineage and like, yeah, I guess. So people are very data lineage. You have a very specific surgical thing you wanna go look at. Yeah, definitely go see the data lineage and because it helps you with that. But if you wanna zoom out and see everything's okay, here it is. You saw a bunch of lines going across all these different tables. So what you really want is a list of things. This is the most important table. This is the most critical one. You literally want is, at the end of the day, people want is a list of things.

Look, we're working on column-level lineage at the moment. Under the covers, we store everything at a column level. When you transform data, because of the way we work, because we use a config engine, we store this field, this column went to this column, and this is what we did to it. We just don't display it. And the reason we don't display it at the moment is whenever I think about visualizing it, I go back to what I know. I go back to lineage graphs. I go back to the London Underground. The circle, this line, and the circle, and this line, the circle. And I go, when I do that for tables, it's messy enough, right? Mad person's knitting to a degree. When I do it for columns, it's horrendous. It's a hairball, like you said. It looks beautiful and it says I'm very busy, but it doesn't tell me anything. And yes, I can filter and isolate it. I can say, click on this field and show everything. But that is what I'm used to doing. But it's an unnatural behavior because now what happens is I see a bunch of circles and a bunch of lines. But the semantics is that the logic, the meaning, everything we've done is embedded in that line. It's embedded in a bunch of code that's not displayed on that picture. So, what do I have to do? I have to drill down. I have to drill down and see the words, the logic, the code that did that change. And so, for me, I keep thinking about it more as a contract, as a brief, is why isn't column-level lineage just text? Why isn't it just a document that said, hey, You got table and table's made up of a bunch of things, customer, employee, and supplier. And then we go and filter it based on the customer type to give us a list of customers. Oh, and then we're gonna go grab customers from here and customers from here, and we're gonna join them and merge them, and we're gonna do that based on first name, last name, and date of birth. Oh, and by the way, we know that's actually a really dangerous mechanism. Why don't we just actually give somebody a story, a set of words that tell them what's happening? Because That's what they're going to understand and they're going to do that quickly because we know as humans we can scan text.

So there's two things here. I think from one, the underlying lineage and column, like you should keep track of that. And by the way, this is why I always constantly say that your first application of a knowledge graph is of a data catalog of metadata. Like everything you just described, like that should be represented as a graph. All the, the level granularity in a graph, right? So that's why your first knowledge graph application should be of metadata and data catalogs should be built on a knowledge graph architecture. Now, the way to display that information, to convey that story is different. It could be in the form of a visual graph, or as you just said, in a particular story. I think from a technical mindset, people are like, oh, I wanna see the graph. But they're like, whoa, but what is the task you're trying to go solve? If you're doing a cloud migration effort and they just give you this hard book, what do you want to go do? Maybe What I should go do is find the easiest things that have the highest value first and figure I can do that. Okay. So how, what do you, those are the types of questions you're doing. Then the typical, if I change this column here, what is that going to affect? If this dashboard is wrong, where could that be coming from? Do you really need to go visualize that? Or do you really need to know the list of things that's going to affect and the people you need to go to contact? Again, it's a list of things. This is a story they wanna go say. So we really need to understand what is There's a task you wanna accomplish, who's gonna accomplish it, and then there's so many different paradigms that we can use to accomplish that. And so the visualization is not always the thing to go do. Again, it is pretty sexy, pretty cool, but at the end of the day, like from a metadata perspective, I think all this has to be in a graph. And I always tell people, if you're not doing, if you're not managing your metadata in a graph, you are basically reinventing and putting and trying to put into a graph in some other weird way. There's a graph, period.

Again, let's carry on with the constraints, the things that are stopping people adopting this graph, these graph patterns. And one of the interesting ones is you mentioned Google Search and the fact that Google Search is built on a graph. And I don't think it was originally, was it? I think I remember listening to a podcast, probably one of yours, where they talked about the— originally it was a page ranking mechanism and then they went and bought a startup.

So that was Freebase. So what happened? So what they— what Google did was they innovated on search. But think about on search, it was just the 10 blue links. And then over a decade ago, you start seeing what we call these knowledge panels. You actually try to— they try to understand the intent of the question and they probably give you the concepts that you're actually talking about. And that change of the paradigm, that was because of the knowledge graph that they built. And actually they take the marketing credit for using the term knowledge graph, even though it's all built on all these technologies from that, from the academic point of view, where my academic household, the semantic web, that's where all this all comes from. You would basically, if you look at all the folks at knowledge at at Google at that time, they all came from the semantic web academic community where they're just bringing in all these technologies.

All right. And we'll come back to that because I think one of the examples you used on one of the podcasts was fire, as in fire a gun or fire a person or light a fire. So I'll come back to that in a minute, but I just want to go down around that access to technology. For what we're building, we build it on Google Cloud. We made a decision that we are single cloud, not multi-cloud, and that had some benefits and it had some downsides. And so we use a bunch of Google services, BigQuery, Spanner, a whole lot of serverless things there. Make our life so much easier and so much cheaper. But the thing that really amazes me is we cannot get a graph engine out of Google. Now, what we do is we have a graph model under the covers, and we think relationally. We don't think in terms— we come from a relational background, not a graph background. So what we've done is created some graph-like relationships in a relational model in Spanner. But I really wanted to use a graph database, and the natural way of doing that would be to use a Google service, but there is not one. There's no service in Google Cloud that gives you a graph database. Now there is Neo4j and third-party that you can stand up as a quasi-serverless capability, but that doesn't suit us. We want native Google stuff. So why do you think that is? Why Google, whose whole world is based on a graph, the one service I can't get from them is this graph database. And then if I look across all the cloud providers, you typically have to go to a specialist provider who hosts their graph database on that cloud provider. It's not seen as a native service, like a relational database, like a message queue, like a blob storage. It's not a first-class citizen in terms of a technology stack.

No, this is a great observation. I think it's just the markets, right? I think this is not where the market is talking about, and I hope, hopefully this will change. And I mean, the other one, Amazon is the only one that has Amazon Neptune. right? And the other cloud providers don't have— Google, Microsoft has some stuff, but they have a couple of things there, but not very popular. And then, as you've noted, Google doesn't have anything. So yeah, I guess it's just where the market is right now. And my hope is that— I've been working towards this for almost two decades. I'm still here. I passionately believe that this is the right way. I genuinely believe that the right way to go manage enterprise data and metadata at scale is in the the form of a graph, because if you don't, you're going to end up just repeating it in a different way. Specifically, you want your metadata to be a first-class citizen. And also, very important thing is that you want your identifiers to be a first-class citizen. And I think this is something that people don't really realize, is that why is data integration so hard? Because we don't think about identity. So we always talk, oh, what is a customer? Of course, their customers can have so many different meanings. Okay. So what are they? So how would you uniquely identify a customer? We don't go about having these discussions. And what is a unique global identifier for this thing? We don't think about it. Now, if we actually thought about unique global identifiers from the beginning, data integration would be much easier because I would know that if I'm creating data, I'm like, well, which identifiers am I going to go use? Can I reuse something that exists? Am I gonna go create some new things? You start thinking about this and then this makes data integration easier. We never think about identifiers from a global perspective. We only think about it from like internally in my application. Oh, primary key. But you don't, don't think about it globally. Now this is one of the reasons why the whole knowledge graph comes from the semantic web world is because the web is all about identifiers and HTTP URI. The URI, right? Universal Identifier. The URL is a locator. That's what actually brings you back to the page that you see in the browser. But the URI is that identifier. We think about those things. On the web, there is no identifier that will take you to two different pages. It's a unique thing. This is one of the things that is missing. And just, I think people aren't even thinking and trained about it. Think about identity. In a way, it's also philosophical too, but we need to have these discussions because otherwise they get pushed all the way upstream and they're like, I got this report about customers, but you don't use my definition of customer because we never defined it and never thought about how to uniquely identify them.

Yeah, it's funny, isn't it, that we will put massive guardrails around our cloud adoption and our databases. We'll put a guardrail in and says your database has to be secured. You can't just go in with a public identifier and access it. You have to actually say who you are. You have to have some form of recovery and backup, and there has to be some way for it to not fail. So we do that, yet we don't do that with the actual data itself. We say, yeah, you can create a unique ID for customer in two different places and they can be different. And then somebody else will do that horrible work. And we love to laugh at governments, but if you think about governments, they've actually been relatively good at building identifiers for citizens in terms of Social Security number or passport or driver's license over here, because they need to actually be able to identify a person as a person at some stage. As corporates, we have much more control over our data, but we don't do it. We let that problem happen to us, knowing it's going to be a problem and not caring. And again, probably speed to market versus resilience. So let's take that URI point of view. Then what you're saying is, does everything get its own URI? So there's just like one bucket of URIs for everything.

So if you look at the standards for the knowledge graph standards for RDF and now everything has a URI. Every concept, every relationship, every instance, everything has a URI. And because you can then point to it, and then even to the point that I can have different identifiers, but if they mean the same thing, then I can say, hey, this identifier, this identifier, they mean the same thing. At the end, it's all about these connections. That's why it's all a graph. So an example for folks listening, just go to schema.org, for example. Schema.org is schema, an ontology that has been developed over the last decade in a community-driven way by Google and Yahoo and Bing and all these folks, which they've defined the schema, the ontology that they want the web to be marked up with. So then you'll have any website that's been created, they can now use these semantic markup. And how does everybody know what the stuff means? Because I use the identifier for it. and you can go through schema.org, you can see every single concept that has been defined, every single property, and they all have a unique identifier. If you think about JSON, that key, the key-value pair, that key should actually be a unique identifier. So if anybody goes in and they like, it's unambiguous. What does this key mean? Oh, it's a unique identifier. I can look up what identifier, I know what it means. And it's machine readable too. So the machine should be able to go figure out on their own, getting it to be more autonomous too.

Okay. So, that schema.org, though, doesn't go to enterprise models. It doesn't say—

But the approach of how that has been created, that's what we should be doing also too in the enterprise. Enterprise models today, oh, the enterprise architects get together, they go draw it, and they draw it on a UML diagram, and then it just stays in a piece of paper and stuff. We should take that to the next level. Like, that model, is very important semantics. Now let's codify that. Let's turn that into something machine-readable. Now what usually happens is that we then turn that into SQL, SQL DDL, for example, but then that stays there and it doesn't get— then they update the model, they update the schema, the SQL schema separately from the model, it's disconnected. But that SQL DDL, think about it, SQL DDL has a very simplistic semantics, right? You have tables and column names, but I want to express more things. This is a relationship. It's a one-to-one relationship. I want to have synonyms. I want to have labels in different languages. I want it like— there's more things that SQL DDL does not let me do. That's where the semantics— that's what a true semantic layer on ontology would be. I can define these things and then you can use that semantic layer to turn it into a JSON schema, turn it into a SQL DDL, turn it into an XSD, whatever. At the end of the day, those are just different syntaxes that represents the same meaning across any type of these indexes. That meaning, that's the ontology that should be independent of any application.

Okay, so you've used the word ontology a bit, and it's another one of those, like knowledge graphs, it's another one that I think I get it, but I think I don't. So again, ChatGPT, give us a description of an ontology for an 8-year-old, or for me. How would you describe to somebody what the hell is an ontology?

Okay. Look, so simply an ontology is just your— the simplest ontology is a schema. So it's a way of modeling your domain. So the simplest way of thinking about an ontology is you have your classes, your concepts, and they have relationships between them. That's already a simple ontology. And then you can think about it as a simple schema, like your tables with your columns. And then an ontology just really represents the way how we see the world. You can start saying more and more things about it. So, for example, I say a customer and an order. A customer is a concept, an order is a concept, then there's a relationship. A customer places an order. So, placed by is an order. So then placed by is a relationship. And I say this relationship is used by customers and it's going to orders. And then I can add more expressive information. A customer can have multiple orders. And then I add more information, more knowledge about that. And that's what an ontology is.

Okay. So it's a bag of words.

It's a bag of words with a very clear meaning, with very clear relationships between how they're all together. Right.

And some form of boundary.

Think about this as a spectrum. So you can have a list of words, which is what a business glossary terms. I have a list of words. Then I can say, hey, these different words all mean the same thing. Oh, so I have a thing, a concept, and you have different words, have different synonyms around of them. Okay, then I can say, hey, these different concepts have some sort of relationship. They're like a broader relationship, a narrow relationship. So then you start getting into some sort of taxonomies. You start defining hierarchies, relationships between them. And then you can start saying, there— a concept can have another type of relationship to another concept that is not a subsumption hierarchical one. Oh, then you start making these kind of these relationships. An order is placed by a customer. And then you can keep going. I can add more and more information to that. So there's that spectrum of what I call expressivity. People should start with— come up with that bag of words list of business glossary terms. And then people start saying, hey, those different words, they all mean the same thing. Okay, then we start relating that. Hey, then these other words now represent these concepts. Here's some hierarchical relationships. You can start building a taxonomy to these things. And then you start saying, hey, these other concepts have a different type of relationship between them. Now you're modeling the graph. That's how you realize you're modeling the ontology right there.

To me, it's like data domains. We talk about data domains, and then I say to an organization, how do you bound your domains? Is it— here's ways I've seen people do it. I see it in organizational structure. So the domains will bound to the way your org chart works. So you'll have a finance domain and an HR domain because you have a finance team and an HR team. Or I may see your domains be based around core business process. So I might see order to delivery. That's a domain and all the things that happen in there, because that's how you think about it.

It's just choices.

What you're saying is ontology is a bag of words and you've got some boundary to say that it goes in there and you can have large ontologies or small ontologies, I'm assuming, because you can extend the boundary or contract the boundary. And the key thing is it's every word, so it's not just the they are words of concepts, but it's their aliases, their relationships, it's everything that gives us a semantic meaning for the words in that bag. Okay, that kind of makes sense to me now.

It's a way of representing knowledge.

And if I go back to knowledge graphs, and again, I come from a technology side, because that's typically where I touch it. One of the things that I see massive value for it is you typically define the left and the right and the relationship between those two things. And then the technology takes care of the complexity. If I say, let's go back to that example that I heard somewhere where I get fired, I fire a gun, and I light a fire. So there's 3 different uses of the word fire.

So they all use the same syntax, F-I-R-E, but they all mean different things. So how do you differentiate this thing when you now bring that back into reality, into our enterprise world? Those should have 3 different identifiers so they're unambiguous.

Thor, the Wind, Fire.

Yes, they have 3 different identifiers. They happen to have the same label. But what I should be talking about is the identity, not the label, because the label, they can have the same labels. So this is why identity is crucial. So if you're like, fire, I'm like, which fire? Here's the identifier for it. Oh, great. I know the identifier for that thing. And that's not the one I'm talking about. So I can ignore it or whatever. That's why identity is important. It's critical, it's crucial.

Let's use an example. So I get the identifier for fire, and I'm going to get back a label, but it's only when I get back the relationship to another term that I'm going to get the context for me. If I see fire is related to gun, I'm going to go, okay. If I see fire is related to employee, or fire is related to heat, it's that second word and the relationship between it that allows my brain to figure out the context you're using there.

Yeah, exactly, because things are defined in the context of other things. In a way. So as you said it, because now think about it as is literally what the triple is. I have an identifier, let's call it, let's call this identifier A, is type of a class, and A has a label fire. And I can say I'm going to define another identifier for the other concept of fire. It's going to be the same thing, but it's gonna have a different identifier B. So the question is, I don't know what is the difference between A and B. Well, A will be connected to some other things, and B will be to some other things, and that's how I, that's the context that you want to talk about.

And then we can also have a relationship between A and B.

Possibly, whatever.

Not to say it's a reuse of the same 4 letters and therefore it's a dangerous thing. One of the benefits of a knowledge graph technology is I don't have to worry about all that complexity because all I ever do is I say I have a left and a right and a relationship between them, and I define those 3 things that there is an A, the word fire. There is a 123, which is the word gun. And then there's a third object, which is that relationship to say those two things are related. And all I have to do is put into the graph those three things. A is related to 123, and then the graph takes care of wherever that identifier 123, wherever gun is used, it takes care of wherever A, wherever fire is used again. It does all that, that relationship mapping and complexity of that many-to-many relationships in the background. For me. And that's one of the things I liked about graphs is I don't have to worry about boiling the ocean, as you say. I can just put in a bunch of relationships, a bunch of two nodes and a link between them, and the graph engine, the database, takes care of all the complexity of everything else it is, and it allows me to answer those questions. The example I remember in fraud many years ago in a previous life was Whiplash for Cash. So what this one was, there were a bunch of people in the US There was normally 5 to 7 of them. And what they would do is they would pretend there was a car crash. They would pretend somebody got whiplash and then they'd get an insurance claim. But what happened is each of the people changed roles. In the first crash, Person A was the driver, Person B was in the other car, Person C was a passenger and got whiplash, Person D was the doctor who assigned the whiplash to them, and they got a claim. And then next time there was a crash. Person A was actually the passenger who got whiplash. Person B was actually the doctor that assigned it. So they keep changing roles. They worked out this pattern, and I'm not sure they actually used a knowledge graph, but that whole mapping of these people are playing these roles in many different things was a good use of that kind of graph model for me, because then they could identify these things will happen. Oh, Person A's been involved in 100 crashes, but each time there is a different relationship. So for me, that idea of being able to put in small bits of relationship into the engine and the engine showing us how it all relates about us having to do that ourselves. That's one of the values of using a graph engine.

Completely agree. And think about this. It's really about the flexibility. And the flexibility enables you to be resilient and adapt to change. So when should you use a knowledge graph and when should you not use a knowledge graph? So if you're focused on, on creating just one specific application, that this is one great defined things and that's it, don't use a knowledge graph. Go build it in a non-relational database. Go create your, your JSONs. Do it in Mongo, whatever. Just do that one thing. That's all you wanna go do. But if you're creating an application that is— that you're gonna give it some requirements, but those requirements are gonna change, you're gonna need to bring in more data, you don't know how that's gonna evolve, if you're starting to go manage integrating data from different sources, then you want to have a stack that is going to enable you to deal with the use cases of today, but also with the use cases of tomorrow. Therefore, be agile, be resilient. That's when you should be focused on a knowledge graph because you can get all those requirements and define your schema, your ontology for today, and then you bring in your data and you're done. Now tomorrow you're going to get new requirements, new things are going to change, new sources you need to go bring in. I can easily— we have the flexibility to go extend that. In a graph perspective, it's really this common denominator data model. I can turn tables into graph. I can turn trees into graph, XML, JSON. So if I do some natural language processing, I do some entity extraction, some relationship extraction, all of that I can turn into a graph and I can just add those things. My taxonomy, that is also can be in the graph. So then I just have this means of integrating all types of data and metadata and knowledge at the same time. Very easily to go expand that. Again, if you're just focusing on a very specific application, you don't— then your goals are— then go do whatever, however you're comfortable with. But if you need to go deal with the known use cases of today and the unknown use cases of tomorrow, where you need to have that flexibility to be able to go very— be able to rapidly, very agile ways extend things, this is when you need to start thinking about knowledge graph because you're going to start thinking about, hey, what a lot of the what-ifs. So you're preparing yourself for the future. You're preparing yourself by having schemas that are well-defined that can be extended. You're defining FHIR so that people can go reuse them and they can also go extend them if needed and so forth. So this is when you should be using knowledge graphs.

So when do you not use a knowledge graph? One of the examples that I tried many years ago, and it was an epic failure, was putting the actual data into a knowledge graph. So you go, knowledge graph's just a database, it's a really flexible database. So why don't we just bypass this disconnection? Why do we say metadata and context and all that sitting in my graph and the physical data sitting on a storage lake or in a relational database? Why can't I just push all my data to the graph?

Itself.

And back then, what I found was it wasn't really designed for large volumes of rows.

The industry has changed a lot. Think about it 10 years ago, even 5 years ago, the scalability of graph database systems were not as well, and they continue to improve, and we'll continue to see this. But I, for one, this was actually my PhD, and this was my, this was the company, and this is the technology we have at data.world, is all about virtualization. Your data, your transactional or analytical data is gonna live in your data warehouse, data lake, whatever, in your Snowflake, in your or whatever, I want to be able to integrate that with other systems so you can have a knowledge graph layer on top of it, which then you would— you can map it and then you can virtually access it. So that's stuff that we do. For example, that was my PhD over 10 years ago, which was like, oh, I can take graph queries and using the mappings be able to translate those graph semantic queries and then translate them to the SQL queries underneath and be able to go execute on a different source such that you don't have to go move all your data centrally into a graph database. You want to have virtualization. And then there's some things that you want to have a, as always, right? Hybrid, right? Depends what some things you would do want to go move. Metadata in general is usually smaller. So I think that natively will live in a graph just because you extract metadata from so many different sources and the metadata gets connected, especially because things move and metadata is keeping track of all those movements. So that naturally is a graph itself too.

So. But I've seen this wave a couple of times. And I think a lot of the vendors, the spin they're putting on now is Active Metadata. Data. And the pattern is fairly simple because when we talk about transforming data, like we do in data management, there's two patterns. We write code to transform it, and we try and extract the meaning from that code into something where we can see it, put it together, and understand it, which is, is the current DBT world again. Um, or we go the other way, which is we define the relationships and we define the semantic and we define that graph effectively, and the code's written for us. Um, and we tend to swap between those two worlds. And I think my view is we're gonna go back to this active metadata, this config-driven code again, 'cause we go in waves. The abstraction, the writing code and then try and suck meaning out of the code, the creating a schema and trying to reverse engineer that schema into a data catalog is where the pattern always ends up. We always end up doing the easy work and then trying to reverse engineer that into the hard piece. Why do you think that is? Is it just because it's easy to write code? Like you said, we know how to write code.

Of course, this is the whole point that we started off with. It's just technology and people don't understand the business, they wanna just connect. This is where I think LLMs are a godsend right now because it will help us being able to make us more productive to understand what things mean. So just here's the exercise. Go off, ask a bunch of folks who work in different business units and literally ask them, what are the questions that you're trying to go answer today? And just get a list of those questions. These questions are what we call competency questions. They're things that they're trying to go answer. Now tell the LLM, here are the questions that people are trying to go ask. Create an ontology, create a business glossary, create a semantic layer out of that stuff. Blah. It will generate something for you. It won't be perfect. That's why you go in and then you try to figure out what it means and then you draw it on the whiteboard and you go back to these people and you say, hey, is this how you see your world? And then that question that you're trying to go ask, you really basically traverse the graph. That's what you mean. This is stuff that we can now very easily go do because the LLMs have kept us very excited. There's so much hype about LLMs and yes, we gotta be careful about all the noise and all that hype, but it is a fact of life that is making everything so productive. This is a pure example of how to go do this. And one of the other things that I've been working on so much when it comes to LLMs and, and knowledge graphs is to really understand what is the accuracy of LLMs when it comes to question answering. So we're talking about, oh, I have my data in a SQL database. I wanna be able to now write questions and that where, where the answer's in a SQL database. So let me use these LLMs to translate text to SQL. This is where you should be really careful because that, that can generate a bunch of stuff. And so a lot of the research I've been doing has been to understand that accuracy. And then also to say, hey, if I invest in knowledge graphs that has all that context, how much does that accuracy improve? And all the research, and we created this whole benchmark that the difference between using a knowledge graph and not using a knowledge graph is 3x based on the benchmark. You have a, a 16% accuracy of all these questions on enterprise schemas, 16% accuracy if you don't use a knowledge graph. And if you use a knowledge graph, that jumps up to 54%. This was with basic prompt and stuff. So two things. One, knowledge graphs, invest in knowledge graphs because it has that really clear context that your business needs. And second, the LLMs are there to make us more productive. By using this, we're going to get out of our technical comfort zone and be able to start doing more stuff on the people side and not get overwhelmed because we have a tool that can help us automates himself.

Yeah, I remember back in the old days when I worked with analytics teams, we used to talk about mixed models. You'd run a bunch of different models, you'd see what the fit is. And then the trick that most of the teams did was when they were explaining to a stakeholder what the model was doing, they typically showed decision tree. Now the decision tree model wasn't the model that was being run and productionized, but it was close enough to what the model they were using was actually doing that they could describe the flow of data. They could say, oh, for the segmentation, this thing it's doing is picking up income and then it's basing it off gender and then location. It gave a, a story, a visual story that could understand it. And I think LLMs are going to give that to us. But like we did back then when you had to test all the different models to see which was the best model for your context, it's the same with LLMs. The text-to-SQL ones we're playing with at the moment, we're seeing how good it can be and how bad it can be. But the thing is, it's not consistent. So simple things like ask it the same question 3 times and get back 3 different SQL blobs. You know, that's a test to you that actually it's not accurate.

Well, in, in the benchmark that we did, we actually did test this stuff, but because you're non-deterministic systems, it's not like I give you a question, I get a, a generated query back. If I always give you the same question, you're always gonna generate that. In the benchmark, we had 43 questions, which we ran for 3 weeks nonstop between 30 to 300 different runs because we also wanted to see the percentage of the accuracy that they get. So this is one of the things that we needed to keep in consideration. Definitely. Definitely.

Yeah, and it's valuable, but it's not a silver bullet. Oh, 100%.

100%.

Just to close this out, so does that mean if we think about knowledge graphs in terms of the pattern of nodes and links, and in terms of the technology that enables us to create those nodes and links and visualize them easier than if we're doing it manually ourselves, it's been around for a long time, like you said, but it really hasn't been adopted in the data world. It's a second or even, I'd say it's even a third-class citizen. I'd say It's a powerful pattern, but one that is hardly ever used in the data domain. Do you think that LLMs is the thing that's going to make it become a first-class citizen?

100%. Our CEO, Bret Hurd, says like startups are grit and luck. And I actually think that from a research perspective, that applies, especially if you're, this is my goal, what I want to do. I've been at this stuff for semantics and knowledge graphs for almost two decades. The web was something that changed people's lives and all, right? The iPhone changed some stuff, right? Social networking changed some things. LLMs has come in, it's going to change so much. My bet here, and this is what I'm literally betting on this now, is LLMs are like that godsend saying, okay, this came out and this is going to be one of the motivators that are good people going to realize we have this problem. How do we solve this problem? People are like, Knowledge graphs. That has always been there. We can go use that. And it's gonna be much more critical in the enterprise because in the enterprise, what is the biggest concern of LLMs? Hallucinations. And then a lot of the topic, the conversations around LLMs are about creating all the— look at these RAG architecture stuff. It's all about text. But really people are not even talking about that much about SQL because that's where a lot of our enterprise data is. But they're not doing that much in there because they know that's a huge issue about the accuracy. And already our research has been pointing that by investing in knowledge graphs, you're increasing the accuracy and this is just gonna get better and better. This for me is the point in time. That's why I'm super excited working so hard on this because this has literally been, the gods have sent us something to say this knowledge graph work is going to start booming because of the LLMs.

And if we take that example of our 70,000 SAP tables, maybe we don't go look at the query logs to see who's querying those columns to see what's most important. Maybe we look at our chatbot that we're using to answer questions, and whenever that bot, wherever that LLM is using some fields and of those 70,000 on a regular basis to answer a question, that's the time we need to go and extend it out with well-formed descriptions, with those relationships, with the aliases, because we know those fields are the ones that are getting used time and time again by the LLM. Therefore, they're the ones we have to enhance. Enhancing them have more value to us because we're answering, we're helping more of our consumers. Maybe it's less about query logs now, more about where the LLMs are heading, and then how do we apply knowledge graph there? And again, going back to hopefully define config, have the config create and run the code for you. It's all about that context. It's all about that config. It's not about the lines of SQL that you're generating as a human because they're abstracted, they're divorced, they're siloed by default. And yes, you can reverse engineer them, but why would you? Why wouldn't you just do it properly at the beginning? I always talk about the web and I talk about iPhone moments. I always forget about social networks. I'm gonna steal that one from you. Web, iPhone, social networks, LLMs, right?

That's been, and that's just been the last 30 years.

Yeah. We just gotta realize that right now the LLMs are Newtons. They're the horrible smartphones that don't work, but at some stage they will, the iPhone will appear.

Think about 1991, 1992 when the web came out. How many people are like, oh, that's a fad thing, pay attention, and look how it changed our world.

I still remember the modem dial-up and nobody in the house being able to make a phone call because you're surfing the web. It's a funny memory. Excellent. I think I understand knowledge graphs a lot more. I definitely understand ontologies, and I'm definitely going to do an LLM trick, right? I'm going to go and give it a bunch of questions until it's typed me some of those things so I can get more examples that refine my understanding of that. But key thing is we're still working in a complex world if we're trying to combine knowledge graph technology with the rest of the stacks that we use in data management, and it's still not a common thing to do. So hopefully that does change. Hopefully we can leverage that out. Excellent. All right, thank you for coming on the show, and I hope everybody has a simply magical day.

Thank you very much.

Datamagicians was another Agile Data podcast. If you'd like to learn more on applying an agile way of working to your data and analytics, head over to agiledata.io.