AgileData #49 - Knowledge Graphs with Juan Sequeda

Foreign.

Welcome to the Agile Data Podcast, where we talk about the merging of agile and data ways of working in a simply magical way.

Welcome to the Agile Data Podcast. I'm Shane Gibson.

Hi, Shane. I'm Juan Cicada. I'm the principal scientist and the head of the AI lab at Data World. It's a pleasure to be here.

Hey, Juan. Today we're going to discuss the pattern that is knowledge graphs. And to be fair, I think I know what a knowledge graph is, but then I don't. I'm really looking forward to this deep dive, picking your big brain of knowledge graphiness to figure out what it is, why we use them, what are all these terms mean? But before we do that, why don't you give us a bit of background about yourself and how you got into the wonderful world of knowledge graphs and data.

Yeah, okay. So if I step back, I've been in this kind of space of not getting involved in knowledge graphs. Before that, I was actually a thing probably, oh gosh, almost now, almost 20 years ago, probably 18. So my background, I'm a computer scientist. I did my Undergraduate and my PhD in Computer Science at the University of Texas at Austin. But early on I got involved into this whole area called the Semantic Web. And it was the early vision of how the web is evolving. And for me, one of the most interesting aspects that I learned was, hey, what does it mean? The semantics part, right? So if you think about the example was on search. So I mean think about 2005, how the web was at that time and search was the main kind of thing that we were doing. We see Google and the example was, if you're searching for Paris Hilton, what do you mean? Do you mean the, the person Paris Hilton or you mean the Hotel Hilton in the city Paris? And which city Paris? So that for me was like my aha moment of like, oh, there, the syntax, semantics and search. And then this is data coming from multiple places and then find fall into the whole area of data integration. And so that's how I got into that as an undergraduate. I found it super interesting at a previous company at that time and just doing data stuff and I realized, oh, semantics, the meaning of things. This is a big gap that ended up being my research where I decided to go do my PhD was really on figuring out how to integrate all these different technologies. You have your relational databases, which have existed since the 70s, and then you have all these, at this time, the technologies that were being standardized by the W3C, which were the semantic web standards, specifically RDF which is a graph data model. Owl, which is an ontology language to find semantics, SparQL, which is a graph query language. And also note that these are just modern manifestations, these technologies being able to implement defined semantics. Knowledge representation has also been something that in computer science we've been doing for decades and decades. So this isn't new. So I was just trying to understand like what is the relationship between these modern technologies, you have relational databases and these semantic web technologies. And that was my PhD and from there we did a lot of theory and, and systems work. And basically one of the Deliverables of my PhD was being able to define a semantic layer in ontology, being able to create mappings from your sources to your target, right? Mappings, transformations, rules, and then be able to do virtualization to take a query in terms of the target, and this case it was a graph model, and be able to translate that over your source model, which was a SQL. And so we did all that and all this semantic virtualization federation work. And that was a company I started back around 2014 out of my PhD company called Capcento. And we did that kind of early on, before the modern data stack, things before DBT or even things before an airflow and stuff. Long story short, the folks at Data World, I've known them for a long time and it was just a match in heaven at the right time, right vision, right timing. And I sold my previous company Capstan to the data world over four years ago. We're on that mission to be able to help organize enterprises data in the structured world. You have all these SQL databases and people. So it's all about first bringing in your metadata and cataloging that and eventually what I call the crawl, walk and run is understand what you have and then from there understand what these things mean and then be able to go access the data in terms of that meaning, which end users and the organization are the ones that talk about. Anyways, that was a very long answer. I can go longer on it, but.

No good answer. So good genie. And I think, yeah, I don't think we can talk about knowledge graphs without talking about semantics and semantic layers. But maybe we'll get to semantics later because that's quite a deep discussion at the moment, especially in the data world because it's all hot. But let's go back to knowledge graphs. So if you pretend you're chatgpt, explain to an 8 year old what is a knowledge graph and I won't get you to do it in the style of a Pirate, you can do it in any style you want.

All right, look, the way I like.

To keep this is that a knowledge graph represents a collection of real world concepts and the relationships between those real world concepts and they happen to be in the way of a graph. So the way to think about this is real world concepts. Let's think about a domain, the E commerce domain. So you have orders, you have customers, you have products and so forth. You have addresses, billionaire. So then those are the real world concepts and you start making these relationships between them. Oh, and a customer places an order. An order consists of a set of lines. A lines has products. So effectively this is, this is your kind of your enterprise data model, your er, schemas and stuff like that. So that's one thing is that you start defining these, these concepts. And this is where the knowledge part comes, the meaning, right? These concepts and relationships between the concepts are first class citizens. This actually encodes what that knowledge is and we can get much more expressive, right? A customer can place multiple orders, right? An order is shipped to exactly one address and so forth. And then the data part, the graph part is really. Now you start bringing in the structure of the data that is also in a graph. If you think about it in your relational world, things are defined in a way to model an application. And here we are modeling data. So it represents actually that domain that end users. Think about it, the real world concepts in your domain. Bottom line, when you start thinking about a domain, something you want to go model, you end up going to the whiteboard and you draw bubbles and lines. It's already a graph. We naturally think about it this way. So we're just making this as a first class citizen. It's really a way to integrate data and knowledge at scale. And why it's a graph is because it's built on the whole premise of the web. Because the web itself is a graph. Everything is linked.

Okay, so let's unpick that a bit. When I'm coaching data analytics teams, one of the exercises I get to do is one based on a great TED talk called how to Make Toast. And the way it goes is you give them a blank piece of paper and you go draw how you make toast. And what they end up doing is they end up drawing a bunch of pictures because no words are allowed. So they draw a bunch of pictures and a bunch of lines and some of them have lots of steps and some of them have a few steps and some of them are funny and some of them are well drawn. And it all Goes back to this how to make toast TED Talk and this idea of nodes and links. And so what we do is we then take that process and we say, okay, how does this work in your day job, right? What's the value stream you do? What do you do with data? Who does what, what's a node and what' link? And so for me, a node's a dot, it's a circle, it's a thing that's done, and a link is a relationship to the next step. And when I think about the way you describe graph and knowledge graphs, what you're saying is we have some core business concepts, customer, product, employee, those kind of things in their circles, their nodes, and then we're linking them to something else to say there's a relationship. And the key thing being the line itself is a. As a first class citizen, where in the data world typically we go, I've got a table and a table. And the line actually is just a blob of code code. It's got many first class citizens embedded in that code. On conforming customer records, I'm cleansing addresses, I'm doing a whole lot of nasty shit to that data. And it's all encapsulated in this lobby of crappy code in the middle of the line. And, and it has no first class citizen. It's a bunch of things, it's a Lego bucket of stuff.

And to add to that is a lot of the semantics, a lot of the knowledge is embedded in the application code, so it's not connected to the data. So the software that uses that data, in that software, human being took knowledge, took meaning of something and they coded it in Java and Python or whatever. And that is disconnected from the data that's being stored. So these things are disconnected. So if I show you the data by itself, you don't know what it means. If I show you the code, you're like, okay, I think I know what it means, but I don't see the data. So I need to put these two things together. You want these two things that should ideally come together. And this is a mindset shift because we don't think about it that way. We've just traditionally been grown up to say, yeah, you have a database where you store your data and then you write code and the code implements all these requirements, but all these requirements have clear semantics. Meaning, so what happens is that the next application comes along and then what do you do? You reinvent, you add again the semantics and you do that again over and over again. And are you interpreting the same way? I don't know. So the idea is that you connect all this knowledge and data together such that knowledge and data together are first class citizen and it is independent of the application. Applications will come and go for the rest of our lives. This date, the data, the mean of it, how your customer, how your company is organized, like that stuff should be first class citizens and they should live together, not dependent to particular application.

Let me give you an example then and see how this plays out in our product. What we do is our lines, our code is effectively a rules engine and we use a pattern from Gherkin, from the testing language which goes, given this data. Dan, Dan, then do something. So let's say somebody's done that horrible bloody thing in an application where they've built a model of thing is a thing of a thing. So I have a table of things and I have another table of thing types and I've got to connect thing table to thing type table to figure out whether it's a customer, it's an employee, it's a supplier. So table of everybody in the world and then a typing table, which is really efficient for an operational system, but incredibly shitty to use from an analytics point of view. What we do is we'd say grab that table called thing that's given this table called thing and these IDs, then filter out where type equals customer and then populate your custom entity, your customer node. What I've got now is I've got two circles and a line. Circle one is table thing, circle two is table customer middle. My blobby bit of code that says filter on thing, type equals customer. But that logic is just a bunch of code. And so what you're saying is we bring out this first class citizen of customer and we probably bring out this thing of type, and rather than being embedded in the code, it's stored as an object with a relationship. And then ideally the code's talking to that semantic model, that graph model, and saying let's use that as config or data rather than encapsulating it in the code itself. So now I have something that's been described, I'm calling that that thing in its context that it has, and using it rather than just embedding it in the code with no context, no semantics, no understanding, apart from the person who wrote it. Is that right?

Exactly. This is exactly it. And this is what These languages, these W3C standards, like OWL, which is the web ontology language, are made to do. You have ways to express all these types of semantics, meaning, hey, this is a class, this is a property. The property has a domain and has a range. The domain can be a class, it could be a union of classes, it can be intersection of things, right? You can get very express around these, around this stuff. And you may not need all this expressiveness, but you have that power there. So at the end, all the semantics, the meaning just become codified. And at the end of the day, this is all metadata.

So that changes the way we work though, because what we're effectively doing is going and populating some form of, let's call it a dictionary, a list of words and a list of context and a list of relationships and then the codes using that, ideally without us. And that's quite different between deterministic and probabilistic. I always get those things wrong. But what we're saying is rather than me going in and being very deterministic in terms of this is the answer, what I'm doing is I'm saying here's a bunch of things, you tell me what the answer is. And that's a completely different way of working. And I have seen some tools attempt that over the years to be that transformation code engine that's based on that way of working. But what I find is that complexity, that cognition as a human to be able to understand how to define these objects and then just call the relationships to do our transformation of data. It's quite a large cognition level compared to just writing code, to writing a set of steps and being able to test those steps. So is that what you've seen? Is that the complexity of this way of working in the data space means actually it's one of the reasons that it hasn't been well adopted.

Things haven't been as adopted as I wish is it's really a human incentive. After a decade looking at this stuff, I'm like, here is the ideal way of doing this. Why don't you do this? Is because it's incentives. First of all, change is hard. And I think one of my favorite quotes from Ludwig Wittgenstein, the limits of my language are the limits of my world. So if all I already know how to do things in SQL and Java, this thing, I'm just don't change it. And if you show me something else, I'm going to compare it to that, right? If I have a hammer, everything's a nail. So you're used to these things. Now what happens is that because you're used to these things, you are efficient in them. If I have a task, I mean task. I'm being tasked to do something. I know how to go do it. And we've been doing all these things for decades and decades. So it's not that it doesn't work, it does work, but the issue is the amount of continuous work that you have to do it again. Oh, here's a change, here's a new application. You do all this work over and over again, but we're just used to it. And this is why all the silos get created. You have one system and then we move into another system. But we're just focusing on our present, on being efficient to do this today. Because I am being incentivized to get the job done as fast as possible right now. And I have the tools, so don't bother me, let me get it done. We're not being incentivized to, to see what are the implications about doing it this way. So I call this always the balance between being efficient and being resilient. We are focused and we are being incentivized to be efficient. We are focused on being incentivized. We're focused on the use cases of today. We are not being incentivized to be resilient. We are not being incentivized to say, I'm going to do this, probably a little bit more work right now, but it's going to help me scale for later because I'm not incentivized to think about resiliency. I'm not incentivized to think about the unknown use cases of tomorrow. And then let's go change. Why not? Because my job is to get this done in one or two years. I'm not being paid to go do this in four to five years because guess what, I'm probably going to leave anyways, right? My bonus is tight for the next one or two years. I don't care about all these things. So we're not incentivized towards these things. So that's one aspect that I think the reason why thinking about semantics and knowledge graphs hasn't really picked up as much as I would like to, number one. Number two, the second one, it's because semantics, understanding, meaning this means that you have to talk to humans and ooh, that's scary. And as a technologist, I have a career in technology because I just want to code and do things and I don't want to talk to people. And this is, I think we've hit the wall. Any new technology that's going to think about Semantics and stuff that where you're not talking to people, all you're doing is reinventing the wheel and coming up with a different syntax. Right? Elt, ETL is the same stuff, batch and streaming, you're just moving data. Databases, data lakes, data warehouses, lakehouses, they're all different types of storage and compute engines. Creating a new thing is not going to solve this problem. What we really need to go focus on is on methodologies, understanding the roles of knowledge engineers, of data product managers. So it's more of the people and process side. And I think that I'm now starting to see hope that is changing. And I would argue that this is really one of the, in the last couple of years, the whole data besh movement really helps us focus on the data product management and getting requirements. I think that's a huge step in that direction. So that's the second thing is it's a people thing. But I'm very, extremely hopeful because one, people are afraid of the people thing because it's manual. Guess what? LLMs help us to automate a bunch of stuff. And I'm not saying just automate everything. It's like it's helping us be much more productive. So that's going to help. And second, I think all large big tech companies use knowledge graphs. Google, Search, everything is in a knowledge graph. Amazon, Netflix, all these things use knowledge graph in the back end. So the big companies have already realized it because LLMs are going to help us accelerate this. We're going to start seeing the barriers to come down and then realizing oh yeah, the incentives may not change, but at least things are going to be faster. People are thinking about, hey, I can do a much better job now faster because I have these LLMs that can help me do it. Okay, that was a lot.

That was a lot. Let me unpack some of that. So we'll roll back and we'll go through it. So I agree around speed to market versus resilience, that sugar rush versus the consequences of that sugar rush. And we've seen it before, as humans we love to repeat the same cycle and wonder why bad things happen or we wear the consequences. We've seen BI tools come out that were self service. Then we saw something like business objects come out with their universe, their, their former semantic layer. So a single pattern of a semantic layer rather than the broad brush. But what we saw then was all the reports going and talking to one thing to get that semantic meaning before it was used. And then we saw tableau come out, the rise of self Service bi and visualization with no universe, no semantic layer. So what we saw is massive enablement, which is a good thing. We want more people to be able to do the work that we used to do. But then we see chaos because with self service comes chaos. So we saw thousands of tableau dashboards and then again we start seeing some centralization again and then we see DBT come out. Yeah, again, good thing. Let's help people who can write SQL to do the data work and they don't have to use these enterprise data transformation tools that are always hard to use. Self service, great democratization. But again, chaos. Now I've got 5,000 DBT models and don't get me started on a rant about the use of that term as a model. There should be semantic crimes where you go and do that you use a word in a completely different way. You should just get put in prison orange suit, you sit in the naughty data prison and whoever defined DBT code block a blob of code as a model goes to naughty orange data prison. But anyway, so 5,000 blobs of code. And so again we see that with self service comes chaos. That's one of the problems. The Second one is LLMs are going to make it worse. And here's a question for you. I constantly get people asking me to use LLMs in our products to auto create the description of a column. And when I talk about AI, I have my standard joke used to be data mining. Back in the day, 80% of data mining is a group by 5% is a regression. Then maybe we might get on to some K means and some neurals and that kind of stuff. But the way I talk about it now is I talk about ask AI, assisted AI and automated AI. And I say Ask AI is where a person's having a conversation with with the machine. Vanilla pretty much. Right? I have a question, it's going to give me an answer. I've got another question. I'm going to keep doing that until I've got what I think I need. I'm going to go and then do the action. So I'm using my cognition and I'm doing my action. Assisted AI is where I'm doing something and the machine's watching and then it's coming back and making a recommendation, right? Saying you're working with that data and based on that data, I think this is the business key. I don't know for sure, but I'm just hinting, oh, I'm looking at this data that you're working with. And I think that's a bunch of people's names. You may want to mask that or secure it. So it's. The machine's looking and it's assisting and recommending me for some stuff that I may or may not need to do, known to do. And the Third1's automated AI, which is the machine doesn't. The human doesn't do anything, so the machine just completely runs it. Something where the machine is actually just doing the work for us and the human's never involved. There's no feedback loop in that kind of. When I think about LLMs on column descriptions, what worries me is I run the LLM. It looks at the data in the column, it creates a description for that column. And I do that for 70,000 SAP columns.

Now our consumers are going to come in and see that description and they're going to see there's a description there and it will be well worded because that's what LLMs do. They give you a sentence that looks like a bloody good sentence. So they're going to trust that actually is the description of that field. But no humans been involved. There's been no review or feedback loop. And so my view is maybe we can use the LLMs as a recommendation engine where the human can look at it and say, yeah, that's good, I'm happy with that, approve. But then we fall into human behavior, which is tick, tick, tick, speed over resilience. So that's going to look at them and go, yeah, yeah, job done. Because there's no consequences to them of that semantic meaning being wrong. So I think that standard pattern of LLMs giving us descriptions for data columns is an incredibly dangerous pattern if we think about resilience. What's your thought on that?

So it always, the pendulum swings on all this stuff. The way you've described things is, from a very technical perspective, active. So the question here is, why do we need descriptions? And then, okay, why do we need descriptions on the 70,000 columns from SAP? How is this going to provide value into whom? Explain that to me. This goes back into the people side. As technologies, we don't ask these questions. We should really be pushing back. If somebody tells me we need a description of 70,000, I should be saying, who the fuck is asking for this and why? How is this going to be useful for anybody? So that is the problem. And I think this is the problem of being focused. So technology, and this is my annoyance. We have to be data literacy. No, screw that. We didn't have business literacy. We didn't have the tech people being able to ask those questions. So that's a big change. So then you go off and you say, here is the problem that we're trying to go solve, and you understand how that's valuable to the company. Then you say, okay, what do we need to be able to go solve that problem? Then you just scope it down and bring it. And then you say, I don't need 70,000, I probably need 100 or whatever. Then you are incentivized to figure out what those things are. So I think the technology is there. We just got to use it and use it for our productivity means. But we are the ones, the humans need to talk to other humans to understand what the heck we're doing and why. And I think this is the, this is what I would call the business literacy. So that's my answer, which is an indirect answer to what you're saying. This is the problem. Looking at the problem from a technology.

Point of view, one of the other problems that I see in the space around knowledge graphs is actually a cemented problem problem. So when you say graph in your head, I'm assuming you're thinking nodes and links. You're thinking those visual graph models that always look like boobies to me. You stand in the middle where the nipple is and you move the, the outside around and it looks like a wobbly jelly. But to me, that's how you think. Whereas when you say graph to me, I typically think bar chart, line chart, and non apply chart. So again, that, that semantic language of graph actually has two different meanings, two different strong meanings, and probably five other meanings that I don't know about. Is that what you find is that when you say graph, do you find that some people start going down visualizations?

Honestly, no. I think if you're, if, if you're talking to somebody who's a, if they do think about it for that, then you're like, it's very evident if we're miscommunicating. And then you would clarify. And I think that. So it's not a thing I, I do. One thing that does happen a lot is that people think about knowledge graphs as senses, nodes and edges. And then they want to go see it. And I'm like, okay, you want to go see the graph. Why? Really? It's because they think it's cool. Because they end like, what problem are you solving by looking at the graph? I'm going to, right? If I show you one node by itself, it's like showing you one row of data or 10 rows of data. Like by looking at 10 rows of data, what is what the problem you're going to solve? You're actually going to go query it, do an analysis, doing a group by, and be able to do that. Right. This is a big pet peeve I have with people. I want to go see the graphics. Okay, I'll show it to you. But so what I'm going to. You want to see the whole thing? Here's a hairball. Okay, what problem did that solve? It's a pet peeve. And I push back so much on it, but eventually people just. It's like some eye candy. You have to go give it to.

Them in the data transformation space. I call it mad persons knitting. Where we see those lines all over the place and things going left, right?

People want to see the lineage. And like. Yeah, I guess so. People, data lineage. You have a very specific surgical thing you want to go look at. Yeah, definitely go see the data lineage and. Because it helps you with that. But if you want to zoom out and see everything, okay, here it is. You saw a bunch of lines going across all the different tables. So what you really want is a list of things. This is the most important table. This is the most critical one. Like you literally want is. At the end of the day, people want is a list of things.

Look, we're working on column level lineage at the moment, under the covers. We store everything at a column level when you transform data. Because the way we work is we use a config engine. We store this field when this column went to this column, and this is what we did to it. We just don't display it. And the reason we don't display it at the moment is whenever I think about visualizing it, I go back to what I know. I go back to lineage graphs. I go back to the London Undergrounds, the circle, this line, and the circle, and this line, the circle. And I go. When I do that for tables, it's messy enough, right? Mad person's knitting to a degree. When I do it for columns, it's horrendous. It's a hairball. Like you said, it looks beautiful. And it says, I'm very busy, but it doesn't tell me anything. And yes, I can filter and isolate it. I can say, click on this field and show everything. But that is what I'm used to doing. But it's an unnatural behavior because now what happens is I see a bunch of circles and a bunch of lines. But the semantics is that the logic, the Meaning everything we've done is embedded in that line. It's embedded in a bunch of code that's not displayed on that picture. So what do I have to do? I have to drill down. I have to drill down and see the words, the logic, the code that did that change. And so for me, I keep thinking about it more as a contract, as a brief is why isn't column level, lineage just text? Why isn't it just a document that said, hey, you got table. And it's tables made up of a bunch of things, customer, employee and supplier. And then we go and filter it based on the customer type to give us a list of customers. Oh, and then we're going to go grab customers from here and customers from here and we're going to join them and merge them. And we're going to do that based on first name, last name and date of birth. Oh, and by the way, we know that's actually a really dangerous mechanism. Why don't we just actually give somebody a story, a set of words that tell them what's happening? Because that's what they're going to understand and they're going to do that quickly. Because we know as humans we can scan text.

So there's two things here, I think from one, the underlying lineage and column. Like, you should keep track of that. And by the way, this is why I always constantly say that your first application of a knowledge graph is of a data catalog of metadata. Like everything you just described. Like that should be represented as a graph, the level grading, the layer in a graph. Right? So that's why your first knowledge graph application should be a Metadata and data catalogs should be built on a knowledge graph architecture. Now, the way to display that information to convey that story is different. It could be in the form of a visual graph or as you just said, in a particular story. I think from a technical mindset, people are like, oh, I want to see the graph. But they're like, but what is the task you're trying to go solve? If you're doing a cloud migration effort and they just give you this hard, but what do you want to go do? Maybe what I should go do is find the easiest things that have the highest value first and figure I can do that. Okay, so how do you. Those are the types of questions doing. Then the typical, if I change this column here, what is that going to affect? If this dashboard is wrong, where could that be coming from? Do you really need to go visualize that or do you really need to know the list of things that's going to affect and the people you need to go to contact. Again, it's a list of things. This is a story that you want to go say. So we really need to understand what is a task you want to accomplish, who's going to accomplish it. And then, then there's so many different paradigms that we can use to accomplish that. And so the visualization is not always the thing to go do. Again, it is pretty sexy, pretty cool. But at the end of the day, like from a metadata perspective, I think all this has to be in a graph. And I always tell people, if you're not doing, if you're not managing your metadata in a graph, you are basically reinventing and putting and trying to put into a graph to some other weird way there's a graph, period.

Again, let's carry on with the constraints, the things that are stopping people at adopting this graph, these graph patterns. And one interesting one is you mentioned Google Search and the fact that Google Search is built on a graph. And I don't think it was originally, was it? I think I remember listening to a podcast, probably one of yours, where they talked about the. Originally it was a page ranking mechanism and then they went and bought a startup.

Yeah. So that was freebase. What happened? So what they, what Google did was they innovated on search. But think about, on search it was just the 10 blue links. And then over a decade ago, you start seeing what we call these knowledge panels. You actually tried, they try to understand the intent of the question and they probably give you the concepts that you're actually talking about. And that change of the paradigm, that was because of the knowledge graph that they built. And actually they take the marketing credit for using the term knowledge graph, even though it's all built on all these technologies. From that, from my academic point of view, where my academic household, the Semantic Web, that's where all this all comes from. Basically, if you look at all the folks at, at Google at that time, they all came from the Semantic Web academic community, where they're just bringing in all these technologies.

All right, and we'll come back to that because I think one of the examples you used on one of the podcasts was fire, as in fire to gun or fire a person or a light of fire. So I'll come back to that in a minute, but I just want to go down around that access to technology for what we're building. We build it on Google Cloud. We made a decision that we are single cloud, not multi cloud. And that had Some benefits and had some downsides. And so we use a bunch of Google services, big grease banner, a whole lot of serverless things that make our lives so much easier and so much cheaper. But the thing that really amazes me is we cannot get a graph engine out of Google now what we do is we have a graph model under the covers and we think relationally. We don't think in terms we come from a relational background, not a graph background. So what we've done is created some graph like relationships in a relational model and spanner. But I really wanted to use a graph database and the natural way of doing that would be to use a Google service. But there is not one. There's no service in Google cloud that gives you a graph database. Now there is Neo4J and third party that you can stand up as a quasi serverless capability. But that doesn't suit us. We want native Google stuff. So why do you think that is? Why Google, whose whole world is based on a graph, the one service I can't get from them is this graph database. And then if I look across all the cloud providers, you typically have to go to a specialist provider who hosts their graph database on that cloud provider. It's not seen as a native service like a relational database, like a message queue, like a blob storage. It's not a first class citizen in terms of a technology stack.

No, this is a great observation. I think it's just the markets, right? I think this is not where the market is talking about and I hopefully this will change. And I mean the other one, Amazon is the only one that's Amazon Neptune. Right. The other cloud providers don't have Google. Microsoft has some stuff but they have a couple of things there but not very popular. And then as you've noted, Google doesn't have anything. So yeah, I guess it's just where the market is right now. And my hope is that I've been working towards this for almost two decades. Absolutely. I passionately believe that this is the right way. I genuinely believe that the right way to go manage enterprise data and metadata at scales in the form of a graph because if you don't you're going to end up just repeating it in a different way. Specifically you want your metadata to be a first class citizen and also very important thing is that you want your identifiers to be a first class citizen. And I think this is something that people don't really realize is that why is data integration so hard? Because we don't think about identity so we Always talk, oh, what is a customer? Of course, their customers can have so many different means. Okay, so what are they? So how would you uniquely identify a customer? We don't go about having these discussions and what is a unique global identifier for this thing? We don't think about it. Now, if we actually thought about unique global identifiers from the beginning, data integration would be much easier because I would know that if I'm creating data, I'm like, well, which identifiers am I going to go use? Can I reuse something that exists? Am I going to go create some new things? You start thinking about this and then this makes data integration easier. We never think about identifiers from a global perspective. We only think about it from, like internally in my application. Oh, primary key. But you don't think about it globally. Now, this is one of the reasons why the whole knowledge graph comes from the Semantic Web world, is because the web is all about identifiers. An HTTP uri, the uri.

Right.

Universal identifier. The URL is a locator. That's what actually brings you back the page that you see in the browser. But the URI is that identifier. And we've. We think about those things on the Web. There is no identifier that will take you to two different pages. There's. It's a very, It's a unique thing. This is one of the things that is missing. And just, I think people aren't even thinking and trained about it. Think about identity. In a way. It's also philosophical too. But we need to have these discussions because otherwise they get pushed all the way upstream and they're like, I got these and I got this report about customers, but you don't use my definition of customer because we never talked about it, never defined it, and never thought about how to uniquely identify.

Yeah, it's funny, isn't it, that we will put massive guard rails around our cloud adoption and our databases. We'll put a guardrail and it says, your database has to be secured. You can't just go in with a public identifier and access it. You have to actually say who you are. You have to have some form of recovery and backup and there has to be some way for it to not fail. So we do that. Yet we don't do that with the actual data itself. We say, yeah, you can create a unique ID for customer in two different places and they can be different. And then somebody also do that horrible work. And we love to laugh at governments, but if you think about governments, they've actually been relatively good at building identifiers for citizens in terms of Social Security number or passport or driver's license over here, because they need to actually be able to identify a person as a person at some stage. Yet as corporates, we have much more control over our data, but we don't do it. We let that problem happen to us, knowing it's going to be a problem and not caring. And again, probably speed to market versus resilience. So let's take that URI point of view. Then what you're saying is, does every thing get its own URI? So there's just like one bucket of URIs for everything.

So if you look at the standards for the knowledge graph, standards for rdf, and now everything has a uri. Every concept, every relationship, every instance, everything has a uri. Because you can then point to it and then even to the point that I can have different identifiers, but if they mean the same thing, then I can say, hey, identifier, this identifier, they mean the same thing at the end. It's all about these connections. That's why it's all a graph. So an example for folks listen to like just go to schema.org for example. Schema orb is schema, an ontology that has been developed over the last decade in a community driven way by Google and Yahoo and Bing and all these folks which they've defined the schema, the ontology that they want the web to be marked up with. So then you'll have any website that's been created, they can now use these semantic markup. And how does everybody know what this stuff means? Because I use the identifier for it. And you can go through schema.org you can see every single concept that has been defined, every single property, and they all have a unique identifier. If you think about JSON, that key, the key value pair, that key should actually be a unique identifier. So if anybody goes in and they like, it's unambiguous. What does this key mean? Oh, it's a unique identifier. I can look up what identifier, I know what it means and it's machine readable too. So the machine should be able to go figure out on their own getting it to be more autonomous too.

Okay, so that schema.org though doesn't go to enterprise models. It doesn't say.

But the approach of how that has been created, that's what we should be doing also to in the enterprise.

Enterprise Models today, although the enterprise architects get together, they go dry and they draw it in a UML diagram and then it just stays in a paper itself. It's we should take that to the next level. Like that model is very important semantics. Now let's codify that, let's turn that into something machine readable. Now what usually happens is that we then turn that into SQL DDL for example. But then that stays there and it doesn't get. Then they update the model, they update the schema. The SQL schema separately from the model is disconnect. But that SQL ddl, think about it. SQL DDL has a very simplistic semantics. You have tables and column names. But I want to express more things. This is a relationship, it's a one to one relationship. I want to have synonyms, I want to have labels in different languages. I want it like there's more things that SQL DDL does not let me do. That's where the semantics, that's what a true semantic layer on ontology would be. I can define these things and then you can use that semantic layer to turn it into a JSON schema, turn into a SQL ddl, turn to an XSD or whatever. At the end of the, at the end of the day those are just different syntaxes that represents the same meaning across any type of these indexes. That meaning, that's the ontology that should be independent of any application.

Okay, so, so you've used the word ontology a bit. And it's another one of those, like not as graphs. It's another one that. I think I get it, but I think I don't. So again.

1,1 GPT give us a description of an ontology for an 8 year old or for me, how would you describe to somebody what the hell is an ontology?

Okay, look, so simply an ontology is just your. The simplest ontology is a schema. So it's a way of modeling your domain. So the simplest way of thinking about an ontology is you have your classes, your concepts, and they have relationships between them. That's already a simple ontology. And then you can think about it as simple schema, like your tables with your columns. And then an ontology just really represents the way how we see the world. You can start saying more and more things about it. So for example, I say a customer in an order. A customer is a concept, an order is a concept. Then there's a relationship. A customer places an order. So place by is an order. So then place by is the relationship and I say, this relationship is used by customers and it's going to orders. And then I can add more expressive information. A customer can have multiple orders and then I add more information, more knowledge about that. And that's what an ontology is.

Okay, so it's a bag of woods.

It's a bag of work with the very clear meaning, with a very clear relationships between how they're all together.

Right. And some form of boundary.

Think about this as a, as a spectrum. So you can have a list of words, which is what a business glossary terms. I have a list of words. Then I can say, hey, these different words all mean the same thing. Oh, so I have a thing, a concept, and you have different words. I have different cinnamons around that. Okay. Then I can say, hey, these different concepts have some sort of relationship. They're like a broader relationship, a narrow relationship. So then you start getting into some sort of taxonomies. You start finding hierarchies, relationships between that. And then you can start saying, there a concept can have another type of relationship to another concept that is not a subsumption hierarch, hierarchical one. And then you start making these kind of, these relationships on orders placed by a customer. And then you keep going. I can add more and more information to that. So there's that spectrum of what I call expressivity. People should start with, come up with that bag of words, list business glossary terms. And then people start saying, hey, those different words, they all mean the same thing. Okay. Then we start relating that, hey, then these other words now represent these concepts. Here's some hierarchical relationships. So you can start building a taxonomy to these things. And then you start saying, hey, these other concepts have a different type of relationship between them. Now you're modeling the graph. That's how you realize you're modeling the ontology right there.

To me, it's like data domains. We talk about data domains and then I say to an organization, how do you bound your domains? Is it. Here's ways. I've seen people do it. I see it an organizational structure. So the domains will bound to the way your org chart works. So you'll have a finance domain and an HR domain because you have a finance team and an HR team. Or I may see your domains be based around core business process. I might see order to delivery. That's a domain and all the things that happen in there, because that's how you think about it. There's just choices. What you're saying is ontology is a bag of words and you've Got some boundary to say that it goes in there. And you can have large ontologies or small ontologies, I'm assuming, because you can extend the boundary or contract the boundary. And the key thing is it's every word. So it's not just the. They're words of concepts, but it's the aliases, their relationships. It's everything that gives us a semantic meaning for the words in that bag. Okay, that kind of makes sense to me now.

It's a way of representing knowledge.

Then if I go back to knowledge graphs, and again, I come from a technology side, because that's typically where I touch it. One of the things I see massive value for it is you typically define the left and the right and the relationship between those two things. And then the technology takes care of the complexity.

If I say, let's go back to that example that I heard somewhere where I get fired, I fire a gun and I light a fire. So there's three different uses of the word fire.

So they all use the same syntax, F, I, R, E. But they all mean different things. So how do you differentiate this thing when you now bring that back into reality or enterprise world? Those should have three different identifiers. So they're unambiguous for the wind fire. Yes, they have three different identifiers. They happen to have the same label. But what I should be talking about is the identity, not the label. Because the label, they can have the same labels. So this is why identity is crucial. So if you're like fire, I'm like, which fire? Here's the identifier for it. Oh, great. I know the identifier for that thing, and that's not the one I'm talking about. So I can ignore it or whatever. That's why identity is important, is critical. It's crucial.

Let's use that example. So I get the identifier for fire, and I'm going to get back a label. But it's only when I get back the relationship to another term that I'm going to get the context. For me, if I see fires related to gun, I'm going to go, okay. If I see fires related to employee or fires related to heat, it's that second word and the relationship between it that allows my brain to figure out the context. You're using that. Yeah, because.

Because things are defined in the context of other things. Things in a way. So as you said it. Because now think about it as literally what the triple is. I have an identifier. Let's call it. Let's Call this identifier. A is type of a class and A has a label fire. And I can say I'm going to define another identifier for the other concept of fire. It's going to be the same thing, but it's gonna have a different identifier B. So the question is, I don't know, what is the difference between A and B? Well, A will be connected to some other things and B will be connected to some other things. And that's how I, that's the context that you want about it.

And then we can also have a relationship between A and B, possibly say it's a reuse of the same four letters and therefore it's a dangerous thing. One of the benefits of a knowledge graph technology is I don't have to worry about all that complexity because all I ever do is I say I have a left and a right and a relationship between them. And I define those three things that there is na the word fire. There is a 1, 2, 3, which is the word gun. And then there's a third object which is that relationship, to say those two things are related. And all I have to do is put into the graph those three things, A is related to 1, 2, 3. And then the graph takes care of whether that identifier 1, 2, 3, where the gun is used. It takes care of where the A, wherever fire is used. Again, it does all that, that relationship mapping and complexity of that many to many relationships in the background for me. And that's one of the things I liked about graphs is I don't have to worry about boiling the ocean, as you say. I can just put in a bunch of relationships, a bunch of two nodes and a link between them and the graph engine. The database takes care of all the complexity of everything else it is and it allows me to answer those questions. The example I remember in Fraud many years ago in a previous life was whiplash for cash. So what this one was were a bunch of people in the US There was normally five to seven of them. And what they would do is they would pretend there was a car crash, they would pretend somebody got whiplash, and then they'd get an insurance claim. But what happened is each of the people changed roles. In the first crash, person A was the driver, person B was in the other car, person C was a passenger and got whiplash. Person D was the doctor who assigned the whiplash to them and they got a claim. And then next time there was a crash, person A was actually the passenger who got whiplash. Person B was actually the doctor that assigned it. So they keep changing roles, they worked out this pattern and I'm not sure they actually used an OS graph, but that whole mapping of these people are playing these roles in many different things was a good use of that kind of graph model for me because then they could identify these things will happen. Oh, person A has been involved in 100 crashes, but each time there's a different relationship. So for me, that idea of being able to put in small bits of relationship into the engine and the engine's showing us how it all relates about us having to do that ourselves, that's one of the values of using a graph engine.

Completely agree to think about this. It's really about the flexibility and the flexibility enables you to be resilient and adapt to change. So when should you use a knowledge graph and when should you not use a knowledge graph? So if you're focused on creating just one specific application, that this is one great defined things and that's it. Don't use a knowledge graph, go build it in a non relational database, go create your JSON, doing Mongo, whatever, just do that one thing, that's all you want to go do. But if you're creating an application that is that you're going to give it some requirements, but those requirements are going to change. You're going to need to bring in more data. You know how that's going to evolve. If you're starting to go manage integrating data from different sources, then you want to have a stack that is going to enable you to deal with the use cases of today, but also with the use cases of tomorrow. Therefore, be agile, be resilient. That's when you should be focused on a knowledge graph because you can get all those requirements and define your schema, your ontology for today and then you bring in your data and you're done. Now tomorrow you're going to get new requirements, new things are going to change, new sources you need to go bring in. I can easily have that flexibility to go extend that in a graph perspective. It's really this common denominator data model. I can turn tables into graph, I can turn trees into graph, xml, JSON. So if I do some natural language processing, I do some entity extraction, some relationship extraction, all of that I can turn into a graph and I can just add those things. My taxonomy that is also can be in the graph. So then I just have this means of integrating all types of data and metadata and knowledge at the same time very easily to go expand that again if you're just focusing on a very specific application, you don't then your goals then go do whatever, however you're comfortable with. But if you need to go deal with the known use cases of today and the unknown use cases of tomorrow, where you need to have that flexibility to be able to go very, be able to rapidly be in agile way extend things, this is when you need to start thinking about Knowledge Graph because you're going to start thinking about hey, what a lot of the what if. So you're preparing yourself for the future. You're preparing yourself by having schemas that are well defined, that can be extended. You're defining fires so that people can go reuse them and they can also go extend them if needed and so forth. So this is when you should be using Knowledge Graph.

So when do you not use a knowledge graph? One of the examples that I tried many years ago, and it was an epic failure was putting the actual data into a knowledge graph. So you go, knowledge Graphs is a database. This is a really flexible database. So why don't we just bypass this disconnection? Why do we say metadata and context and all that sitting in my graph and the physical data sitting on a storage lake or in a relational database, why can't I just push all my data to the graph itself? And back then what I found was it wasn't really designed for large volumes of rows.

The industry has changed a lot. Think about it. Ten years ago, even five years ago, the scalability of graph database systems were as well. And they continue to improve and we'll continue to see this. But I for one, this was actually my PhD and this was my, this was the company and this is the technology we have at Data World is all about virtualization. Your data, your transactional or analytical data is going to live in your data warehouse, data lake, whatever, in your snowflake and your bigquery, whatever. I want to be able to integrate that with other systems. So you can have a Knowledge Graph layer on top of it, which then you would, you can map it and then you can virtually access it. So that's stuff that we do. For example, that was my PhD over 10 years ago, which was like, oh, I can take graph queries and using the mappings to be able to translate those graph semantic queries and then translate them to the SQL queries underneath and be able to go execute on a different source such that you don't have to go move all your data centrally into a graph database. You want to have virtualization and then there's some things that you want to have, as always, right? Hybrid, depends what some things you do want to go move. Metadata in general is usually smaller. So I think that natively will live in a graph just because you extract metadata from so many different sources and the metadata gets connected, especially because things move and metadata is keeping track of all those movements. So that naturally is a graph itself too. So.

But I've seen this wave a couple of times and I think one of the vendors, the spin they're putting on now is active metadata data. And the pattern is fairly simple because when we talk about transforming data like we do in data management, there's two patterns. We write code to transform it and we try and extract the meaning from that code into something where we can see it, put it together and understand it, which is the current DBT world again. Or we go the other way, which is we define the relationships and we define the semantic and we define that graph effectively and the code's written for us and we tend to swap between those two worlds. And I think my view is, yeah, we're going to go back to this active metadata, this config driven code again because we go in waves, the abstraction, the writing code, and then try and suck meaning out of the code. The creating a schema and trying to reverse engineer that schema into a data catalog is where the pattern always ends up. We always end up doing the easy work and then trying to reverse engineer that into the hard piece. Why do you think that is? Is it just because it's easy to write code? Like you said, we know how it works.

The whole point that we started off with, it's just technology and people don't understand the business, they want to disconnect. This is where I think LLMs are a godsend right now, because it will help us being able to make us more productive to understand what things mean. So just here's the exercise. Go off, ask a bunch of folks who work in different business units and literally ask them what are the questions that you're trying to go answer today? And just get a list of those questions. These questions are what we call competency questions. They're things that they're trying to go answer now tell the LLM, here are the questions that people are trying to go ask. Create an ontology, create a business glossary, create a semantic layer out of that stuff. It will generate something for you. It won't be perfect. That's why you go in and then you try to figure out what it means and then you Draw it on the whiteboard and you go back to these people and you're saying, hey, is this how you see your world? And then that question that you're trying to go ask, you really basically traverse the graph. That's what you mean. This is something that we can now very easily go do because the LLMs have keep us very excited. There's so much hype about lms and yes, we gotta be careful about all the noise and all that hype, but it is a fact of life that is making everything so productive. This is a pure example of how to go do this. One of the other things that I've been working on so much when it comes to LLMs and knowledge graphs is to really understand what is the accuracy of LLMs when it comes to question answering. So we're talking about, oh, I have my data in a SQL database, I want to be able to now write questions and that were the answers in a SQL database. So let me use these LLMs to translate text to SQL. This is where you should be really careful because that can generate a bunch of stuff. And so a lot of the research I've been doing has been to understand that accuracy and then also to say, hey, if I invest in knowledge graphs that has all that context, how much does that accuracy improve? And all the research and we created this whole benchmark that the difference between using a knowledge graph and not using a knowledge graph is 3x. Based on the benchmark, you have a 16% accuracy of all these questions on enterprise schemas. 16% accuracy if you don't use a knowledge graph and if you use a knowledge graph that jumps up to 54% just with basic prompt and stuff. So two things. One, knowledge graphs invest in knowledge has that really clear context that your business needs. And second, the LLMs are there to make us more productive. By using this, we're going to get out of our technical comfort zone and be able to start doing more stuff on the people side and not get overwhelmed because we have a tool that can help us automate some stuff.

Yeah, I remember back in the old days when I worked with analytics teams, we used to talk about mixed models. You'd run a bunch of different models, you'd see what the fit is. And then the trick that most of the teams did was when they were explaining to a stakeholder what the model was doing, they typically show a decision tree. Now, the decision tree model wasn't the model that was being run and productionized, but it was close enough to what the the model they were using was actually doing that. They could describe the flow of data, they could say, oh for the segmentation. This thing it's doing is picking up income and then it's basing it off gender and then location. It gave a story, a visual story that could understand it. And I think LLMs are going to give that to us. But like we did back then when you had to test all the different models to see which was the best model for your context. It's the same with LLMs, the text to SQL ones we're playing with at the moment, we're seeing how good it can be and how bad it can be. But the thing is it's not consistent. So simple things like ask it the same question three times and get back three different SQL blobs. Yeah, that's a test to you that actually it's not accurate.

In the benchmark that we did. We actually did test this stuff because you're non deterministic systems. It's not like I give you a question, I get a generated query back. I always give you the same question. You're always going to generate that in the Benchmark. We have 43 questions which we ran for three weeks non stop between 30 to 300 different runs because we also wanted to see the percentage of the accuracy that they get. So this is one of the things that we needed to keep in consideration. Definitely, definitely. Yeah.

And it's valuable, but it's not a silver bullet. 100% just to close this out. So does that mean if we think about knowledge graphs in terms of the pattern of nodes and links and in terms of the technology that enables us to create those nodes and links and visualize them easier than if we're doing it manually ourselves? It's been around for a long time, like you said, but it really hasn't been adopted in the data world. It's a second or even. I'd say it's even a third class citizen. I'd say it's a powerful pattern, but one that is hardly ever used in the data domain. Do you think that LLMs is the thing that's going to make us become a first class citizen?

100%. Our CEO Brett Hurd says like startups are grit and luck. And I actually think that from a research perspective that applies, especially if you're. This is my goal, what I want to do. I've been at this stuff for semantics knowledge graph for almost two decades. The web was something that changed people's lives. The iPhone changed some stuff. Right. Social networking changed Some things LLMs has come in, it's going to change so much. My bet here, and this is what I'm literally betting on this now is LLMs are like that godsend saying, okay, this came out and this is going to be one of the motivators that are good people going to realize we have this problem. How do we solve this problem? People are like knowledge graphs. That has always been that we can go use that. And it's going to be much more critical in the enterprise because in their price. What is the biggest concern of LLMs? Hallucinations. And then a lot of the topic, the conversations around LLMs are about creating all, look at these rag architecture stuff, it's all about text. But really people are not even talking about that much about SQL because that's where a lot of our enterprise data is. But they're not doing that much in there because they know that's a huge issue about the accuracy. And already our research has been pointing that by investing in knowledge graphs you're increasing the accuracy. And this is just going to get better and better. This for me is the point in time. That's why I'm super excited working so hard on this because this has literally been. The gods have sent us something to say. This knowledge graph work is going to start booming because of the LLMs.

And if we take that example of our 70,000 SAP tables, maybe we don't go look at the query logs to see who's querying those columns to see what's most important. Maybe we look at our chatbot that we're using to answer questions. And whenever that bot, wherever that LLM is using some fields and of those 70 thousands on a regular basis to answer a question, that's the time we need to go and extend it out with, well, form descriptions with those relationships with the aliases. Because we know those fields are the ones that are getting used time and time again by the LLM. Therefore they're the ones we have to enhance. Enhancing them have more value to us because we're answering, we're helping more of our consumers. Maybe it's less about query logs now, more about where the LLMs are hitting and then how do we apply knowledge graph there and again going back to hopefully define config, have the config create and run the code for you. It's all about that context, it's all about that config. It's not about the lines of SQL that you're generating as a human because they're abstracted, they're Divorced, they're siloed by default. And yes, you can reverse engineer them, but why would you, why wouldn't you just do it properly at the beginning? I always talk about the web and I talk about iPhone. Moments I always forget about social networks. I'm going to steal that one from you. Web, iPhone, social networks, LLMs. Right.

And that's just been the last 30 years.

Yeah, we just got to realize that right now the LLMs are Newtons. They're horrible smartphones that don't work, but at some stage they will. The iPhone will appear.

Think about 1991, 1992 when the web came out. How many people are like, oh, that's a fat thing. Pay attention. And look how it changed our world.

I still remember the. The modem dial up and nobody in the house been able to make a phone call because you're surfing the web. It's a funny memory. Excellent. I think I understand knowledge graphs a lot more. I definitely understand ontologies and I'm definitely going to LLM trick.

Right.

I'm going to go and give it a bunch of questions until it taught me some of those things so I can get more examples that will help refine my understanding of that. But key thing is we're still working in a complex world if we're trying to combine knowledge graph technology with the rest of the stacks that we use in data management. And it's still not a common thing to do. So hopefully that does change. Hopefully we can leverage that out. Excellent. All right, thank you for coming on the show and I hope everybody has a simply magical day.

Thank you very much.

Data Magicians was another agile data podcast. If you'd like to learn more on applying an agile way of working to your data and analytics, head over to agiledata IO.