Listen now | Recording of AURA - Juha Korpela & Shane Gibson on 31st July 2026

So yeah, let's, uh, let's get going. What's, what's top of mind for you this month?

Well, Shane, um, I mean, context. Context is the word of the year, right? And, uh, I've, I've been doing, um, all sorts of context-related work recently a lot. And I mean, it's coming out of my ears. I mean, I love it, of course, you know. Well, I'm, I'm starting to ramble immediately. We just started. Is that okay?

Ah, that's fine. Look, wait, well, I'll keep an eye on the chat. Anybody's got a question, jump in with a question, that'll steer where we go. Otherwise, as it says in the brief, we'll just, uh, we'll talk shit about what's top of mind at the moment. And yeah, exactly, context is definitely top of mind as well.

So exactly, exactly. Yeah, it's just a tiny thing, Shane, because, uh, you and I, we've been working on data modeling things. And if, uh, anyone's listening, you might know or might not know yet that Shane and I are also writing a book on data modeling. Hooray! Which is going to be awesome. once it's done. But I've done data modeling now for a long time, about a decade, I think. I've been in data for 15, 16 years. About the decade, the last decade of that has been sort of focused on data modeling things. It's been very at the core, and I love doing data modeling, super fun. But a few years ago, I was working with a startup, and in that startup, we were sort of figuring out how to market this data modeling thing, right? And this was, you know, 2020, 2021, before AI, like before LLMs, right? And before this new context washing everything. And one of the marketing guys told me that, you know, data modeling doesn't sound sort of sexy, it sounds old-fashioned. How about we just come up with a new term, new kind of category for these things that we're doing? I said, oh no. And he said, I have an idea. Let's start calling this contextual metadata.

Oh.

Well, back then I said, oh no, I don't want to do this. It's data modeling. I've always done data modeling, so why can't we just talk about data modeling? Everyone knows what that is. No one knows what contextual metadata is. But I mean, now, if we had come up with that like last year, that would have been completely different.

Oh, would it? I mean, it's just, you know, it's rebranding to a degree. I mean, we've always had, you know, when we talked about data warehousing automation tools, you know, because I've been around for a while and we've been through waves, you know, we talked about active metadata. We tried to describe something where you could define the context and have the, the tooling then generate the code, um, and we never really came up with a term that that fit, fitted. So we were like metadata generated or metadata driven, active metadata. Never really landed. I mean, for me, I think about data design because in my head that's what I'm doing. I'm like designing data like the way I design apps or the way I design dashboards or the way I design a house. You know, I'm designing something that's fit for purpose, but I've never really tried to push that as a new term because, as you said, everybody talks about data modeling. Although when you say to somebody data modeling, they think physical data modeling, right? They always think physical. They don't think conceptual. They don't think logical. They don't think at a business level. Um, and so that's kind of interesting for me is that I always think data modeling goes straight to physical modeling. So if you said to somebody context modeling, like, what's that? What's that going to look like? Like, what, what does their mind go to? Uh, and yeah, there was a big scrap last year between semantic and context, right? Which word was going to win?

True.

I now think of when anybody says semantic, I think of a BI semantic layer, you know, the layer that's at the end of the data between the data and the last mile tool. It's creating some metrics, maybe definitions, it's creating a very light definition of concepts, but nothing that's got a high level of context. It's late. You know, we should be defining that context at the beginning and hydrating everything else off it, not slapping meaning and understanding at the end of the work. So yeah, I think as a market, right, we still— we're in this cool space right now where nobody actually knows what the answer is and everybody's experimenting. And yes, lots of vendors are buzzwashing, but when was the last time we had this kind of disruption? And the data domain life.

Exactly. That's cool. Yeah, yeah. I mean, it's good, kind of good times to live through this. It's also very good times to be in, in this sort of context-related area of data models and knowledge management and so on and so forth. So I mean, busy times, but good times also, and sort of good to see that there's a lot of interest in these topics. And, you know, but I, I, I also noticed that it seems to me that context kind of won the terminology context, as it were, that semantics is kind of one thing, but overall to me it feels like that context is sort of a wider term, that it's all sorts of information that usually an agent, but why not a human being, might need to perform a task, right? And if the task involves, let's say now, utilizing data in some way, consuming data from whatever place to do inference or make decisions or, you know, whatever that might be, you need metadata around that data, and that metadata is context for that data. So in order for that agent or a human being to perform that task based on that data, they need metadata around it. One part of that metadata is, in my opinion, the semantics of that data, meaning, you know, What does a customer mean when this says customer at this table? You know, what does CLV mean? Oh, customer lifetime value. Oh, how has that been calculated? But as you said, the basic semantic layer idea is just that you're kind of gluing on top of everything definitions, usually only definitions of those metrics, right? So when people talk about semantic layers, they talk about layers in which metrics are defined. Customer lifetime value is calculated like this, but they don't talk about anything relating to the term customer itself, which I think is very funny that you put a lot of effort, a lot of thought, a lot of processing power into defining and maintaining what customer lifetime value means, but none into defining what the customer is in the first place. So it sort of, you know, feels a little bit weird, but it is what it is right now.

Yeah, and I think, you know, semantically, semantics shouldn't be the last mile between the data and the last mile tool, right? That's that's actually not what it should be. But if I ask anybody what it is, uh, that, that's what I get back. And yes, some vendors are trying to blend it and talk about semantics in a different way and back to its more truer meaning. But I go back to, you know, if it walks like a duck, quacks like a duck, then it's mother duck. I mean, no, it's a duck, right? Um, and so yeah, you ask a bunch of people that were, you know, that what do they mean when they hear that term. And then, you know, it's like branding. Everybody's— you know, your branding isn't what you say it is, it's what people say about you. And so I think with me, context is now, um, the thing. And then of course we've got library sciences and ontologies and taxonomies which have been well defined, um, for many, many years. I mean, their domain seems to have— well, I suppose their domain is around definitions, right? So actually they eat their own dog food and they actually defined what their thing was. And again, I'm seeing a lot of ontology washing, you know, uh, and, and I tend to do it. Like, I kind of look at a glossary and I go, oh, is that an ontology? And then I think, ah, that's not. And, and you've done a bit of work around that, right? A bit of research around, uh, where you see ontologies fitting compared to context and metadata.

Yeah, yeah, yeah. I've tried to climb that, uh, learning curve, and I've put a lot of effort on that over the last maybe a year and a half or so. Massive thanks to Jessica Talisman, who has been, first of all, writing her excellent Substack, which is my kind of favorite source for anything related to ontologies. But also, she's been kind enough to answer some of my questions, because I like asking stupid questions. And she's been commenting on my posts on Substack, and so on and so forth. So it's been really, really educational. And I I am sort of a true believer nowadays in that idea of the semantics and the ontologies, as you said, as they have been defined for decades now. I think there are standards that exist right now that are good, and, well, you know, they are known to be good because they have been already used in some various contexts for decades now. So, I mean, why wouldn't you use those standards? Why would you come up with your own? I don't know. There are issues, of course. Some of them are related to that learning curve, which Jessica is doing a lot of work on, sort of making that a little bit easier now. But when it comes to actual ontologies, I am sort of not anarchistic, but slightly rebellious relating to the term ontology, because I feel like the work that we are doing, actually writing the book on modeling business concepts, is actually modeling an ontology, because an ontology is made of entities and definitions of those entities and the relations between those entities. And I know that, as you said in the beginning, a lot of people, when they talk about data modeling, they talk about physical data modeling. Physical data modeling has nothing to do with ontologies. There's no ontological information there at all. But when we talk about conceptual data modeling, like you do with with the book that we're writing together. In that context, data modeling is ontology modeling, and I've been trying to write that down in different ways, in different places, in my Substack and some other articles and whatnot. And some people seem to be grasping the idea. I don't know.

Yeah, look, when I— because I was like you, I thought, okay, I need to learn some new stuff. So I started reading on ontologies. I found it hard, hard to read.

Very hard.

like the language, it became like— I don't like academic papers, right? I kind of just, for some reason, when I start to read them, my brain turns off. And I found the writing around ontologies kind of in that bucket, right? So I found them hard to read and digest. But after I read a bit, I was kind of sitting back going, it looks like concepts and defining concepts and relationships. Like, it can't be that simple because we've had conceptual modeling in the data world for years. And if ontologies are really— that's the core of it— why have we had 2 domains with the same patterns but done very, very differently? And then I got confused because then we go into taxonomies, and then whenever you talk about ontology, you, you then get RDF and OWL. And for me, they seem to be like technical implementations. Like, if I wanted to define an ontology, I would use those patterns as a technical way, just like if I was going to do a physical data model, I could do it as a star schema or third normal form or, um, you know, Data Vault, right? I have a bunch of technical ways I can store it. Uh, and then there's the terminology thing. So I think— I can't remember who it was, but maybe it was Marco Walwyn, or maybe it wasn't, but, uh, he— somebody was writing about— they used the word concept Then from an ontology point of view, the use of the word concept is incorrect. It's been defined and this is how it's defined. When we talk about conceptual modeling, we're not actually obeying those rules. Then the last one is when we talk about conceptual modeling, we define a concept and we define a relationship. Where I typically see us weak in the data domain is around the definition. You know, our definitions of a concept are very, very light, and our definitions of the relationships are very, very light. And one of the things reading Jessica's stuff that kind of came out for me is there are effectively tests within ontology around that definition, you know. So the way in my head I think about it is— I could be wrong, but this is how I think about it now— is when I have this definition of a concept, of a thing, Can something fit in and pass that test or fail? That's the key. And when I'd say there's a relationship between things, can I then test that this is meeting that relationship or not? So, you know, customer relates to product, right? That fails every test because, because the word relate isn't testable, right? It was like they're just— they kind of not stand next to, you know, customers in a store and there's a product on the shelf, it's related to them by distance. That's not what what they're talking about. They're talking about really strong definitions of those relationships. So I think there's a lot we can get from there. But then just to kind of close out my thinking on this, when we moved into the data lake world with Hadoop, big data, and then we moved into the democratisation or decentralisation of transformation code with dbt, in terms of the data modelling world, we kind of made a mistake in that a bunch of people, and I probably was one of them, became grumpy old people who said, oh, that's not how you do data modeling. And we didn't teach people how to do it. We said, oh, you just leave it to us because we're experts. And I'm kind of seeing in the library science or ontology world the same behavior. And again, just looking at it, distilling it, could be wrong, don't think I am, that there is this whole domain expertise, trust us, we know how to do it, you can't do it. And it's kind of right, but it will lose because we know that decentralization and democratization always wins. So I'm kind of intrigued to step back and watch what happens on that side of it. And then of course you've got all the vendor washing.

Yeah, I fully agree. I have also kind of identified similar undercurrents, shall we say, in the ontology discussion that, you know, this is our thing, this is how we've always done it, what are you doing here, you know, messing around with our terminology. which is exactly what we said when dbt came up with the idea that the data model equals a SQL query, basically. So I have personally been writing several rants about, you know, how I hate the fact that dbt started calling queries models, and I also hate the fact that Databricks started calling Unity a catalog, but that's another topic. But, you know, there is some of that, and I agree there is a risk of doing running into the same mistakes that us data modelers did back then. It's also, I think, inevitable because of the vendor washing that, you know, when Databricks says the Genie ontology is an ontology, then okay, you know, there's nothing you can do anymore. They didn't ask an ontologist, you know, they just built it, and now engineers will be using it, and they will be talking about ontologies to each other, and the terms just get out there, and they morph into something completely different. You can't help that. What we can do, I think, is to keep the discussion going so that at least there's a chance for someone who's interested in this, this sort of backgrounds and these concepts themselves, and what do these methods mean, how they actually work, that this content for that actually exists somewhere, that you can actually run into it, you can listen to these sorts of discussions. figure out, oh no, I've been doing this, but now I've understood that, hey, I'm actually doing, you know, taxonomy or whatever. But it can't be helped that the terminology of any sort of niche area of expertise will sort of water down eventually when it's picked up by the mainstream and the vendors and everyone. And I don't think we can help it. It's useless to fight against it.

Yeah, it's like my pet one on Mendelian architecture, which—

Yeah.

By the way, it's a classification system, right? And I watch all the people vibe coding these stupid diagrams and putting them on LinkedIn, and I just lose my shit. And then I go, I'm just not going to post because I'm boosting their stupid thing. And then I stepped back and said, well, actually, what is the problem? And for me, the problem was there was— it was classification of a layered architecture, which has had value for years, you know, something that has the kind of raw-ish type data, something where the data is designed, and something where it's consumed. But what it wasn't describing is the rules. And so I did a little work around a template for layered data architectures to say, well, when you say it's bronze, like, what's the rules? What can I put in there? What can't I? And that kind of flipped my thinking to stop ranting about these things and start helping. And so I think you're right. I think picking up something like an ontology and saying, okay, if you're a data person, here's 3 things you can do that leverage the ontology pattern, but honour the actual term of it, right? That kind of goes within those rules. So, you know, an easy one would be when you define the relationship between concepts, between entities, you know, it has to be a strong— I always get my English wrong on this— noun or verb, right?

Verb.

I should have listened to English. English and stats, right? Those are the, like, I, uh, I used to really hate stats at school, so I didn't do it, uh, for very long. And then I went to work for SAS, the statistics company, for 5 years, and I was like, that was a mistake. And then, uh, with English, it's like I didn't really listen in English class either. And now with the LLMs, I need to get much better at it. Um, so I think that's it, right? We need to actually help people, uh, take patterns or take things that are useful that kind of follow the rules or the terms and honour them rather than just letting the vendors make up shit. But we're not going to stop the vendors making up shit. And, you know, we still haven't got a definition of a data product that anybody agrees with. So, yeah, so, yeah, I am a pedantic semanticist because, you know, without that, I don't have the context of what you're talking about.

I do immensely enjoy the fact that we are— we have a lot of discussions going on right now about the semantics of the word semantics, which is, I mean, just amazing. This is— how did we get here? But, you know, it's the irony of the universe or something like that.

Yeah, yeah. And Ramona's just jumped on, so, uh, no doubt we're gonna get some questions from here. Before we do that, um, one thing I've been seeing, uh, is if we think about visualizing data models, right, conceptual models at least, and maybe a logical and maybe a physical, you know, I'm old enough to remember Erwin and those, God, and what was that, EA Sparks and those horrible enterprise tools that we all used. ER Studio.

Yeah.

And then we kind of came with the modern data stack, right? We had some newish vendors come up, that were kind of doing it in a slightly different way, but not really. One thing I've seen right now is, is, uh, the number of people that are vibe coding, uh, canvas-based modeling tools. Yeah, where you can just boxes and lines, right? Uh, it's gone crazy. And a lot of them are becoming open source, which is even better, right? Because you don't actually have to pay, you know, enterprise subscriptions every year for it. But what are you seeing in that space? Are you seeing the same, that actually every team and every person is building their own Miro version of a modelling interface?

Yeah, I think that's happening, yes. And that was also sort of inevitable, because when code became almost free to create and new apps became almost free to create, the UI itself cannot be a moat, as it were. I mean, what the data model is visually, it has boxes and lines. It is essentially a graph with nodes and edges, which, you know, has interesting philosophical kind of interpretations, coming back to the discussion we had moments ago about what the difference between an ontology and data model is. But disregard that for now. It has boxes and lines, right? And if you want to create a data model, a diagram of data model, you need something where you can place boxes and lines. And now I can open Claude, and I don't know anything about actual UI design or any web applications or anything, but I created for myself, my own purposes, a— what did I call it— a data product mapping tool that shows, you know, data products and which is a source of what and so on and so forth. And I bet I could create a data modelling tool that creates the boxes and lines diagram in, I don't know, a couple of hours. So there's no moat there for a vendor anymore in the UI. Now the big question is, I think, and has been, and When I used to work in this business of data model tooling, what I was also kind of sort of trying to drive was the idea that the big thing is the repository of those objects, of those models, of those entities, and of those relationships. What do you do with those? That's another thing. That's not a UI thing, you know, it's okay, it's boxes and lines, but boxes and lines are just a visualization of kind of statements that you have. a customer makes an order, that's a statement basically. It's a triple also in the graph world, but it's a statement that we can sort of store in our sort of repository of facts, and then we can pull it out in different ways, one of which is sort of visual diagram. So I think the interesting part in data models and ontologies and everything is in what do we do with the data. Well, it's sort of metadata, but the actual kind of information that exists behind that diagram rather than the diagram itself. And I think that these sort of vibe-coded diagramming tools like mine that I did for data model design, data product design, these are sort of one-off things. People don't generally create kind of applications that have persistent data banks of all the different models that you've created for different organizations and all sorts of metadata for everything. It's just that, okay, I need this visualization right now, so I'll just, you know, boxes and lines and some words on the canvas. So that's the easy part, but that's not the interesting part, I think.

Yeah, I come back to everything's context now, and then it's how do you capture it, how do you store it, and then how do you use it?

Artifacts, yes, yes, the artifacts are the point. Yes, the artifacts are the diagrams, but the data content, yep.

Yeah, so for me, the, the artifacts are visualizations that help you tell a story, help you understand something quicker than looking at code or looking at text. And so that's why I struggle with this, like, yes, I can use a canvas to draw the data model, but actually that's not how I think, because there's a whole lot of work I have to do before I get those nodes and links. The nodes and links are really just giving me a visualization back where I can look at it quickly and go, yeah, That's what I think it is. And then what do we do after it? So what happens is often those nodes and links are done as documentation upfront and then nothing's generated off it, right? A human then goes and writes code in dbt, or it's done at the end once we've already made all the decisions to visualize it really, really late, right? And the cost of change is so high. And so for me, it's about how do you use that, those bits of context in ways that are automated now. So for example, I'll take my information product canvas when I work with a customer and I'll put that into Claude and it helps start the journey of building out the information product for me. And we just had this really thing, weird thing, well, not weird thing, interesting thing where we needed to do some work around fine-grained security, right? So we needed to bring in more levels of fine-grained security around the information products themselves and the data within it. What Nigel would— because I was using the canvas as the starting point for the design process, the context of the personas from the canvas was being captured. And then when we were presenting the products, we had a hub and you could see each product and who the persona it was targeted at, right? Because it just made sense. So it captured who this was for. So let's surface it and you can filter by, you know, the type of persona in your organization. So what Nigel did was then he then used that to bind the fine-grained security against, because it was context that had already been captured and was reusable. So instead of going and creating roles and a whole lot of other stuff, we captured that context that was already there and we just reused it. And so I think that's where we're going to get the change in the way we work is by capturing these things as lots of little bits, storing it in one, what I call a context plane, so in one place, Mm-hmm. and then hydrating it time and time again Yes. And providing a feedback loop. Yeah. Oh, well, actually, we've got a whole lot of users that are coming in and using information products, and they're not personas we captured at the beginning. Okay, so now I need a feedback loop to say there's more personas using that product. And then you can, you know, it's that create it, hydrate it, feedback loop, update it. That's kind of what I'm seeing. So yeah, I think, I think we're about to change completely the way we work, and that's going to be interesting.

Yeah, I fully agree, I fully agree, and I've always thought so, that the amount of data modeling we've done over the years, and what I myself have done, the amount of data modeling that has resulted in just those diagrams as separate documents or pictures somewhere, and the amount of knowledge and information that we've sort of lost there, That diagram contains these statements, these facts that we should persist somewhere as part of this. And I am also completely with this idea of a knowledge plane, as I like to call it, a context plane, if you now want to use the sexier version.

Oh, we're going to have an argument. Is it knowledge or is it context? Is it wisdom? Is it insight, understanding?

Anyways, a separate plane on which this information needs to live in. And then, as you said, hydrate it. Yes, okay, present it as a graph or as a diagram when needed and present it as code for some, you know, application, whatever that might be. And even though I sort of love doing sort of visual data modeling with those diagrams and everything, I nowadays think that the point of the diagrams is simply to act either as sort of information capture method, because it's easy for humans to see that, okay, this connects to that one. So it's easy to kind of visualize that as a diagram. It's easy to draw it. So we create, or actually we don't create information, we capture information from people's heads by drawing that diagram. Fine. But the process should never end there. We should take the information from the diagram, put it somewhere safe, persist it. And then later when we need to tell people, this is how our organization works, this is, you know, what trial customers are and how they relate to subscriptions or whatever, then we can sort of take that information from there. The original diagram can be lost for all I care, that doesn't matter anything, because we can then sort of generate a new diagram based on what we had stored earlier. And I think that everything— if I can only think back to many of the organizations where I created, I don't know, dozens of conceptual models around various topics, if only there had been a single repository of knowledge, of information, into which all of that had been fed, That would have been cool. But now, you know, if you have 2 dozen data model diagrams and you change the word customer to client, you're not going to remember to change that in every single diagram. So that's not going to work. And this is exactly why my argument about this sort of live-coded data modeling tools, because that's not the part that people usually do. They just want something to create a single diagram with, and that doesn't help.

And I think the other thing is, is that context plane in my head has to hydrate everything else.

Yeah.

So when I— if I decide to rename it from customer to client, because that is actually the term that is resonating within the organization, I actually want it to rebuild all the data for me. Yeah, I don't want it— I don't want any LLM I'm using to have to go and talk to the context plane to figure out that there's a mapping of customer to client. Like, just blow the data away and rebuild it and call the fricking column. Because what we know is if we point the LLM at a set of data that's physical, that is well named, it does a much better job than if it has to go through another layer. So why do we have these BI semantic layers where, you know, we rename the column into a pretty name? Just fricking make the column a pretty name at the beginning. Now, too many years ago, we couldn't do that. I remember we like— What was it? SQL Server, one of them had an 8-character limit on the column name. There was always constraints we used to have that we don't have anymore. Then the feedback loop. For example, when I go and model something, I work with Claude and I have a guess of what the conceptual model is going to be. It now goes and looks at the actual data and it validates it for me whether we've got it right or not. It goes, well, we think this is the relationship and now I've gone and looked through the data and actually, 80% of the times that is the relationship, but there's 20% where it's not. Or we have a process, a funnel, right, a series of steps, um, and it goes and says, yeah, actually, you know, this percentage of the data follows those steps, but here's all the outliers. It's doing that work for me, it's giving me that feedback, and that feedback needs to go back into wherever we store the context so that it's now captured, uh, because it used to be in people's heads. So That's, you know, one of the things that I'm seeing is the ability to kind of daisy chain all these little bits of data work that we used to do separately and use it as a feedback loop or reinforcement model. But it's so new and it's so hard and I keep— you keep getting lost. It's like, oh shit, which bit's feeding what? So yeah. Hey, Joe just came on there, so I thought I'd say hello.

Hey, Joe. Yeah, I want to catch on, kind of, Take you on that comment about sort of catching those different bits and pieces of the data work. I just read today a really good article. By the way, thanks, Joe, for pointing at the Atlan page. Did you check that, Shane?

No, not yet.

Atlan is documenting what they're doing. Super interesting. And there was an earlier article by Prukalpa, who is super cool with this, and she was writing about the different versions of this sort of context plane thing that they had built At Atlan. Now, Atlan, of course, is a vendor who tries to sell this solution to others, but she was writing about the things they had done internally. And also, the website that they have now up is sort of documenting that process, which is super interesting. Anyways, one of the kind of key things there was that they are trying to find all the places in which context is born and get them to feed into that sort of central context plane, whatever that might be called. And I think this is one of the kind of things that we sort of miss. And I've been ranting about this in many different ways and forms over the years, and recently on my Substack and everything, that we're so focused on individual solutions that we lose a lot of— first of all, we lose the big picture of everything. No one knows what's going on. But we also lose these pieces of information that are discovered during the processing of that individual solution. We should build everything so that from every single kind of project, every single solution development initiative that we might have, whenever a new little piece of information is discovered, we figure out that, oh, actually we have now trial customers who might have subscriptions, but they don't pay anything. That information should be immediately fed into this sort of enterprise knowledge layer in which it would be reused for other projects. And that's also the only way, I think, to maintain that kind of context layer. We can't build it so that we have some sort of, you know, head context engineering team or whatever who spend all their time kind of typing new facts into some knowledge graph. That's not going to work. It has to be built so that the feedback loop also goes up from the individual initiatives into the kind of centralized context layer thing. And it's not centralised actually then, it's sort of, you know, bottom up and top down at the same time.

Yeah, I think, yeah, if we tried to approach it like data catalogues did with data stewards and data governance, yeah, we got, we got stuck, please. I think this is where tooling will help eventually, in that I think that information needs to be captured as we're doing the work in the tooling. So if I think about when I'm working with Claude, the number of conversations I'm having and the amount of context that we're sharing. If I have to then take that and put that somewhere else, if I have to remember to distill it and put it somewhere else, yeah, it's not happening, right? I carry on. But if there was a way, as I was— we were doing that work, it captured it for me, um, then that would be useful. And I was working with a team a while ago, and they were starting their AI-first journey. And, um, one of the things we were experimenting with was they were using the LLM as a diary. So they were at the stage where, like, I have a maturity model where I talk about, you know, ask AI, where you just use it to ask questions and that helps you a little bit, you know, what's this code doing, what should I do here, but you're doing all the work. And then you move into assisted AI, where it's either watching what you're doing and making recommendations, or you're using it a lot heavier to do the work you want to do. And then automated AI, where it just does it without you. And so they were right at the beginning. And what was— what we experimented with was, well, as they were doing the work, using LLMs in an Ask AI format, could we get them to effectively diary on the side with the LLM what they were learning? And then could we use that as an input to then generate this context layer, uh, context plane, right, the knowledge, um, rather than them having to document it and put it into wiki or trust, you know, getting them to have to harvest all the code every time we would understand what it was doing. And that We kind of did that as an experiment at McSpikey and never really got to the end of it, but I'm intrigued by that. Then the other question is, if you're an expert in your business, in your organization around business process or conceptual models or any of those things, how happy are you going to be to have that, that understanding and knowledge distilled out of your brain? Because actually that might be your last moat, right? Like, that might be where your value is.

Yeah.

Not in an evil, I'm gonna go and— like, I once knew a guy, he was a contractor, and I'd worked in enough organizations where he kind of came in and out of these different organizations, and every time his code was different. Like, just the structure of it was different. It was like he, he got it and put it through. It was pre-LLM, and he got it and put it through and said, write this code like a pirate, write this code like in French. And, and I was having a beer with him once, and I said, Like, why do you do that? And he goes, because then nobody can read my code. And I'm like, serious? And he's like, yeah, yeah. And so I think it's gonna be interesting to see how many people obfuscate their knowledge because they, they— that's where the value potentially is. So, uh, yeah, it's going to be interesting time on that one as well.

Yeah, yeah. I mean, it, it is already being said everywhere that knowledge is the, uh, kind of moat for enterprises. It's not that who has access to the fastest models, LLMs, whatever. It's actually who has the context to give them. And this information that you have about, first of all, your own organization, how it works, that's of course something that you need to keep to yourself, especially those parts that are unique to your organization. I think it was Vin Vashista last year at Sales Executive Week. He was speaking about the cost of data quality problems. And he said, yeah, I mean, if you have bad data somewhere, some transactions have faulty numbers, that means that the decision that you make based on that data is wrong once. All right, so fine, you suffer the consequences from that once, and then you go forward and fix the data quality. But if your ontology is wrong, you will never make a right decision again. Because everything is based on the context that you have about your organization. And that's sort of interesting how different the consequences are between kind of data problems and then sort of knowledge problems. It sort of makes sense. But one thing I wanted to say about gathering the knowledge also is that I still kind of would like to see more emphasis on perhaps not on the idea that we expect ourselves to extract information from all these humans and then replace them, but rather kind of capture information that they don't even themselves kind of know that they know, as it were. There's all sorts of tacit information that is not that people kind of keep it to themselves knowingly, It's just that they have some sort of, you know, way of working. We've always done it this way, but they don't even know why it's being done that way. And capturing that would be beneficial, not just for the organization, but I think for everyone, all the people also in that organization, to have that information somehow visible that has never been visible, not even to the people participating in the work.

Yeah, I mean, there's this idea of decision tracing, which is—

Exactly.

When I looked at that, it's figuring out how decisions are made from the data that was used to make them and what happened when that decision was made. So the example I read was, kind of resonated with me, is the policy says you can only provide a 15% discount to your enterprise customers at the end of quarter, but everybody knows that if you've got a million-dollar deal and you go to the sales manager, they'll give you a 20% one.

Yeah.

And so the policy says 15%, but the data says at the end of quarter, there's always a 20%. so it can infer it. In the last session with Anna, she kind of came up with something that I've been kind of mulling over. So she talked about the fact that they replaced their hospital system, and, you know, the majority of their core business processes go through it, and they don't have access to the data yet. So they've got that whole, you know, replacement of system and new data problem. But she started thinking about all the all the context that was being captured as part of that replacement of the system, because, you know, all the business processes were reviewed, all the— everything was being kind of discussed as you move it from the system to that. And so she thought, well, why don't we treat that context as the way we would treat data? Why don't we create context transformations to capture it and transform it and store it so it's reusable, just like we do data transformations? So I'm really intrigued that when we do the next, next Aura session, like, see where she's going with that, because I keep thinking about it. But then I think, well, when we do data transformations, the data's there, like, we know where it is. There's been decades of where it lives, how to get it, how to transform it. If I think about that context of how an organization works, yeah, there's a wiki, and lots, lots of vendors are now trying to, you know, scrape wikis and that kind kind of thing. But yeah, there was an Anthropic paper around this which I kind of talked about pre-starting the recording. Pissed me off because it kind of had made one line around star schemas, and now everybody's telling me you have to use star schemas for LLMs, and that's just, in my view, bullshit. But one of the things that I did pick up from it that I found interesting was, um, they said they tried to harvest all the context, uh, using an LLM out of all the stuff they had, and their evals were low. As soon as they had a human write the kind of rules of the context, they got a better response. So, you know, I'm always a bit—

yeah. But that's the thing. That's why the book that we are writing is going to be so important. Because now let me state this clearly. I think that we cannot expect to catch all the context automatically, no matter what sort of context harvesting systems we create. I think there are going to be, because some of that information is completely tacit, some of that information is not written down directly anywhere, and some of that information needs to be agreed on. We actually have to bring people together, make them talk to each other so that they can form a consensus that didn't even exist before that session. That's what we do when we do conceptual modeling sessions. So we have to be able to also produce knowingly produce and capture that context which sits at the core of the very enterprise. And these are going to be the core entities that you have in your data models, the whatever sort of unique ways of dealing with some parts of your business that do not come from industry standard models or standard documentation or your ERP documentation or anything, or even the data. There will be this kind of hardcore of vocabularies, of relationships between terms, of policies, rules, definitions for this and that, that need to be kind of curated and possibly even completely captured by humans. And around that, it is then more safe to kind of add all sorts of context by harvesting it automatically, which cannot be done— I mean, at an enterprise scale, I do not believe it is possible for human beings to create that context, that That's right as well. So there has to be some sort of AI-assisted, you know, whatever context harvesting mechanism in place. But at the core of that, I think there has to be something that humans need to verify, need to agree upon. And that is where we need people like you and me working on the conceptual models, which are not the only way to do that, but they are a very, very good method, a method of capturing that sort of Information.

Yeah, I think the key for me is curated. I think it's going to be automation, curation, feedback loops. So the idea, you know, when we've done conceptual modeling in the past, it's a guess. You know, it's like no plan, you know, was it no plan ever passes the first engagement with war or whatever.

Yeah.

We do a conceptual model and then we go and look at the organization and the data and we're like, damn, there's this stuff survived, this stuff didn't. I think that that feedback loop is going to be much faster for us.

Yep.

And I think we're going to get better at definitions. So, and I think LLMs can help with us, GenAI. I think we can now create a definition and then get it to look at the data and tell us whether the data fits that definition or not. And then we have to make a decision about, is the definition wrong or is the data wrong, right? I think that's going to be really valuable. I think we've got to be careful about large ontologies up front. you know, when we had enterprise data models and we said we needed to take a year or two, it was right, but it was wrong, right?

Yeah.

It was actually the right thing to do, but actually our speed to market, our value was too late to the party. And so, yeah, I'm still struggling with the idea that, you know, ontologies seem to be large work up front, and I think when we get agility into that practice, then we're going to get really really good value out of it.

I think John Giles has it right with the data town plan idea. I think he has the basic idea basically there in that latest Elephant book, that at the very high level, there's just a bunch of different patterns of information that exist. You just kind of pick from the existing patterns. Oh yeah, we actually deal with documents because we have contracts or whatnot. And then you basically just write down, somehow store the terms and definitions that you use for those. The pattern of information is pretty much similar. And then you can cover on a super high level pretty much the entire enterprise at that kind of very, very core level where you maybe have, I don't know, 20 entities throughout the whole enterprise. Not that you create a massive enterprise data warehouse— sorry, massive enterprise data model. covering the entire organization at the attribute level, which is never going to happen.

Yeah, I'm a big fan of Christopher Alexander's book around architectural patterns. When I started getting into the idea of patterns, and Ramon always says to me, what's a pattern? I needed a good ontology for patterns. And so, but I love the way that he describes in his books this idea of a town, you know, and then a part of a town, a suburb, and then a street. and then a house, bunch of houses, and then a house, and then floors within a house, and then rooms within a house, and then furniture within a house. And that level of extraction coming down, and I love the way John Giles has done that with data. I don't like the way that he has extracted it out to a thing is a thing is a thing entity model, right? I've always struggled with those, and that's just because I don't think that way. And so I think that classification has value, but it's not how I think when I model, when I design. So I think again, it's this idea that there's lots of tools, there's lots of patterns you can use, pick the ones that work for you. You know, Joe's book's coming out around mixed model arts and this idea that actually there is no one data modelling pattern to rule them all, we all mix them anyway without, you know, a Data Vault bigot will always have some form of persistent staging area, typically, right? They will typically have a consume layer that's not Vault, yet we talk to Data Vault practitioners and they're like, everything's Vault, and you're like, well, actually it's not. So I think you've got a bunch of patterns, you've got to put them together, and the interesting thing right now is those patterns are moving so fast because we've got, you know, new ways of working effectively, it's coming out of the LLMs and the technology. So Yeah, it's kind of interesting, right? Like, I just can't keep up, uh, with what's changing and, and what, you know, what's useful and what's not, and time to experiment. Um, you know, so like you, where you've been playing around with the knowledge plane, right, and actually being able to build some stuff with customers and test it in reality. And I think that's the other thing is, I was talking to somebody about it last night. Um, so somebody else that's running a fractional data consultancy and they've built their own platform And, you know, the conversation was around speed of change because, you know, they kind of came along the same journey we had. And this idea that, yeah, I could use a platform that somebody else had built, you know, Databricks, Snowflake, whatever, but the problem is they're so disconnected from the people actually doing the work that the speed of change is too slow. If you're building the platform at the same time you're using it, And especially with LLMs, you find something that doesn't work, you then basically build in a reinforcement process, or you change something really quickly because you're using it and it's not working and you need it to be fixed from then on. And so I think this abstraction of vendor to practitioner to usage, right, I think that's going to cause some problems in the future. But the problem I have with that statement is now I'm saying everybody should build their own tooling and their own platform, and I don't agree with that either.

No, no.

Wow, but you know, but maybe because we can now build a data model visualization tool in 2 hours, it's good enough for me to see a model and I don't have to pay hundreds of thousands of dollars. I can build a data catalog tool that allows me to see the context, the metadata, whatever, for, for an entity without having to pay millions. Maybe that is what the change in the market's actually going to be.

Yeah, but then we get back to what Ramona said about artifacts being kind of that, that, uh, okay, yeah, we can build tools to produce individual artifacts, diagrams, whatever, but then actually what we need is the information behind those artifacts underneath them. And that has to be something that we can manage. And this is the whole context layer question. So how do we manage context at an enterprise scale, not on the scale of an individual agent? or, you know, individual database or individual, you know, whatever BI layer solution it is, but at the level of the organization. And I don't know, I'm still sort of waiting for a pattern, as you like to call them, to appear that will show that, hey, this is how you do it. I have sort of my ideas, but yeah, I'm working on some of those, but I mean, it's all just guesswork at this stage. I just told you, because before we started recording, I just told Shane that I have been today looking into case studies involving these context layers and knowledge graphs or whatnot. I can't find almost any, not, you know, it's empty space.

There's so many moving parts, and I think that complexity is part of the problem, and it's absorbing so many patterns that were siloed in the past and not integrated. And just think about what you said. I mean, really, it is 4 steps, right? It's how do we capture this context? How do we store it? How do we view it in a way that we can make sense of it? And then how do we hydrate it to do something else? And that sounds really simple, but then we're— we sit there and go, well, do I store it as a relational model? Do I store it as a graph?

Right.

I got some choices around that. Do I store it as a graph model in a relational technology? Because I can do that now. Or do I actually need a thing that has native nodes and links? Do I capture it from my source systems? But what about all the conversations? What about the emails? There's so many places that it's created. Yeah, we struggle enough now with, you know, 1,000 SaaS platforms in a company, and we can't even bring that into a centralised data warehouse. And then when I visualise it, I can do a map, but I can't do a map for everything because I want to visualise how a metric works. and its relationships to other metrics as part of my business strategy. Something like a metrics tree is a graph visualization, but it's different to a conceptual map and it's very different to a lineage, even though it's a map. Then I often don't want it as a map, I want it as something else. That's a good question. Can you distill it down into 5 visualization types that's reusable? Then I've got to hydrate it. My dashboard has to hydrate every one of those things. And so I start to put everything into buckets and then it gets too complex. It becomes a multidimensional problem and my mental model breaks. Yeah. So I think—

I know what you mean.

Yeah. Yeah. Yeah.

Yeah.

Yeah. Anyways, breaking down to smaller things.

Yeah. Yeah. Anyways, I think the key question for me that I like to do most of my thinking and working now around is context management. or knowledge management or semantics management or whatever. And that involves how do we get it, where do we put it, and how do we use it, right? Yeah, that's what it comes down to. And I don't think there are very good answers right now for any of those. So it's very interesting to think about these and try out things.

I think there's that 4th one though. I think there's how do we get it, where do we put it, how do we validate it, and then how do we use it.

Okay, yeah, true, true.

Yeah, yeah, yeah, because you always want to be able to Yeah, yeah, easy, easily validated as a human. Create that curation, right? Is like, yeah, that's good, that's good, that's right, that's wrong. Yeah, that— yeah, how do I get that feedback loop?

Did we have any questions, by the way?

No, no, we haven't. I was waiting for Ramona to give us a question.

So yeah, come on, ask us anything. And then, then it's just 2 guys, uh, talking about all sorts of random stuff.

That's the Aura way, right? We either get questions and then that's what we go and discuss. or we just—

I should have bought a drink or something.

Yeah, what, what's top of mind, uh, for us this month, right? It'll be interesting to see if next month whether it's still context. I mean, I'm impressed though that context is now coming out of your mouth, uh, earlier than knowledge, right? Yeah, now like, well, knowledge. Yeah, semantics, right? So I've influenced you on that one.

Yeah, possibly, possibly. Maybe you have to go with the flow, you know.

Ah, yeah. Yep, yep. So I think, uh, I think the other thing is, if, if I was in the data world with the amount of chaos we have now, um, I think there's quite a few people that are actually experimenting and sharing. Um, you know, like Ramona just said, she's in the trenches too. Yeah, completely different level, different type of experimentation than I'm doing, um, but really, really useful. So, yeah, that's the key, right? Okay, can you please talk about how a knowledge graph is distinguished from an ontology? We're 5 minutes to go. Do you want to go on that one?

Yeah, because I was just talking about this with a client just the other day. So I've tried to— and this is by no means an academic definition or anything— but the way I try to explain this is that in a knowledge graph, You know, we have sort of 2 layers. This is the closest to academical that I can get, and they are usually called the T-box and the A-box. These are sort of 2 layers on top of each other, right? And now the knowledge graph is the union of these 2 boxes. Don't ask me why they're called the T-box and A-box, doesn't matter. There's 2 boxes, right? One of those boxes contains information about types of things that exist. That's the top layer. That's the ontology. Ontology tells us that there's a thing called customer, there's a thing called order, and customer makes an order. And then the lower box, the A box, I think, is where the actual instances live. So there you have Shane as one customer and Juha as another customer, and then the specific orders that they've made, and they are connected to kind of those specific orders. But the trick with the knowledge graph is that when you have the ontology on top, you then sort of hang these instances from the ontological concept in the, in the T-box. So there's the ontology and then there's the instances, and put those together, you have a knowledge graph, which I think is super important because now we have so much of this vendor washing about— sorry, ontology washing from the vendors, and they are often calling things ontologies and knowledge graphs with only that instance layer of that. And I don't think you can call that a knowledge graph. It doesn't have knowledge in it. It's just a graph. It's just a different kind of a storage method for data. You could have a relational database, you could have a graph, but what makes it into a knowledge graph is putting the ontology on top. When you get these 2 layers, that's the knowledge graph. That's how I see it. I don't know. Now you can shoot me, Shane.

No, I haven't answered this question myself. I think Sanjeev Mohan did a good post a while ago where he went through and explained it all, and I read it and I kind of forgot it again. Um, yeah, and then I get confused between graph databases and knowledge graphs. I, I get confused in this. I think one of the, the things that I'm still struggling with is this idea that a knowledge graph can hold the values, right? I, I get that, right? So I have a, a node in my graph effectively which is, uh, customer, you know, and that might hold Shane Gibson and it might hold Agile Data Limited, right? Because they're values that are customers. And I can kind of see that. And where I struggle then is how would I use that as actual data? Like, you know, how do— and when I'm building an information product, does my knowledge graph, my physical graph with those values, replace my BigQuery table? I mean, what's interesting for me is Google have now come out with graph on top of BigQuery, so I can now take my relational tables in BigQuery or my columnar ones and I can actually make them graph. Yeah, but I'm not sure what that does for me, but I can do it.

As far as I understand, and I haven't done this, I've just read the documents and tried to understand what they're doing, I think what BigQuery does there is just the instance layer.

No, I think it holds the values as well.

No, no, no, no, no, I mean, yes, the values, yes, specifically the values, but not the concepts.

Ah, it doesn't hold the definition of the node? No, no, because that goes in their knowledge graphy thing that they've come out with as well. So, so now, yeah, and I think that's going back, I think that's the point you're making is I now, when I, when I look at my Google Cloud stack, I have this knowledge graphy thing over here, Sorry, no, I had this definition-y thing over here, which is my ontology of, hey, I have a customer and this is the definition of a customer. And then I have this graph node over here which holds the values for customer, but those 2 things may or may not be linked. And I haven't played around with it enough yet to know, have they actually coupled those 2 things together or not?

To my understanding, not, but I might be wrong. You're the GCP expert here in this call. I have no idea how it actually works.

Oh yeah, no, no, I'm just trying to think how I take the the ideas and then make them real.

Yeah.

We've got a couple of questions. Ramona made a comment, problem is mixing physical storage with logical and calling it one thing. Yep. Well, we had that with conceptual, logical, and physical models for so many years. What would you put in an article called Ontologies: The Good Parts? I would ask Jessica Tellisman to write an article called Ontologies: The Good Parts, and it would probably be a great article.

No, let me just take that one as well, because I sort of tried to write that article a while ago. What I think is the sort of core thing that we have to understand about ontology is that there's things, the things that exist, customer and order, and there's relationships between things. Customer makes an order, and there's definitions of those things. Customer is someone who buys products, and order is whatever. And those are the good parts of the ontology. Everything else is syntax. or specific methodology, or, you know, so on. But if you get the things, the relationships between things, and definitions of those things, then you have the good parts of an ontology.

Yeah, and that's again something that when we do our conceptual models, we had nodes and links, not a lot of definitions that were really robust. All right, do we see a way for a knowledge graph to be used to implement an MDM program? That's a big question and we're out of time. So maybe, maybe next time. Maybe next time. Yeah, maybe in the next DORA I'll be a bit better at asking people for questions early and then we can explore it.

So let's go on to the next one.

We can take that one away and research it because now we've got this term master data management and we've got this term knowledge graph and ontology and then how do they blend together? Uh, that's interesting.

Malcolm Hawker has been writing about this very topic recently.

Yeah, and he's a, he's a big MDM guy. Um, so we'll see from there. Excellent. All right, well, hey, thanks as always. Good to catch up. And it's just, we're doing it live now, so that's kind of cool. And, uh, we'll see, see you next month. So your, uh, last Friday of the month is, uh, when we, we do our Aura sessions together. So, uh, anybody wants to go on to the Substack put a question in for, uh, for that session when I list it there. Um, anybody that writes it up first, we'll do some research before we turn up and, uh, potentially be a little bit clearer about our definitions. Or maybe not. But yeah, the key thing is if they want to, the audience can drive what we're talking about. Otherwise, we just catch up like we normally do and have a good chat. So, uh, good to see you again.

Thanks, likewise. See you soon. Excellent.

All right, catch everybody later.

Bye.