Sarah Levy, CEO of Euno, explains how semantic layers solve data governance's core problem: nobody trusts the metrics. A fintech company she cites had asset counts that varied wildly across systems. A semantic layer centralizes certified definitions so analysts stop duplicating work and AI tools can query data in plain language.
Chapters
Just talk me through what are the big challenges in data governance right now.
Wow. So I think the biggest challenge is that so many business leaders cannot trust the numbers that their data products report. Right. You know, I think that's the biggest. You invest so much money in the data stack, in building data pipelines, data engineering teams, you know, a warehouse, BI tools, all of that, just to make data driven decisions. And in the end you don't trust the numbers. I think that's the biggest issue right now, if you ask me.
Sure, yeah. So I can say, oh, there's a big problem. If you are spending all this money on your data solutions and then. Yet. Well, actually I don't really trust the answer at all, then it's a complete waste of time. So I know one of the solutions you're sort of interested in in order to get better trust in data is the use of a semantic layer. First of all, can you talk me through what is a semantic layer?
Yes. So it's actually, you can consider it a mart or a store where you park all the certified or governed or official definitions of your calculations. For example, if I led a real estate department in a fintech, huge fintech company and one of the major KPIs was the number of assets that we own. So number of assets can be calculated from various systems in different ways. And we had actually experienced, I experienced it myself, we had about 300 assets that we managed and the numbers that we got for number of total assets from different systems ranged between 270 to 320. That's a huge mistake. So a semantic layer is, is a sort of mark where you will have the official definition of total number of assets. And if I want to know it, I will use the semantic layer to get the right context from the data. So there's lots of data out there in tables. And if you want to get the right context for this data, you use a semantic layer. It provides you the context for that data so that you can know which table you need to query to get the answer that you want.
Okay, so the idea is that you've got an official definition of important metrics that you need to calculate. Okay, I like that. And I guess if you don't care like 10% either way, then it maybe matters less on how many assets you have. You said some 270 to 320. It's like if that's good, about 300 is good enough, then you maybe care less. But if you want the exact answer, then you need an official definition. So talk me through. Are there any other benefits beyond just having a single source of truth? Like, why would you. You want this semantic layer.
So, you know, actually, right now, almost every BI tool has a semantic layer. It's not a new concept. If you use tableau, you build metrics in tableau, you build them in workbooks, you can build them in data sources. If you use looker, you have semantics. In lookml, you have the equivalent almost every BI tool. So it's like the bread and butter of analysts to create calculations, to create definitions, to build, you know, define new terms as they go. So every data system already had lots of semantic definitions. This is what captures business logic. The reason you want a semantic layer is because quite often there are lots of duplicated and inconsistent definitions. Many of the definitions are siloed or trapped in some analyst workbook or a spreadsheet or something. If you want to reach consistency and alignment across an organization, you want everyone to speak the same language. You want to build this central source of truth, this semantic layer where you have the official definition that everyone can trust. This is certified. This is the right definition for total revenues for daily active users. There might be lots of other copies out there for experiments or ad hoc analysis for things that were built and abandoned, but that's where you find the truth. This is why it's so important. It's important for two things. Because you want to know that the number that you get is the right number. And when we're facing the future, and I guess we'll catch that, you know, we'll elaborate on this a bit more. If you want to adopt AI tools, if you want to have like a ChatGPT for analytics, and ask, you know, what's the number of daily active users, what's the interest in the past month? And you want to trust the number that this tool gives you, you have to rely on some source of truth, some certified set of definitions to train those models.
Yeah, I can certainly see how if you've got lots of different analysts working on similar problems, they're all going to calculate things in similar, but not quite the same way. I think I've even done it myself. I've had to go back and calculate something I knew I calculated last year, and then I've probably done it the same, but maybe not. So having that single standard definition is going to reduce that sort of duplication workload. Okay, Actually, this leads to a question in, like, how do you make sure that you only have one version? Like, the implementation seems like it's maybe the hard part Stopping analysts doing all this duplicate work.
I mean, you touch. I think the most important part, I think today almost every organization understands they need a semantic layer. But at the same time, you rarely see well built managed semantic layers. And the reason is it's actually a hard implementation process. And let me try to summarize what it consists of. So first you need to curate the right metrics. You have thousands of metrics that were built over the years in a large scale in our enterprise. They're all buried everywhere in BI tools, in data applications, in data science notebooks. So you need to find maps and understand which ones matter, which ones are the most important KPIs you need to resolve all inconsistencies and duplications. If you have three versions of something, understand what is the right version that you want to add to this. After this curation, mapping understanding, which measures capture business value which you can delete just to keep the environment clean and clear, then you need to code them. That's a big migration process. But once they're coded, what does the workflow look like? So you cannot just tell the organization, listen, now we have a semantic layer. These are your destinations. That's what you're going to use from now on. Stop creating new stuff in your notebooks, that's now your dictionary and that's what you can use. Because things change all the time. A day later, there are already 20 new metrics out there buried in all those BI tools. So if you want to really build and manage a semantic layer, you also need to develop a workflow that takes into account that things will change all the time. I think these are the main challenges. So curating, creating it and then building a workflow that keeps maintaining it up to date, consistent as you go.
