Put Semantics Upstream, Not in BI #substack #shorts

When you codify meaning in a model that exists. So I might have glossary terms with definitions, metrics with definitions, those like as semantic assets, and then they're going to build them in code. And so we're talking about, okay, when you create it in YAML, can we reference and create a tag in Databricks that has the UUID from the catalog? So when the collector picks up the metadata from Databricks, the UUID is there for both, and then we can run a job that creates the relationship between them automatically. So that's what we're thinking about now is. We have discovery, we have definition, we have codification, and then we have the serving layer. And so we're definitely opinionated that semantics exist upstream of the serving layer. The BI people all want us to build our semantics in there, and I'm like, no, you can read my semantics. I will feed you my semantics, but I will not build them there. And, and, you know, and then because the semantics live upstream, we can serve them. So we have to think about how does it get carried along? Mm-hmm. Okay, does this exist here? And then it's become a tag in Databricks, and then how does it get fed into Tableau or Power BI or Omni or whatever we're going to use. And so part of the problem I've been facing is a lot of BI tools that are like doing really cool things assume that the person creating the meaning is the person doing the viz. So like building in our semantic layer, I'm like, no, the meaning happens earlier and by a separate person. And I'm so frustrated because I'm like, you assume they have enough context to create the meaning at this time. I need to create it way earlier and differently, and I just need them to codify the meetings. I'm very frustrated that a lot of current tools, I get into it and it won't work with our process.