#231 Manage Your Data Better with Shinji Kim, CEO at Select Star

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power Your data and AI transformation journey on datacamp.com.

Active metadata is one of the critical parts if you want to automate more of your processes. If you want to do more streamlined data governance, data documentation providing this context I would say is really a core part of making that happen. Foreign.

Welcome to Data Framed.

This is Richie.

Here's one of the most annoying conversations about data that I've had far too many times.

Hey Richie, can you do an analysis.

And answer this business problem for me?

Sure boss.

Where's the data? I don't know, probably in one of our databases. And then I spend more time hunting for data than I do analyzing it. Rather than grumbling about this, today we're going to learn how to solve data Discoverability. Along the way, I'm also hoping to learn how to solve other pain points like how to properly document data sets and how to avoid spending all my time maintaining a dashboard that no one actually uses. Our guest is Shinji Kim, the CEO and founder of Select Star, a data discoverability platform. She was previously CEO and co founder at Concorde Systems, since acquired by Akamai Technologies, where she worked on distributed stream processing for large sensor data sets. And before that, Shinji set up an ad experimentation framework at Yieldmo. So this mix of a software and data background makes her ideally place to help automate away some of the challenges of working with data. So I'm very keen to hear her take on this.

Hi Shinji, thank you for joining me on the show.

Thanks for having me Richie. Excited to be here.

Data governance is just such a huge topic. Maybe can you just tell me why it's important?

Well, it's becoming a lot more important now because of Gen AI. But to go back in the history, most of the data governance has been implemented and been considered mainly for organizations to meet with data privacy regulations, gdpr, ccpa, or even before just to protect the individual's privacy and to make sure that your company confidential data and customer confidential data are kept intact and is accessed or Utilized correctly. I would say that that is a part of data governance. At the same time we also see a lot of needs of data governance because people want to utilize the data. If the data is not organized or managed well, then people will say, oh, we don't have governance, we don't have a good way of saying what's this data? Or how is data actually getting processed and be available that's trustable inside the system. So I think this like term, quote unquote, govern also encompasses that. And today it is also being utilized for a lot of data management side of the issue. And that's why it's a big topic and why it's also very important. I would say today's data governance is really around providing the right way of accessing the data. And when we say right way, there is still the part of protecting the data and making sure that it's used for the right purposes by the right people. But also the right way of using the data is about giving you the indication of what data is the one that I can actually use and all the possibilities that I can use this data for. And the utilization of data is starting to increase. I would say up until the whole waves of ELT and cloud data warehouses, cloud data lakes, data has been collected a lot, but not necessarily have been utilized a lot. But I think we are now going into an era where more and more enterprises are leveraging and utilizing their data. They can finally run the 360 degree analysis of their customers because all of the applications are starting to provide, allowing users or customers to export the application data into one place to join them together and analyze them together. So when you're trying to utilize the data, it's really important to understand which are the right ways to join these tables together and which are the right curated or prepared data sets that I can actually use. And this is where also the governance comes in.

That's a really broad range of things. And from compliance, it's making sure that you're not breaking any laws through to actually making sure that all your data analysts, data scientists, actually have access to the data in order to be able to utilize it and do stuff that's going to make you money or help your customers. So I like there's such a broad range there. I think I interrupted you.

No, I was going to say and then I was going to move to Genai because there is a lot of usage of data by the data teams and almost like an explosion of leveraging data into a lot of day to day business decisions. Now we are trying to plug our enterprise data into Genai so that machines can help make us decision or help us automate operations. And if the Genai module is just looking at the data as a raw data set because it does not have the context of which data is the right one to use for which use cases, that's where the governance should really come in. To ensure that there is a good data quality and integrity in the data that is provided or is accessible by Genai or others. But also there is a good amount of context associated with it so that the data can be used for the right purposes that it's designed for and that is collected for.

That's really interesting that generative AI is providing another use case for data governance. It seems like all these large language models are the very much hyped cool thing at the moment, but actually a lot of the secret to their success is that underlying data governance stuff that's going to produce about the extra performance. You mentioned the idea of data quality and context of data. Can you go into how do you define data quality? Like how good is good enough?

So to me there are a few things about data quality. I think whenever somebody says data quality, the first thing that everyone thinks about, especially if you're technical, is is my data fresh? Is this data that was collected the right way, in the right format that I wanted, does it have the cardinality or is it filled with a lot of like blanks? So there are these types of technical data quality that you want to get through, but that is, I would say like one half of the data quality. The other half of the data quality is to ensure that the data is at that context of data. Where is this data right to be used for and how it should be used. Because when somebody says, okay, I look at this dashboard, but this data doesn't seem right. Our data quality is not great. That's what one person may think, but that might be because that the analyst put it together, did not necessarily realize that they were using the wrong data. The data itself is totally fine and it is up to date and looks great, but may have had some field or filters or different things that I would consider as if you think about organization data quality, it's important to consider what is the correct data to use because that is what encompasses of the technical data quality you mentioned. It can be very overwhelming to think about. Oh, I just need to have all the data up to date at the same time. If you understand where each data sets are being actually utilized, how often and used by whom you may realize that the SLA and the technical data quality that you want to keep track on, you may not have to do that for every single data set inside the company. It is very expensive to have that type of, I guess, checks and qualities and everything else you do want to have them. However, for your most used data, most critical data that powers and is being utilized for automation, like your pricing engine or your recommendation systems, for example, yeah.

I can certainly see how if there's a data set that isn't really being used by anyone, or maybe not being used by anyone at all, you don't care whether it has 99.99% uptime or not. But if it's something about like, how much money does your company have, like, what's your current revenue, then yeah, you really want to make sure that number is correct all of the time. You mentioned the idea that it's often quite hard for data scientists know they're using the right data set. This seems like something that's incredibly important. You want to be using the right data, and that's a very important part of data quality. So why is it such a big problem for data scientists to find which is the right data set to use for any particular analysis?

To put it simply, we have too many data sets. So in the past when we didn't have all these data sets, a lot of data analysts and data scientists were able to just export the raw data and then prep it to create the reports. And that was fine. A lot of this data analysis has been done in like Excel spreadsheets in the past. And we lived fine, but now we want to have live dashboards, data that is enriched from multiple data sources, that is up to date, and that does provide a richer and more accurate results for our decisions. And I would say in the past, the complaints, and this could still be the complaints today, but in the past, the biggest complaints that I've always heard from the data team is that they spend too much time on prepping the data because they were accessing the raw data. Today there are all these transient tables. You have all DBT and others to make your own jobs, to prep the data automatically. And that's all great now, but you have created a lot more tables and joins and views inside. It's a matter of understanding which one is the right model to use. That's becoming more important. But it is hard because you have to now consider hundreds or thousands of different tables that are created for analysis and you need to figure out which one is the right one to Use.

Once you start looking through like hundreds or thousands of different tables, I can see how that becomes a very difficult problem to discover what's going on. And certainly even in like a fairly small enterprise or small company, you're going to get a pretty large set of data there.

Yeah, I would say in a way there are two parts. One is if you've been in the company for like, I don't know, more than six months, then you might already have a pretty good idea which are the trusted data sets and which are the main data sets that I can work off from. Another part that I see as an issue that happens a lot is that you have a data scientist or data analyst or engineer that have built a lot of these models. One day, if that person leaves the company, it's very hard for the next data scientist or the rest of the data team to parse through what have been the main sources of truth that the person was using. There might have been documentation or whatnot, but it's always really hard to dig through all the work that they've done because a lot of it is embedded inside of their code in their SQL queries. I would say overall, the phase that we are in the industry is starting to mature as we are adopting more software engineering practices. But overall, in the past and still in many companies, a lot of data teams work in their local environment. They run different SQL queries and then embedded it for their specific dashboards. So a lot of the tribal knowledge and then the understanding of using which data sets, how to join them, what to filter on, a lot of these are embedded as like a tribal knowledge in organizations today. And this is why it's hard to find the right data.

I can certainly see how people leaving the company or having comments about how the data set was processed, being stored in a code comment somewhere, that's not really ideal for understanding what was going on. So you mentioned the idea of documentation. What does good documentation look like and who would be responsible for creating it?

I think documentation is one of those assets of the company that helps you operate more efficiently and effectively as organizations grow. You cannot just have a onboarding or transition meeting every single time. You would want to have a documentation that people can read and contribute and make it so that you're kind of like a knowledge base that is being shared in terms of data. A lot of that it trickles down to what are considered as your core or tables, like in your data mart. How are these data sets related to your current business processes or your business operations? Because most of the data is a reflection of the tools that you are using or how the business operates internally. It also has a lot of the, I guess, indicators of, let's say if it's about product analytics, then a lot of the data is about data, the user behavior and activities and so on and so forth. If it's more about financial analytics, it's about the sales data and operations data that comes from different ERP systems and so on and so forth. So being able to map those in a high level picture, I would say is one of the more important documentation. But it could also be very obvious to people. I think the part that gets a little bit trickier is mapping that internal data, internal either business process or analytics model with the data sets. So that's one part. The other part of documentation is more of like your data dictionary. So your important tables. Now you know that these are the core tables or denormalized tables that everyone should use if they were to run an analysis for a user or customer or an account for you. It is going to be important to actually document the fields. So all the attributes and the way that all the different columns that makes up of that data set. So like what this is represented in or what, like maybe it could be a sample data, what's the grain of the table? What is the unique key that you can use? Things like that, I think it's important as part of the data dictionary or the table column. And then last but not least, what a lot of companies really also try to have is a distance glossary. So this helps mapping the terms from the business processes with the data sets, but also to share the notation of what has been marked or as a field X. In businesses this can be also called, you know, Y, Z or something else. And a lot of that I think comes into the acronyms that the business uses. And it gives you an understanding of that linkage between whenever we have this acronym there is a definition and then you can have the definition into how that is represented in data as metrics. So I think that that linkage is also very important. So kind of three main parts that I would say I would consider as important in data documentation.

I love that idea of business glossary, which just explains all the nonsense that your colleagues talk to you about and you don't quite understand. Certainly I have to say, like moving from I've been in a lot of sort of different teams who have done data stuff and then I've worked with sales people, then I work marketing people and they all have their own different Sort of ecosystems of terms and just having all those things written down are. That sounds incredibly useful. In general, I think people don't like writing documentation. So who ends up being responsible for this? And are there any kind of shortcuts to make this easier?

Yeah, in a way, people that are responsible for usually becomes, quote, unquote, the data team. And it makes sense because they are the ones that have created the data models. So I believe the main in a lot of companies, they also call them data stewards or data owners. You created this model to transform the data from this raw sources to your transient tables or to the reporting tables. You have your own idea of how that data should have been prepped or aggregated or transformed. So that needs to be embedded into your documentation. So if you think about it that way, then what about the source data? So a lot of the times the source data is, I believe, should be documented by either you can pull the documentation from the source, like application, like if you have Salesforce, a lot of the APIs have documentations of the fields of what data is coming through. But then either the product manager or the developer that have created the initial data model can also document the data. So it's actually both the producer and also in a way, consumer, because consumers sometimes have a lot of say in how the data model should be structured and then the intention of collecting or transforming that data and the way to make this work or make it easier for the team. Now, there are multiple options, but here are the some of the things that we've seen and that have worked well in the company. So first and foremost, having an ownership of these data models or data tables defined, I would say is quite important. And you can do this by who has been querying this table the most, probably because they requested that table to be created so they can document that table. For solxr, we look that up by if you are one of the top users of that table. The other way on documentation, not just by asking people to own the table, but to more or less help the documentation, is to try to automate the documentation. The two ways that we've been doing this is one, by looking at the data relationships that is in the pipeline. If there is a raw field that came from a source that's been already documented, and then there are other transient tables that goes into the reporting table, but it's actually using the same data, we can trace that field through a lineage and propagate the documentation. And there are two benefits onto this. One is you don't have to Worry about documenting it from the first place and it will use the same documentation that somebody else, which is the original writer have written. The second part is using Lineage. You can actually keep the documentation up to date. Because if I were to, let's say make a different change, that new documentation can be propagated to other fields that it's linked to. I think utilizing these models to associate which could be the similar or related documentation and actually propagating them throughout the data model increases the data documentation rate much higher than before. We personally have seen just with lineage, about 300% increase on field documentation fill rate across our customers. Now that is only if you have documentation somewhere. If you have no documentation at all. Something that we recommend our customers to use is Genai. So Genai, by looking at the and having the schema information can process put out some documentation for you. For us at Select Star, we also utilize and feed the Gen AI with other types of active metadata like its source columns, its downstream dashboards, so that that object there is a full context actually being fed into Genai to create more richer, accurate documentation. A good example of this is that we have a customer that were documenting their, I believe it was a Looker Explorer field which is like all Looker views. Right. But still none of their view fields were defined. But because we fed in these Lineage metadata and also the downstream dashboard metadata, we were able to give them a unique documentation for 30 different count fields.

And these are all supposed to be the same field.

They were all different because the actual view or the raw field it was coming from was different. And in Looker there is a lot of different ways to create your own measure. It could be like sum of X or count of X. But for that customer they all had those field names as count. So providing that context of what is it counting, what is it actually counting for three different fields was actually very helpful. So Genai is definitely coming in as a really handy tool and it really also helps gives encouragement for the users or the owners and the stewards to not necessarily feel like oh, I have to start from scratch and where do I even start? They can just review what Genai have generated for them and edit as they need to. And I think that's a huge uplift for a lot of data teams.

Absolutely. I mean I couldn't imagine having to scroll through like 30 different fields trying to work out which one was the right one that I wanted. That seems like a horrible situation. So I like that documentation side of things automatically taken care of by AI and then you can be more productive. So that does seem incredibly helpful. You mentioned the idea of data linge and I want to come to that in a moment. But before we get to that, I want to go back to something you said near the start, that having context around data is incredibly important. And this seems like metadata is going to be an important part of the context. So can you talk me through what sort of metadata you need to capture about your data?

Well, metadata defined as a data that describes your data. So I would say anything that we interact with as a data, whether that is a file or record or anything, there is always some type of metadata that's associated with that object. Inside of database there is some metadata tables called the information schema that keeps a record of all list of databases, schemas, tables, when they were created, when they were last updated, who created them, how big is the table, how many rows there are? A lot of these are, I would say the part of metadata. Even your video files have a metadata that has your video file name is X and how long is the video for, like so on and so forth. So that's kind of like the metadata there. And when you look at metadata, metadata tells you about that. Like when I think about data in general, data can be just like one point where one factor, even if it's a data set, it is one point of thing. But if you use metadata and if you are starting to analyze metadata, you can look at and try to figure out what the correlations are, or you can lay out how many tables were created. And then if you start looking at the activity logs, which we consider as part of the metadata, but different type of metadata because it talks about what actions has happened on the data set, you can gather a lot of information to build the context around usage of that data set. So I would say combination of those in the buzzword active metadata makes your metadata that describes your data that can be its own. But as your data changes which your activity logs are going to tell you, you can make your metadata to be quote, unquote active, where the metadata that describes your data sets are up to date and has more information that it's associated with. Because for any data sets, the data set itself can be useful. But if you want to join it with other data sets because it's related, or it can be augmented with more context, it's a lot more valuable. So I would say active metadata is one of the critical parts. If you want to automate more of your processes, if you want to do more streamlined data governance, data documentation Providing this context I would say is really a core part of making that happen, which we basically all really what we do at Select Star is mapping and making metadata to be very active in our customers data ecosystem.

I love that idea that just looking at metadata properties like how old the table is, that's going to tell you is this going to be useful to me or not? Or maybe this was created by someone in marketing. That field that was just labeled count is probably going to be something to do with the marketing metric. I also like this idea of active metadata where you just making use of the metadata to do further analyses and getting some value out of it. How does this use of metadata then lead into the idea of data lineage?

There are I guess three parts of like active metadata that we consider as quite important. First is data lineage. So data lineage is about tracing where the data has originated and it has gone through different pipelines to be consumed by different users or applications. This allows you to see the dependencies of data sets. If there are data pipelines and other data assets that queries or brings data from one place to another. The second part is what we call popularity, which is data usage. If you were to monitor which are the most important data sets or if you were to try to find out which are the important data sets that we have inside the company, and if we have thousands and tens of thousands of data sets, do we really need all this? Data usage can really tell you that answer. We've also seen a lot of different use cases when you combine usage and lineage together. Together when you want to do some finops analysis or if you were to do any data migration or wanting to let's say implement like a data contracts, both the usage and lineage have been really helpful. The last part around the active metadata I would say is really important, I would say is understanding the entity relationships. So in the past in the relational database world, where to come from more like a transactional databases, we have worked with primary keys, foreign keys and we can see how the data models look like by knowing where to join. In a lot of data lakes and data warehouses there's no such restrictions or implication of that. So understanding the join keys and understanding which are the related data sets that I can use with this data set that I just gathered is quite important. And I would say the entity relationship diagrams, if you can refer it and we refer this from a lot of the select queries that run within the data warehouse. This is also really important to be able to understand all the other possible ways to look at the Data. And so I think those are the three main things that are quite important. Going deeper into data lineage. Data lineage is more around the. You can think of it as I have a dashboard. Where did this data come from? I have this table. If I were to change anything in this table, will this break anything downstream? I think those are the two main use cases that we see the most on data lineage. At the same time, we are also seeing more interesting use cases when you are starting to combine lineage of multiple tables to understand what are the shared data sets, fields that are being currently queried or being utilized versus not. Which gives you an understanding of if you were to try to prioritize during your data migration work, if you were to try to remodel your data because you feel like we have this like a giant denormalized table, how many of those columns are actually being utilized? I think this is another part that we've seen customers splitting their tables or remodeling their tables just overall. And when you use the understanding of data lineage as well as the usage metrics like the popularity, this just becomes a lot more handy and one for planning the work and also to communicate other teams if you need to rely on others to change their models.

Some interesting problems that I hadn't really thought about before. So that first one about if you've got a data lakehouse, normally in the traditional sort of data warehouse setup, all the tables is obvious kind of how they can be joined together. But in a data lakehouse or in a data lake, it's maybe less obvious. You just don't know whether you can do an analysis involving multiple tables. So that's. Yeah, that's a crazy problem. And I can certainly see how having that lineage that describes how things should fit together, that's going to make people more productive. But the second one was also very interesting that if you've got a very complex data pipeline, then if you change something, you're probably going to break some dashboards somewhere and then everyone's going to hate you. I suppose. So you need to make sure that you're not going to break stuff. Can you talk me through how the workflows now change for people developing data pipelines or developing data products, how that's going to change once you have the data lineage.

So one is that you can look at the data lineage to understand if you were to make a change to a column or table, which other tables that may need to migrate from using the old column to a new column, or for them to be aware that this new Data set is available, things like that. So one is the planning the change and being able to communicate that change to the downstream users. The other way that this is also being utilized in the workflow for SelectStar is through our Git Actions integration so it can be integrated into your CI CD with our Lineage API. So that every time there is a PR that makes a change to the column or the table definition query, like a DDL or DML that creates the query or updates the query that refreshes the table, basically we can post an automated comment PR comment that lists out all the downstream impact that PR could have. And this is not just about a here are five tables and two dashboards that is definitely one part of it. But then the other part will be like for each table there are two users of this table that is like a monthly active users. There are five dashboards consumers that is on the downstream of this. So having that understanding and before the code gets merged is also an automated part that we see as a game changing workflow for customers today. So a good example of this is that one of our customers, Zometry, they mentioned that integrating the Lineage column level lineage into their CI workflow has made them to sleep through a night and allow their data engineering teams to focus on more proactive work rather than reactive work of trying to fix the data pipelines when their data models on the production side breaks. When I say production, I'm talking about the production data before it comes into their data warehouse. Cross team collaboration perspective, like the data teams with the product engineering teams. This can really also be super helpful on the workflow side of it.

I can certainly see how that could be a real game changer if you know what you're going to break before you break it. Rather than just receiving a phone call at like 9pm on a Friday night saying hi, we don't know how much revenue our company has because the dashboard's broken, the data pipeline's down.

So yeah, or it could be as more critical and for this company, Zometria was very critical because they were maintaining a data pipeline that feeds into their pricing engine. So 24, 7, if this pricing engine does not operate correctly, they can lose a lot of revenue.

One thing that seems to have been a recurring theme in this episode is that things the data team does is going to have an impact on what the business team does and maybe the business teams want to have some kind of say over how the data is defined and things like that. Can you talk me through how data teams and business teams can work efficiently together.

The more that we work closely with data teams, the more that I see that a lot of modern data teams and where data teams should go overall is acting as a partner with the business teams. And many times I think this is something starting to become more true. But in contrast, a lot of the traditional and in the past, many data teams still do act as if they are like a service organization or like an IT team to the business teams. And I think there are a few ways that we've seen how this can be changed is first and foremost, having the data teams to be aware of the business goals and the context of the business so that they can ask back the questions, how important is this analysis for what the business is trying to achieve? And to be able to suggest the different ways of looking at the data, because they are experts in how the data is modeled and can apply their business knowledge on perhaps providing even better analysis than what the business has asked for. And I think that the process of this, in a way, does require some type of embedding of data analysts or business analysts to be alongside with the business teams. And I think a lot of companies are making that trend, and I think that's really great to see. It's also, on the other hand, important for the business teams to be aware of the data models that are underneath the dashboards because it tells them how the number has been calculated. And it is always the frequently asked question when they get this number, oh, so how did we get this number? So I think it's on both sides. And a lot of teams are making more strides to make that happen, where data discovery, data governance, and data documentation are all very helpful to make that happen.

It's interesting that you say, well, business people need to be able to just dive in and see how things were calculated. And it seems like this is a really tricky design problem, is designing things so that both the technical data people and the maybe less technical business people are happy. Do you have any tips or tricks on how you can design so that less technical people can understand how something was calculated?

So I would say that a lot of the times the business teams are accessing data through their BI tools Within the BI tool, there may be ways to share the fields that were chosen to run and the formula that were created for that dashboard. So that is definitely one part that they can see it. So that's one. And then the second part is the documentation. Right. Like we talked about the business glossary, they're very familiar with all those business terms. If that business Glossary includes the metrics and the calculations of that metrics, then it is very much on state and that everyone can refer to instead of thinking, well, I calculate my revenue this way. Did you use something else? You can always have it to be pointed into the documentation. So I think that's another part. But at the end of the day, especially for business teams to be more self service and can be grocking more of the data models, they do have to have that interest and curiosity around data and the way to make that happen. And this is just overall, maybe the human nature. I believe that one of the most important things to make that happen is for the business users to be able to use data for some type of analysis or some type of thing that they weren't able to do themselves before. This kind of comes into like training and enablement and everything else, but giving them the tools, space and the support to be able to play around with Data Explorer, the data that the organization has within. I think it is super critical to kind of get more people up to speed because I believe that a lot of people do have this interest. A lot of business teams, people are very good at Excel or they want to utilize more data, but they just don't know how. And I think giving them these examples of, okay, well here is the dashboard. Now, if you want to see it slice by this, like you can answer this question or what other questions can we answer? And I think there is a lot of this interaction that data team can also have with the business teams, which might happen already. But having the business teams to be able to utilize the tool themselves I think is also a key part of getting them up to speed onto datamos.

So I like that idea. Seems to have gone back to the idea of business glossaries. Again, having the calculations made transparent so that everyone can see how they're done. That just seemed like a good way to help different teams collaborate better.

Yeah, and in a way, because it's hard to document everything. One of the things that we exist and the part that we are trying to provide is that if you have anything that's embedded in the tools, like code, SQL queries, things like that, like we'll parse them automatically to take it out as your documentation or to your lineage, so that everything is laid out clearly and you can have that transparency with other data teams or team members as well as your business teams.

Okay. Yeah. I do like the idea of automatic documentation because again, it's better than having to write it myself. Wonderful. Okay, so before we wrap up, what are you most excited about in the world of data governance at the moment?

I mean, right now I think this market is really interesting how it's evolving, but with the introduction of and a lot of the usage around gen AI, using GenAI and AI overall for data governance as well as leveraging that data governance for AI, I think both are very interesting. But for me, I gotta say, yeah, being able to like for SelectStar, we were able to get a lot of benefit and acceleration by using Geni for our customers. And this is something that I'm very excited to keep forging on.

Cool. Yeah. Get AI to solve all the nasty problems that you don't have to do yourself and make data easy to work with. I love it.

Yeah, it's really converting a lot of manual processes to more of automated fashion. The other part is we talked a lot about documentation, but another part that is also very important in governance is tagging, semantic level tagging, whether like how sensitive this data is or if you were to organize your data, what are the gold, silver, bronze tables, things like that, what are the tables that you should mark as to not use or to be deprecated. A lot of these now can act as the semantic for your search results, your documentation were your chatbot and assist or SQL query generation. And a lot of this like the fact that we can decouple more into the semantic tagging documentation and then utilizing those on top to create and automate more workflows is why this is very exciting to me.

Absolutely. Yeah. I can certainly see how having something that says, okay, this table is about to be deprecated, that's going to be really helpful advice. Anyone who's thinking of using it, and that's going to make it much easier to find the right data.

And then the other part about governance, the traditional side of data governance, around regulations, complying with the rules, legit data privacy and so on. There's a lot of, I say the legalese and the ways that can be now processed through Genai to build a better and more refined policies for different types of data that comes into the picture. So that's another part that I feel like will be the step change from before in the industry.

And do you have any final advice for anyone who wants to get better at managing data within their organization?

I would say the key is in the metadata. Looking into metadata, whether you're doing it manually yourself or using a tool, is that all kinds of completely fine. But I feel like as you start diving into more metadata, you'll realize that there is a lot of insights that you can get around managing the data. So, yeah, that's my main advice. And we are happy to help.

Wonderful. Thank you so much, Shinji.

Well, Richie, thanks so much for having me here. This was a lot of fun conversation. Great questions here. So thanks again, Sam.