How Local Government Should Approach AI

All right, TGIF, and welcome back to In Case You Missed It, our weekly roundup of what's hot in GovTech. Outside of the weather here in Austin, Texas, where it is right now 105 degrees, we're going to be talking about something else in the market that is hot, and that is artificial intelligence. And Joe, I mean, this is something that comes up everywhere we go. I know you've been at some events, I've been at some events, What are you seeing kind of in this AI landscape today?

I mean, everyone's talking about it. I think a bunch of my phone calls today were on AI. So, I mean, no matter what you're looking at editorially or talking to industry partners or government, it's a topic that comes up. And that's why we're so excited to have our guest here with us today.

That's right. And that guest is Juan Sequeda, Principal Scientist and Head of AI Lab at Data.World. And today we're going to break down how agencies should be approaching AI as they hear about generative AI and all of the other advances that are happening.

This isn't a new thing.

This is something that's been around for a long time, but there's some recent advancements that are changing the game for many agencies that are leaning in on it. So let's welcome to our virtual show here, Juan.

Good afternoon, Dustin and Joe. Thank you so much for having me.

Yeah, thanks for joining us. I guess first question for you, you know, you're a first-timer on the show. We'd love to introduce you to our wider audience and learn a little bit about your background and your role at Data.World.

Yeah, well, so I'm Juan Cicada, as I mentioned, principal scientist and the head of the AI lab. In a nutshell, so from the AI Lab perspective, I help run innovation. So basically, Data.World, we're an enterprise data catalog platform. So we help organizations organize your data, basically like we create the library for all the data you have in your organization so you can help search, discover it, govern it, and make sure everything is running kind of very smoothly. And a lot of my work is to really ask what's next? What, let's push the boundaries around that from the innovation perspective. And so I spend time with customers to be able to understand where they're going and how can we help them understand the sales and marketing to figure out where the market is going and influence the product team to get them ready for the next things. So that's a little bit of what I do. And also one of the other things, I also have a podcast called Cataloging Cocktails, the honest, no BS, non-sales-related podcast. So just like you all, I do this every Wednesday live at 4:00 PM Central.

Awesome. So one of the things we'll make sure to drop it into the, into the show notes for sure.

Maybe we need to move ours, Joe, to the happy hour time slot like Juan. I think that's a great, a great idea.

That's very cool. Uh, look, we were just talking obviously in the intro, it seems like everyone we're talking to is looking at generative AI. It wasn't on their strategic plans 4 or 6 months ago. Everyone's trying to wrap their, wrap their arms around it. What do you make around all the craziness related to generative AI?

So, okay, so when it comes to craziness, it's the moment to get excited. There is hype about it, but at the same— so get excited about it. Like, oh, if you have not been playing around with ChatGPT, literally stop what you're doing and go play around with it. Like, I think that you really need to understand why there is hype around that, because that hype is real. Like, it's not— it's— there is tangible value right now. about it. It's all about productivity and efficiency gains around this. So once you start playing around with it and having fun with the craziness, then start thinking about how is this going to provide value for my day-to-day work, for my day-to-day personal life, and so forth, because it is providing tremendous value around it. So the whole ChatGPT is really— it is an inflection point in, I would argue, almost in humanity. The same way we saw the web, what we commonly call the internet, back in the, in the early '90s was a big inflection point. It is another iPhone moment. Uh, I remind, remind people that this is iPhone 14. Uh, the first iPhone started 15 years ago. So what we're seeing today is going to evolve. It is going to get better. The web that we do today, that we take for granted, started back in 1989 by Tim Berners-Lee, and now we just take it for granted. So this is one of those points in life, and we are extremely lucky to be alive and living this right now. Bottom line, if you are afraid or like, what's going on with this, go to ChatGPT, go play around with it. And people say, oh, is this going to take my job away or not? Like, AI is not going to take your job away. The person who uses AI is going to take your job away.

Well said. Yeah. So, you know, I want to talk a little bit, uh, the kind of the most technical question I have for you is around knowledge graphs, because, you know, you've been a major proponent of that, have built things on this, and really have been an early pioneer. Can you break down a little bit, you know, what are they, how do they fit into this conversation, and, you know, what are the, what are the uses, and how should agencies kind of be thinking about these in the context of, uh, of how they might use them?

Yeah, so, so my, my very simple definition of a knowledge graph is that you're integrating data and metadata at scale and it's in the form of a graph. But before we go into the knowledge graphs, let's look at some of the other common things that we may compare it to. Let's think about, for example, relational databases. If we think about these like superheroes, a relational database, superheroes will have strengths and weaknesses. The strength of relational databases is that they are well-defined systems that are used to go create applications when you have very clear requirements. That's what their superpower is. They've been around for 30, 40 years. So much of the applications are built on them. The weaknesses is that they're not very flexible. If you don't have very clear requirements and then you want to start, you add things and change, you want to change things, you need that power of flexibility. That's where the relational databases, that's where a lot of the weaknesses. Another technology we see around is, for example, JSON. Well, look, JSON is one of those things that I need to have that flexibility. Definitely JSON gives that flexibility to go do things, right? Very rapid development. It's schemaless. I can just go do things very quickly. Now, when it comes to weaknesses, it's schemaless. That means that I don't really have to find— I kind of swung the pendulum to the other side, right? It's like the relational database world is like I have very clear requirements, very strong schemas, and then with JSON, I go to the other side and say, well, I just want schemaless, just be able to go add things and then you can very quickly do rapid development around that. That's that other swing of the pendulum. Now, another thing that is hard with JSON is that it's really hard to do interoperability. Yeah, I can share JSON data, but then if you share my data and I don't know what it means and I got to go create code to transform it, so then there's all this transformation, these meaning, is stuck in code that people are doing. Now knowledge graphs come into play is because you are literally combining all that data and metadata, the semantics and meaning into one place. It's in the form of a graph because when you think about it, a lot of the stuff that we— when you start drawing things on a whiteboard, you're drawing the models, it's a graph around that stuff. Then you can start transforming tabular data into a graph, tree data like JSON is a tree, you can put that into a graph. It's this really interoperable model to go do transformations. Um, and so, so that's one of the, the strong points. So just, it really enables you to go integrate data at scale coming from so many different places. Now it has some weaknesses too. First of all, it's a paradigm shift, right? Not a lot of people think about graphs, right? People very think about tables. So that's a paradigm shift you go do. And it's new technology. Uh, I mean, it's been out there for a long time. A lot of— half of the web pages have RDF graph data in there. But it's still something that people are not common known, RDF and OWL and SPARQL and so forth. But hey, quick tip, because all of this has been on the web, ChatGPT, they learn everything from the web, from the internet, so it already knows this technology. If you want to get up to speed quickly with knowledge graphs, you can actually build knowledge graphs and do things with ChatGPT. It's fascinating right now.

I love your enthusiasm as well as your depth of knowledge, and obviously our audience does too. since we're getting a bunch of comments rolling through. One of those is, do you anticipate ads bombarding generative AI? Will the tech become less effective to do ad-driven responses?

That's a very interesting question. Actually, one of the folks I actually had on our podcast is a guy called Tony Seal. You should follow him on LinkedIn. He talks a lot about knowledge graphs. One of his hypotheses is that Because ChatGPT, these large language models, at least from OpenAI, they've actually learned from the web and the web has all this structured data in it already. This may be an incentive for people to start adding more structured data on the web. The next crawls or the next models that are learned from the web actually has that structured data. They can start using more facts inside of it. But then as the comment right here says, maybe that's going to be a form of people putting spam or ads in that. So I'm a big fan of history, right? Uh, we gotta learn our history, otherwise we're gonna repeat it. And history has shown us that people find all the loopholes around that. So my guess is it's probably gonna happen, and, and I don't know what the solution is, but I'm sure people are gonna be thinking about that. So great, great thought right there. Great question.

Go ahead, Dustin. So one of the things that, that, uh, we saw in past work— and this is pre-generative AI— we were looking at AI adoption like 2018, 2019, and all of the challenges came back to data. So what are some of the most significant challenges that you are seeing around data challenges in government that may be preventing government from unlocking that full potential of AI?

Oh, so, so here's the thing is that folks at organizations don't have their data house in order, right? So they don't even know what data I have, and you don't even know sometimes what that data— if you do know what data you have, you don't even know what it means. So this is kind of that first step. It is important to be able to keep track and understand what you have. Now that's one side kind of from the technical perspective, and what we see a lot is that we kind of focus so much on the technology perspective, I'm like, but wait a minute, we can't just jump into the technology. We need to also understand what is our goal? What are we trying to accomplish? What are the use cases? And this goes more into the social side. So you also need to keep track of what are the problems I'm trying to solve right now? What are the goals that I'm trying to go accomplish? And then start tying those 2 things together. Because if you don't, you end up saying, I want to go keep track of all the data I have and you end up boiling the ocean. Then you just do work for the sake of doing work that you're actually not helping accomplish the ultimate problem, the goal within your organization. To sum up here, I think it's one, people don't know what data they have, they don't even know what it means, but there's a disconnect of that with what are the problems you're trying to go solve. I always say start cataloging the questions that you have. What are the problems you're trying to go solve? That helps you scope down to figure out what data do I need to solve those problems? Do I already have those types of data around that? and then trying to make sense of what that data means. That's not just looking at the data by itself. You got to start talking to people around these things. AI can come in and help us go do this to help us be more productive, but we cannot expect that there's just some magic wand that's going to say, oh, just generate beautiful data and let's go solve these problems. We're not there. The reason why is that because a lot of this stuff is stuck in humans' heads and it's sometimes not even shared. or it's implicit, right? How many times we say a word and we think we mean the same thing, but that one word means multiple things, or we use multiple different words that don't mean the same thing. This is a human interaction that we need to be able to go get involved here.

Yeah, absolutely. And, you know, I think you, you know, pretty articulately said, I mean, data is really foundational to unlocking all of those, you know, AI use cases that agencies are gonna, you know, want to do. And as we know, you know, one of the biggest data types in government is a non-OCR PDF, and that creates a lot of challenges when it comes to, You know, trying to bring intelligence. I mean, what would you say to someone that's in the public sector today that understands the potential of where things are headed? Maybe their data environment's not in the right order. Maybe it's, you know, in silos kind of all over the place. I mean, any kind of best practices, words of advice or recommendations that you have on just some of those initial kind of things that they should be doing with that grander vision in mind?

Yeah, so the honest, no BS, my motto from the podcast, and I will acknowledge that this will sound salesy, but I genuinely mean this, is, You need to have a data catalog. Now, does it mean that you should go off and start buying right now a product? Like, you should literally start your catalog by, by having a spreadsheet. Let's just keep track of these things. And, and once that starts to break, once that doesn't scale, maybe you start realizing, okay, that's where I need some technology to help me out. But you need to start cataloging these things. Imagine going to a library where you just walk into a building and you're just seeing a pile of books. Dumped on the floor. You need to start organizing that, right? So a catalog is that means of starting to organize information. Now when you start organizing that information, you say, following analogy with the books, it's not like, well, let me just start from left to right and go do everything. You probably have an objective. I'm looking for some— so you have some techniques to be able to kind of scale the problem to something smaller. So first of all is you need to have a data catalog because the data catalog is going to enable you to be AI ready. To be AI ready means you need to know what data you need to be able to go use for your AI. If you don't have a catalog, you're not going to be AI ready. That's number 1. The second thing is thinking about this AI world that we're in right now and going back to the previous question that came up from Tim on LinkedIn I saw, is this AI, the common thing, it hallucinates, it lacks it lacks the context of your organization. You think about it, it's like these 2 worlds. One has this general world, general knowledge of the world, and then there's all this internal knowledge of your organization that these large language models generally doesn't have. The way these things bridge is through the knowledge graphs because the knowledge graphs provide all that context that you have in your organization. What you really want is in addition to a data catalog to help you organize organize the data you have, you want that data catalog to be on a knowledge graph architecture because that is what's going to prepare you for that AI world. Without it, honestly, and actually I'm not even saying this like it's other customers or users, it's like if you don't have a data catalog on knowledge graph architecture, if you're not bringing in your data, if you're not governing it, you are setting yourself up for an AI failure.

What do you think the landscape's gonna look like over the next year?

Oh, um, okay, so one, we're going to get a lot of the, the hype. We're going to see what's hype and not. So first of all, uh, if you're not focused on business value, then people will realize, okay, that's just cool and we've had enough of that because, hey, we're spending money on these things. So people are excited, but we're gonna— and we're in these economic times where we're really making— being very careful on how we're spending money. So I think we're going to start focusing on things that are providing direct value and we're identifying what those things are and put the noise away. That's number 1, I think it's going to happen. Second, I'm also seeing a separation of the types of data for given use cases. I think that separation is going to happen with unstructured data and then structured data. A lot of the cool things that you're seeing right now and examples that are on the web, on LinkedIn and so forth, people are doing things given text. If you think about all these vector databases and embeddings, that's all because you have a bunch of text to go do. I think we're going to see a lot of advancements over that. Then it's going to be separate from when you have a lot of structured data, data that's in SQL databases or spreadsheets or CSV files. That side of the world needs more work because that's where we need to invest more into the meaning of it because we're not going to be able to do that. This is the data that we go use to make our calculations, adding things up and presenting it to the board and stuff like that. And that we have to have a lot of scrutiny. So this is where the governance and be able to understand where the semantics come in very, very, very, very, very clear on that side. So we're going to see those 2 separations. I think people will start talking, oh, is this more on the unstructured, on the text and images too, or we're seeing seeing things on the structured side. I believe that we'll start seeing more advancements faster on the unstructured side because for the structured, we need to really sit down and do more of that work. I think that's where I find more exciting because that's a lot of the work that I do right now, especially in the AI lab. We're actually working on projects on how to benchmark all these chatting with the data systems. I want to go chat with my relational database. That's where we wanna go, but there's a big gap and we're really trying to understand what that gap is, how big that gap is, and how to start bridging that gap. And anyway, that's, if you're interested more, you can find me on LinkedIn and see all the work that we're doing in the AI lab at data.world.

Yeah, we definitely wanna dig into the AI lab stuff in the future 'cause I think that's really exciting, especially that you have one carved out. I mean, it's such a trending topic. You know, one of the other things that we hear, I mean, government as a regulated industry has to balance its fair set of risk, and, and they have to maintain, you know, compliance and public records and other types of things as a part of that. What is the safe way to start testing and experimenting with some of these things? I mean, where do you kind of see, like, you know, the, the risk profile, I guess, going from, uh, from an AI standpoint? I mean, would you just tell agencies to, you know, maybe have an acceptable use policy and don't put anything that's PII into, uh, into ChatGPT to test? Or any advice kind of for the agencies that are that are wanting to test but maybe now they don't have the right governance structure in place to deal with the potential risk for, uh, for using it?

Yeah, so, so I think, so first of all, it's time to have grown-up, adult, honest, no-BS conversations around these things. So, um, right now I'm sure that we'll have like the, we'll have the tech folks really excited about it, the business folks be doing, the legal and privacy teams are going to be afraid around this stuff, right? We need to get everybody in the room and have these adult conversations and actually engage the legal and privacy teams as fast as possible. Because what's gonna happen is that if you don't, you start doing things and they come in later on, they have not gone on the journey that you have been on. They don't understand it and they're gonna say no immediately and start turning things off. So to avoid that, you're on a journey. Join, invite them to be on your journey too. And let's make sure that they have all these, like you start asking these questions. So I think that's num— that's one of those things. Second, really start creating— I know committee is not the favorite word, whatever— to create a group, a community, a diverse community, a community of legal, privacy, the business side, the technical side together, so you can all figure out what are the dos and what are the don'ts. Understand what is proprietary, what is clearly proprietary, what is clearly not proprietary, where maybe the middle ground, right? Try to understand those guardrails and It may be things as simple as, I mean, quote unquote simple as, look, we're going to let people go start playing around with this stuff. Yeah, sign up for ChatGPT, but then everybody does it individually. Now you have 200 expense reports you have to go through. It's like maybe we just have something that we go figure out with Azure or something like that. Have this committee to be like a help desk. People are going to start playing around with these things that you want them to ask. If they don't have a place to go ask, they're not going to go ask those questions. Be able to go ask those questions and then start cataloging those is what they're doing. This is evolving all the time. Something from one of our customers, what they started to go do is that they actually created their own chat interface, their own interface over ChatGPT. They partnered with Azure and then they provided to their organization a skin version of ChatGPT. They control it. They know everything that goes in. They keep track of the prompts of everything. Actually, in the government space for a Freedom of Information Act, you probably actually need to keep track of all the prompts because because that could be asked. So you want— you need to be able to keep track of all these things, right? Um, yeah, understand what is proprietary, what is not proprietary. And I think what's really important is to start building a community, right? Let people share what they're doing, socialize it, share the interesting stuff they're doing, share the stuff that is not working, right? Go try this. Um, other examples are to consider the outputs. For example, if you're generating an image, what do you know? Can I use this image? Maybe not. There's legal copyright issues around that stuff. But maybe you can use that image to have a discussion that can inspire us to go do things or whatever text. Understand how you should use the outputs. Look, let's acknowledge that there will be problems. You're using the technology to help you solve problems. And this is probably going to generate new problems. Be mindful about that. And it's opening the conversations that we need to have. We have new conversations about problems of privacy, problems about copyright. Let's be adults about it. Let's just bring it up and figure out— let's not just go hide and say no, uh, because at the end of the day, like, they're— if you're not using the technology, you're going to be behind. So we just need to figure out how to go use this technology and just remind ourselves this stuff is evolving, as I mentioned. Just my phone. We take it for granted, but there's been 15 years evolving around all these things. It's going to evolve. And then at some point, you know where you figure out, are you an early adopter, innovator, or laggard, right? You know where you're going to fit into that stuff. Anyways, that was a long laundry list of things right there to go.

I love it. In fact, I was joking with Dustin last night. I said, is it too early for my 3rd grader to start experimenting with this?

It is not. And actually, Early this year, so at TED in April, I think one of the best TED Talks that I saw there was by Sal Khan, right, from Khan Academy. And so Khan Academy, they actually had access to GPT-4 like last year, early on, one of the first people to have access to it. They, when we were at TED, they actually, he actually presented and gave a demo of what they actually released within a couple months ago or whatever. Try— they take— they, they took GPT, ChatGPT, and said, imagine if every single child had its own tutor. Imagine if every single teacher had its own private teaching assistant. How better would the world be? And there's actually so many studies. There's a 2 sigma problem, uh, uh, research around like if you give everybody tutors, how things would improve. Now they've designed it in a way that it's very Socratic. So you can say, well, the concern is like it's going to do my homework for me. Yeah, well, say, hey, write me an essay about Rome or whatever. It will start writing the first couple of sentences and it'll tell you, what do you think should come next? They started to organize it. This is how they figure out what the guardrails are. It is important to be able to provide— this is a new set of tool. Just remind ourselves, people are afraid, oh, we can't let people use calculators because they're going to not learn how to add. Same type of stuff. Again, history. Let's go remind ourselves of what's been going on in history.

Well, I appreciate that perspective. I know earlier you shared with us your LinkedIn, but anywhere else that we should go to keep up to speed with what you're working on?

Yeah, no, well, come to our— learn more about data.world, our enterprise data catalog on a knowledge graph architecture, just data.world. The AI lab, we're going to start releasing more, kind of a lot of the progress that we're doing, and we'll be sharing that on data.world and on blogs. Find our podcast, Cataloging Cocktails. It's the Honest, No BS, Non-Salesy Podcast. We've been doing it for— this is now year 4 that we've been— that we're on it. Uh, also follow me on LinkedIn. I've been really posting a lot about this. Uh, I've been traveling to different places and love to go meet with folks.

Well, Juan, it's been a pleasure to have you on and share this knowledge with our audience. So thanks for joining Dustin and I today.

Thank you for having me. Appreciate it.

Have a wonderful weekend.

You too. Bye, everybody.