Generative AI Use Cases in State and Local Government

Happy Friday, everyone. It's Joe Morris here. Dustin Haisler won't be joining us this week, but look for us next week. I've got an exciting guest here that's going to be joining me here in a moment. I think we're going to be talking about the hottest topic going on in state and local government right now, and that's generative AI. And I have the pleasure of hosting Vidhu Shekhar, the author of, I think, the state and local government's first book on generative AI. And the book's titled Generative AI Use Cases in State and Local Government. So It's a pleasure to bring a guest on anytime, someone who's focused in state and local government, who's got that same passion that I do. You know, maybe you can tell us a little bit about yourself and your background in public sector.

Awesome. Hey, Joe, thank you so much. Um, you know, I have been a, a longtime listener, uh, of your show and of course following GovTech for a very long time. Um, and I am super excited about this opportunity to be on the other side of the fence for a change. Um, I actually started off my career in architecture, believe it or not. Uh, not the coding type, but the building type. Um, and, uh, pivoted a while ago when I went to USC— you see the Trojan flag back in there— um, into public administration. And since then, that was the start of my, uh, career in public sector. I worked a little bit in the city and the county of LA before moving up here to Sacramento. And since then, really been focused on everything, all of the major domains within public sector. And so much so that, you know, I feel like public sector is a place where you have a lot of challenges, but also that opportunity of innovation is probably a lot more than what you would see in pretty much any other industry. Yeah.

You know, so I look at it like this ChatGPT kind of took off and all of a sudden it went from something that was trending online to now we're talking about it in state and local government, we're talking about it in private sector. So what prompted you to write a book about how state and local governments could use generative AI?

I don't think I was in any different place than any of our listeners or you and, you know, other folks. November, when everything in the news was about OpenAI, ChatGPT, right? Uh, this is the new revolution. I'm like, I need to get my hands onto this, right? And you have no idea how many sleepless nights I had to spend because you go during the day and you don't get access to it, right? It was so overloaded with users that the only time you could go in and actually do something was after hours. So I was doing a lot of like, you know, fun stuff, right? Uh, asking it random questions. Funny enough, since my background is in public administration, I was asking a lot of policy-based questions, right? And I was getting some really interesting responses, especially given how the data is a little couple of years older, right, in the language model. A week went by, maybe I think I was in my second week of doing the same thing, and my wife got really annoyed. And so she's like, if you're going to be spending so much time on this topic, you might as well write about it. And so that was actually the starting point on why I decided to finally put it into some sort of a curated manual of sorts, right? Because I think it's important for us in government to know that thematically there are multiple things ChatGPT or a large language model can do, right? Whether it's summarization, whether it's content generation, or whether it's just that chatbot interface, right? But within each department or within each agency in a government organization, there could be multiple variations of this, right? And as I kind of started going through that, I was intrigued at the possibilities this opened up. And Dustin could not join us, but in his last webinar, right, I think he made a really good reference just across the spectrum of the use cases. I believe he had a handful in there that covered not just a part of government and not just the chatbot components that mostly come to mind. And that's where I really think the intrigue for me was to actually go deeper into this. And as I started going deeper, I also got sort of the AI hallucination component surface up, right? And I'm like, okay, Got it. Like, let me actually pause and take a step back. And I was wondering, like, do I edit this out? Do I not edit this out? Like, what do I do with this? I ended up including it in the book, uh, in pieces of it with a callout that this is where I think it's repetitive. Um, I did try to edit out inaccuracies to the extent possible because I think it's important for us to know that this is an evolving technology. Right? And that's a critical message I would love for folks to understand, especially in government, because we gotta test it out before we really put it out there.

Well, as you looked at it, I mean, you can kind of see where there's some points where there's a collision course between government and generative AI. As I travel around the country, you hear a lot about workforce challenges, talent gaps, Speed, efficiency. Where do you think some of the main benefits, uh, to leveraging a platform like OpenAI or generative AI more broadly across the public sector?

Uh, I actually have a slightly different view than you. I don't think it's a collision course, and the reason I say that is because I think generative AI can almost be used to the extent of an argumentation tool. Right, across multiple fields, whether we look at healthcare, whether we look at government, and even within government, right? When we look at sort of legacy systems, COBOL programming needs versus other system needs, it comes down to we just have not enough people to do that work, or not— the knowledge base is either retiring or even worse, right? And this is where I think the generative AI components come in really handy. where they can look back and the large language model can really self-train itself on some of the nuances, right? I strongly believe that human in the loop is the way generative AI really picks up, especially in public sector, because it can augment and be a force multiplier for a public sector workforce. Whether you think about, you know, the Codex-type components where you can ask for it to generate code for you, or whether you think about like just image generation. You know, back in the day when you go onto a government website, you would struggle and it would almost be like, you know, the old school, just pure HTML pieces, right? You go onto a government website now and it's— the look and feel is no different than a consumer website.

Mm-hmm.

And I think that accelerated motion can be even taken a step further where now the user experience of government websites and services in general can really become responsive to the end constituent, right? And I think that's, that's the utopia that I think, um, we always think about going towards, but generative AI is going to really accelerate that whole motion of having a really laser focus on service delivery for the constituent in the most efficient way, right? And it's going to challenge us. I'm not going to deny the fact that it's going to challenge us on multiple fronts.

Mm-hmm.

You know, the people front for sure, but I think the policy front is untapped. And, um, you know, being a policymaker in, in today's day, uh, is going to be especially challenging, uh, because you see the advantage generative AI can bring, but also the notions that it has to be responsible across the board, ethical, right? There should be equity across the board. We need to make a conscious effort to not let the biases influence the system in the same way that we sometimes see it do, right? So I think it's a loaded path ahead of us, but I think I am optimistic about it.

Well, I share some of that optimism. I think it's important to know, like, and like you said, while this is dominating our conversations that we're having both professionally and maybe personally, this is an emerging tech in the truest sense in state and local government. What do you think some of the challenges may be that governments, or hurdles that government's gonna have to get over in terms of their approach to adopting?

You know, it's a very good question, one that I tried to kind of at least bubble to the surface, uh, in the sort of the closing of my book, right? I really think it's about, uh, the 5 dimensions, right? The, the traditional people, process, technology, and then the data and policy, right? And we have to look at a maturity curve along all 5 of these, right? It will come down at the end of the day. Any large language model, any AI model is only as good as the the dataset it's working off of, right? But still, you gotta have processes in place, controls in place to be able to inform the appropriate use, which goes back to the policy piece, by the right people for within the technology, right? And so I think it's gonna be that sort of continuously evolving loop. The other barrier, and this is just, the same thing we experienced when we sort of transitioned to cloud technology, for example, right? Security is going to be a massive concern. I mean, I cannot emphasize the chances that generative AI can help with security, but also the risks it poses if something goes wrong, right? If you were following some of the NIST guidelines, NIST AI-101, came out last month, and the risk management framework specifically calls out security as a major piece of this, governance being another core component. So when I, when I sort of look at barriers in government, I think these two, that security piece and the governance piece, are probably going to be key to unlocking that potential, right? If we can figure the governance piece out, if we can figure out the right controls on the security side, the uses and the potential can be then sort of exploited in the best sense of that word for the better of all constituents.

Your book outlines 100 use cases in state and local government. By no means do I expect you to cover them all today, and we want people to check out your book, but maybe you could share with the audience a couple standout use cases for ChatGPT?

Sure. The one that I probably had the most fun with was around social media. When we look at— just this goes back to the government website piece. Having ChatGPT generate social media content is fairly easy. I also think the whole document summarization piece is applicable across just so many different departments and agencies within government. Uh, there's 2 sides of that too, right? It's how you can potentially unlock the Freedom of Information Act, the FOIA mechanism, for people to be actually able to consume the open datasets that's out there, right? So I think that's a very big piece of it. But on the flip side, think about, uh, all of our state, city, county workers, all of the policies that they need to be aware of, all the standard operating procedures appeals management, permitting processes, right? How do we, um, make that steep learning curve a slightly less steep learning curve? And this is where I think it was another use case that I've used a couple of different places, you know, through the book. What I also tried to do was align it to general government functions. So you'll see some things around general IT, you'll see some things around constituent-facing services, obviously around infrastructure. Health and Human Services. There's a little bit around public education as well, which I think was another interesting one that I want to highlight because there's a lot of concern around how plagiarism and things around that are going to be handled. How will generative AI impact the delivery of public education? And I see that as a massive opportunity because, you know, I have young kids like many of you you guys, right? And I see them at very different learning stages, even within their own classrooms, right? And I think generative AI could be a very good tool to support those learning curves. There's a lot I mean, I could spend hours talking about it. But perhaps I would like to do it over a glass of wine.

All right, well, I'll take that up on you since you're local. But yeah, I know we're at the very beginning of the conversation, let alone adoption. We're seeing a lot of efforts in the private sector, plugins, tools being built around ChatGPT as an example. Are you aware of any jurisdictions within the state and local landscape today that have either acknowledged the use of generative AI or are doing anything around it yet?

Not specifically that are using it in probably a public-facing way. We are very much in the dipping the toe phase of this technology. There's a lot of excitement. I know of governments across the world, you know, Europe sort of leading the charge on this, because they want to test out how does this stand up in sort of the strong data privacy environment that Europe has, right? The GDPR notions around data and things like that. Obviously, here in the US, we've seen a tremendous growth in excitement across from all of the private, you know, platforms that's available.

I'd certainly echo that. A lot of questions and a lot of buzz. So that's kind of the lead-up, right? So we're very much at the beginning of this. Earlier you mentioned cybersecurity, and there was a reference to it in a recent article. I think just yesterday there was a ChatGPT plugin in certain cases now has access to to the live internet, not just looking backwards. And they're saying, hey, one of the things we want to be careful with is the cybersecurity implications, ransomware and things. You know, when you look at our market today, whether that's education or government, we're seeing K-12, you know, get hit by cyberattacks. You kind of just hit on it a little bit earlier. But is this like the next level that if you're a CISO, or IT leader, that you really have to have a heightened sense of awareness now in terms of what this might mean as it relates to cyber?

Oh, absolutely. Absolutely. But here's the interesting thing. I think the amount of technology we use in our daily lives, it's not just the CISO or the cybersecurity professional that needs to be aware of this. Every single person should have an understanding of when they use this technology, one, what is the information they're getting? And put their own filters towards understanding that, right? And I mean, it does open a door to bad actors, probably a lot more should they start actually figuring out ways to exploit this technology, right? But this is also where I go back to the NIST 101 AI risk management framework is where you can put in some guardrails. As much as I love that it can go and do stuff with live internet, I think the biggest opportunity, especially in our context, uh, education, government, uh, and perhaps even to a large extent in healthcare, is we have a trove of data. How do we shift through all of it in, in sort of a way that we can get those insights bubble up to us to then take action, right? And I think it's the taking action part that I think still requires a human. I mean, I think that's where the thinking is and that's where people are highlighting that, that, hey, don't let the AI do all of it. You know, make sure you have sort of that human in the loop concept along the way.

I love that force multiplier aspect that you were talking about earlier. Right. And I think that resonates with where we see our audience or what we see in our research data. You look at the CIO priority list, you see automation, you see workforce, you see these things kind of now elevating and moving up that priority list and your book outlines a number of tasks where government is already leveraging automation to perform. Some of them great, others where we've had a challenging time, be that maybe a frustrating experience with a chatbot or a, you know, an IVR system. We're pushing the buttons but not getting to where we want, and then we want that human intervention.

Right.

We just want to talk to a real person. Do you think generative AI is going to help in terms of changing the public perception of dealing with an automated system with government?

I believe so, right? Without being sort of the social psychologist or anthropologist on this, but here's where I see the maturity curve, right? You know, things which we thought were sort of more the cognitive reactionary AI pieces. As you mentioned, IVR is a very good one where you kind of build out the flow. If this button, then this button. If you hear this prompt, then you respond to this. And it's limited. That language model is very limited. This is where the large language models just blow it out of the water. I mean, think about when you think about GPT-3, I believe it was like a 170, 175 billion um, transactions or the reference point for that. GPT-4 is essentially 150 trillion. Now don't quote me on the actual 150, it might be slightly more, but we went from billions of, of data points like trillions of data points, right? That's where it unlocks the potential of sentiment analysis, being able to not have the first in first out Which is partially the frustration we face in most data centers or most call centers, right? You got a call in and you're like, well, this person's busy. You are 15 in the line, right? It can start understanding the sentiment and maybe dynamically shift it, right? It can also start understanding sentiment while you're on a call expressing your frustration with an internet provider, which I'm sure all of us have expressed that frustration. And I use that information to then highlight an appropriate response. So I think that'll be there. The other big piece is the non-call center piece. How quickly can you actually find the answer without having to get on the phone? And I think that's the other piece that I think will help just general citizens like you and I get more comfortable with the notion of this is actually being helpful. And not yet another guardrail before I can talk to a real person.

Yeah, I think we just want answers, right, in the quickest and simplest way. So I think you're right, if people, people get them however they get them, uh, the satisfaction will go up.

Very big reason, you know, major search engine names became verbs, right? We don't say we're gonna go search the internet, we usually just say, hey, let's go Bing it, or let's go Google it.

Yeah, yep. So when you look at this all in, what factors are either going to shape or influence, uh, the trajectory of generative AI or ChatGPT in state and local government?

Um, I actually look at the metaverse as a really good example of this, um, because the hype cycle was very similar, right? If you think about it, you know, NFT started taking off Uh, there was trillions of dollars of investment that went into the metaverse, uh, only in my opinion to see it plateau out a little bit and then actually take a dip. Now, where I do see generative AI being different from that is the peak of the hype cycle is still not in sight. You know, um, I think we are entering into a phase where this hype cycle is unlikely to just plateau out. It will soften, but I think the adoption curve is going to start picking up. And what I mean by that is, is the technology is actually going to find its way into different tools, right? I mean, we've heard a lot of the technology companies, software providers, start putting it in as part of their own software, right?

Mm-hmm.

Um, I could just rattle off the list, you name it. You know, Microsoft started with the Copilot thing. Salesforce has it in there. Slack has it in there. Google's Bard got launched yesterday. They have other plans as well. And so this adoption into the core technologies that we use is gonna really help push this forward. The second piece is going to be the hardware side of it and not the, just the software side of it, is there's a lot of computational power required to really exploit all of the generative AI components, right? And I think this is where the Intels, the NVIDIAs of the world, all of the semiconductor companies will start seeing a lot bigger play. And then I think the downstream impact of that is going to be, you know, how we respond to certain things, situational awareness, scenario planning, all of those will now become much easier. And so, you know, it kind of brings me back to another use case around when we think about starting to do multiple versions of a plan and doing that stuff, it's going to become a prompt rather than us trying to like code it out and say, hey, here's this long code, run it through.

This— the wave to me feels very similar to the adoption wave, or has the potential to be very similar to what we saw with cloud in the public sector or mobility in the public sector, where if you see it and there's people that start to do it, and then you don't really fully grasp that the full transformative nature of it until you get to a certain point, but it feels like it has that same transformative potential.

Absolutely. I don't think generative AI is going to be the EV1. Yeah, I think generative AI is going to be the equivalent of a Tesla. Yep.

So now, admittedly, and I think we both probably are geared a little bit this way in terms of this, like, rational optimism, But in terms of this technology and certainly the state and local government market, but are there any programs or areas where maybe something like ChatGPT you wouldn't recommend people use it for?

You know, there's a lot. There's a lot, especially with anything in public safety. I would recommend exercising a very high degree of caution. It's very easy for the AI model to pick up on the biases on the internet and build it in. And I think public safety is one of those places where we gotta be extremely, extremely cautious. I think it's the same thing with healthcare as well as human services in general. Because, you know, that's an area where I don't think the AI can take out the bias by itself, right? Uh, it could also enhance the equity gaps, uh, sort of inherently due to that. And so those, those are the few places where I'm, I'm gonna definitely recommend our, our listeners to exercise a lot of caution. Uh, and again, I mean, I keep going back to the, the NIST folks because They put together a very good cross-section of experts across multiple agencies or, and, you know, industries. Um, and their risk management framework was, hey, be careful. Like, don't be scared of this, but be very cautious. Uh, it's like when you get onto a very high-powered motorcycle, you better put on all that protective gear because, uh, you know you're gonna fall. It's just a matter of when.

Right. That's very, that's very sound advice. So where can our audience go to keep up to speed with you and what you're doing and, uh, to purchase your book?

Oh, uh, you know, the part of the best place is, is LinkedIn if you want to keep up with me. Uh, my professional work, um, one, makes me travel a lot, which I, I really like, uh, and I interact with governments across the country, uh, and so I would love for folks to follow me and connect with me on LinkedIn. With the book, it's on Amazon. I kind of went the self-publishing route. It's available both in a hard copy version and a Kindle version. Please feel free to order it. You'll be contributing to my kids' college fund. And if they don't use it, it's probably gonna be used for my next toy, which in all likelihood, I'm actually gonna look at buying a a hardware development kit around generative AI. And so, so, you know, it's gonna be a fun little thing like that.

Well, very cool. And we appreciate you coming on and joining us on In Case You Missed It. We'll make sure we drop a link to your book in the show notes on government technology. And until then, I hope everyone has a wonderful weekend.

Joe, thank you so much for having, and say hi to Dustin for me.

Will do.