Beyond the Hype: AI's Impact on Public Sector Agencies

All right.

Hello everyone, TGIF, and welcome to GovTech's In Case You Missed It, where we talk to the movers and shakers inside the government technology market. Today we're speaking with Chris Hein, Google's Director of Custom Engineering for Public Sector and SLED, to find out more about what he's seen firsthand as it relates to the hot topic of the year and probably next year, artificial intelligence. So welcome back to In Case You Missed It, Chris.

So glad to be here. Thanks for inviting me back.

So in case people missed your previous episode with us, share a little bit about your journey in GovTech and kind of what you're working on at Google today.

Yeah, happy to. So I have— my job here at Google is to run the engineering organization for our public sector, right? And so, you know, what I think of is really like trying to take the things that Google does particularly well and translate that into something that the government can actually make use of. And so really looking at like all these amazing, you know, technologies and innovations that Google has come up with, what is applicable to government? How do you then start to put that in front of agencies and get it into the hands of constituents in a way that can really make their lives better? And so much like you just mentioned, in terms of the topic of the day being AI, that really has been the crux of the last 2 years or so of my personal journey is thinking about AI, thinking about how does it impact government technology and how can we try to be The best partners for that as we possibly can be over here on the Google side.

Oh, that's awesome. And you're right, I think we all feel that way. I think over the last, whatever it's been, 18, 24 months, that AI has dominated certainly the events that we all produce, the conversations that we see the technology community having, even line of business leaders having. I think that's what makes this episode so exciting is that we can kind of go beyond just the hype and talk about what you are actually seeing out there in in the field. Maybe you can kick us off by just talking about some of those real-world use cases that you're currently seeing and involved in.

Yeah, absolutely. You know, I think it kind of classifies into a couple of ways, right? Because there's some things that we've been doing that, you know, are kind of a natural outgrowth of a lot of the projects that the governments have been working on for, you know, 5 to 10 years now, right? And so those would be things like, you know, the classical chatbots or, you know, virtual agents that do allow you to kind of have a better constituent-facing experience. Sometimes it's an internal employee-facing experience that allows them to kind of get access to their data. And so that's everything, you know, we've had great success with that, you know, both in Illinois where I live, we've been working with them for 4 years now on some of those technologies. New York and a bunch of other places have really leaned into how do you make it so that the the constituents can get information easier and faster. So some of that is kind of coming from, you know, classical, you know, natural language processing AI and being able to do some of those things. But where we're seeing it move to, and what's really exciting about kind of that, that next cusp, is how do you start to introduce generative AI into that conversation as well? And so that's where we're starting to see, you know, even counties like Sullivan County, uh, in New York that, you know, with a very minimal IT staff was able to basically point a generative agent at their knowledge base and then allow their constituents to then ask questions of that virtual agent with no pre-training, no, like, every single intent and what it's going to be able to respond to. And it's already reduced call flow for them. I think the numbers are somewhere like around 60% in terms of people that would have been calling into their, you know, 311 system and instead are able to just get their questions answered on the website itself. And so that ability to really take something that is low, very low hands-on, right? Very little work from their perspective, but ground it on kind of the datasets that you're asking it to. Because we do know that hallucinations are a problem and we need to make sure that we're eliminating those as much as we can. So that's one big category is just that natural progression of kind of constituent-facing applications. But then there's a couple others that I would mention, you know, Covered California is a good example of using document processing AI to be able to really speed up. You know, like they process over 50,000 documents per month. And so being able to get through that level of document flow is something that previously was, I think, 71% of them had to be validated by hand. And so those are things that like we're now working directly with the state to be able to take that And automate that with different document classifiers. They kind of, you know, it's a little bit like OCR on steroids, right? To be able to really take it and add some intelligence to the processing system so that you're really getting to a point where that is no longer— backlogs are no longer caused by the fact that there's too much paper, right? We can now get past that point where paper is the thing that is preventing us from being able to help our constituents really, as best we possibly can. Those are 2 outgrowths, and then we can talk about some other ones that are maybe in the formation process, which is also looking at how does— if you take something like a transformational technology like generative AI, and then you rework how you would build a system. This is definitely something that's still a work in progress, but it's something that's really fascinating is to start to think to yourself, what's it gonna look like to start to not just around the edges add artificial intelligence, but really at the core rebuild a system and think about, you know, with multimodality, the fact that you can use images, you can use pictures, you can use videos, what does that start to change in terms of a government technology workflow? And so those are some things that are kind of in the coming soon realm, I think, of where we're at today.

Love it. You know, I think a lot of the conversation around AI, you know, the benefits are always pegged at productivity and speed and other things that you kind of mentioned, but there's a lot of other things that I think you've alluded to and have seen that are maybe not as obvious as benefits to leveraging some of these new generative AI solutions. Do you have any examples of those that you can kind of share with us as well?

Yeah, I think that there's a couple there, and I think one of them, which is surprising sometimes, is equity. And so if you think about the fact that, you know, government has to serve all of its constituents who who are not always going to have the exact same abilities, right? And so that could be anything from language ability to, you know, voice-to-text and things like that for folks that have whether it's hearing or sight disabilities, being able to add that ability for your websites or your agents to be able to interact in the language that the person can most clearly understand. You know, we're doing that a lot with the City of Dearborn right now, which has a significant portion of their population that's not natively English-speaking, and being able to add that ability to do kind of real-time translation on the fly really can add that ability for that equity of service capabilities. Same thing for the state of Minnesota. You know, they've got a large Somali population that the government did not feel like they were doing a phenomenally good job of being able to service from a technology perspective. And so they've been working with us on some of our language translation services to really be able to meet those folks and make them feel like the government understands them and isn't forcing them into a language that they're not particularly comfortable with. So equity is one kind of surprising benefit that you can get out of using AI in these places. Another one is more around some of the climate and resilience in our infrastructure. And so this is where it goes beyond just necessarily your constituent-facing, but we're really excited with the state of New Mexico to announce a project that we worked on with them called HydroDelta. And HydroDelta is this idea of using overhead satellite imagery to look at the actual ground below and then using artificial intelligence to be able to detect changes in that ground. And so for New Mexico, it's well as a lot of our southwestern states, water is a key issue for the next, you know, 20 years. And so what they did was they were looking for basically leaking pipes, 'cause if you've got potable water and it's leaking onto the streets, that is a huge source of just lost water. And so what we're doing is being able to identify those things with them, and we're really excited about what the next phases of that can look like, 'cause right now it's a bit of a manual process, right? The AI can find something, Then somebody's got a human goes in and does some review to see whether or not it's worth sending a car out there to take a look at it. But as we move forward on it, we're really looking for how can it be doing alerting and being real-time monitoring of some of those systems. Another example kind of in that vein of using vision intelligence to try to improve things is even, and we've been doing this for a while now with the city of Memphis, is helping them to identify potholes in their streets. And so those are things where just using, you know, they're using kind of like a Street View type car to go down their streets and log and look for any potential potholes. And beyond that, they're now doing additional projects to look at ADA compliance. Again, coming back to equity and accessibility, be able to say, are there places where the curbs are incorrectly leveled, right? Do the sidewalks have the appropriate cutouts? so that folks on wheelchairs are able to get into them. And so there's these really interesting kind of ancillary places. It's not just the chatbot use case that folks tend to focus in on these days that do provide a significant amount of value to those cities and states.

You know, as I hear you walk through these examples from the various cities and in your experiences, I'm struck by 2 things. One is you see like state and local government being a target-rich environment, right, for these types of innovations that you're covering, but also that like AI or generative AI can appear or feel a lot like the silver bullet remedy for a lot of public sector's problems. How do we kind of manage reasonable expectations? So what reasonable expectations do you need to have and what challenges are you actually seeing out there regarding the use of AI in government?

So I think for one, it's important— I couldn't agree more, Joe. Like, this is not magic fairy dust, uh, that you sprinkle on a problem to make it go better than it used to, right?

There are—

AI is a huge field, right? And as we, we put together a Venn diagram of where that may or may not intersect with a, with a statewide problem, you need to be actually really looking for what those benefits will be and how you can clearly articulate them, right? Even down to how Google thinks about this. You know, I sit on our AI Responsibility Council for our public sector. And as we think about responsible uses for AI, one of the key ways that we want to— that we look at it is first you have to come up with the value statement. What value is AI bringing that's differentiated from how you might solve this problem another way? Right? So if you can solve a data challenge with just normal analytics, right, great, go do it, right? Like, don't try to use AI because it's shiny. That's not really what Google's in the market for. But much more around starting to build the muscle memory of identifying these places that AI can do something that brings significant value that existing tools and techniques are just not going to get you to. And so part of that, and one of the biggest challenges of that, is that that muscle memory doesn't exist in most of our government workforces today. And no shame on them, right? Like, this— a lot of these tools have only existed, some of them, for 6 months or less, right? And so what I've really been trying to help a lot of the agencies that I have a chance to work with understand is we've got to start to help build these safe playgrounds, right? These places that the employees themselves, the folks that have been doing this job for 20-plus years, how do you make sure that they get their hands on the tools in a safe, protected way that allows them to go explore and to go see, does this change things, right? Does this actually add a significant amount of value before you go and try to revolutionize everything and you know, just throw all your money and resources at a specific project, really trying to see, hey, let's start with the people who understand this the best. Because that's what I think is so interesting about generative AI as a differentiation as we look at— look forward into the future. Because what it does is not what, you know, a typical machine learning system would have, which is fairly specific results every time you ask it the same question. Right? It will return you the same results every time. Generative AI is less like that. Generative AI is kind of this partnership between you and this almost alien intelligence that we're constructing in these models. And so you need to have explored it. You have to have played with it, right? And so it's one of those things where if you've got a new employee in your agency and you've never had them sit down and work with the most experienced people in your agency, of course they don't know what to do. Of course they're not adding value because you haven't had them work directly with the folks that have that experience on the ground. And so by really getting those folks hands-on tools and hands-on the different AI services that are available to them today, they will unlock what's possible moving forward, right? And again, they're busy folks. You don't want to just overburden them, but showcasing just like some of these easy wins that they could utilize to make these services better, Give me my AI lights have decided to dim on me. By focusing on some of these things, it will give you that chance to potentially see what that best use case for AI might happen to be.

So, you know, a lot of agencies are kind of at the foundational stages of setting up, right? We've got some that are more mature that are moving from pilot to enterprise rollouts. You've got a lot of customers that are kind of in that category now that are moving towards enterprise rollout. But for those that are like in the foundational stages and they're trying to set up maybe some of the initial guardrails and figure out like what jobs are most applicable to be leveraging these types of solutions?

Where do they start?

I mean, any thoughts on like what are the bare minimum things that need to be confronted maybe upfront before you pull the trigger on deploying an AI solution within your agency?

Yeah, I think there's a couple of things to start with, right? So one is making sure that you've got a pretty good level baseline of, you know, knowledge of what we're dealing with here, right? Because there's this whole new nomenclature that you start to get into, right? Like I mentioned hallucinations earlier on, right? Which I think is funny because if you— if we were having this conversation 2 years ago and I brought up hallucinations in parlance of government technology, I think folks might have wondered what I was doing earlier in my day before I joined the podcast. But now it's this like, oh yeah, we talk about hallucinations, we talk about machines hallucinating, and that's a normal thing for us to have a conversation about. But while it might be normal for me because it's what I do all day, it's still not normal for the folks on the ground doing the real work. And so by providing a level of overall training and enablement that can kind of hit them and make sure that they know what they're dealing with, right? And so when we are working with these systems, how do they— how should they think about the data that is feeding them and whether or not there is implicit bias inside that data? And so that's something that we need to help them to start to look at and say, wait, why do I have to worry about bias in my data? It's just my data, right? Like, what would I have to worry about inside of that? But by taking them through kind of these thought exercises of thinking about How is the data collected? What's the types of statistics that will live inside that, that you may or may not want to represent when you put this thing out into the world? And so there's some of those, you know, like, you know, ethical and responsible things that I think is really important for the agencies to work through is step one. I think one of the other big steps is then, you know, and this is something that Google itself takes very seriously, is just like, How do we educate the population so that there is enough overall understanding of these systems? One of the things I'm really proud about, proud of right now is that Google is just launching here in November what we're calling Launchpad for Veterans. And this is going to be like a focused coursework for our veterans, which is where a lot of our government workforce ends up coming from. And so by giving them a lot of these digital skills, We think that's going to help to empower them to know what's real in this world, what's not, and giving them a bit more ability to go and work with these kinds of tools without fearing whether or not they're going to work well for them in that use case. I think the other side of it in terms of what we should be thinking about here is, one, this is incumbent on us here at Google, but one of the things that I love about working inside of the state and local government areas is really just sharing the patterns that are working. And, you know, it's one of those great things that if you solve something in a unique way for one state, there are 49 other states and a couple of territories that would love to hear how did that happen and what can I— how do I get my hands on that exact same thing? And so this is something I think we need to be really loud and vocal about both the successes and the failures. And that the failures part is hard for, for us, both in the private and the public sector, because it's hard for us to admit sometimes that, that we've tried something and it didn't work. But if we do a better job of articulating that, that will build some public trust to say, hey, this one wasn't the right use case for this. But over here, we've gotten a significant amount of value out of using an AI service. It is one of those things where I hope that we can, as an overall group of folks, we can start to talk more broadly of these great things that are happening and shout them from the rooftops when it goes well, but also don't be shy about the things that we're doing when it doesn't end up adding value. Because then you start to get these kind of downward spirals where folks are like, oh, I've heard about that and it didn't go well for this city, you know, but they won't talk about it. They keep throwing money at the project. And now, now we're in a position where nobody else wants to— it becomes tainted in some ways. So those are a couple of the ways that I would look at trying to, try to help in this particular way.

So I have a 2-part question for you, and one part's easy. The first part is an extension to your last comments there. Given the free-flowing sharing of information you mentioned, you know, tackling the hard parts, is there something today, uh, from when looking at AI that you think could be— could benefit CIOs across the state and local landscape to be demystified or decoded? If we were to share something editorially, what's that— what's that gap today? What is it? Is it a policy gap? Is it a technological understanding? Is it a Is it something else?

So one of the— so I'll get on a soapbox for a second, if I may joke, because you sparked me. So one of the things that I worry here is we— generative AI will answer the best of its knowledge based on the knowledge sources that you've given it access to. And so one of the things that I've heard really, really frequently is this this pushback of, yeah, but like 90% of the time this thing knocks it out of the park, but that 10% of the time it doesn't, and we just— that's not going to work for us. We can't have a 10% failure rate. And I don't disagree with making sure that you've got a metrics-based evaluation of a technology service. Absolutely you should. But the thing that's hard is that I always like to come back and say, How accurate are your people at this particular task? Right? Like, are you comfortable with the statement that the current— that the status quo is better than what I'm offering you? Right? We've seen this even in language translation, right? Because you'll pay a human translator for something, but you don't necessarily— like, again, you're not going to pay a secondary translator to check the first translator's work. And so Sometimes you'll have this like, oh, we put it through the, you know, the Google Translate, if not the translate.google.com, that's a different system than what we work with for our public sector entities. But you put it through the Translate APIs and you're like, well, it's not quite up to snuff. And I, again, I want us to have the best of all possible translations, but I also want us to be working on an apples-to-apples kind of comparison, right? Let's make sure that we're doing almost a scientific double-blind experiment to say, all right, great, but let's evaluate where you are today and see whether or not this does get you to the next place. I do think it's important that that's not to say that we don't have to then educate because we, again, I come back to people expect computer systems to be like like specifically correct, right? And so this is a different paradigm to say that sometimes they're not. And that's weird for us because we're not used to dealing with computer systems that are sometimes right and sometimes not. And so we have to rethink some paradigms in our own workings with them. But I think it's really important that we don't stop because it's not 100%. Because I think that that's really holding us back from having the success that we could potentially have with these amazing tools and techniques that are out there today.

Well, I love you and I love that, rather, and thank you for sharing. Now, the easiest question you're going to get from us is how do we keep up to speed with all of the things that you're doing at Google and what's coming?

Yeah, so I think y'all will put up a couple of the links in terms of where to go look at some of the Google-specific AI things. There's a great search engine also where you can find some of that information. You know, I invite you also to come along with me on my LinkedIn. You can share that link as well. Over the next couple of months, I'm going to be trying to post a couple more videos and things like that where I articulate some of the ways that I'm looking at the different public sector use cases and what both AI and just technology writ large can do for them. And so would love to have folks check me out there as well. Those would be a couple of places to go find out more about what's going on in our world. Awesome.

Well, Chris, thanks so much for joining us, and, uh, until we see you again on another episode, uh, stay safe and take care.

Thank you so much, really appreciate the time.