The Future of AI - Understanding Where AI Is Headed and How To Secure It

All right, TGIF! Welcome back to In Case You Missed It, our weekly news roundup with Government Technology Magazine. Joe and I are back, and we've got another AI episode for you. And, you know, Joe, AI has been all of the rage across the country. We've got a series of different episodes that we're going to be doing doubling down on this, but it's what agencies are talking about, and they need guidance. They need to know what's happening within the space and, you know, Security continues to pop up. What are you seeing here, Joe?

I mean, a tremendous amount of conversation. Obviously, even on Monday, we were going through our editorial insights across government technology. You can clearly see that's what our government audience is gravitating towards.

Absolutely. And, you know, as we think about all of the advancements with generative AI and all of the landscape advancements with kind of the category of artificial intelligence, whether it's deep learning or natural language processing, This is a wild time to be around, and we haven't seen anything yet. And so as I was thinking about looking to the future and, you know, trying to think about how we secure that as well, I thought of an old buddy of mine, Dan Charbonneau, who's a senior sales engineer at Vaultree and one of the foremost security experts that always scares me with my own security, to come on and kind of break down what we need to know about everything that's happening. Plus, he's built his own LLM, and I'm sure has a few running right now with some AutoGPT on top of it. Let's invite to the virtual stage Dan Charbideaux.

Hello, everybody.

Hey, well, thanks for joining the fun on a Friday before a long holiday weekend. This is your first time on In Case You Missed It, so maybe just to kick things off, share a little bit about your background.

Sure. I've been mainly in information security, done a little bit of everything from being a CCIE, MCSE, 20-plus years ago at the lowest level to, uh, ending up building global security programs. So I've built 2, one for a financial industry and then one for a global e-discovery firm. And in the middle of all those things, I've also done, uh, work at Cisco, been on the vendor side, uh, many years at Splunk, and then most recently I've probably spent the past 2 or 3 years exclusively focused on security and some emerging technologies.

So what do you make of everything that we're seeing right now around generative AI?

I'm probably as big a fanboy as anybody else, and that's what got me down the security path of it. I'd say 6 months ago I started playing with it, and when I got it to write 80% of the code for a crazy little device that monitors for seismic impact in my neighborhood for explosions from a quarry, it blew me away. Started using it for some of my personal stuff and then scared the crap out of me.

As you look at all of these advancements and you look at everything that's happening with generative AI, We hear the security risk just around like, oh, your language that you input is going to be part of the foundational model used to train these things, but it goes far beyond that. And so tell us a little bit about, you know, as agencies are thinking about generative AI, what are those new security risks that they need to consider outside of just text becoming part of a foundational model or code becoming part of a foundational model?

We'll start with sort of my personal journey and then maybe the enterprise government view. In the past, if you wanted to get access to all of my things, you'd have to hack my computer, slowly access things. If you wanted to impersonate me, that's yet even harder task to do. And then even getting the exploit or getting me to do something so you get that access. So now imagine if I had a personalized AI, transit trained on all my email, my text, my location, my phone number. I mean, everybody I talk to. If somebody ever got that file, they'd be able to impersonate me, trick my bank. trick my company, and trick my friends. And it's not far-fetched because in the past 6 months you can already look at the use cases. It's been done to beat a bank's IVR system for voice recognition. It's been done to do a real-time FaceTime call and trick a person's family. This is just the very beginning of it. So when you start thinking about that, and then me handing my data to a third party to manage that, you got to start really thinking about that security because that is a different level of risk than we've, uh, had posed to us to date from all the things that make our lives convenient. Now you take all of that personal risk and look at it from an enterprise or government perspective. Now you've amplified it, right? All my personal problems on top of all of my company secrets and the access I have to and what it can predict. Traditional security tools, they're not gonna get us there, and generative AI is part of the reason for that, right? I don't know how far you want me to go, but if I start thinking it through, When you see how capable they are in their infancy, imagine 24 hours a day, 7 days a week, somebody training and leveraging something like WormGPT, DarkGPT, DarkBERT. There's several models out there that are focused on, say, social engineering or phishing, getting me to click the button. So, you know, those emails you get that are normally misspellings and terrible, those are rapidly going to become amazingly good, and it's already happening. So from that perspective, now imagine that attacking your government, your corporations, 24 hours a day, 7 days a week, everybody that's there, your entire family, and then every downstream or supplier to them. I don't think security programs are ready for that, and we haven't really implemented very many primary controls, and we're just at the beginning of trying to figure that out.

You know, you— I don't know, your past episodes and certainly Crossroads editorial coverage, I'm sure it comes to no surprise to you that government's top challenge, top opportunity is cybersecurity. So given everything that you just outlined and some of those, you know, scary and concerning threats, any recommendations recommendations or thoughts on what government should be doing now to begin to look at solving some of these issues maybe before they get too far out of hand?

Yeah, so there's— it's a multi-domain issue, right? It's multiple dimensions to this problem. The area that I've been focused on is the infrastructure first, because if you're not secure at the most elemental level, everything else you do on top of that's probably not gonna work. So if we look at the way we run software today in the cloud, or, you know, wherever it's running, if somebody can ever access that file, I'm toast. If they get my keys, I'm toast. So there's some new emerging technologies coming out that enable us to maybe start to address that. One would be confidential computing, right? It enables me to run my workloads in the cloud or run my models in the cloud, which I've already had several large language models running in what we call secure enclaves. Effectively, that removes the cloud service provider and everybody else from that tree of chain of trust. so they can't access the data and they also can't modify the code. I can start to have some base-level trust. Then there's new technologies, like think of somebody like a MirrorTab, where when I go to the web browser to go to my chat interface, which is how you're going to use it, I'll be getting— think of a streaming session to my phone, so no residual data will exist there. You have to start looking at new architectures and technologies to start protecting it. Because as I said, with the generative AI attacking you 24/7, it's going to get in, so we need new controls. to provide that level of assurance, not only for my personal data, but for corporate data. The next part might be something like a VaultTree. There's something called fully homomorphic encryption. It's the holy grail. It's being able to search data while it's encrypted without ever decrypting it. That doesn't truly exist yet, but there are things like VaultTree which have fully functional encryption while using it. Effectively, that means I could send my query, to the, to the model. It's encrypted. The model is able to execute against the encrypted data and send the response back to me. What that means is at no point ever is any of my data exposed. So even if I took over the box, I wouldn't be able to use the model. I wouldn't be able to see what you asked. I wouldn't be able to see the response. This is— these are some of the techniques I think we're going to need to start really looking at. And I think outside of large language models, that's pretty much going to end up needing to be everywhere because of the generative AI risk.

Yeah, there's no playbook for the situation we're in today. And I think, you know, to one of your points that you made to Joe's question around kind of those Nigerian prince emails that now all of a sudden have better grammar, can you break down, you know, like when you think of things like WormGPT, break down what that is, you know, and what it can do? Because this is more than just a ChatGPT that can write, you know, clever social engineering, you know, things. This is much more malicious and goes a lot deeper than, you know, what you can find on the dark web.

Exactly. I apologize, I need to go. My dog is going crazy. Give me one second. This is terribly embarrassing.

Yeah, no worries. So, so Joe, you know, you're seeing a ton of stuff here as well as we think about, you know, the, the security stuff. I mean, I was just in Texas, uh, at our state technology event there, and, you know, it's funny, we were focused on kind of what was going on in the market, and then all of a sudden generative AI comes into the conversation with a bunch of legislation and You know, agencies are legitimately curious about it. So, you know, kind of back, Dan, to that original question around, you know, WormGPT. What is it? How should agencies be thinking about these new tools that hackers have?

Well, what they should be thinking is, one, they're probably already doing some sort of phishing exercises, right, trying to get people to click on something they generate internally. Good campaigns or spear phishing campaigns have gotten a lot easier, but they still mess up. What you're going to end up with is en masse, spear phishing. So think of, instead of me crafting it to go after Dan Charbonneau, I'm going to have it learn all of your social, all of your Twitter, anything you post, learn a lot about you, and then I'm going to use that to decide how I'm going to present something to you, right? That's taking all the effort out of the guys who are trying to attack you, and now it's going to be happening en masse. So that's where we have to go to the assumption they're going to be much more successful and on a scale and pace that nobody's been Absolutely.

So, uh, so fun question for you. You know, you've kind of mentioned, like, we think everything is happening right now is overwhelming enough, right? But this is kind of a warm-up to, you know, what's gonna continue to evolve with the rapid exponential change that are, you know, that's happening here. And, you know, when I looked at emerging technology in the past, like blockchain and other things, we always are like, you know, 3 to 5 years, we've got time to prepare for that. Quantum, you know, coming up. You know, just looking to next year with AI and generative AI, you know, what are you— what do you see here?

Well, I thought it was gonna take a bit longer. I have a friend of mine, I can't mention the name of the company, and we were talking about this, and, you know, they had to bring in the big guns after an issue. And he goes, Dan, we're already seeing what you're talking about. It's not chained together yet. And when I say chained together, think of a phishing campaign with a technical attack campaign, right, where I've got different models really good at different things, one thing controlling it and then training it, doing all the next steps. They're not seeing that fully yet, but they're seeing better phishing campaigns, they're seeing better reconnaissance and attack against their basic infrastructure, they're seeing more downstream supplier attacks starting to happen. So I would've said 6 months ago, you know, 12 to 18 months. I'm thinking in the next 12 months we're gonna start to see a very large uptick in successful attacks. And it's really because they'll have trained them, tested them, and start just pointing at a target and going.

You know, you're outlining, I mean, some of the awesome potential around things getting better. We talk about that a lot in the context of how one company's marketing is going to improve and how people's copy is going to improve. And you just spent a decent amount of time outlining some of the concerns and threats and how those are going to improve. And, and likely get through many people's filters, both personal and technological. I want to pivot a little bit and look at AI from 2 vantage points, both the resident and then government. Starting on the more personal front, what potential do you see here in terms of how personal AI would impact how an individual accesses government services?

Well, I'm really excited for the government to start using it because I hate reading rules and regulations or even trying to find them to begin with. say I want to do a permit or a process, um, 80% of the time is spent searching for it and then trying to figure out what it means. So what I would love is to be able to go, you know what, I want to build a pool in my backyard, or whatever the problem is, and go to a ChatGPT that's been trained on my locality, all of their rules and regulations, the state, federal, anything that'd be relevant to it, ask it, have it bring me back the citations, but then explain it to me like I'm 12, because I— the language in these documents is abhorrent. And if you're not used to reading them, it's very frustrating, right? So that would be amazing. That would just— I'd love it. Then maybe other, from the personal perspective outside of permitting, a lot of times when you're looking for data and you want to analyze or interpret it, think of something like, um, if you look at Code Interpreter on, uh, GPT-4, the abilities of that are insane. Now imagine if I had access to the public data from my area to ask any question I want, maybe start trying to figure out where's the next, uh, sewage problem going to be because it's got enough data to figure it out. I don't have to be a coder. I can't think of a better one as it relates to the data I'd wanna use it for because I don't spend a lot of time doing that, but I'm sure Dustin has crazy ideas there.

Yeah, there's, it's a wild landscape, I think, as you think about, you know, what's going on on the personal front, the consumer front of it. So I wanna shift to kind of the second part of what Joe was talking about with the enterprise, because, you know, the number one challenge, you know, outside of cyber today in government is workforce. We don't have enough people, we don't have the right skill sets, you know, there's data literacy challenges that still exist. And so, you know, when you think about AI and some of these applications and kind of, you know, as Andreessen said, you know, software's eating the world and, you know, AI is now eating software, right? So it's like, how does AI impact government, like inside of government? Any thoughts on that? And also like, what does that set government up for?

Um, from the usage, from the government perspective, delivering a service to the citizens, I think of things like real-time translation, right? I haven't seen this in a government setting yet, but where I can real-time watch a movie without subtitles and it fixes the way the lips move. Now imagine people who don't speak very good English, their access and understanding of what they need to do, that's going to be insane. And you'd never be able to hire enough translators or people to do that. Right, right. It just wouldn't work. Um, let's see, how else would it impact the government? I imagine when, uh, you get calls about, uh, that's a good example locally here. We've had this, um, Board of Equalization and our property taxes increase. That took me a very long time to figure out all the rules and regs, what I need to do to go file an appeal. If they had all that already indexed and they could provide that as a service, that'd just be— I'd be much happier. Yeah.

Or what if people didn't even have to go to the government website and they just asked, you know, their personal AI to go and pay their taxes or go find Dan speeding tickets and, you know, use it, you know, against him?

That would be the panacea when we can trust personalized AI, right? When I know that my queries, my data, and my access is only available to me, that's when I'll trust a personal assistant. And that goes back to why I still don't trust it. I run my own models on my own local laptop. I use GPT for some controller stuff, but I'm worried about the inference data I send because the way most of these services work today is it's a trust model. I have to trust that OpenAI is going to not train on it to safeguard my queries and responses. I really think we need to get a little bit past that, and there's ways to do it, and that's part of the, you know, securing it with Vaultree and confidential computing to where maybe governments can run their own large language models powered by a foundational model from OpenAI, but never send the data to OpenAI. I mean, it's possible. Now, the compute power, things like that, we'd all have to work out, but I also see as the advancements come and transform architecture or maybe net new, versions of large language models that maybe don't use that, that your GPU power is going to go lower, it's going to become more probable. I was reading about a company that has 30 micro models where they can run all of them on a single server, and then they have their large, you know, controller model. And the reason they did that is so that they had domain expertise in each one of these so it wouldn't hallucinate in its responses, you know, by— in other words, not understand what I'm asking and respond with something that's completely crazy. And they're doing that all on-premise, right? So I see it going more that way. And just from a personal perspective, I think that's, uh, you know, the way the government should go is have some government foundational models that understand how their rules and regulations, all these things work with the micro models to ensure when asked the question, it doesn't hallucinate.

Well, you kind of, uh, answered part of this, but as you look to the future and you look at how AI is going to integrate with that government frontline worker into the workforce at large, will you see software-based roles, uh, or will there always be a human in the loop, you think?

I think for anything that's low risk, you don't need a human in the loop. Anything that's high risk, maybe retitling, right? Anything that deals with actual money or property and the reassignment thereof, I think you're always going to need a human in the loop.

Yeah, I think we agree with that as well, and so do a lot of the, the civil service, uh, you know, agencies that are out there. So I kind of have a 2-parter for you. I'm going to start first with just advice for agencies. So everyone's testing it today. We've done surveys and, you know, over half of respondents are using it, even if they're not telling their agencies that they're using it. So what advice do you have for agencies today that have employees that are doing this, whether they know or not, to maybe empower them to do it more securely? Or, you know, is it a guidance policy procedure thing here, or do they need to look at kind of technical solutions right now as a part of this?

Ah, it's like a third rail. Um, if you ban it, people are going to do it anyway. It's just that's not going to work, and it's not ready right now for prime time for most organizations, including the government. Why? How am I going to secure my vector databases with maybe custom content? How am I going to secure my chat logs going out to a third party. So knowing that they're gonna use it anyway, I'd start off with, here's what you can and cannot send. I'd look at helping find ways to monitor it because you're approved to use it, but we need to find when you do something bad, right? And then say, you know, training and guidance to start. At the same time, they should have a group coming together to go, what types of questions, what types of data are we going to have in here? And then how are we going to secure that? Is it going to be on-premise? Is it going to be into a cloud? And look at new technologies. You know, the traditional antivirus firewall endpoint detection response, network detection response isn't going to be sufficient.

Yeah, and the second part, which I have to throw this in here because it came up yesterday, so we're talking about AI, we're talking about all this stuff that's happening, and someone asked one of the legislative representatives about quantum. And so as we think about, you know, re-architecting the security environment for this new generative AI wave, where does quantum intersect this? Is it an architecture for both? You know, how should agencies really realistically be approaching that technology in the context of what's happening today? 'Cause you see a variety of different thoughts on it, but it's also hitting, you know, this exponential curve of progress. So just wanna get your final thoughts on that one.

Final thoughts on that one. Quantum, when I, about 12 months ago, I'd have told you I wasn't terribly worried about it, you know, spending a year working with different organizations selling confidential computing, We found that it actually is a very large concern for many and coming forward rapidly. And this goes back to the only— there's a couple of ways you could do it, right? There's the quantum key encryption stuff for doing key exchange as a level against it. But when you start looking at new technologies, you know, the 2 biggest ones I see, you use something like a Vault Tree, which is a combination of homomorphic encryption with some elegant encryption sidesteps. You use something like Confidential computing combined with those encryption and key exchanges, that's where you need to go. Both have different approaches, right? If you go confidential computing, you create an encryption control plane, the ability to swap out those algorithms and those key exchanges quickly across all your applications. 'Cause let's say they chose one today, how long would it take you to implement that?

Hmm.

Right, so there's a benefit of confidential computing. On the other side, you use something like a VaultTree. Well, now I'm encrypting the data that is using symmetric key encryption, and now it's no longer, you know, an attack against the RSA or, you know, the other ones that are more quantum susceptible. You still have your key exchange problems, but, you know, there are protocols to handle that. Keep in mind, not an expert in cryptography, but that's— those are the things that we're seeing that people are looking at to do.

That's insightful. So, Dan, the easiest question that we've thrown at you all day is, how does our audience keep up to speed with you? Where do they go to, you know, keep tabs on you?

Uh, the best place would probably be LinkedIn. I haven't been, uh, publishing or posting much, but that'll probably be changing here very shortly. I've been talking with Dustin about, uh, several blogs about, you know, personalized AI risks and what you can do in healthcare and finance, and then maybe an overarching security architecture to address it.

Oh, great. We appreciate that. We appreciate you coming on, and we also just threw out another link for the organization that, uh, you referenced, uh, throughout our call, so we'll make sure that our audience, uh, can gets in touch with you there as well.

Thank you.

Well, Dan, it's a pleasure. We appreciate you sharing your insights with us and our audience today and joining Dustin and I. And until then, everyone have a wonderful long holiday weekend.

Thank you.