EP185 SAIF-powered Collaboration to Secure AI: CoSAI and Why It Matters to You

Secure running. Hi there. Welcome to the Cloud Security Podcast by Google. Thanks for joining us today. Your hosts here, as ever, are myself, Tim Peacock, the Senior PM for Risk Analytics and Cloud Threat in Google SecOps, and Anton Chuvakin, a reformed analyst and senior staff in Google Cloud's Office of the CISO. You can find and subscribe to this podcast wherever you get your podcasts, as well as at our website, cloud.google.com/podcasts. If you enjoy our content and want it delivered to you piping hot every Monday for when you're walking the dog, commuting to work, doing whatever, hit that subscribe button. You can follow the show, argue with us and the rest of our Cloud Security Podcast listeners on our LinkedIn page. Anton, is today an episode about trigonometry? Is it about cosine?

It's not about sine and cosine.

Ah, so I'm going on a tangent.

We're not going on a tangent. Yes. So see, I did set up for this joke, so I'm not lost. I can't only joke about things. Listeners, I hope these jokes hit a Okay, but now we have to go continue this.

No, no, no, we don't. We can stop. We don't have to go in circles about this. They're going to think we're real squares if we keep making geometry jokes. It's well outside our area of expertise. Listeners, this is an episode about AI and standards and how those things come together.

And it has, believe it or not, at the end, some useful advice on AI governance.

That's so mean.

No, I mean, I really mean it. It's not just very fun. But at the end, there's also something you can use even if you're not an AI researcher or dealing with AI security. There is stuff about, dare I say, governance.

Oh, he dared to say it.

I know, right?

Maybe with that, listeners, let's turn things over to today's guest. Today we are joined by David Labianca, Senior Engineering Director here at Google. David, thank you so much for joining us.

Thanks for having me.

We're in a world where the the risk of AI is both broad and deep, and we've made a lot of headway with our SAFE framework. Can you give our listeners the 90-second tour of SAFE, explain why it's so popular? I'm not saying it shouldn't be, just how did it get there? And then talk about where do we go next with this thing?

I think it's a great question. So part of the problem, I think, with any new domain is folks don't know what they don't know, and it's the major challenge. And I think the beauty of the SAFE framework and its initial 6 pillars is it makes really clear for the integrator, the developer, the enterprise user of AI, what they need to worry about and what they need to do well. And some of the really good news is they just need to do foundational security really well to at least be on a level playing surface for the use of AI. You want to make sure that all of your classical security programs, your detect, your hunt, your understanding your risk and what type of data you're using in any system. All of those things are super important when you're using AI, just like for classical systems. There are definitely more unique elements of security controls and invariants that you need to worry about as you get deeper into AI, how you fine-tune your own use of other people's models, what's the supply chain controls. But that's all secondary to whether or not you know you're using AI, you know what you're using AI for, and then that you're sure you're doing the core blocking and tackling really, really well. And if you were going to invest in an area, I think it's still nothing really changes. Your company should be doing the core blocking and tackling of cybersecurity controls and information security controls as best as you can. And that will set you up for a much better pattern. Yep, there'll be another unique risk, there'll be other unique challenges, but at least you'll know, oh, I'm not going to lose control of the nodes executing this and be training somebody else's malicious LLM for them because I no longer control my infrastructure. So I think that's why the SAFe framework is so popular because it doesn't start with the whiz bang. It doesn't start with everything's changed. It highlights the fact that there is change and there is difference. And thinking about how you use AI as a help for the defender is something you should worry about and think about and take advantage of. But you need to start from the basics. And I think, again, I think that's why SAFe really has landed and resonated with folks.

I mean, it's always like in a somewhat humorous take, sometimes I would say that it's likely a SQL injection in an LLM. application rather than robot rebellion is what you need to worry about. There are definitely things that would bite you today or tomorrow rather than things that theoretically may cause problems later down the path in the later centuries or later decades or whatever. So this is useful. I mean, we don't just run away on fun sci-fi pursuits. So we also launched something called the Coalition for Secure AI. And I'm not sure, do we have an approved acronym, COSAI? I think COSAI is what we call it.

COSAI is the approved pronunciation of it.

COSAI.

Cool. Okay, awesome.

I love it.

You tested it? I guess I missed that training. So this is a collaborative effort to address AI security challenges. This is not a standard. This is not a document. This is actually a collaboration. So what are our goals and expectations for COSI? And how would we know we succeeded? It's kind of a tough question, the success part. Like, do we have a secret success metric? If it happens, we succeeded.

So it's a great area to focus on a little bit, because I think it builds from our release of the SAFe framework in the first place. As the SAFe framework was published, you guys will keep me honest, June or July of 2023, we started to get, we were already getting them, but we got even more inbound questions from customers, peers, entities around, hey, how are you doing security at Google? Right? Like we know there's got to be more than just what's in the SAFe blog post that you've posted. How do we get to a point where we can secure our own ecosystems and our own solutions for AI and And that message got louder. We got more and more questions about it. And by November of 2023, it became really clear is that folks need access to the underlying best practices, learnings, guidance patterns that will allow them to secure their own use of AI, whatever that means, whether they're deploying an open source model, whether they're doing fine tuning, whether they're just using ChatGPT or Gemini as an API to build their own models. For one of their services, doesn't really matter. They need to know how to do it, and they don't want to trip over all of the same, you know, shin kickers that Google or other companies who've already gotten there have to. And we're really, you know, privileged in a lot of ways. We have 10+ years of experience of maybe what we now would call working with classical AI and what the security controls look like for that and how you deal with the overall impact to your practices. And people really wanted that content. But additionally, we knew from our experience of sharing Google learnings in the past that there's no one way to do it, and what works for Google doesn't work for everybody else. So we knew we were going to need a really large breadth of voices in the room to share all the different ways they approach the same problem and the same and the different solution that we provided for that problem. We needed their voice in there, and we need to make sure it both really provided influence from others who are building AI, others who are integrating and controlling and governing AI solutions, technologies, vendors in the rest, as well as consumers and users of AI at an enterprise or an entity level. Because only when we had those groups together would we start to be able to really do that. Well, here's the real best practice, not just a best practice that works when you have Google's ecosystem and you have Borg underneath it, not just working for OpenAI because, you know, you all are experts in AI. You needed to have that really broad story. And that's what led us to, well, then the only way to do this is a neutral third-party forum, right? That's the way to do this. And it came to our real mission because the mission here is we want to democratize access to the security learning, security best practice, security understandings, and security patterns necessary, because everybody should have the ability to know what's necessary to deploy secure AI, to use secure AI. Obviously, there is no homogeneous way to apply that, like pixie dust and a magic wand and go, now everybody's secure. But if the content's out there and the best practices are out there and as much as possible materialized into open source solutions that then all of the great companies in the world can compete around providing their own solutions on, like, then we put people into a position where They're much better prepared for how to move forward. They've got really broad solutions to choose from, not just, oh, this worked for Microsoft, or this worked for Google, or this worked for Anthropic. And that was our goal. And that's where we started from. We were really, really lucky that as November, December, January came around and we started talking about this with initially a very small number of our peers and our partners, it resonated really strongly. Really, really strongly. And we had more and more voices like, yeah, this is the thing. Like we've got to really get this stuff into the hands of our users. We want to avoid that moment that happened around cloud adoption by enterprises where adoption was way leading way out over its skis before controls had caught up with people learning around how to control cloud, you know, the early 2010s and that kind of stuff, which we've done an amazing job getting past, I think, but Like, how do we get them ahead of that? And the goal of getting them ahead of that is like, we gotta get real stuff into their hands. But additionally, it's not just one voice in the room. It's never gonna just be COSI because there's amazing other work going on. And what we really wanna do is fill the gaps and provide the meta guidance across all of these other really wonderful entities that are doing work in the space, whether it's OpenSSF and the great work they're doing around AI provenance and, and the statements around how you provide a proof around that provenance, whether it's the Frontier Model Forum, the FMF, really focusing on what do you do about really capable AI and how do you secure the futures of AI, but all the, the much broader mission they have than just security of an AI.

So when we talk about that broader mission, David, one thing that is so notable to me about COSI is who you got to join you on this mission from the broader community. I heard Microsoft is in and I heard OpenAI is in. So how did you do that? That's awesome.

We, again, like I mentioned, we were just super lucky, right? Like, as we started talking to the Microsofts, the OpenAIs, the Anthropics, we had 14 launch partners as of our launch at Aspen last week. We have more already than that number. This same kind of goal of making sure everybody had access to this best practice and learning, and that was broad, just really resonated, right? Like, we had early, early, early feedback from companies like Intel and IBM and NVIDIA about, yes, we've got to get this content out there. You know, the Anthropics, the OpenAIs, the Microsofts of the world, just really, really invested in like, how do we get this message out there? And as a whole, it was an easy sell, right? Like going to these groups, you know, like, like anything else, like why this forum? Why not the infinite number of other forums? You want to answer those questions. We wanted people to pressure test that because we didn't want to start something that was just a lot of names on a billboard and go, hey, success. We wanted to make sure there was a problem that we were going to address and that people saw that as something they wanted to invest in. And that really, Tim, that was our luck. is that this problem domain at that meta level of dealing with all these problems and integrating all the great work from other forums, everybody saw it as a core problem.

So this is— I don't want to shorthand it to it was just luck, but I think it's also the broad appeal of the problem because ultimately everybody is looking at this from their sides and everybody wants to solve it, but it is solved collaboratively or not at all. I think that part is somewhat clear. So I want to also kind of compare and contrast it with sort of 2 types of entities perhaps. So there are of course other organizations. There is Frontier Model Forum, there is Open Source Security Foundation, there is CSA, and there's OWASP. So like sort of groups of people working on various types of things. And there are also standard bodies. In this case, I'm thinking of ISO and NIST, which are of course quite interested in AI as well. So, how does it all fit together? Like, I don't want to go and say, yeah, well, we need another standard, because ultimately it's not so much about writing a standard, it's about collaborating. But how is this— how does it fit together?

Are you sure we don't need another standard? We've already got so many. What's one more?

Well, as a person with a big ego, I would obviously agree with you and say, as long as my name is in it, I'll go for it.

Ah, the Anton standard. We actually have an Anton standard. It's, David, if somebody wakes you up at 3 AM and says, what framework should I use for securing AI? What would you recommend? That's the Anton standard.

Whatever Anton has published, that's what it would be. What has Anton talked about last? That'll be my one. No, so look, I think it's a really great question. When you look at what's needed, right? Because it's driving kind of our hierarchy of needs for the COSI group. People need best practices. They need them as quickly as possible materialized into open source and code so they can leverage them simply and easily. And then over time, they need those things to materialize in standards. And because of that, there is in our minds no difference from the fact that we want to collaborate deeply with the OpenSSLs, with OWASPs, with the CSAs, as much as we also then want to collaborate and provide input into and help expand the stories of what a NIST or an ISO or any other entity in this space globally, right, is looking at from a controls and standards standpoint. We believe that the best form of success for CoSci in some ways is other people, like, stealing like an artist from the stuff we publish and using that as part of what they deliver and the types of problems they talk about and the way they build standards. You know, it would show we've done a really great job of providing the core materials required to get somebody successful if people then start to crib from our stuff and say, hey, that's going to be part of our standard, right? Like, that's an amazing outcome. That's a level to your earlier question around, like, what's success look like? Success is use. Success is people coming to us and going, this is helpful. Long-term success is, and then look, it was so helpful, we materialized some of this stuff through our partnership with the different standards bodies. I think like you guys are, the fun you're having, standards isn't the place to start at this moment. There's really amazing work going on in standards. We need to participate as an entity, as an ecosystem, as a company, and make sure we give them all the support we can. But right now for CoSci, it's like, let's get best practices and code materializing that out in people's hands and then collaborate with those other bodies and help make their efforts more successful.

That all makes sense. And I have struggled to get through some of this with a straight face, so I'll continue on that thread. How do we keep up with the pace of change of AI to make sure that this stays relevant and stays successful? Because AI, I hear, moves quickly. Yes, maybe. I think it does.

Some of it goes back to where we started with SAFE, which is that there are basic core problems you need to solve for, right? And general cybersecurity practices and making sure you know your inventory and your assets and making sure you have control of your network inbounds and authentication across all of the ports and authorization and constraint of use by either your models or your applications. They don't just have all the privileges in one. Like, those things are applicable no matter what goes on in the AI domain, right? How you work with third parties and how you work with SaaS and open source and all like those. Critical aspects of security are required as we go forward. There's absolutely going to be dramatic change over the coming months and years around the new ways we train models across all of the ecosystem to either provide more controls, more defense against prompt injection, more capability. Like those things are going to happen, but they're not going to invalidate those core principles of do you know how you're you're adapting your defenders program to deal with the fact that you probably have AI usage now? What the AI does might be very different, but like you've got AI, you now need to think about it. Have you thought about how your security program is impacted by its own use of AI. Like, if you have an LLM-style model looking at a log stream, you need to remember, well, your log stream is potentially a prompt injection vector for now your security program, right? Like, that's not going to go away. You may get more controls from your LLM that it doesn't matter, but like, you always need to worry about that. And then the overall risk governance aspects of like how you deal with new capabilities, like those ways to think about that will continue. And then There will be dramatic change. There will be ground shifts and paradigm shifts underneath us as we move forward or beyond us, and we'll adapt to those as an ecosystem. But we think these core aspects, Tim, like these are things that people just need to get going on and get moving on or continuing moving on because there's some companies doing an amazing job on this stuff. So that's why, yep, it's there. It's something we're aware of. We're not worried as much as aware that it's going to keep shifting. So let's focus on some of the stuff that's really stable as we go forward.

Yeah.

That actually makes sense as an answer because I think that over-obsession about the rapid pace of change have led people to not do something that is the same since the '80s, and they lost because of it.

Right.

I'm thinking credentials, MFA, authentication, the stuff that in theory was in security books in the '80s, and it's basically a mistake in the 2020s. So, that's why I want to bring it back to practical matters. And I think that was launched with much glamour, but there's also practical work to be done. It's not just a committee of people to chat about AI. There are projects. How do we actually plan to deal with the proposed projects, the outcomes? How do we expect to see the benefits? I'm thinking more about something that a mid-sized bank CISO can use in a year or 2. So this is kind of like my practicality question/projects question.

Yeah, so like any forum, right, that's your big worry, right? Yep, you've launched. Now do you deliver something, right? And that's where we're all focused now as part of the board and getting our workstreams formally launched. and getting the workstreams established in our Technology Steering Committee. The reason we selected these 3 inaugural workstreams, right? So preparing the defender for the changing environment, AI security governance, and then the overall how you deal with the security supply chain around AI and these blended systems and with classical approaches. Because there's one real need today, right? You know, like how you as an engineering director or a CTO of a mid-sized bank to, as an example, think about What's my posture around security for AI? How do I think about it? What's my risks? Like giving folks in those positions the questions that they need, and then the ways to map those to classical risks or classical control domains or assurance domains they think about. Like that's super important. People just need that because to your points earlier, people are adopting this now, and they want to know they're going to have something. So as well as like the defenders, lots of companies have already adopted AI. Giving defenders a way to now look at what do I do for watching for threats and vulnerabilities? How do I keep track of the fact that, hey, there's problems in this ecosystem and this domain? Because every ecosystem has problems, right? There's nothing about AI that's going to be new around vulnerabilities. It's a truism in technology. So getting that stuff into people's hands quickly is why we've selected those workstreams. We think it's the real need. And again, it came from those listening sessions that we started with before we jumped into this. It's like, what were people asking for? What were their big problems today? There are big problems beyond this, and there's a lot of additional stuff that we think we could do and we could bring.

So that part makes sense. And I think that the governance project for CoSI is the one that's, I want to say, the most exciting, but governance and exciting, I'm having trouble uttering the words. So I would say that for many organizations in the early stages, it's the most pressing because confusion about AI governance, AI, shadow AI, AI by employees versus gen AI versus other stuff, It's all kind of a mess. So it's a tall ask, but can you give us a 3-second— okay, 30-second answer how to magically fix AI governance? I think we asked Phil Venables that question. He kind of answered. So we're gonna test David on that too.

I'm sure he has a better answer than me. So by default, I start from know your denominator, right? Know where you're using AI, know what business processes you've put AI into. What you've impacted with the use of AI, right? Now, granted, to your point, Anton, it's difficult if you've got lots of shadow usage, but you still have to ask that question and at least know the knowable. The second bit is what data are you using in it? Or is it just vanilla AI from the ecosystem? Are you using people's foundation or trained, tuned models from Hugging Face and the rest? Or are you using your own data? And if you're using your own data, do you understand how you've done your fine-tuning, what the impact is, or if it's just, you know, prompt engineering, how you've used your own data in the space? And then the third bit is, have you started to prepare all your control functions for that use of AI? Does your assurance and auditing teams know where this change is and now what they need to think about differently? Do your hunt and detect teams have in their mind that, hey, I've got this AI model, I've got to now worry about changes on Hugging Face or problems with the providers or just the challenges in my own usage? I think those 3 steps get you starting to move forward towards a governance program for the security aspects of AI. And there's obviously a lot more than the security aspects of AI for governance, but that's where I think you start for security.

I think that makes a ton of sense. And so I hate to do this. I do this every time. Anton, why is it always my job to do the thing I hate doing? But I have to ask our traditional closing questions. One, what's your favorite tip for helping organizations adopt AI safely? And 2, can you recommend some reading material for our listeners?

So on the one tip, it's— I'm a broken record. It's just do your classical security really, really well. Keep focusing on the core blocking and tackling. Make sure you're doing your monitoring, making sure you're locking down access to your systems, doing your identity controls well. None of that becomes less valuable. Honestly, it becomes more valuable because now you've also got data exposed in maybe different ways in those systems. That's my one tip for everybody. As far as reading materials, I think right now, keeping track of 2 different aspects of the AI domain are the most important. And it's not one particular thing to read, but it's keeping track of how NIST, ISO, EU bodies are looking at regulations and standardization around AI. Keep track of what they're looking at, is in control, where they think problems are, how they think about risk classes. I think the EU draft AI Act or the AI Act now has a really wonderful thought around risk classes and what that means. Learning about that. And keeping abreast of that's super important. And then the second piece is there's some really wonderful research and white papers coming out of the different coalition board members around secure supply chain, around keeping track of your models, around using your models for defense. And I think keeping abreast of where are the folks who are doing arguably a good job around the use of AI today, how they're leveraging it, how they're moving forward with the space, I think that's the other aspect that, again, if you're learning about AI, where are standards going and regulation going? And then how are others doing it and doing it well? Those are the 2 things that really set you up for the future.

I actually love it, and I think that it's nice, crisp, and useful. So thank you very much for being on the show. This was really insightful, and I think it would help make the world of AI a safer and more secure place. I mean, to me, this is very uncommon for me to have some kind of a loud proclamation in the end, but at this time, I feel very moved by David's take on this. So I'm saying it.

Hey, thank you both, Anton and Tim. It was wonderful to be here, and thanks for the moment to share this conversation with you.

Thanks so much for joining us.

And now we are at time. Thank you very much for listening and, of course, for subscribing. You can find this podcast at Google Podcasts, Apple Podcasts, Spotify, or wherever else you get your podcasts. Also, you can find us at our website, cloud.withgoogle.com/cloudsecurity/podcast. Please subscribe so that you don't miss episodes. You can follow us on Twitter, twitter.com/cloudsecpodcast. Your hosts are also on Twitter at Anton_Chuvaikin and _TimPico. Tweet at us, email us, argue with us, and if you like or hate what we hear, we can invite you to the next episode. See you on the next Cloud Security Podcast episode.

Security.