Project Atlas: Wiz's AI Vulnerability Research

Security.

Hi there. Welcome to the Cloud Security Podcast by Google. Thanks for joining us today. Your hosts here today are myself, Tim Peacock, a Group PM on Google SecOps, and Robert Patrick, another Group PM on Google SecOps. Robert, thank you for joining us. Super stoked to have you guest host with me today. You can find and subscribe to this episode wherever you get your podcasts, as well as at our website, youtube.com/cloudsecuritypodcast. If you enjoy our content and want it delivered to you piping hot, subscribe, ring the bell, do all the things that a podcast host is meant to say at this point in the show. Robert, what are we talking about today? This is a fun one.

So we're talking about using AI to hunt for threats and using some very different techniques, using multiple agents, multiple models simultaneously. Fascinating insights that the team has found.

So I think you might be underselling it. We get a story here of cross-tenant data access. We get a story of, well, also cross-tenant data access, both found in hours in major platforms using AI.

Yeah, this is, it's kind of both amazing and terrifying.

Yeah, yeah, it's both, isn't it?

So there's a lot being found here. Yeah.

So maybe listeners, with that acknowledgement that there's terror afoot, Robert, anything else to add before we turn things over to today's guest?

So one thing I have been top of mind for me in the last few days is With the launch of the latest version of CodeMender, using the 3.5/Cyber model, which is the first model of Gemini where we focus on cyber specifically, uses a technique of using multiple agents to build a single report— different skills, different thought processes. And I think the story we're about to hear talks about what it is to use different specialties, different agents, different models even, together to do something a little bit special. So I'm looking forward to hearing about that.

I think we're at a really interesting moment where companies like Google, which are both a lab and an app provider, companies like Wiz, who are an app provider on top of other people's models, are at this moment where we're announcing these things, which are in some cases a new model and in other cases a conjunction of models in an application that makes it look like one model. I think our industry, our marketing teams, Even product management teams internally, I find, we're really— I've never felt such a struggle personally to keep up with the pace of change and the pace of new concepts for how these things connect to each other. And so, maybe with that, listeners, I will admit that I struggle to keep up with this, and this is a guest with whom I struggled to keep up as well. So, without any further ado, let's turn things over to today's guest. Today we're joined by, and I've been warned this is hard to get right, Nir Orfeld, the head of VM vulnerability research at Wiz. Nir, thank you for joining us today. Super stoked you're here. We've got some exciting news in the vulnerability research world, and you're actually joining us from Vegas where you're presenting this, this week. So this is super timely and awesome. I want to talk about Project Atlas, which is your vulnerability discovery work, and your first Zero-day cloud hacking competition. Could you first tell us about your cloud hacking competition? Why did you start that? What did you discover? And then tell us how this turned into Project Atlas.

Yeah, of course. So actually, the story begins like, I don't know, 5 years ago. We've been doing cloud vulnerability research for around 5 years, like since the early days of Wiz. And we had like pretty good success at finding cross-tenant vulnerabilities. Now, cross-tenant vulnerabilities are vulnerabilities where we, from our account, are able to access the resources of other customers using the same cloud provider, whether it be Azure, AWS, or GCP. And this is the worst kind of cloud vulnerabilities. This is as worst as it gets. And to be completely honest, We had a playbook. We had a pretty simple playbook where we found a zero-day vulnerability in some open-source software that the cloud providers rely on, such as, let's say, PostgreSQL or MySQL, major database engines. Then we used the same vulnerability to execute code on the cloud provider. And the same vulnerability being a zero-day that affects open-source software, it actually affected all of the cloud providers. And from there, we'll try to find vulnerabilities in places that other researchers don't look for, because you need to cross the first security boundary to execute code and then to poke around. And we had, I think, more than a dozen of these kinds of vulnerabilities across the years. And being completely frank, having a playbook makes it much, much easier, but it's also very worrying. Because we have this playbook, we report these vulnerabilities for the open-source projects and also the cloud providers. But maybe some less honest or more malicious actors can use the exact same vulnerabilities to compromise customer environments, which is very worrying. And we noticed that we are basically in a very small group that uses this playbook. So after working with all of the cloud providers like Microsoft, Amazon, and Google, we decided to actually point the researchers to places that we deem important. We noticed that because these projects such as PostgreSQL, MySQL don't have bug bounties, like if you find a vulnerability in these projects, then you don't get anything basically. Then the skilled security researchers are not incentivized to actually look for vulnerabilities in these projects. So we wanted to fix that. So we brought together all of the cloud providers in the same room. We created an event where we published a list of targets, the targets that we and the cloud providers deemed critical for cloud providers. And these are the backbone of the cloud, the open source projects that every cloud provider uses, every cloud provider relies on, but security researchers are not researching.

Hmm.

So we published the list. We had like, I think in total of prizes, like was it $5 million? Like we wanted to really incentivize the researchers. And once a researcher submits an entry, he flies out to London where we had this event where they actually execute the vulnerability on stage. And then they'll come to a disclosure room with the maintainer of the project and all of the cloud providers in the same room. thinking on how they can mitigate this vulnerability on the spot. It was a very, very special event. But in this event, we also noticed a shift where a lot of the vulnerabilities being found and reported to us were found by AI.

Hmm.

And we were curious about it because we've been doing vulnerability research for more than 5 years, I don't know, 10 years. And thinking about it, and in every year when I've been doing vulnerability research, there was someone saying, in one year from now, we won't be needing to do vulnerability research. Someone will invent a machine that automatically finds vulnerabilities. I mean, vulnerabilities are like a repetitive process, and we'll have a program that automatically finds all of the vulnerabilities.

Hmm.

Over time, you become skeptical hearing this. But in that competition, it was the first time where we believed there was a shift. And it's actually something that is happening. It's something that's possible. It's possible to use large language models to find complex zero-day vulnerabilities that have an insane impact, particularly on the cloud environments. And basically, they're like the start of Project Atlas.

So that's a really cool start because you came at it from such a place of both economic incentives and good for the world, but also a very clever way of breaking cloud providers in the first place. And then you just stumbled into, oh, wow, people are already doing this with AI. That's great.

Yeah, absolutely.

Huh. That's remarkable.

Yeah. The way you use AI in Atlas seems a little different to many folks. I think a lot of folks use their one core model they rely on. They say, this is the best thing that we can use right now. But I think your thesis is that will change. The top model for X changes constantly. Can you walk us through how did you go about this? Because you do something a little different here.

Yeah, of course. In the start of Project Atlas, we didn't come from AI backgrounds, so we needed to learn everything from the real basics. At first, we had a very naive way of building the system, but then we started to create evaluations, like tests that our system needs to go through to make sure that we are progressing. Because without these tests, you have no idea if you're going in the right direction. I mean, you could have a hunch that says, yeah, this is the right thing to do, and this is something that will dramatically improve the system. But if you don't have any evaluations to test it, you have no idea to actually measure progress. And something that we discovered along the way is that usually our intuition is wrong. The way that I look at vulnerability research as a human being is different than the way that AI is doing vulnerability research. But during this evaluation phase, we discovered that to really have a robust and predictable system, we need to break down the process into kind of deterministic or well-scoped tasks. Like, first, understand the codebase, then split it into bite-sized pieces, then find a vulnerability in each bite-sized piece, validate the vulnerability, make sure that it's actually a vulnerability and it's actually exploitable, and then write the proof of concept or exploit. And this is the process that took us a lot of iterations and a lot of evaluations to reach to. But then when we looked at what would happen if we use different models for each stage, because we wrote these evaluations for each individual stage, we wanted to make sure that not only we have a good outcome end to end, like we find the vulnerabilities, but we are like finding the— we are doing each step optimally, like each individual step that I just talked about. And while doing these evaluations, we discover that for each step, each model is slightly better. There isn't just one model that is like best at all tasks.

Mm-hmm.

But some models are better at validating vulnerabilities. Others are better at finding them. Others are better at exploiting them. And this is like a continuous process. Every new model that comes out, we run it through our rigorous evaluations to make sure that if it performs even a little better at a certain task that we have in our system, we just replace it. And I think that's kind of the beauty of having a system that uses multiple models. Because I'm not scared that there will be a better model tomorrow. I'm not scared that we'll have a model that finds all of the vulnerabilities because Me as a defender, I can just use it. And actually, as the model progresses, the system compounds with it.

And you don't feel guilty about firing the model when it's not as good?

No.

Zero guilt.

That's good. So how do you see this changing life for vulnerability researchers like yourself? It kind of sounds like you're describing the end of the road for a thing that you've been a part of for a long time.

Right now, not really. I think it's really like—

No?

It heightens the ceiling of what we can research on, and both the scale, the amount of vulnerabilities that we can find. If up until now I was tasked to find a vulnerability in an open source project, it will take me a month of dedicated 12-hour days where I look at the computer and try to find the vulnerabilities, and I couldn't do anything else. Not only at work, but in general. My life was completely dedicated to finding these vulnerabilities. Now I can find hundreds of vulnerabilities in a month or even in a week, which I think it's like, and the impact you're able to do on the world or on security in general is insane. But also you can be really, really ambitious. You can find vulnerabilities that up until now you didn't have, You would say, okay, I will invest, I don't know, a quarter, like 3 months looking at this piece of software. And no one guarantees you that you'll find a vulnerability. But now that the economics actually changed, it's different. I think I have a really good example of this. Sagiv from my team, a really talented security researcher, literally one of the best in the world, had this dream of finding a vulnerability in GitHub. The source code version control, like I think number one.

Everybody's heard of GitHub.

Yeah. And he had this dream, but GitHub is closed source and you need to reverse engineer it. Reverse engineering is like makes the pace of vulnerability research even like pre-AI like 10 times more. Like he will need to invest, I know, literally months in reverse engineering the GitHub binaries in order to Maybe somehow find a vulnerability, and he has like really good intuition to finding vulnerabilities. But it just takes time to understand closed source binaries. But with AI, he was able to ask questions about this closed source software. The AI was doing all of the reverse engineering, like the tedious work. Like the AI didn't find the vulnerability like completely autonomously, but by asking. questions, he was able to find his dream vulnerability in 48 hours, which was—

Sorry, hold on. He found— and is this the thing that we want to talk about? This massive RCE in GitHub? He found that in 48 hours?

Exactly.

Okay. So, all right, listeners, Nir has taken us one step ahead in our story here, which is you guys found a massive remote code execution in GitHub. which is used by the entire world for source and version control, and you found it in 48 hours using AI?

Yes, that's correct.

I did not know this part, listeners. This is genuine surprise by me. So tell us more about how this happened.

So basically, again, it really shows how AI lifts the ceiling of the quality of the things that you can research as a security researcher. As a researcher, you have a bucket list of things that you want to find vulnerabilities in. You have your dreams. And with AI, it becomes suddenly much more possible. So what Sagi did to fulfill his dream, he set up something called IDA, which is like a software that helps you research closed source binaries. Essentially, closed source binaries are assembly. It's machine language. You as a human, you're not programmed to understand this machine language. It's something that you do get better with time. But still it takes— I don't think there's a single human that is better at reading assembly code, which is designed for machines, as fast or as good as researching source code. So we used IDA, which is the flagship product for researching these kinds of assembly languages. He connected an MCP, an interface that LLM can interact with, and that's it. He basically said, okay, explain how this service works. The LLM explained. Then he asked, okay, but let's double down on that. I really need to understand how that works. And then the AI took a few minutes reading that assembly language and explaining it in human language. And then this was a iterative process. And I think even in the first couple of hours, he had his direction.

Hmm.

He has this idea of, okay, as a vulnerability researcher, As the time progresses, you become much better at smelling blood.

Yeah.

When you're onto something, you have this hunch that only comes with experience. And Sagii is like a very, very well-experienced researcher. So he smelled blood kind of instantly. And then again, writing the PoCs and making sure that the vulnerability actually works. And at first, the vulnerability was so simple. It was just one command, like git push command. that, like, with a couple of flags, and it just worked. And at first I didn't believe, like, it's too good to be true. It was so simple and yet so powerful. And then we said, okay, this works on our local instance of GitHub that we created for this research. No way this simple command will just work on github.com. And it did.

No way.

It literally did. And yeah, of course, right after that, we proved that this vulnerability works on GitHub.com, and we actually had access to other customers' or organizations' repositories, which again, something that really ties back to the cross-tenant vulnerabilities, but it really shows the scale of which, how big cross-tenant vulnerabilities can be, where you can actually access the internal source code of basically every organization in the world in a research engagement that took 48 hours. Again, it's insane.

What I love about that is it's not just, you know, Sergey said, okay, AI, find me the vulnerability. It was really a collaboration between him and the AI, and it relied on his instincts. And so I don't know, there's, I guess, a part of me, like the human side of me, that thinks, oh good, there's still a role for us in this world.

Currently, yes.

Currently, yes. Okay, that's good. So speaking of cross-tenant vulnerabilities, Robert, why don't you ask about the next cross-tenant vulnerability we want to talk about? Because I think this is a really juicy one.

Can I actually go back to something that I've been thinking about? You mentioned there that you discovered that AI thinks about the problems differently, so the way that a researcher would do. Can you give us some examples of what does that look like? What is it that the AI is doing that's different?

So I think one really concrete anecdote in the team that really emphasized to us why we should really evaluate everything and never introduce changes to the system based on a hunch. And let's take an example. When I was taught on how to do vulnerability research, the first thing that you're being taught is that you don't just dive in into a source code, but you do some recon. You really understand what's the attack surface and where an attacker could maybe insert its input and what's the threat model and all of that. So when designing our system, the way that it worked is that we took a source code, we split it to a couple of components. Then on each component, we ran a recon phase that writes a summary on that component and then actually try to find the vulnerabilities based on that reconnaissance phase. Because this is the way that myself as a human would approach the problem. And we had this test where we have like, I don't know, like 50 or 60 zero-day vulnerabilities. And we always check how many vulnerabilities the system is able to find. And at that point we were at, I think around 65 or 70%. We have this chunk of vulnerabilities, run the system, we find 65% of them, and then at a certain cost. And then Yuval from the team asked, what would happen if we just drop the recon? Like, I would assume that maybe we'll have some performance hit, maybe we'll find only like 60% or 50% of the vulnerabilities, but it's really interesting to see how it impacts the cost. So I said, okay, let's just skip the recon phase. Our results jumped from 65 to 85 and our cost literally halved. And it really shows that for me as a human, I said, okay, I will need to do a recon and really understand how things work. But for AI, it just confused him. It was looking at the wrong problems.

That's really interesting. We used to, in a past team, we used to build money laundering models to find, catch money laundering. And we found this really interesting thing, which was we had this theory that you should take all the rules that people typically use to detect money laundering, and you just throw them in as features to a traditional machine learning model. And we didn't— we weren't very successful. We were kind of surprised how it wasn't terribly successful. And then we decided we should try, maybe try stopping doing that. And we stopped. Instead, we described the world in very generic phrases, very generic ways. And the model just ran with it. And actually, rather than find lots of things that people had found before, it found a terrifying number of things that people had never found before, because of course it was being limited by the rules that were there in the first place. So it's almost as though you have to let the AI be free to find the things that you're not even thinking about.

100%. And it's something that, as a human intuition, it feels wrong. So yeah, I think that's one particular case that really taught us that You never introduce changes or do anything without having evaluations that actually support your thesis.

Hmm.

It's a very scientific approach when it comes to, which is different than how we approach vulnerability research until this day, which is really freeform. You look at the source code, you follow hunches, you smell blood, but now you really try to break that process to small scientific pieces that you can actually So I want to ask about one more cross-tenant vulnerability that I think you're announcing this week.

Is that right?

Yeah, tomorrow actually we'll have the talk.

Oh wow. Okay, so this will air next week, so you'll have already announced it when this comes out, but you found a single key that unlocked every database in Azure Cosmos DB. Is that right?

Yes.

Tell us all about that. That's amazing.

Okay, so basically I have Azure Cosmos DB. It's, I think, Azure's flagship database service. It has like tens of thousands of customers. Azure itself is built on top of Cosmos DB. It's like a recursive thing where you have other services, like main services in Azure that are built on top of it, like a really core service. But the beautiful thing is, is that you can actually create your own local instance of Cosmos DB, and to make sure everything works before you go to production. But for us, having the binary in that way basically means that we can research it using AI. This time it was even easier because it's written in a language called C#, like .NET. It's kind of easy to take compiled .NET or C# code and turn it back into readable source code.

Okay.

which our system is best for. We trained it, we used evals specifically for researching source code.

Hmm.

So we got the source code, we ran the system, and again, less than a couple of hours, we had an exploit that basically escapes from the database to the machine actually hosting the database, which again, it really ties back to what we talked in the beginning on zero-day cloud and finding cross-tenant vulnerabilities. This is literally the It's the playbook for us. Yeah. Find some source code that we can research that will impact the cloud provider. In this case, it was Microsoft proprietary source code, not an open source project. Research it now using AI, which makes it much, much scalable in a sense. And you're also much more free to do experiments because let it run for a couple of hours. For worst case scenario, it doesn't work. but in this case it did. The exploit worked locally, then it also worked in a Microsoft— in real Cosmos DB. We're able to take the query that allows us to escape to the hosting machine and run on Microsoft's managed infrastructure. But during our, like, time here at Wiz doing vulnerability research, it's really important for us to demonstrate impact. Like, hmm, execute code on a machine that is dedicated just to my tenant, or like my tenant owns this machine, like the impact Right. But so after we executed code on the machine, we started to actually explore the environment around us. And this is where we found some internal services that while poking them, we were able to retrieve this master key, literally a master key that opens every database in Azure. And then we said, okay, but I can have like a name of a database and I can unlock it both for read and write. Like a complete database takeover. But I still need to know the name of the database.

Hmm.

So we poked a little further, and then we found out that there is another internal service that allows us to name an organization and then get a list of all of its databases. So basically I could say Wiz, get all of the Wiz databases, and then unlock each and every one of them. And that also includes Microsoft itself. Microsoft is an Azure customer. It's built many of these services like Defender and others on top of Cosmos DB. So basically, theoretically, an attacker could just list all the databases owned by Microsoft, use this master key to gain read/write, and that's pretty much it. And it's also another engagement that took less than a week.

That's wild.

Which in a non-AI-enabled world, only the first part would have taken us weeks of work to understand how we actually escape this, how we should escape this sandbox. Basically, an AI broke the sandbox, like Atlas, the system broke the sandbox. And from there, it was just a matter of poking around and really understanding how the service works behind the scenes. And from there, it was That's wild.

How was the Microsoft engagement on this one? Were they responsive? Were they happy to hear this from you? Yeah, I assume.

So this is not a— like, we had multiple engagements with Microsoft, and they have one of the best bug bounty programs and teams in the world. Like, we are constantly in communication with them. They're also one of the, like, the key partners on Zero Day Cloud. And so we have a really close relationship with the Microsoft team. And being like, Like Frank, this is, if not even our first vulnerability we found in Cosmos DB. My first vulnerability in Wiz was named ChaosDB, which was a vulnerability we found in Cosmos DB that allowed us cross-tenant data access in 2021.

No way.

So it's like the next chapter.

So it's an old friend, Cosmos DB, at this point.

It's an old friend. We joke that when they saw an email coming from Wiz saying cross-tenant data access in Cosmos They thought it's a mistake, we're sending an old report. But no, it's like a new one.

Well, Nir, this has been a really fun conversation. I suspect we could go all day talking about the future of AI and VM research, but unfortunately we can't. And I have to ask our traditional closing questions. So one, do you have a tip to help teams improve how they're using AI in their vulnerability research? And then 2, do you have recommended reading? And that reading could be literally anything, just not Anton's blog.

Okay. Let's think about, like, when using AI for vulnerability research, something that we found that it's the best to invest in is that right now most of the models are pretty good at finding vulnerabilities and the actual work or like the most substantial amount of work that we put in is actually validating the vulnerabilities. Because AI being like a statistic model, it outputs, I don't know, like hundreds of vulnerabilities. From there, only 20 are real genuine ones that the maintainer needs to fix. So actually investing in good validation pipelines that take all of this pile of vulnerabilities and actually finds the gold, this is where we found the best ROI. Because if you have good validation, good validation, you could run crazy experiments. And in the end of the day, if you have a good validation pipeline, you could find more novel vulnerabilities. So that's really where I would put most of my effort in.

Amazing. Okay. And then recommended reading?

Recommended reading.

Other than obviously your blog posts about ChaosDB and all that.

I really like Ian Rand books, like Atlas Shrugged, one that I recently read that I really liked. I mostly read fiction, so I have nothing like—

I enjoyed Fountainhead when I was, I don't know, 17 maybe. I read it a long time ago.

So I also enjoyed Fountainhead. Another fiction that I liked? Nothing that comes to mind.

All right, well, 287 episodes and this is the first time somebody recommended Atlas Shrugged. Nir, thank you so much for joining us today and best of luck with your talk. Well, later this week, but listeners, it was last week. So time travel for everybody. Thank you very much.

Bye.

And now we are out of time. Thank you very much for listening and of course for subscribing. Please subscribe so you can get new episodes piping hot. And if you love our content, please drop us a review on your platform of choice.

You can find this podcast on YouTube, Apple Podcasts, Spotify, or wherever you get your podcasts. Also, you can find us on our website, cloud.withgoogle.com/cloudsecurity/podcast. You can argue with us on the Google Cloud Security Community site, googlecloudcommunity.com.

You can follow us on X, x.com/cloudsecpodcast. Tweet at us, email us, argue with us, and if we like or hate what we hear, we can invite you to the next episode.

See you on the next episode of the Cloud Security Podcast by Google.