EP83 What Does reCAPTCHA Actually Do and How Does It Do it? Product Manager Explains

Security.

Hi there. Welcome to the Cloud Security Podcast by Google. Thanks for joining us today. Your hosts here, as ever, are myself, Timothy Peacock, Senior Product Manager for Threat Detection here at Google Cloud, and Anton Chuvakin, a reformed analyst and Senior Technical Security Staff in Google Cloud's Office of the CISO. You can find and subscribe to this podcast wherever podcasts are sold, as well as at our website, cloud.withgoogle.com/cloudsecurity/podcast. If you like our content and want it delivered to you piping hot every Monday morning Pacific time, please do hit that subscribe button. You can follow the show and argue with your hosts on Twitter as well, twitter.com/cloudsecpodcast. Anton, we are talking about a topic near and dear to my heart and my financial security today. Anti-automation, right?

Bots, fraud payments—it sounds really exciting. And of course, at some point, the conversation falls down all the way to the level of finding invisible buses.

Invisible buses talking about money laundering and evading KYC. I mean, this is, I think, a really fun episode because you get to hear from somebody who is at least as passionate about their job as Anton and I am. And it's so much fun because Botter. Not only does he sit next to me at work, he's one of my favorite coworkers. So with that, let's welcome today's guest. We're joined today by Badr Salmi, a product manager for reCAPTCHA, and listeners, the person who sits next to me at work. Can you imagine what this man has gone through having to sit next to me in the office? Badr, thank you for being willing, despite all of the time we spend together, to join us on the show today. Thank you for having me. Let's start off with a relative softball. What is reCAPTCHA? Aren't you that annoying thing where I have to click on the tractors and train the AI? Like, come on, what is this?

Thanks, Tim and Anton, for having me. I'm really glad to be here. So yeah, I'm the product manager for reCAPTCHA, and reCAPTCHA probably is one of the products that a lot of people on the internet have interacted with at some point in their life.

Do you have a guess of what fraction of human beings have interacted with your product?

A lot.

A lot. Okay, a large fraction.

Yes. But like a lot means like probably like billions would be conservative estimate, right?

Yeah, in a conservative estimate, probably So you're another one of our billion-user products, it turns out.

Yeah. So there's over 5 million websites using reCAPTCHA. So you can imagine, if you factor the number of users per site on average, then you're probably in the billions indeed.

Remarkable. So what is it? What does it do? Why do websites use it? What are those 5 million getting out of it?

So actually, I'm glad I can explain that because being a product manager with this, I'm so passionate that I've actually done that line even with my friends and family. But basically, reCAPTCHA like in essence is a protection against automated attacks and bots, right? So bots can be used on the internet for many things and many nefarious things and many good things, right? So what reCAPTCHA is trying to do is really block bots from doing bad stuff on people's websites and potentially leading to damage for users of those websites. One of the largest use cases is called credential stuffing, which stems from the fact that some people reuse their passwords across websites. We just talked about this.

No way, right?

No way. Is this like one of those quintessential 1980s security threats which we have to fight with in 2020s?

You know what's great about credential stuffing? It might be from the 1980s, but it didn't have a name until the 2010s. And I happen to know the guy who created the term.

And I think I know that person too. Do you want to enlighten us a little bit, Tim, there?

Well, it's one of the creators of the company I worked at 2 companies ago, Shape. And fun fact about the term credential stuffing, Anton was actually the only Gartner analyst to look at that term in a deck and say, what? Wait a minute, you guys made that up. You were the only analyst to ask what that phrase meant. Everybody else was like intimidated into silence by how credibly it was presented. But fun fact, Anton won the analyst award at the office that day for being the only person to say, what the heck is credential stuffing? So reCAPTCHA fights credential stuffing. That's where we were.

Among other things. Yes. Among other things. Yeah. You can think of anything that a bot can try to do on a website and abuse. Credential stuffing is usually used for account takeovers.

Mm-hmm.

But you can also use bots to create fake accounts in bulk. You can try to check out very expensive sneakers that drop only at a certain time. If you want to go and get all the PS5s out there, for example, you'd create a bot, inventory hoarding and things. So anything that's rare that humans will try to get their hands on and can be resold at a premium is something that you could try to get with a bot. And I think one thing I want to say, because you touched on that point, Tim, since 2018, there is an invisible version of reCAPTCHA, which is called reCAPTCHA v3, and it's gaining extremely— it's very popular to the point that hopefully if you think a little bit about the last few years, you should have solved less and less I'm not a robot challenges.

I've certainly solved fewer of them successfully. In fact, they've gotten so hard sometimes I wonder Am I a robot?

That's a deeper question, right? But yeah, hopefully you should have solved less and less of these as more and more companies and websites of all sizes have embraced v3, which is more user-friendly and in that kind of frictionless UX that a lot of companies are trying to achieve today.

And the way you explain it is instead of looking at pictures of the buses and turtles, you have to look at invisible pictures of the buses and turtles?

That's right, exactly.

Thank you. This is really— no.

No, what does it do instead? There's no invisible buses.

So yeah, no, joke aside, what happens is you get, as the site owner, you get a risk score between 0 and 1, which is the likeliness of that request to be from a human. And you can instrument that in any way you want, which is much harder for bots to defeat because they'd have to reverse engineer the detection engine in that case. There is no more puzzle to solve, but also it allows a site owner to create their own challenges. Usually they opt for 2FA. That could be an SMS, an email, You name it. And the great thing as well is that it's more accessible in the sense that if you build that 2FA path for the user, you can choose to make it accessible for everyone who's using a screen reader and so on, right?

Ah.

Because it's hard for people with a screen reader to solve CAPTCHA, even though we've made it WCAG compliant last year and everything. It's still not ideal, but the v3 is great for that. And it's much harder to reverse engineer too.

So you get basically the same benefits while the bosses are banished.

Yeah. And also, no, you get even more protection because it's harder to break by bot makers, right? Because you don't have a challenge to solve anymore, right? So you can't train an AI, for example, to solve these things and so on. So better protection, better user experience as well, and so on.

Because obviously somebody would use AI to crack reCAPTCHA, right? Like, that's a great use case.

Yeah. Is it really though?

Kind of an abuse of AI superpowers, no?

Oh yeah, 100%. So you're right on. So it's like every time I talk to people about like, oh, SMS is very safe, right? And they tell you, well, SIM swapping is a thing. Well, yes, it's a thing, but it's a very small amount of people who get targeted by that because there is a very high-value account behind it. But it's like spear phishing. It's a lot of money that's invested in getting spear phishing, right? But at the end of the day, if you do scaled attacks, You don't want to deal with all of this stuff, right? So correct, it's really a reward and investment equation. V3 for scaled attacks makes it more expensive for attackers, right?

I think we've got into threat model question without asking our question number 2, where you guys threatened me with legal. So go and ask it and I'll be quiet.

I will ask question 2 since listeners, Anton is pouting right now because we're leaving out—

I'm pouting for sure. I am pouting.

Question 2. Botter, before we get into threat models, we're talking today not just about reCAPTCHA and reCAPTCHA v3, but Account Defender in capital letters. What is Account Defender in capital letters?

Capital letters?

Well, actually, it's small. It's not capital letters in the branding.

Oh, capital letters in my notes. It's small letters? Okay. What's Account Defender in little letters?

Yeah, so Account Defender is a product we launched late last year, actually, And it's actually a natural evolution of the product. So the way reCAPTCHA has been instrumented over the years, we mentioned that earlier, right? It's been protecting contact forms for small websites or WordPress site owners that they've used with their plugin. And then it's evolved naturally to get into the login aspect, right? Because there is a very high reward for attackers to get through and hijack an account. And so what we've noticed is that As we launched reCAPTCHA Enterprise a few years back and we got direct access to a lot more customers that we talked to, one of the biggest requests there was like, how can we better protect our authentication? Not beyond the credential stuffing piece, right? Because imagine now we're really good at blocking bots. So one behavior we see is that if you have no bot protection, you've got a lot of bots trying to attack your login button, then you install a bot mitigation solution. that goes down. But if you're actually a large site and there is a high reward in hijacking accounts on your platform, you'll see that now they'll start hiring humans to solve these challenges or go through and hijack accounts. So what account defenders try to do is instead of trying to simply answer the question, is this a human or not? Account defenders are trying to answer the question, is this the right human trying to connect or log in into their account?

Not just Any human, the right human.

Exactly.

Right. So impersonation rather than just a takeover act, but more like, is this being used by somebody else? Sorry, I didn't get the right human versus wrong human. Of course, if somebody takes over the account, they're the wrong human, right?

Yeah, exactly. So think of the attack vector. If we weeded out the automated aspect thanks to reCAPTCHA Enterprise, it told you this is actually a human. Instead of letting that pass through, Account Defender is going to tell you, there is a deviation in the normal behavior we see for that user, you should challenge them. So it's risk-based authentication, basically. It's a new model, right?

Oh, okay. Like adaptive auth.

Exactly. And the way it works is that you can pass us what we call a hashed account ID. So basically you generate a hashed account ID. So of the identifier the user uses to log in, and that allows us to keep tabs on that hashed account ID every time they come back. And when there is a big deviation, either because you change devices or location and hundreds of other signals we look at through clustering, we'll be able to tell you like this time there's something suspicious going on and we'll have a recommended action which tells you, hey, you should actually 2FA this user. So we've moved from just bots to risk-based authentication, which tells you, you know, you should probably challenge this human because they're probably not the right human trying to log in. And that's a really nice evolution, I think, of the product.

That's an awesome evolution. One of the things we saw at a former employer of mine that I won't name, was once we got sufficiently good at stopping automation, the adversaries just switched to manual labor.

Exactly.

And we had this killer good signal for detecting what we called click farm, which was window size. So I would love to know what your signals are, but please don't say on the show because then we'd have to edit things.

Maybe we can do that over coffee.

Yeah, we'll do that over coffee next time.

Yeah. When you think of like The way we thought about it is we took a step back and thought of what are the steps that an attacker kind of goes through in order to be able to hijack the account of a user. They go first and harvest all of these leaked credentials, right? So we have password leak detection capabilities now that you can put on top of reCAPTCHA and Account Defender that will tell you, hey, the user is actually using leaked credentials now. Yeah, we GA'd the product last month actually.

Wait, that's awesome. Congratulations.

Yeah, thank you. So we're super happy about that. And then the second thing is, okay, now that I have a billion credentials to test, I'm going to create robots to actually go and test them on large websites. So now you install reCAPTCHA and that stops those bots. And then the next thing is we want to make sure that this is the right human trying to log in. We've also actually done another one last thing that is pretty great that we've launched recently is we've launched what I call like the investigation API. So now that you're equipped with a hashed account ID, we're doing some clustering in the background and then you can query that API with the hashed account ID of a suspicious account and we'll return you a cluster of accounts with similar behavior.

Cool.

So if you think of, yeah, that's really cool because like—

That is really cool.

Yeah.

I totally get it.

Yeah. It's like a good investigation tool because a lot of our users, right, at the end of the day, not everybody has enough money to like hire, you know, a SecOps team that can go and dig deeper into like attacks. So simplifying that clustering for them and being able to like start from that seed that they've identified as being bad and expand from there through clustering that we've done is saving them a lot of money and time.

So That's incredible.

So I think we kind of went through a bunch of things that I would call part of the threat model, but do you have a more formal kind of a structure so that maybe our audience would understand the threat model that this solves a little bit better? Like, I get it, like, I have an account, I have a password which is 4 characters and a username that's easily guessable, so a machine can easily get in, so I'm going to lose my account. Sure, like, account takeover is pretty clear, but like, what other Threat scenarios does it cover, and what does it handle really well?

Yeah, so I think there's 2 main cases, right? So we've just talked through account takeover, which is probably the most widely spread one and also the most nefarious one, because account recovery for anyone who has gone through the process with any company is really hard, right? So making sure that we block account takeovers for companies is really important because it's a lot of costs on support, And for users as well, the fact that the company kind of let them down makes them feel bad, even though at the end of the day, the whole threat is based on the fact that users are reusing their passwords. But passwords are just an imperfect security system, but it's good enough that everybody can understand it. I think the other case that we covered that is really good and I'm very excited about developing more is fake account creations. As we've had more and more digital natives, especially during the pandemic, we've had this explosion of services that have moved online. You have a lot of these services offer free money basically when you sign up or when you refer accounts and people and friends and so on. So fake accounts have become a big problem. Think of it as well for misinformation, manipulation of reactions, et cetera. And so what Account Defender helps you to do is another thing, which is detecting fake accounts. at the creation moment based on, again, hundreds of signals we're looking at. The cool thing as well is that through the register process, everything is an API response. So you can earmark that account and let them actually go through, look at the attacks that they're trying to perpetrate within, and then actually decide to shut down the account. And what you can do as well is use the investigation API we mentioned earlier, and we'll send you back a whole cluster of these dormant accounts.

Oh, cool.

That's actually cool. Wow. It's way cooler than buses and like we capture circa That's right.

These are huge revolutions, Bader.

Yeah, I know. It's something I'm extremely passionate about. I've been 7 years at Google working anti-abuse, protecting search users. I was in Safe Browsing before, protecting users against phishing and malware. And now being able to actually do a lot of that for companies to be able to protect their accounts and the data of their user and livelihood sometimes of their user, the money of their user, it's like, that makes me super excited. So that's why also I think we've brought that approach of, there's 2 things. There is empathy that we brought from the user perspective, it was like how bad it is for a user to get their account hijacked and so on. But also the empathy into the developer, the DevOps team, the SecOps team. With the shift left now, there is a lot of DevOps teams that have to do the job of security counterparts. And simplifying all of that workflow for them is actually really important for us and making sure that they have the tools they need there. And one last thing I wanted to add as well is that a lot of different attacks actually start from these 2 things, from either fake account or an account takeover. The goal at the end very often is that they can use the accounts to do things like using stolen credit cards to purchase things. So you can have payment fraud coming out of these. There's like a million things that you can do with a hijacked account. You can go around KYC, for example. So for example, let's say you've been banned for a certain accounts on like a crypto exchange. If you hijack someone's account, you can leverage their KYC to make transactions, withdraw their money, etc. So there is a lot of like really bad consequences for people. So protecting your accounts is the number one, I think, priority for most security teams out there.

So wait a minute, wait a minute, wait a minute. What you just said kind of blew my mind because I was in this space for a long time. I share your passion for it. I hadn't actually thought about this as a sanctions Evasion mechanism before doing account takeover. Geez, that's gnarly.

Yeah, well, that's true, right? Think about, like, say tomorrow you get booted out of a platform. I could just, quote unquote, right, if I'm like an attacker, go and like get someone else's account and circumvent all of these KYC that they've already passed.

Yeah, of course.

Yeah, that's pretty nefarious. Attackers are very smart. Like, after 7 years of actively trying to figure out how we can get better. I am impressed by the level of ingenuity we see there or of innovation we see in the space.

Yeah, that's something I love about this job actually, is you're playing against somebody in our space.

Yeah, actually, yeah. It's like I'd recommend anyone who's working in this space to read about game theory in general, right? Because at the end of the day, you're working like it's like a chess game where you need to think of how the person in front of you is going to to react to your decision and then think of the next step. So that's kind of the thinking we bring as well when we think of all of these tools we're building. We're thinking of the SecOps team and what tools are they using and what data they need to be able to make their next move against adversaries.

So you've kind of skipped many questions now answering our recommended reading, which is traditionally a closing question. So listeners, that's it for today. No, I'm just kidding. I do want to pick more at this. Account takeover question, though. Like, what are the other monetization paths with a stolen account? You could cash out rewards points. You could apparently evade sanctions. What else are they doing concretely to make money out of these accounts?

Well, one of the big ones that kind of encompasses the case, what we've talked about, is the fact that you can resell these accounts on the black market. Right. So I moved to the US a year ago, and I didn't know of all the regulations around gambling, for example. Right. That's right. Right. And so think of the value of, like, an account For online casinos and stuff like that, the value on the market is really high. Also, like other things like crypto, for example. Unfortunately, when you withdraw crypto from someone's wallet, there is no mechanism to claim that money back like you may be able to do with, for example, traditional banks.

That's kind of the point, though, right? Yeah. It's supposed to be optimized for criminal use cases. No?

Exactly.

I know you both are quite opinionated. I personally have self-custody over my crypto. Keep them on my wallet, and I would recommend probably anyone who like you know doesn't do that to do it.

How many cubic meters of Arctic ice are you personally responsible for having melted?

Trust me, not many. I'm not. I don't have that many.

That's mean. Okay, Anton, I think you have the next question here before we beat Botter up about his coin usage. Indeed, indeed.

And I think I wanted to very briefly go into my somewhat favorite area from many years ago, namely payments. And of course, I'm going to mention PCI compliance because. Oh no, it's just a tweak team. But it also, reCAPTCHA and the tech you're building also helps with a lot of the payment fraud, right? So would you say that you are kind of a key element for solving this problem or you're somewhat related? Like, where does reCAPTCHA and payment fraud connect?

Yeah, that's an excellent question, actually. So I think if you think of the attack model for account takeovers that we've talked about, it kind of applies similarly for payment fraud. Because very often it starts with the same modus operandi. It starts from they're harvesting a lot of stolen credit cards and they're going to do, the first thing they do is they use bots for card testing. So one thing you can do there is that reCAPTCHA allows you to block the automated nature of the attack where just bots go and test these cards over and over to figure out which ones are valid, which ones have been canceled, et cetera. And then again, working with our customers, one of the things that customers started asking is, can you actually go a step further and start telling me, is this potentially chargeback? Is this potentially fraud? Can I get more data so you can help me protect myself? And that's why actually the shift you'll see with our product strategy over time has been from being only a bot solution to being a full fraud solution. And to be a full fraud solution, we've done the first piece on account protections, but now we've moved into the payment fraud. So We've started doing more into, and that's a whole other topic we can discuss in another podcast that I'd love to cover. But basically we're looking into how we could solve for that problem exactly. How can we bring in more signals that would allow developers and businesses to detect payment fraud by leveraging reCAPTCHA? So yeah, it's a lot of exciting things to do with the product given that it's deployed inline already at the point of attack through the JavaScript you implement on the page. If you're not deploying reCAPTCHA at the edge through a WAF. So a lot of exciting opportunities ahead. So yeah, we're kind of evolving towards that full fraud solution as opposed to only bot management solution.

Okay, so I look forward to having that conversation on the podcast as well. And God, the number of times I said deployed inline in my former life and all the cool things that meant. You're giving me flashbacks.

Yeah, I'm really excited about it. Yeah.

So how about I ask a dumb question? Can I? Am I allowed to ask a dumb question?

I don't think we can stop you.

Well, there is that. Fine. Yeah. So when I think for a second about protecting accounts, of course, in many cases, Google says, just use MFA. Use multifactor auth. And it's like, is this together with 2-factor auth? Is this instead of 2-factor auth? Or is this like— how does it relate to multifactor authentication?

Yeah. As we mentioned earlier, I think we've moved into the invisible piece. So you don't have challenges to solve anymore. So now as a developer, you get that score and you decide, okay, I want to block traffic. I want to block this because it's a blatant bot. And then you're like, I want to let this user pass through because Account Defender and reCAPTCHA tell me that there is nothing suspicious. And then you have the in-between, which is in that gray area. And there, this is where multifactor authentication comes in as the preferred way for our customers to actually challenge users or bots in that case. And so we actually offer a 2FA solution right now in beta. It's one of the features we offer on top of the account protections bundle. So password leak detection, reCAPTCHA Enterprise, Account Defender, and multifactor authentication, which gives you that full kind of account protection. So bang on, actually, 2FA is also my preferred way of doing it because as much as I know SMS is universally understood as well, it's something that my parents, who are not very tech savvy can use and understand by using any service out there. And that's why I think it's such a popular way to do it. Or email, which is a better cost-efficient way to do 2FA, because SMS is very expensive for some companies. But yeah, 2FA is a key part of it, of course.

Makes sense. Okay. So now we actually are just about at time. So we will ask the traditional closing questions. One, recommended reading. And then 2, One tip for improving, I guess, account defense would be the topic today.

Okay. I'd say, so is the reading actually anything that I've read or something?

Yeah, it's absolutely your call. We have had people recommend the books.

There's a swear jar if you recommend the SRE book though.

There's an SRE— if you recommend an SRE book, you would be number 70 to recommend it.

Okay. Actually, my favorite book, technical book I've ever read was called Code. Yeah, it goes from explaining a lot of very technical—

Wait, called Code? Just Code?

Yeah, just Code. And it's basically a book that starts from telling you how Morse works. And at the end, it explains to you how the internet works and how systems work and how binary works. And the author is Charles Petzold. And I highly recommend it because I think as a product manager, very often I have to actually simplify We have to explain technical terms in plain words. And that book was for me something that allowed me to put a lot of technical concepts into perspective in simpler words. And it's also just beautiful to see the evolution of technology through the eyes of the author.

Amazing.

Yeah.

Badr, thank you for the thoughtful tip there.

Yeah. And then for the tip for account takeovers, I'd say use reCAPTCHA and Account Defender.

Use reCAPTCHA, it's fine. Absolutely. We will not hold it against you. We have a vendor hat. Here, mine is visible.

Well, actually, I'm going to make an argument that over 5 million websites and growing are trusting our product today. And we're seeing more and more people, and more and more people are trusting v3 as well. It's actually a majority of our traffic is going through v3 now because I think they understood that it's harder to break. So yeah, that would be my pro tip. And use 2FA also. It's safe, usually.

Usually.

Amazing. Badr, thank you so much for joining us today, and look forward to having you back on the show really soon.

Yeah, thank you for having me. That was fun.

And now we are at time. Thank you very much for listening, and of course, for subscribing. You can find this podcast at Google Podcasts, Apple Podcasts, Spotify, or wherever else you get your podcasts. Also, you can find us at our website, cloud.withgoogle.com/cloudsecurity/podcast. Please subscribe so that you don't miss episodes. You can follow us on Twitter, twitter.com/cloudsecpodcast. Your hosts are also on Twitter at Anton underscore Chuvakin and underscore Tim Pico. Tweet at us, email us, argue with us, and if you like or hate what we hear, we can invite you to the next episode. See you on the next Cloud Security Podcast episode.

Security.