Double-clicking, but not on fire hydrants, with bot fighters
21 Jun 2021 · 34 min
Sparky Taves from Adobe, Randy Jingaleski and Brian Lozada from HBO Max discuss how they fight credential-stuffing bots at scale. Adobe uses invisible risk scoring with reCAPTCHA Enterprise instead of outdated image CAPTCHAs. HBO Max protects smart TVs, mobile, and browsers by combining invisible detection with manual analyst review to tune thresholds.
Chapters
Security.
Hi there. Welcome to the Cloud Security Podcast by Google. Thanks so much for joining us today. Your hosts here are myself, Timothy Peacock, the product manager for threat detection here in Google Cloud, and Anton Chuvakin, a reformed analyst and a member of the cloud security team Here at Google. You can find this podcast wherever podcasts are distributed, as well as at our website, which we have finally launched. It's at cloud.withgoogle.com/cloudsecurity/podcast. If you like our content, please consider hitting that subscribe button on your podcasting platform of choice or on our website. You can follow the show and argue with your hosts on Twitter as well, twitter.com/TheCloudSecPodcast. So Anton, we've got a super fun episode today. We have 3 guests in 2 segments talking about reCAPTCHA, which I think our guests are gonna learn is not just clicking on the fire hydrants anymore.
Indeed, and this is our first episode where we combine 2 mini episodes. So please let us know in comments or however you wanna let us know whether it worked out okay or not, because we usually have 1 guest or a set of guests for the duration, but this time we are doing the whole 2 little mini episodes in 1 combined by the same topic. Fighting bots and reCAPTCHA.
I love the topic of fighting bots because I got my start in the security industry at one of the leading bot companies, which is now part of another company. But it's super fun to get back to this. And I loved hearing both of our guests use the phrase credential stuffing because we invented that when I was at Shape.
Yes, and here's the funny one. I was the first analyst who Shape mentioned this number to, presumably. At least that's the version of the story I've heard. And I was the first to call them on it and said, what is this? There's no such term. And I think somebody from Shape Shab told me, you know what? We invented the term and we kept telling it to people and nobody questioned us. And you're questioning us. You must be good. And I said, yeah, well, I am.
That's right. It was Schumann, Schumann Goesmujunder, who gives you the award every time. But it was Summit, one of the founders, who actually invented the term.
Ah, and the term is now a standard term. I mean, it's not as widespread as, I don't know, EDR, but it's definitely something that people know other vendors mentioned in their materials and slides, webpages. So it's kind of an industry term at this point.
Yeah.
Company has a credit to inventing the term that stuck in the industry, which is kind of a big honor.
So with that, I want to introduce today's guest. His name is Sparky Taves. He is a product manager for Adobe Identity. Thank you so much for joining us today.
Thank you.
Let's start off with an untraditionally softball question for the show. Just tell us, what are you currently working on at Adobe?
So I spend my time helping keep users safe and secure. I've held various positions in identity, privacy, and compliance. done some AI bot work, but my biggest focus over the last more than a decade at both Microsoft and Adobe has been on identity systems, and in particular, focusing on securing accounts, keeping their data secure. As part of that, I've done a lot of work with passwordless, MFA, and risk scenarios. And also at Adobe, I own a lot of our platform APIs and developer experiences. These are sort of composable building blocks that can be a great force multiplier but it's really key that they are secure by default.
The secure by default is one of the things I love about working on cloud and helping people move towards that.
Definitely. So when I hear identity, I typically would think of human hackers who are trying to impersonate, take over accounts. But we wanted to touch on bots. Presumably, that's the other side of the threat. So could you give us a bit of a threat assessment perhaps of how you see the bot threats?
Sure.
Well, human attackers use bots, and they use them as a force multiplier. If I have your username and your password or I'm targeting you, there's a set of steps I might do, but realistically credential stuffing is by far the most common form of account takeover. It's not nearly as effective as some of those more targeted attacks, but it's cheap and it's easy to do.
What is credential stuffing? 'Cause I don't know if everybody knows that term yet.
Credential stuffing is where an attacker has gotten ahold of a list of usernames and passwords, usually from a breach. They're very easy to get. There are literally downloadable files of millions, if not billions of these that you can get for free on the dark web, and then they start spraying them against all sorts of different sites. Unfortunately, humans generally reuse passwords unless you're using a password manager. And as a result, if I get timothy@gmail.com and some password, if I go and try that against hundreds, maybe even thousands of different websites, there's a fairly good probability that at least some of them are gonna come back and let me log in and get access to your account. Bots are the number one force multiplier. Most attackers don't sit there and try 1,000 sites, but if you've got the right script, you can try all 1,000 with the click of a mouse.
Were you about to use the word combo list in there?
Yes.
Yeah.
I have the combo list here on my computer, and a common refrain I tell people when I'm giving, for instance, internal security talks is, I likely have your password on my computer.
I'm sure you have mine.
Realistically, unless you're using a password manager, I probably do. I have a billion passwords.
Well, if Hunter 2 is in the list, you got mine.
Gotcha.
I'll be right back. I need to really quickly go try a few websites with that.
So this is an interesting problem, right? We've got these users out there that are being victimized by breaches on other websites, bad guys using forced multiplication at scale, and you've got to secure your user accounts on your website. So how do you think about that problem in the context of Adobe, and what's the best practice for securing the user experience without people's lives becoming miserable?
Well, account security is a very multilayered approach. There's no one thing that really solves this problem holistically. The biggest thing you can do is multifactor authentication or passwordless. It is the gold standard for keeping accounts secure.
But aren't they seen as a friction by users? Sorry for an interruption, but I've had this debate on social media way too many times and I get beat half of the time. People say, no, no, no, no, no, no, no, it's too much work for the users. They won't do it.
It is too much work for the users. And this is something that in the enterprise setting, admins can enforce it. They can make you do it. In the consumer settings, across the industry, I've seen quotes as high as 2% of users will opt in to MFA. And often I see reported values of 1% or lower opt into MFA.
Whoa.
I, as a security professional, am horrified by that. When, when you don't have MFA on, you're protected by a password and passwords alone suck. It's really interesting talking with you here at Google. I saw just a couple of weeks ago how Google was in the process of sort of opting a whole bunch of users in the consumer space into multifactor using the Google Search app and some other things. This is an interesting solution to the user experience challenge using push notifications, using apps users already have. I, and I'm sure much of the rest of the identity industry, is watching that very closely to see how that works out. If it does, you can bet that I'm gonna be pursuing figuring out how do we involve MFA in the rest of Adobe's suite of apps and allow Photoshop or Acrobat to receive a push notification and authenticate our users for every login.
Oh, now that's fascinating. You've got Acrobat on almost everybody's desktop. So you really do have a pretty wide base there. That's a fascinating opportunity.
We talked about, like, how this is thought about and practiced. But I'm sure there are tools involved, right? So roughly, what kind of security products or maybe practices with products are involved? Now, naively, and again, in my former Gartner days, I used to cover this a little bit. So I know that some of the clients would bring up web application firewall if it's about the website security. I was also smart enough to know that WAF isn't really the answer to bots. And I think Tim, by the way, also knows this, right, from his past experience. Still, what kind of tools and kind of practices around tools are involved in securing this?
In an ideal world, we want to allow real users to log in quickly and easily without any additional friction, and we would want hackers to be stopped by additional friction. The trick and the main tool, I think, in my arsenal is trying to determine when do we think it might be a user and when do we think it might be a hacker. reCAPTCHA Enterprise is one of the products we use, and we have a whole bunch of systems that go into a machine learning algorithm. And in real time, as users are logging in, we consult reCAPTCHA, we consult our internal logs, we consult all sorts of blacklists, and we come up with a score where we guess, How likely is this person a bad actor? And that's a combination of how likely they are to be a bot, is this a familiar location for a given account, and a bunch of other things that sort of roll into that. In a way, you could think of that machine learning algorithm as a way to selectively avoid friction for good users. If you get a good enough score, we're able to say, okay, you can use your terrible, scary password alone and we'll let you log in. But if you don't meet that certain threshold, we can very selectively add additional friction in varying amounts to double, and in some cases triple check to really make sure that it is the correct user logging into their valid and self-owned account.
So what does that friction look like for you? I'm guessing it's not, you know, click on the fire hydrants or put the dog the right direction up.
Well, no, it is not. And a note on that, traditional CAPTCHA, clicking on the fire hydrants and such, is very dead. In the modern ML world with artificial intelligence and generative adversarial networks, they are effectively broken. Machines are as good, if not better at humans at solving them. Instead, you want to use signals to identify when there's risk and then trigger things that are much harder for a machine to do. We at Adobe use multifactor authentication, and we'll sort of selectively enroll a user in MFA for a given login. But there's also cases where companies might want to require a cost proof or throttling experiences or other forms of friction where they can do things that a machine learning algorithm can't necessarily solve, but it still either stops or slows down the attackers appropriately for the scenario.
I think that's exactly what I was seeing. Anton was alluding to the fact that I was an early employee at Shape Security, which kind of did the first successful invisible CAPTCHAs for the web. We did this at incredible scale, and all of what you're saying checks out with our experience back then. One of the things I loved when I found Shape was the CAPTCHA-solving businesses. You know, these are businesses that offer like, take your credit card, we've got professional services to help you integrate it with your bot app. The CAPTCHA solvers out there really put one over the top. So looking ahead rather than back at things I was doing in my early 20s, where do you think this is going? Where do you think the next big things in identity and securing user experience for good users versus bad users is going?
Honestly, I think passwordless is the future. Passwordless done right combines all of the benefits of multifactor authentication with an almost magical user experience. The user either gets a push notification on their phone or they can tap an authenticator or their watch or something to a reader and they're immediately and securely logged in. It's virtually impossible to spoof, it's virtually impossible to hack, and it's a really convenient user experience. As we pursue this world, we're gonna start to see it applying to things way beyond just logging into a website. In our future passwordless world, that same authenticator, again, be it your phone or your watch or whatever, is gonna unlock your house door, your office door, your laptop. It's gonna start your car, it's gonna pay for your coffee, coffee, authenticate you to your bank or at the ATM, and sign you into your Adobe account or your Gmail account. We're almost to the elbow curve of the passwordless world. And I'm really excited to see not only the experience, but the security come to the industry and users as a whole.
But to be a bit contrarian here, so you have a reasonable degree of confidence that this can be built securely. And by securely, again, I know it's all relative. And I know it would not suffer the same problems passwords suffer. That part's clear. But won't it suffer possibly bigger but very different problems? Like you mentioned spoofing as one, device possession being necessary drops convenience, device theft, device loss. I haven't thought about it all that much. And I know it would solve password problems specifically. Sure, they'll be gone. But how can we be relatively sure that new bigger problems won't be created?
So it does create new bigger problems. But a lot of very smart people are thinking about how to solve those problems and sort of nipping them in the bud. Device loss or device theft is a very valid case, but most passwordless schemes require multiple factors. So it's not just possession of my phone, it's also the ability to unlock my phone, my unlock gesture, be it a biometric or a PIN or something like that. In the watch case, you have to authenticate your Apple or your Google Watch when you put it on your wrist. And while it remains in that authenticated state, it's able to act as an authenticator. But if it's removed from your wrist, it's virtually useless to an attacker. Now, spoofing is one thing that exists today with passwords, and it will probably exist tomorrow with passwords. If an attacker can convince you to log in to your account for them, it's pretty much game over for most schemes. The one advantage that passwordless has that many other mechanisms don't have is because passwordless is able to consider things like the secrets being used by your browser's TLS connection and other sort of ephemera that can help a well-implemented service identify man-in-the-middle scenarios, which are very common in sort of the spoofing and phishing cases.
I like the move towards possession being 9/10 of the law when it comes to, you know, logging in. And Anton, to the question of like, well, what about devices getting stolen and rubber hose attacks? Yeah, sure, but that's so much less common.
Yeah, for sure, for sure. But I'm not thinking about a threat. I'm thinking about like you lose a device, you cannot log in. With passwords there in your mind, you can log in without having anything in your possession. So it's a convenience I'm worried about, not the security in this particular case.
It is a convenience, but I wanna turn that around into a security argument. The challenge with something like a password is you don't know when it's been stolen. It's still in your head. You have your only copy. The fact that some website somewhere got breached is invisible to you and undetectable. Detectable to you, as opposed to if I lose my phone or I lose my watch, yes, I lose my ability to sign in. There's a whole host of problems that come with that, but it's a physical thing in my life that I've lost, and I'm generally pretty quickly aware that that's happened. And as a result, I can start to take the set of actions to acquire a new phone, to secure my accounts, to do whatever it is I need to do. I have that obvious triggering point in my life.
Okay. No, that actually makes sense to me. This is kind of a solid argument in favor of this. And again, I'm not trying to say passwords are better, let's keep them. I was just kind of figuring out how my life and how other people's lives would look like if passwords are gone and you have a device that works like that.
You know, Sparky, you should be really proud. You got Anton to admit that something was a good point. You should put that in your perf packet. Let me put it that way.
I will do that. I'll make sure it's noted in my next review at Adobe.
I'm such an optimist. Come on, Tim. You're pushing me too hard. So let's get to our fun final questions. We usually ask all our guests to suggest one practical thing for the audience to help them, well, fight the fight, in this case, fight with the bots. So, Gan, any practical tips that the audience can do pretty much right now to help them fight bots?
Absolutely. First off, you need to identify bots. Depending on your particular scenario and solution, there might be a bunch of signals you wanna consider. Clearly, we're here talking about reCAPTCHA Enterprise. That's one really valuable signal. There may be others that are very particular to your case. Once you've identified the bots, you need to figure out what is the correct way to add friction to your attackers. In some cases, it's just throttling them, slowing them down. In other cases, you have to do more frictionful things to your valid end users, requiring MFA, requiring cost proofs, whatever. But stopping to think about this ahead of time and figuring out how are you gonna identify the bots and then what are you gonna do is the path forward for you and your business.
Perfect. Thank you very much.
Awesome. And then do you have any favorite reading on bots aside from like Do Androids Dream of Electric Sheep?
Oh, Do Androids Dream of Electric Sheep is my answer. Philip K. Dick is a wonderful author.
Whenever I look back on my time at Shape, I feel like I was a Blade Runner.
Are you sure you're not?
Are you sure I'm not?
No.
Me neither. Awesome. Well, hey, Sparky, thank you so much for joining us today. This was incredibly fun to dive back into You know, the deep recesses of my brain on bots, and it's a great chat. So thank you so much for joining us.
It was my pleasure.
All right.
Well, Anton, what did you think about that conversation?
What I found fun is that this is a much tougher problem to solve because we are kind of dealing with automation, but we are dealing with automation created by creative humans, right? The bots are built to bypass defenses, to pass through web application firewalls, to really achieve the business purpose for the attackers. So to me, that's interesting that while it sounds like a counter-automation problem, it's really counter-creative human who builds the automation problem. That to me is fascinating.
Yeah, you're not really fighting the bot. You're just fighting the bot author, and the bot happens to be the proxy for their intelligence.
That sounds so fancy.
You know, when I was at Shape, we really felt like we were Blade Runners out there retiring the bots. It was a lot of fun. So our next 2 guests, really excited about what we're going to hear from them. One of the things I find fascinating about both approaches and both use cases is the diversity of devices that they're able to achieve protection from and against. And so really interesting problem of how do you protect when you have such broad, as John would say, attack surface? Well, folks, I'm super excited to introduce 2 more guests to talk about this topic today. I'm joined by Randy Jingaleski, a senior staff security engineer at HBO Max, as well as Brian Lozada, the CISO for HBO Max. Uh, guys, thank you so much for joining us today.
Awesome. Thank you for having us. Really appreciate it.
Our listeners are not always familiar with what our guests are up to. So could you give our listeners just a quick deep dive on what you're doing at HBO?
So I work on application and product security for HBO Max, part of WarnerMedia's direct-to-consumer security team. I got involved with this just a week before the launch of our platform in May of 2020. HBO Max has seen tremendous growth in the meantime from a recent ad-supported variant that went live last week, covering an additional 39 countries at the end of June. And then WarnerMedia's entire 2021 theatrical slate releases in tandem on HBO Max and in theaters, just given COVID-19 impact. So it's been really exciting to be part of a team that directly affects millions of customers and kind of guards these brands that people really love. I do a mix of program and operations work, so working pretty directly with how customers experience our platform. And then before this sort of chapter in my career, I spent a bunch of time in application security consulting. I was part of a small boutique company that got acquired by a very large consulting firm, so kind of saw both ends of the services spectrum there. And then what feels like a lifetime ago, before any security day work, I was a software engineer working in defense and insurance. So this is a bit more glamorous by comparison.
Can you tell us about the problems in securing user accounts you're working on at HBO today?
One of the challenges, I guess, transitioning from the work I used to do, which is kind of pen testing behind closed doors, you write up vulnerabilities for the customer and sort of throw them over the fence. With HBO Max, we have millions of active users, so if we introduce something that negatively impacts the wrong population. You know, of course we want to mitigate attacks and risks but not hinder the actual customers in any way. So if something like that happens, we're really going to hear about it. And our customer support team is fantastic. You can reach them by all these different channels. So we've found the user base to be really vocal about that. It took some getting used to.
So people are motivated to watch their shows, huh?
Yeah, that's something you come up to speed on very quickly. Yeah. So we We've seen a lot of just customer outreach at times about potential security issues or just friction being introduced to the experience that was unfamiliar. We try to be really careful about what we do and don't apply just to make sure that the customer experience is as seamless as possible, but at the same time trying to guard what we have.
And then Brian, from the CISO perspective, how do you think about, you know, these problems of securing user accounts versus friction and user experience?
For us, we're always looking at, you know, the customer side of the house, right? How can we actually secure that customer experience? How can we innovate on behalf of the customer? And introducing friction into an experience is something that we don't want to do, but we also want to have that right balance so that there is not any sort of impact from a negative perspective for that customer's account. So when we look at these things, trying to find the right balance, it takes the right talent. to actually look at it from a different perspective. Partnering with Randy and asking Randy to help HBO Max solve this problem, I loved how Randy's approach was, these are the things that we could do to not introduce friction, but also give us that right balance so that we could secure that customer experience. We don't want customers to have a poor experience, their account get taken over. That could lead to some sort of signal online, impact the brand, and also impact customer acquisition.
Mm-hmm.
We're trying to bring more customers into our experience. We want them to enjoy our content. We have a lot of content out there. And if there's no trust, if the customer doesn't trust us because either their account is being taken over or we put some friction into the experience that it's hard for them to log in, we're not going to have that customer acquisition growth that we want. You know, asking Randy to help us out, man, he took a really good broad approach and then very tactical on saying this is how we could do this without putting in friction. And, you know, so far it's, it's worked out great, and I really appreciate the help that he's provided.
Getting one layer deeper into the details, it sounds like y'all are using some kind of technology to detect account takeover at login time. What's going on there? How's that working?
I'll kind of throw some praise towards Google's way there because generally as part of an overall like anti-automation and botting strategy, I kind of look at one pillar of that as what you might call in-client bot detection. Basically some client-side code executes to fingerprint users, or in some cases not actual users that are bots. Then that's passed to the backend where we expect that information and can take action on it. It's very interesting. If you look back to where we were at launch time last year in May 2020 and the tools that we had available at that time, one thing I've really appreciated about this team is we've tried a lot of different tools and solutions and approaches, sometimes even writing certain things in-house. And at this point, our kind of go-to slate has really evolved, but I think progressed at the same time. So reCAPTCHA Enterprise from Google has been our go-to solution for in client bot detection. We've been really happy with that and just a mix of understanding scores that we get and just information from how we're reading our clients. One of the challenges but also exciting parts of HBO Max is that we support such a wide range of devices. So if we purely had a web application, it would be a little easier to do certain things.
Mm-hmm.
But we have support for various Smart TVs, and then Android and iOS, you know, not just phones, but also tablets and smart TVs, you know, all sorts of different devices. So being able to roll reCAPTCHA Enterprise out, we've taken what I'd imagine is one of the more complex overall implementation strategies, but that's something that we're using and just very happy with today that allows us to get information on what's happening in our clients, help pare down automated attacks that we definitely don't want. Something that I guess sticks out about reCAPTCHA, and this is even something that we contended with in our own organization, is that you say Google reCAPTCHA and people automatically think of sort of the old, like, checkbox that might pop up with a bunch of pictures of bikes or traffic lights.
Yeah, the click on the traffic light. I get stuck in those and I lose faith that I am indeed a human.
It's really interesting. And it's sort of, I guess, because that was around for such a long time and is still really prolific. throughout the internet. One thing that we really had to kind of go on a social campaign about within our org when we wanted to try out reCAPTCHA initially is just that we're able to do it in an invisible mode. So, you know, our actual customers don't see reCAPTCHA in action. I suppose you could look really hard and use the JavaScript developer tool to see that it's there.
Sure.
But at this point, it's really a seamless experience. The client passes information that we derive from reCAPTCHA to the backend, and we have logging and can take other actions at that point. So I think a mix of just being invisible and seamless, but also something that we can depend on. But of course, we still get those questions about whether users have to click pictures at any point in our experience, which they really don't. I kind of chalk that up to a combination of just the accuracy of Google's product, but also a combination of we just did a lot of analysis before we ever started actioning any traffic based on these scores. So we have tons of other logs throughout the experience. And initially we did a lot of work monitoring only with reCAPTCHA. We sort of took that data and said, okay, how can we verify who was labeled as bad and who was labeled as good and sort of establish what thresholds we want to take further action on?
How did you get to ground truth in this case on whether this was working or not? I'm always curious about how security professionals actually at the end of the day feel confident in knowing what's actually going on.
It's definitely a challenge with just a platform that's as wide-ranging as ours, but I kind of could simplify it as getting very familiar with logs. And I think this is sort of something I initially did for this reCAPTCHA analysis when we were just verifying and then sort of presenting to other teams that, you know, this is what we wanted to move forward with and this is why we were confident in the data. Essentially what we did was we took samples from each of the different score tiers that are provided by reCAPTCHA. You know, you have kind of your lowest confidence that someone's a human to your highest confidence. And we, we sampled just different sessions that had been labeled in each of these categories. We went to all the other logs we have. So it's kind of a combination of our WAF logs, you know, our web application firewall. You can see all the HTTP traffic going through. And then even before this, just as part of a team that does operations work, you kind of get used to staring into the abyss and making sense of WAF logs as far as what's going on.
Staring into the abyss. You know, we try in this podcast to be encouraging to people who might want to get into the industry.
I'm not sure staring into the abyss is, uh, it might be a little more akin to, uh, The Matrix being a WarnerMedia property. There's at least one character on there who could kind of stare at the Matrix text flowing by and make sense of it. So maybe that's a more positive spin.
I think that was Tank.
Yeah, yeah.
I'm picturing you sitting there like Tank with, you know, all the screens now. You know, looking into the future, and hopefully not into a Matrix-like future, where do you two see the problem of protecting user accounts against bots going? Because I used to work in this world and it really felt like we were headed towards a Blade Runner universe, which I don't know if that's one of your properties or not, but that's where I saw it going. Where do you two see it going?
I kind of see maybe a natural evolution of what we've started to see even with reCAPTCHA, just very positive. Like if you look at the old experience of clicking around on pictures to, you know, moving towards an invisible product today. And then one thing that we've really found a lot of value with in reCAPTCHA Enterprise is not Only the out-of-the-box model being fairly accurate, but just that we're provided the capability to annotate and tune our model once we lifted to enterprise. So I kind of see that like one part of the experience, just taking a temperature check, if you will, on user sessions as they're going through the experience and completing different actions that are prone to fraud or attack or even misuse. I think the obvious example is kind of logins and credential stuffing. But what's just been exotic about HBO Max is We have other parts of the application where we might be taking some unorthodox payment methods as we expand globally. We also have promo codes, and you've even seen some of them appear in new video games and whatnot. So even promo code fraud is a potential attack avenue.
Uh-huh.
Exercising these invisible tools in different actions that are prone to fraud, that's kind of one thing that we're starting to do now and just trying to roll out completely throughout the platform. But I see us eventually evolving to a spot Where if you think about what we do manually today, it's kind of beyond just the basic actioning and automatic things that we do when we get information from the user. You also have just more complex cases where you consider like an entire session and see like, okay, somebody went from completing this action at this time and then they went over here. It's really interesting because as like a human analyst, we sort of get a feel for what's genuine and what's not. Just from looking at so many different user sessions, you can see if somebody boots up the app, they're searching for something to watch, and then there might be a manifest of something before it starts playing and pulling from the CDN. I kind of see that manual work maybe moving more towards automation in the future.
Mm-hmm.
It might be some sort of machine learning magic thing that's a little over my head at this time, but I think that would be really cool.
If you've listened to an episode of the podcast before, you'll know that Anton and I both see machine learning as very far from magic, at least today. Brian, where do you see this stuff going in the future?
From a bot perspective, look, threat actors, they are innovating just like we are, right? So they're trying to use this tool to say, how can I exploit the experience? They're looking at our protections. I mean, they know this, they know what we're doing. So they're going to try to cater those attacks or customize those attacks using bots. A few key things that I think, just to piggyback on what Randy said, is like looking at the entire user journey or that experience. What other areas of that experience, from logging in to selecting content to actually delivering that content, could be susceptible to a bot and how we can give visibility so that we could counter that? I think we're going there. We're definitely going there. But threat actors look at that.
There's a threat modeling piece to all this.
Oh, absolutely. 1,000%.
It's not just throw the tool out on the website. Figure out what I really need to protect.
Figure out where you really need to protect, and again, where friction should be introduced versus where it shouldn't be introduced, right? The invisibility piece is incredibly huge for a consumer-facing product. Like Randy said, we had to go on that social campaign to kind of like get it across, you know, certain folks' threshold of friction. And, you know, we used our powers of persuasion. You know, we waterboarded them, uh, and stuff like that. Just joking, joking, right? But we used our powers of persuasion of actually showing them data, how this is actually going to work, continuing to do that throughout the experience and saying, this is where we want to introduce the right amount of friction or put some sort of visibility and tools, is going to be very crucial for us to continue to counter these bot attacks. Threat actors are creative. They are innovators as well, right? So we need to look at this in that lens of this is not a set it and forget it.
Mm-hmm.
Threat modeling is a constant focus. As we improve the experience, as we continue to innovate on the experience to offer the content out to the world, as security practitioners, we need to look at at that and say, where can we facilitate the protection of that experience without introducing friction that's not going to allow customers to come in? So I still see a lot more creativity and innovation happening in this space on both sides, both the threat actor side and both on the security practitioner side as well.
Totally. I used to work in this space, and watching the bad guys innovate against our defenses was fun, honestly. It really felt like we were the cat and the mouse for real. And, you know, with these attacks, they're so often economically motivated. A lot of stuff driving them.
I agree with you. Like, you're going hand to hand with a threat actor and it is fun. It also keeps you constantly thinking of ways to counter that as well. So it keeps you innovating. It keeps you thinking creatively, thinking out of the box and how we could use our partners and tools to facilitate that security experience for our customers.
Well, Brian, Randy, this was a ton of fun today. Thank you so much for joining us here on the Cloud Security Podcast by Google. Really appreciate your time and your thoughts on all of the things reCAPTCHA.
Awesome. Thanks, Tim.
Thank you for having us. Appreciate it.
And now we are at time. Thank you very much for listening and for subscribing. You can find this podcast at Google Podcasts or wherever you get the podcasts. Also, you can now find it at our website. Website. Hear it? We have a website, cloud.withgoogle.com/cloudsecurity/podcast. Please subscribe so you don't miss fun episodes in the future. You can also follow us on Twitter, twitter.com/cloudsecpodcast, twitter/anton_chuvakin, and twitter/_timpeacock. Tweet at us, email us, argue with us. And if you like or hate what you hear, we can invite you to the next episode. See you on the next Cloud Security Podcast episode.
More from Anton


EP289 Software Engineering vs. Software Craft: How Google Scalably Eliminates Classes of Vulnerabilities


EP288 CISO Tested, Board Approved: Cloud Resilience with General Mills CISO Noah Korba


EP287: Creating Trust at Global Scale with Local AI: Reken-ing with Shuman Ghosemajumder


EP286 Building an AI-pilled, solo vibe-coded, Clickhouse-based SIEM with Dan Lussier


EP285 Scaling Lessons from Leading the NSA to Defending the World with Morgan Adamski


EP284 Closest Alligator to the Canoe: How Transforming SOC Became P0 for Lloyds Bank


EP283: How Google Cloud CISO Chris Betz Uses LLMs to Defend Billions of Users from Vulnerablities


EP282: Debating Coupled or Decoupled SIEM with Alex Hurtado and Christopher Witter


EP280 Hyperscaling Cloud Security: Wiz Joins the Cloud Security Podcast by Google