AI-Driven DevOps Data Security and Privacy with Amjad Afanah

Get ready to discover some of the most actionable DevOps techniques and tooling, including performance and reliability for some of the world's smartest engineers. Hey, I'm Joe Colantonio, host of the DevOps Toolchain podcast and my goal is to help you create DevOps toolchain awesomeness. Hey, today we're talking all about shift left strategies for sensitive data protection and privacy compliance. I found a new new company that just came out of stealth called Hound Dog AI that I featured on my new show and the founder is cool enough to join us to take a deeper dive into why the solution and how it's going to help you with security. You don't want to miss this episode if you don't know. Amjad is a serial entrepreneur with deep roots in CyberSecurity. He led DCHQ, which is a cloud management startup, to acquisition. Ian founded AppSec AI pioneering AI API security scanning. He really knows his stuff. But being frustrated with the reactive approach to data security, he recently launched Hound Dog AI to allow to align security with evolving code bases. Like I said, I think it's a really great topic. Something a lot of people don't know about and how. You need to really be thinking about this with your DevOps pipelines. You don't want to miss this episode. Check it out.

Hey, if your app is slow, it could be worse than an error. It could be frustrating. And one thing I've Learned over my 25 years in industry is that frustrated users don't last long. But since slow performance isn't sudden, it's hard for standard error monitoring tools to catch. That's why I think you should check out bugsnag, an all in one observability solution that has a way to automatically watch for these issues. Real user monitoring. It detects and reports real user performance data in real time so you can quickly identify lags. Plus you can get the context of where the lags are and how to fix them. Don't rely on frustrated user feedback. Find out for yourself. Go to bugsnag.com and try it for free. No credit card required. Check it out, let me know what you think.

Hey Amjit, welcome to the Guild.

Thanks so much for having me Joe. Appreciate it.

Awesome to have you. Like I said, I found you on my new show. As I was going through LinkedIn, I noticed this company and I was like oh, this sounds cool. So I guess before we get into it, I'm always curious to know about founders. It seems like you're a serial entrepreneur founder. Why or what? Inspired you to create Hound Dog AI? And how did you see your experience in data security to shape your vision for this platform?

Yeah, I appreciate it, Joe. Yeah. So like you said in the introduction, this is my third startup. You know, before founding houndog AI, I was one of the co founders of apisec AI, which was an API security scanning platform and then, you know, transitioned into doing different things after that. And right before I started Hounddog AI, I was with a data security company. And like most data security vendors, I found that most DLP and data security platforms focus almost entirely on discovering data in production, whether it's structured or unstructured data. And then based on like whatever they discover, they help you kind of apply access controls by either asking you to kind of install a proxy to any connections or traffic going to the database and that's how they are able to kind of protect the data in production. Right. And while I was there, I started hearing a lot of feedback from various privacy and security teams where they were kind of the overwhelming feedback was, you know, it's great that we have these controls in production, but how can we prevent these pii, the leaks of personally identifiable information from day one, from the very start, so that we don't have to deal with the repercussions of cleaning up the logs, doing the code changes, reviewing the, who accessed the logs, reviewing those, you know, access, doing the access log reviews, doing the risk assessments, in some instances even like doing customer notifications depending on what kind of data they were they discovered in production. And so they wanted a more proactive approach to stop it from before the code is even merged and pushed to production. And then on the privacy side, that's like, you know, specifically for gdpr, right? It's often like kind of centered around documenting all the systems, all the applications that are having user data. And you have to kind of document the data flows, where the data is coming from, why you're collecting it, who are you sharing it with, that's what it really boils down to. And believe it or not, but most companies rely on like manual surveys. So they would send out surveys to like their business units and they would cite somebody to say, you know, what systems does this business unit use, what kind of user data are they collecting, etc. And as you can imagine, it's like a very error prone manual workflow and it often doesn't keep up with the evolving code bases, especially for the internally built applications. So that, you know, privacy teams are often the ones that are the most blindsided by changes because They're a bit even further removed than security from what's happening in development.

Love it. Yeah. I used to work for a healthcare company and every quarter something we had a list all the third party libraries we're using in our software. So it's something similar and it never was right, always was a pain for sure. So this sounds like it helps you with certain aspects of it. I guess I like to keep it simple to start off with. Maybe we take a deeper dive if someone's listening and they're like, okay, what's a personal identifiable information PII leak? Is that just a Social Security numbers? Like what is that all about?

Yeah, yeah, they come in different categories. So even within PII there's like account data, there's like usage data, there's stuff that is about your employment history, there's stuff about your education history. Right. So all of that is essentially like personally identifiable information. Information where you could infer who the person is by looking at that data. But you know, obviously there's like companies and security and privacy teams usually apply very sensitivities to that data because not all data is alike. Right. Like you collecting a phone number or an email is very different than you collecting Social Security or credit card number. And so you know, the most critical types of PII are actually like, you know, passwords and tokens. Right. Because like if you leak that, that can be immediately used for hacking. But obviously like the less severe, you know, the, the like high sensitivity but less critical, maybe the stuff that I talked about which is like Social Security, credit card information, protected health information, you know, things about like your medical history, your symptoms, your health, diagnosis, etc, and then it goes down into like the more usual bread and butter PII like date of birth, you know, address and, and so forth. So that's kind of the high level, kind of the explanation of what, what is pii? Yeah.

All right, so once again used to work for a large company and because it was regulated we had to use masked data, not real data from customers. So in development when we were testing, we were using this mask data. So how does this PII information that get gets into production if you're developing using like a lot of data that's not real data, like how does it get there? I guess.

Yeah, no, a great question. Right. So what we look for is not the actual exposed data in your code base. Right. So what we're looking for is the code logic that is about to handle sensitive data. So that's the exact distinction between what the SAS scanners can provide today. So most SAS scanners can help you detect exposed secrets. Right. If you have actual passwords or actual API keys and tokens in your code base, they would apply some kind of like reg X pattern that will say, oh, okay, this matches this kind of password or this kind of API token that is typically used with aws, for example. Right. We on the other hand are examining a different thing. We're examining the functions, the methods, the classes, and we're looking at the names of these things to see if, based on the name of the function, is it likely to handle Social Security or credit card number. Right. And then we apply the abstract syntax tree and very advanced kind of code analysis techniques like inter procedural analysis and tate analysis to kind of basically follow whatever triggered a match. So like if we detected that this function is called ssn, so most likely it's handling Social Security, we'll track that variable across the code base and see if you are kind of invoking it inside a log. So you could do log console insert ssn, Right. Or you could do, you know, save into file and you know, insert SSN or even worse, you know, send to third party systems. So you can call like a Sentry or datadog API and start sending that stuff there. And so that's essentially what we're doing. We're not looking at, we don't care what kind of actual data you use when you're doing development and testing. We're looking at the actual code logic to see what is it about to handle when, once it's actually deployed to production.

Gotcha. So you can expose issues that a hacker may be able to normally penetrate before it gets into production. Is that does that my understanding correctly?

Right, right, exactly. So basically the two main kind of use cases are a, that developers make mistakes, they sometimes overlog things. So sometimes they want to debug something and they say, oh, okay, maybe if I kind of dump this variable, which you know, has a whole bunch of information into the log, it will help us with debugging once it's deployed in production. Right. So then in, once the app is actually deployed in production, it will start logging and, and then like 10 other tools that ingest those logs will start collecting that information. And all of a sudden security team that wants to kind of clean up those logs, you know, that have like 10 times more effort or resources that are needed to just do that cleanup task. And then it depends on what kind of data you're overlogging. Right. So obviously like the extreme case of hey, you know, we actually like logged customer passwords. Right. Or customer password hints, which actually happened a few years ago. There was like the, a kind of a breach associated with one of the big tech companies where the clear text password hints were kind of exposed and were publicly kind of available as part of that hack. And so in those kinds of cases, you know, that triggers a whole bunch of like incident response and customer notification and you have to kind of go through like a whole hoopla of trying to kind of comply with the whatever frameworks and make sure that you're paying the compliance fines and all that stuff. Right. So we're helping kind of avoid all of that by, hey, you know, developers make mistakes, our scanner can detect those mistakes. And then the remediation is often like very simple. Right. You either omit that data or to your point, you mask it. Right. So if you apply a masking function to that data, we'll actually know. Okay, like this is fine to log because it's being masked when you're logging it. Right. But if you're doing it in plain text, that's when we flag it as a vulnerability, basically. Yeah.

Nice. Yeah. I know a lot of developers, when they create APIs or IoT headless devices, a lot of times they don't think about the data because it's not a front end. And it seems like this would help with that as well because it's able to flag things before it actually gets out there in the wild for sure.

Yeah, absolutely. Yeah. So one of the features that we provide in addition to just like flagging vulnerabilities, is the ability to do what we call sensitive data visualization. And so we will basically, from the very first time you scan a code repository, will be able to map out all the sensitive data elements that are being processed in your code base. And then you can actually drill down and view a visualization of all the files that process it and the data syncs where it's actually ending up. Is it ending up in a different third party API? Is it ending up in a log? Is it being stored in a database, in a data warehouse? So obviously not everything triggers a vulnerability. But that visualization is also very helpful for you to kind of just get a snapshot of, oh, okay, like we're processing Social Security and it's being, you know, stored safely in a database and is being encrypted and etc. Etc. That visualization will help you kind of basically kind of understand what's happening to your sensitive data. Yeah.

Cool. So if you're being audited, if you have a regulator that's coming in and they're like, I'll prove to me that this is compliant or something like that. Could you use that report to show them, hey, look, here's what we've done and here are the things that we can prove our either fits a law or is compliance. It's not leaking user data.

Yeah, 100%.

Yeah.

I think the privacy compliance is the one where that's most stringent and is kind of completely centered around dataflow documentation, which is like those documenting your processing activities. Right. So that, that one is, seems to be the most urgent when it comes to using a feature like this. But there could be other security audits, for example, like we went through SOC2 and I'll be honest, like SOC2 is a bit lenient. It doesn't necessarily like have clear cut criteria as to, hey, you know, give us kind of a sensitive data map so that we understand what data you're collecting. It doesn't have a clear guideline like that, but obviously if you submit it as part of that evidence gathering for the risk assessment control, it definitely can help make your case that, hey, you know, we're, this is the only data that we're collecting, we're not collecting anything else. And it can definitely be used in the, in these use cases as well.

Nice. So, you know, anytime I talk to security folks or developers when they have to deal with security with all these scanners, they get a lot of false, false alerts. So I know you mentioned this is a little bit different than a normal, you know, SAS kind of scanner. So how does it work? So you don't get maybe false positives or how do you know it's really detecting data and that it's really true? Positive detection, I guess.

Right? Yeah, we passed, you know, several months before we actually launched. We kind of focused primarily on that challenge. Right. And so our scanner comes with predefined patterns. So these are like regular expressions that. So I mentioned the Social Security number. Right. So will have a regex pattern that will say here are the different combinations of what a developer may call a function or method that's about to handle Social Security. Right. So with our predefined patterns you get about like 80 to 85% coverage. Right. But then what we do is we also have an AI integration. So we integrate with OpenAI today. And what happens is that our scanner collects all the tokens, all the names of functions, variables, methods, et cetera, and anything that doesn't trigger a match against our patterns, we actually send off to OpenAI, using a very finely tuned prompt that we've been working on for months. And you know, in essence, we'll basically ask OpenAI, hey, based on the names of these things, which one do you think is handling sensitive data? And give us back like a confidence score. And we only return back the ones that have 100% confidence. And based on our testing, you know, the accuracy has been upwards of 95%. So we have, you know, a number of customers that have kind of also attested to the high accuracy of that kind of workflow because we're only returning the very, very high confidence results. And so what that translates to is that, you know, you can use our scanner without AI. And like I said, it should cover the most common pii PI fi, which is the financial information, the protected health information stuff. But if you have something custom, like let's say you're a government agency and you have like federal registration number or, you know, U.S. contractor number. Right. Those are like not common things that we would have out of box definitions for. Right. But OpenAI is actually smart enough to say, oh, I think this is actually sensitive and has been consistently returning those true positives for us. And so that's how we're leveraging the power of AI for that. Yeah.

So does anyone ever get freaked out, oh, you're sharing this with OpenAI, like, is it now going to know? Like, hey, this is a method that I can let someone know this is could be a vulnerability to other people? I don't know if that makes sense. You can always hack it backwards to say, all right, it says it's a vulnerability, but now it's listed somewhere and someone could find out now the name of functions that may not be obvious that could be hacked for personal data.

Right? Right. Yeah. So what we said to OpenAI is just the names of functions, methods and all that stuff, so we don't actually send it the actual code. And yeah, basically like the reverse engineering thing. Right. That, I mean, that somebody would have to kind of intercept kind of your request, going to OpenAI for it to kind of understand what that request is about. But the OpenAI model. Right. Is not going to kind of save that information. Right. It's just kind of making that assessment on the fly based on the naming of these things to tell you if it's handling sensitive data. So it's an evolving space and we understand that a lot of companies may not be comfortable using OpenAI just yet. And so most of our customers start out without this AI integration, especially if they're in kind of a standard, kind of fintech, health tech, insurance tech type of space for which we have upwards of 90% coverage. Right. But if you are in a more kind of niche space and you have like your own unique set of type of sensitive data that you process, AI is going to be like a multiplying force for you because you don't. Obviously we allow you to add your own data definitions, but we don't want you to spend your time doing that. That's been like the Achilles heel of a lot of the DLP and data security platforms where most customers end up being frustrated to kind of, you know, having to kind of define their own definitions. That defeats the whole purpose. If you already know what data definitions, what is what, what it's actually called, then like task of discovering it is. You've already done that task already, right? We are trying to help you discover things that you may not know existed in your code base or you don't have to know what function or the method that is handling that data is called. Right. So that's kind of the, the whole point of our scanner.

So going back to my, my original thing when I said we're using third party libraries, so most likely the developers don't necessarily know what it's doing. Does this help you know that you're using a third party, a library that isn't being compliant? Will it help point that out as well to say it's not oz, but it's definitely this, this is not compliant, we can't use it, or this is definitely leaking something?

Yeah, no, great question. Right. So I'll say instead of like third party library, I'll talk about like third party applications. Right? Because okay, that's the main thing that we're kind of addressing. I think the library is more like the software composition analysis type of tools and all that stuff. And we're not covering that. Right, but the third party applications is exactly the problem we're trying to address. Because both privacy and security teams alike are often blindsided by integrations that developers enable in the code base. Right? Because they often need to debug things, they may need to, to send things to like Sentry or Datadog or other tools that may help them. And you know, oftentimes a lot of these teams may not, may be blindsided by the fact that this integration even exists. Right. And so like I said, privacy teams are even farther removed from that and they kind of rely on the people's knowledge. Right. So they'll send a survey to the engineering Team saying like, hey, just document all your integrations. And you know, whoever is filling out that survey, if they didn't talk to the engineer who enabled that integration, maybe missing that one. Right. And so that's the whole point of like relying on a source of truth that is actually the code base that's off, that's changing. Right. And because you can get a snapshot of what the integrations look like today, but in a week or two weeks, depending on how frequently the code base changes, that stuff could be changing. And there was like a recent survey by the State of Data Integrations or something like that that indicated that, you know, majority of companies are adding systems that handle user data on like as frequently as a, as weekly basis. And so it's like almost impossible to keep up with all the things that are handling user data and just keep relying on manual methods like surveys and spreadsheets to kind of keep up with that. There needs to be a way to kind of plug it into your CI pipeline and have like the assurance that like, even with frequent code changes, you're able to keep track of the data flows, the third party kind of integrations, as well as like PII leaks and logs and piles and so forth.

Nice. How hard is it to integrate to a CI pipeline? What's the flow to get this up and running? Is it long process? A simple process? How does it work?

Yeah, no, extremely simple. If you go to Docs, Hounddog, AI, you'll see our documentation and we integrate with all the major CI pipelines. GitHub, GitLab, Azure pipelines, Jenkins, CircleCI and so forth. And basically our scanner runs as a docker container. So you basically create like a CI config that invokes that docker container. We can spit out the results in various formats. We can for, for GitLab and GitHub specifically, we can actually spit up the results in their security dashboards. So you can view the vulnerabilities right there in the, the GitHub advanced security dashboard or the GitHub vulnerability report. And then we can also obviously send the results back to our cloud platform, which allows you to kind of do more sophisticated workflows like, you know, file JIRA tickets or get notifications via Slack email and be able to have a consolidated view of all the code repos in like one platform. Yeah.

Nice. So, you know, we mentioned multiple times you've founded many companies and so I assume you have a lot of ideas. You looked at a lot of different venues, a lot of different things to get into. So I'm just curious, did you see like on the horizon an increase in the need for data security with different privacy compliances that countries and, and states might be coming up with? Like, and is that, was that one of the driving factors? Maybe a little bit about like where you see it going with the evolution of data security and privacy compliance and why people need to get on top of it now, if at all, basically.

Right, right, yeah, exactly, yeah. So I'll just say that from the data security standpoint, right, that use case is all about like minimizing, you know, security risk, improving your security posture and most importantly like reducing costs. Right. Because we have like an ROI calculator on our website and it was based on extensive research that we've done with our existing customers and other kind of security leaders. It can take like at least 80 hours of resources to just remediate one leak in one PII leak that is discovered in production. Right. That's because it requires the code updates, access log reviews, the risk assessments. You have to kind of do the, kind of the, the incident response and all that stuff. Right. And so we estimate that and for every 100,000 lives we, we scan, we find like at least 10 vulnerabilities and that's been consistent for all the customers found. Obviously they're not all at the same severity. They're usually like medium low severity type of issues. But still like we're finding a lot of these vulnerabilities because this is the type of stuff that you know, where you find PII hiding in logs and in third party systems as the most predominant use cases, examples. So it's a pure cost cutting measure. For the privacy one, you're 100% right. But that Europe kind of consolidated towards GDPR and that's very established. They're actually been finding companies like crazy. Last year in 2023 they, the total GDPR fines were more than 2.2 billion upwards from I think.09 billion the year before. So the, the, the GDPR is going after companies and finding them and that's been like something that, you know, a lot of the companies that operate, they don't have to even operate in Europe, but if they have European users, that's a huge concern for them, that they don't want to be fine. Right. In the US it's to your point, it's very much scattered, there's like state by state kind of regulations. But you're absolutely right that it's only a matter of time before the US comes kind of comes together with one kind of federal guideline similar to GDPR but for the US when it comes to privacy regulations.

Yeah, I'm just a one person operation. I get contracts sometimes for different companies in Europe that say am I gdpr, GDPR compliant? So I would assume it affects everyone. How hard of a sale is this to someone? I'm just curious to know, does everyone get it? Like who do you go towards? Who do you, who's the perfect person to tell about the solution? If someone's listening and they're like, all right, I get it, but how do I get buy in? Who should I tell first about this solution? Yeah, I know, if that makes sense.

100%. 100%. Yeah. So we have two buyers. One is like the, like the CISO and like the VPS directors and security, specifically AppSec because even though we do data security, but we, our scanner looks and feels like an AppSec product because it plugs, it's a code scanner ultimately and plugs into CI pipeline. And so they're the ones who will be able to kind of manage it and get up and running with it like the fastest. And then on the privacy side, it's more like the chief Privacy Officer and the various kind of directors, VPs of privacy and sometimes it's like legal, sometimes it's the GRC kind of types of departments within companies. And yeah, in terms of who really gets it, it's the ones who've been like bitten by dealing with some kind of leak. Right. Like from the security side it's more like, oh man. Yeah, we dealt with like a leak in logs and like it was nasty and it like, you know, wasted like I don't know how many days and like it, but we reduced the productivity of the engineering team, etc. Etc. And then on the privacy side, it's actually much easier to kind of convey the message because the status quo is really bad. Right? Status quo is really manual reliance on tribal knowledge on all that stuff and to kind of give them an ability to rely on a more reliable source of truth, which is the code base that's evolving and changing and they don't have to kind of rely on these services. They can like be keep track of like the unpushed changes and be proactive about things that for example, the privacy teams want to have an input on. Right. So they want to make sure like if you're introducing something that is of high sensitivity that it's being lawfully processed, that is asking users for consent and is doing X, Y and Z and they don't want to kind of be blindsided by something that is already pushed to production and try to kind of patch it after the fact. Right. So, so that these are like the, I would say like the kind of the, the whole sales cycle for us. Yeah.

So I should have asked this question earlier. You kind of touched on it. I check in code, the scanner runs, it finds an issue. Is the developer alerted right away? How long does it take? Is it a dashboard? And if so, do they even know how to fix it when they get alerted? Like they're like, okay. Or does it give some suggestions how to resolve?

Yeah, yeah, great question. So it's really up to the AppSec team that configured that workflow. So in the CI pipeline they can say anytime there's an issue of severity critical. Basically I want to like feel the build. Right. We often don't recommend that because you don't want to like slow down things, but you know that's an option. And then as far as like assigning issues, like I said, if you're using our cloud platform you can use like the whole JIRA ticket approach and you assign JIRA tickets. A lot of companies prefer that model if you are like a GitHub, GitLab kind of security dashboard user. So both GitHub and GitLab have their own security dashboards where they can actually have a consolidated view of all the issues that's covered not just from our scanner, but all other code scanners. That's another way you can assign things to developers and we provide like detailed remediation strategies. The good news is that the type of issues that we're finding is not hard to fix. It's more like, like I said, you either omit the data, like don't log Social Security into logs, you don't need it for debugging. Right. Like that's, that's common sense. But let's say like, oh, but we do need to collect this kind of pii, like email or something you could do that. You can like use UUID or like do some kind of masking. So we don't want you to not be able to debug things, but we want to make sure that you're doing it safely. So we provide all these mediation strategies in whatever issue we find.

Yeah, nice. So I have in my notes, I don't know if this is accurate. Is there a free scanner for early access customers? If someone wants to listen to us, like at least want to try it first before I bring it up to my, my organization. Is that an option?

100% yes. So go to our website, Hounddog AI. You can sign up for the free scanner. It will, after you fill out the form, will just like redirect you to the Docker hub page and you just a docker container, basically. You pull the docker container and you just like point to the directory that has the code repo you want to scan. You scan it, it outputs the results directly in the console. You can save it in a markdown file. And we support like JSON and other types of formats as well. And for the free scanner, we are, we've limited the features to being able to create like or generate that sensitive data map that allows you to see what all sensitive data flows exist in your code base. And for the paid version, that's where you need to kind of use the exact same scanner, but basically add like an environment variable that plugs in our Hound Dog API key that will give you as a paid user and that will unlock other features like the vulnerability detection, the generation of record processing activity reports, the ability to kind of do all that stuff in a consolidated manner in the cloud platform. So that's kind of the distinction between the two. Yeah, the free and the paid version. Yeah.

Awesome. Okay, before we go, is there one piece of actual advice you can give to someone to help them with the DevOps security PII efforts and what's the best way to find contact you and learn more about houndog AI?

Yeah. So in terms of like, you know, best practices, I'll be actually hosting a webinar in the coming weeks. I'll be talking about like both preventive and detective controls. So we, we are more on the detective controls where we help you detect these things as you kind of develop.

Right.

But I think on the preventive controls, I think training is the most important thing. Right. So, you know, educating developers and you know, there's a lot of security awareness trainings, but. And oftentimes, you know, developers are, you know, overwhelmed by the amount of information that's being thrown at them. But this is one of the things that is like, just easy to kind of communicate because, you know, the developers may not know that what they're doing has like major repercussions. But you know, oftentimes just like omitting or masking goes a long way in terms of preventing a lot of the, you know, the repercussions of that. And yeah, people can reach out to me at amjadaldog AI anytime and I'm always here to answer any, any question. Yeah.

Remember, latency is the silent killer of your app. Don't rely on frustrated user feedback. You can know exactly what's happening and how to fix it with bug snag from SmartBear. See it for yourself. Go to Bugsnag.com and try for free. No credit card required. Check it out, let me know what you think.

And for links of everything of value we covered in this video DevOps toolchain show head on over to test guild.com forward/p151 and while you're there, make sure to click on the Smart Bear link and learn all about Smart Bears awesome solutions to give you the visibility you need to deliver great software. That's smartbear.com so that's it for this episode of the DevOps Toolchain Show. I'm Joe. My mission is to help you succeed with creating end to end full stack DevOps toolchain awesomest. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@testguild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value to help can improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head on over to test guild.info and let's make it happen.