Using edge models to find sensitive data

Foreign.

Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI related tech is changing the world, this is the show for you. Thank you to our partners at Fly IO, the home of changelog.com Fly transforms containers into micro VMs that run on their hardware in 30 plus regions on six continents so you can launch your app near your users. Learn more@fly IO.

Welcome to another episode of Practical AI. This is Daniel Whitenack. I am founder and CEO at Prediction Guard where we're safeguarding private AI models and I'm really excited today because I'm joined by a friend who I've had the pleasure of getting to know a little bit over the past weeks in the startup community. Joined by Ramin Mohammadi, who is the AI and ML lead at TaoSite and also an adjunct professor at Northeastern. Welcome Ramin.

Hi and thanks for having me.

Yeah, yeah, it's great to have you here. It's been cool to visit the Boston startup community a couple times and participate in a few events together. I've been really fascinated to hear about some of the things that you're doing at TaoSite, so I'm excited to dig into those a little bit. I'm wondering if you could share a little bit with us about kind of you're really thinking deeply about the intersection of AI and privacy, but specifically as related to privacy and personally identifying information, but also personal health information. So Phi so Tausite is thinking deeply about how companies are handling very private sensitive data and knowing about where that data is, which is actually a huge problem. I'm wondering if you could talk a little bit about I was kind of shocked when I heard one of your presentations and you're talking about just the size of this problem and the scope of this problem related to Phi. So could you give us just a little bit of a sense of what Phi is, why companies handling phi is kind of a problem and some of the challenges related to that?

Sure, I can do that. So first to give introduction what's a Phi or Personal Health Identifier rules. So based on the HIPAA rule there are 18 identifiable which can lead to identify an entity or a person within an healthcare organizations and this information are valuable and being targeted by hackers. One of the reasons is that they have high value because they contain your sensitive personal information such as medical history, Social Security number and insurance details which makes it very valuable on black market. They also use this for monetary gains. So hacker can sell this stolen phi to criminals who use it for Identity theft, insurance fraud or other legal activities. They also use it for exploitation and extortion. Basically they use this stolen health information to use for blackmailing and blackmailing individual organizations. So 133 million healthcare data was breached in 2023. Which means one out of three Americans life was affected. This means about 160% increase in compared to 2022 and about 240% increase since 2018.

So like for our listeners, if at least if they're in the U.S. one out of every three of those listeners has had some portion of their health information exposed in some type of breach. Is that right?

That is correct. That record came out in 2023.

Yeah, yeah. That's insane. And you mentioned hacking. How is this data being breached mostly by hackers or by sort of mistakes of organ? What is the combination of ways that this data is getting exposed?

That's a great question. So actually based on the report On 23, 78% comes from hacker of the network storage where the data resides in healthcare. And there are a small amount which is about like a 2% happens by when someone stole a laptop for example and the laptop contained phi or sending an email via email or basically phishing emails, stuff like that. So there's like a breakdown, but majority comes from hacking. 78%.

And are these companies sort of mostly healthcare companies or like who has this data and how is it given?

So that's, that's really interesting because technically healthcare organization like hospitals are only I think accountable for 30% of this incidence. And the remaining 70% happens by hacking through the business partners like third party organizations that they have some sort of softwares or the storage for keep tracking of the most medical data. But at the end the cost comes to the healthcare organizations.

So that cost, what are the implications like what happens when this data is breached? What is the bad case scenario? I guess.

Sure. So let me first tell you that how overall cost of this has been. So healthcare cybersecurity has spent about 28 billion over five years span and we are still not able to predict the phi. And the way that it is that when an organization getting hacked or that there's some data breached, depending on the states, there are some cap. If for example if you have been breached and you have lost more than 500 entities or life basically data, you need to go public about it and also you will basically get sued and also you need to also get fined. So we have this wall, which we call it wall of shame unfortunately and government post the names of the organization where they got hacked or they lost basically phi data and this wall being constantly updated and the last things any CISO or CIO wants is that to see their name on that wall.

Yeah, and their brand is hurt by that. But also there's fines, right, for this, this sort of breach.

What's up, friends? Do you remember when ChatGPT launched? I do. It felt like the LLM was this magical tool out of the box. However, the more you use it, the more you realize that that's just not the case. The technology is brilliant, don't get me wrong, but it's prone to issues like hallucination on its own. But there's hope. There is still hope. Feed the LLM reliable current data, ground it in the right data in context. Then and only then can it make the right connections and give the right answers. The team at Neo4J has been exploring how to get results by pairing LLMs with knowledge graphs and vector search. Check out their podcast episode about LLMs and Knowledge Graph throughout 2023 at Graph Stuff FM. They share tips on retrieval methods, prompt engineering and so much more. Don't miss it. Find a link in our show notes. Yes, check it out. Graph Stuff FM Episode 23.

Could you talk a little bit about, like let's say that I'm a healthcare company and I want to not be on that wall of shame and I want to do the best practices and all of that. Like what is the reality of. You know, I know that of course you have been thinking very deeply about solving these issues with AI and machine learning or some related issues, but let's just say that doesn't exist. Like what are the choices that a company has in terms of, and what are the challenges that they face in terms of securing this data?

So currently healthcare organization have series of tools, some of them, like maybe four or five tools that they're going to do the same task. And the way that these traditional tools work is that they have series of pattern like a regex, for example. And when someone tries to download the data, if this matches, for example, with some rejects that they have, it will say hey you transferred phi or hey you downloaded phi. So those types of files which they're coming directly out of EMR electronic medical records are easy to detect. The problem is with what we call dark phi, a phi that resides on your network, on your machines, but you are not able to detect it because the patterns that you are using are basically not capable of detecting that. You know, we have, for example, organization that has millions of patients in your network. And I don't know if you probably. You have written regex before. Regex is always as good as the person who's writing it.

Yeah, yeah. And it also, what is that? That meme? It's you decide to solve a problem with regex and then you just end up having another problem, which is regex.

That's correct. So current from the prospects that we have talked with is that updating these rules is costly and requires a dedicated engineers or IT professionals. And no one likes to write regex.

Very true.

At least I can say that. So that's a problem. They have tools in place but, but you know, it's incapable of solving the problem.

And when you say dark phi is this, I'm assuming there's like, okay, you might have a regex for a Social Security number or something like that. But if I just think of like a doctor recording a dictation of a patient visit or something like that, there's a lot of natural text in there about diseases and all of those sorts of things. So is that more natural text sort of health information? Is that what leads to kind of the dark phi or does it also have to do with, you know, oh, it's easy to detect in this file format because I know the pattern I'm getting. But then someone scanned in this document and it's in like a PDF or something. And my script doesn't know how to scrape these different data types. What could you kind of go into? I'm super fascinated by this idea of this dark phi sitting around.

So the first thing is that point out that is 80% of healthcare data are unstructured. Unstructured means that from image to audio transcript, like all sorts of PDFs. And what we have seen also on the healthcare, it's a variety of data extensions. So you will be surprised that you will see file extensions that they don't exist. But what the clinicians or researchers do is that when they edit a file they put dot the last name at the end. So you like. I think in the last study we did, we found 8,000 extensions on our basically prospects environment.

So literally the personal information is in the file extension.

That's correct.

That's great.

Personal extensions, Personal information could be in the extension, but also random file extension that they might use in order to basically bypass some rules.

Okay, I gotcha. So they're doing a workaround because there's like this annoying tool that prevents this data from being transferred around is blocking this file type. So if I just change the file type, that's Correct.

So one thing also here to point out is that healthcare lives on data. Clinicians need to access that data and you should not stop them basically from doing the job. You just need to have a better way to detect and basically maybe for example, encrypt the file. So if someone else has stole that file, they cannot open the file. Our goal here is not is to prevent clinicians from doing something, it's just to make it more secure.

Yeah. And maybe that starts to get a little bit to kind of transitioning to how AI and machine learning fit into this puzzle before we exactly describe how it does fit into that puzzle. I think there might be a lot of listeners out there that are very intrigued by what the challenges might be of applying AI or machine learning in the context of healthcare. Like what are the unique challenges? If you're a data scientist and you're building a model or wanting to use an LLM or building your own model to use in a healthcare context, what's unique about that context that makes it more challenging to. Maybe it's on the deployment side or the model building side. What are some of the challenges related to working in healthcare, specifically with this technology?

This is an interesting question as someone who's keen on MLOps.

Yes, exactly.

Say the main challenge of ML is the whole project. But for us these challenges are a bit more than some other AI technology due to the space which we are in. For example, we don't have access to real patient data to train our model model. And no healthcare organization will agree to let you use the data as it.

Said so and probably if one of them agreed to it. I'm guessing you couldn't use that same model for a different organization because you trained on specific data. That's sensitive for one organization, is that right?

That's also correct. We have a huge data heterogeneity problem. And that comes like for example, one organization, it's for cancer organization. The other answer is like it's a good dental organization. The status are different. Other challenges that I can say is that you also cannot collect or transfer any data to the cloud. That means everything needs to happen at the edge. Data labeling is highly difficult. Even human level performance has about 8 to 10% labeling error for detecting PHIs. Again, for example, you have lots of types and extensions. Data normally contains bias. Certain demographics have higher amount of data than others. Model development is confined by first model performance and then optimization metrics. And model deployment on the edge has its own difficulties which we can talk about later, I think. Lastly, unsupervised model monitoring makes it more challenging to detect drifts.

And just to kind of define a couple of those things. When you say the edge, what is your. Because people might have different definitions in their mind of whether that's a, you know, some staff member's laptop or a desktop in a lab or something like that, or like a phone or a microcontroller somewhere. Like people have a range of that. So in the context of healthcare, what is the edge environment?

Edge could be from laptop that clinician is working with, from the desktops, could be the tablet for example, that you're using and could be the storage server. Storage basically.

Gotcha. But all sort of on site with some healthcare data center or where staff are working on site.

That is correct.

Gotcha. And when you say unstructured model monitoring, what do you mean around that? So what are you monitoring for? You mentioned drift. So I'm assuming that that has to do with like this Phi might also be changing itself in terms of what it's like there's a new form type or there's a new. New thing that starts being collected. Is that. Is that what you mean by model monitoring? And that drift element?

One of the things that we will say is that the change in the distribution of the data as you scan across different groups within the same whole organization. For example, there's a group for radiology versus there's a group for like a normal pcp. And these data have different distribution and we need to be able to detect any drift in the data distribution as early as possible. Sometimes you also might find something like concept drifts where it's more like contextual. Maybe a file that under certain scenarios consider as phi. In some scenarios it's not actually phi, it's. There are some rules over here which makes it more difficult.

Yeah, yeah. Contextualization I guess is a challenge. Interesting.

What's up friends? I love Backblaze. I'm happy to have them as a sponsor. Backblaze makes backing up and accessing your data astonishingly easy. This is a service I personally use. Go to backblaze.com PracticalAI you get unlimited cloud backups for Macs, PCs, businesses for just 99 a year. You can easily protect business data through a centrally managed admin. Protect all the data on your machines automatically, easily deploy across multiple workstations with various deployment options. You can add on enterprise control, including granular access permissions, advanced single sign on group management controls and compliance support. They even offer multiple restore options including rapid recovery in the event of data loss. Or ransomware that sucks. You can access your backed up data from anywhere in the world using their web app or their iOS or Android app. You can even restore by mail. They'll give you a hard drive with all your data shipped to your door. You buy a hard drive, restore, send the hard drive back within 30 days and get a full refund. Get one year file retention and version history over 55 billion with A B files restored for customers so far, visit backblaze.com practicallyeyes they know where you came from and continue to support the show. This is a service obviously recommended by me, but also by New York Times Inc. Magazine, Mac World, PC World, LifeWire, Wired, Tom's Guide, 9to5, Mac and just so many more. You receive a fully featured or no risk trial@backblaze.com PracticalAI Again, they're supporting the show. Go there, play with it. Start protecting yourself from potential bad times. Start today.

Yeah, so maybe we could get a little bit now kind of into some of how you've been thinking about and approaching this problem and thinking about it from the Tao site perspective. So in the context of this edge environment, in the context of this unstructured data, in the context of the constraints that we just talked about, how did you, you and your team specifically think about applying AI and machine learning in the context of detecting phi and maybe also like what is your goal here? Is your goal to stop breaches? Is your goal to provide sort of insights about this phi? How did you decide on what the main problem is you wanted to solve and why AI or machine learning was relevant to solve that problem?

I'd like to actually first tell a short story.

That'd be great. Yeah.

Six months ago I was on a going to Dell, the AI summit in Austin. I got to the airport and I was passing the TSA pre where the TSA agent asked me to check my bag. It turned out that the machine picked up on this pre workout container which I had in my bag. The agent used this device on the box and the result was positive. It was like oh man, I need to call for this special unit to come and check this. I was like sure. Then this special unit with something like hazmat suit. Oh geez. They came and used a kit with bunch of different reagent to more specifically test my pre workout. They started sampling from the pre workout and added to bunch of these test tubes. Long story short, the agent was like yeah, you're good. He was like this happened to my cousin. Also these pre workouts are causing false positive so we are like that special unit with bunch of different model trains to find and protect the dark vhi. While the current tools in the market are like the first first and the second machine which leads to false positive or unknown false negative. At TAO side we do see this problem as a personal problem. It's our phi that's being targeted and clearly the current tools in the market that cannot protect it. So key part security rule says that you must do a complete and true assessment of all your risk and vulnerabilities to ephies. It is so fundamental to what we need to do. And AI is such a critical piece of taking advantage of the newer technology around solving what used to be a labor intensive problem that are could be much easier if you can define the scope of the problem and you can have machine learning models which can run effectively and accurately in a calibrated manner. That's what we do. We take advantage of the AI to find sensitive data. I think we get to the point that where risk and threats and vulnerabilities are going to be detected at the edge using the AI as opposed to gee, I have all these heuristic rules which is how do we lots of stuff today when it comes to recognizing patterns. So at our site we use AI for example to recognize when the sensitive data is in unstructured content. It doesn't require us to say hey, there's a keyword here, there's another keyword here that would be how you do heuristic programming. Combination of these three words must mean this. Combination of this four must mean this. You never get to do all the rules, you will never get to the variability you need. So in our model, for example, one of our model with about 50 millions parameters model that can be set to recognize this stuff, you will never in years of programming get that much logic into your regex. Right. So now the other main factor is for us is to ability to run these models right at the edge where the data is being created, emailed, printed, copied or faxed. We bring the AI to the data rather than taking the data to the AI, which most of the current AI solution do that. By doing so we can ensure that our data is always protected, agnostic of hardware spec or network connections.

Yeah, so all of what you said makes a lot of sense in terms of the approach and how you're applying AI and machine learning. But also in my practical sort of data scientist mind I'm like oh man, that's really, really difficult to sort of have these deployments of, of models. Especially against sort of heterogeneous types of data, run them on the edge, run them across a diverse set of hardware. From your perspective as a practitioner, where do you think was the most challenging of those issues? Was it having to do with like the deployment targets and the diversity of those? Was it having to do with the types of models that you could or couldn't run in those edge environments? Did it have to do with the actual training and labeling of the data? I imagine all of those are really difficult problems to solve and you had to tackle all of them. But what were you, maybe what was some of the hardest problems to solve with respect to those things?

I definitely will say the first, but the most challenging problem is the data labeling and data creation because we don't have access to real patient data. So we need to create our own curated data set, which we need to ensure that we don't introduce bias creation bias in that data. The other thing comes around the model training. So our solution need to be able to live alongside other programs that are running on a given machine within a certain performance boundaries. One example I give you is that it has set of rules. If there is an application surpasses certain memory or cpu, it will block that application. So you need to be sure that all these ML models that you have or this ML pipeline that you have always remains below this basically boundary.

And is that because like these are essentially, I mean, I might say mission critical, but these are sort of life critical systems. Right? Like they're using these to treat patients. Right. So if they.

That is correct.

If you pull the memory and the thing stops working, then it's potentially a life threatening type of situation, or at least a very concerning situation in the healthcare context, right?

That is absolutely correct.

Gotcha. Yeah. And in light of those constraints, of course some people now might just say, oh well, we've got all these LLMs now and they're great at doing all of these things, but I'm guessing a lot of those aren't sort of fitting for this sort of environment, these memory constraints. So where do you go with that? Is it looking back to sort of traditional NLP sorts of things? Is it model optimization? Is it a combination of those? How are you balancing the constraints? But also kind of looking forward to these new generations of models and that sort of thing.

Regarding the LLMs, I was reading about this Phi 3 by Microsoft, the small model.

Yeah.

And even that model requires certain amount of core RAM or gpu, correct?

Yeah.

None of the healthcare organizations have computer with those specs. They all have like 4 gigabyte of RAM maximum and some legacy CPUs. The other problem with LLM is that it introduce some additional risk to the health score. Some clinicians or researchers, they're using tools like for example chatgpt to copy paste patient data to get through some summary extraction which it's not how it should be used. I know for example, I know actually your company prediction guard, you are trying to solve a problem like that.

Yeah, there's certainly a lot of people pasting things into chat interfaces. That's very concerning. I'll definitely say that. Yeah, for sure.

That is great. Now, when it comes to model optimization, we take series of approaches to be sure that our model are optimized for such an environment. This could be from knowledge distillation or student teacher networks quantization and model pruning. We do technique combination of all of this to ensure that every model that we have lives within a certain boundary.

Gotcha. Yeah, yeah, that makes sense. So it's very important and I'm guessing the model architectures and the approaches that you, you can only go so far. It's not like you're going to take llama3.70 billion and do these optimization techniques and fit it into 4 gigabytes of memory and run it on a CPU. So it's super interesting and I think, I don't know what is your view as, as maybe you observe in the marketplace, people are exploring these open models, exploring bigger models. But at least in the space that you work in, the only way that you kind of move forward is with small models or customized models, optimized models. How do you view that kind of shifting into the future? Do you think there will always be this sort of diverse set of environments in the healthcare space that you need to optimize models for? Will they eventually get over their hurdles of using kind of the cloud or large models? How do you see that developing moving forward into the future?

The report by Schneider Electric indicates that currently 95% of AI workload operates on data centers. They have forecast that this number to go to 50% between edge and cloud by 2028. When you're monitoring the current developments in the market, you can see that most of the chip manufacturers are moving towards creating much stronger chips or machines where they allow to run the AI at the edge. For example, Intel's meteor lake or the aipc.

Right.

But it will take quite a while for healthcare organization to have that change adapted because you know, it requires budgets and I think healthcare organization, they go through machine update once Every five years. I don't think they do it over all data machine only maybe certain machines. But definitely I do see the future, that you can bring much larger models right at the edge. But I don't think we are there yet not.

Yeah, I appreciate that perspective because some people, I think in our listener base, they're constantly overwhelmed by this news about these new big models. But it's harder to get this sort of story of a practitioner on the ground working with specific companies in certain constraints. There's still quite a diversity of constraints that practicing data scientists or AI engineer has to work within. So I think that viewpoint's very important as you kind of look at what you've done with with Tao site and these tools that you've built in detecting Phi, helping companies know where their Phi is, reducing false positives, figuring out how to run these models on edge devices and all of those things. Do you have anything that stands out in your mind in terms of you don't have to mention specific customers or anything but success stories or really things that you're proud of that you're glad that you've been able to be a part of in terms of helping protect this Phi? Any sort of case studies or use cases that pop into your mind?

Yeah, I can give some example without naming anyone, but we were in this meeting and this CISO was in the call and it's like I had this laptops that was stolen and I don't know what's on that laptop but because we have our software on that laptop, we give them an inventory and it turns out the laptop contain lots of phis so they never have that view on this type of scenarios that where the phi is or who has access to it. But we can basically give you that, oh, I was on another customer call and it was like we are happy, you know, with the tools that we have, they have less false positive but unknown false negative and we are quite unhappy for writing the rules. Yeah, you know, it takes us a while but when you use our tool there's no rules that you need to write. It's basically out of the box. After you installing our product, it will start scanning all the files and also monitoring what's happened on the machine. So if someone copying or pasting for example Phi into an email or into another file, we can read that, we can detect that. If someone faxing it, we can basically detect that. So this customer calls that they have been on, they also got to be positive and sometimes scary for customers because you know, we are able to Find really detailed information around your network, you're.

Pulling the curtain back. There's work to do once you understand it. Yeah, well, maybe as we kind of draw pretty close to a close here, as you are kind of plugged in both on the academic side, you're plugged in on the startup side with Taosite, you're, you know, investing in this healthcare industry from the perspective of AI and ML. What gets you excited as you look to the coming year? Maybe it's things with Tau site, maybe it's things more generally in the AI community. What, what are you excited about and what do you think is a. Some of the positive things that you're, you're seeing kind of develop over the coming year?

I think there are two things that I'm really interested. One is this development on these large models and the fact that they're getting smaller and smaller. I am looking for the day that I could work with those SLM or small, large models and deploy those to unwrite the edge. And I think the other thing that I'm quite interested and right now actively working on is the federated learning. I know federated learning is kind of in the background. Not many company actively are doing it due to all the challenges that it has and also some security concerns. But for a domain like healthcare where you cannot transfer data and you cannot see the data, I found that absolutely necessary for your models to be able to train themselves and update themselves. So I think those are the two main things that I'm looking for the upcoming years.

That's awesome. Yeah, I'm definitely excited by both of those things as well. And I know Chris and I on this podcast have mentioned federated learning for years. I hope that it kind of comes more to the forefront as people figure out paths to do this. I think that will be interesting. Well, Ramin, it's been great to have you on the show. I think we'll see each other again in Boston before too long, I think. But yeah, it was great to have you on the show and thanks for taking time out of your schedule to share some of these insights with us.

Thanks for having me, Daniel, and see you soon in Boston.

Sounds good. See ya. Foreign.

That is Practical AI for this week. Subscribe now if you haven't already. Head to PracticalAI FM for all the ways and join our free Slack team where you can hang out with Daniel, Chris and the entire Changelog community. Sign up today and at PracticalAI FM Community. Thanks again to our partners at Fly IO to our beat freaking residents, Breakmaster Cylinder and to you for listening. We appreciate you spending time with us. That's all for now. We'll talk to you again next time.