Cybersecurity in the GenAI age

Foreign.

Welcome to Practical AI if you work in artificial intelligence, aspire to or are curious how AI related tech is changing the world, this is the show for you. Thank you to our partners at fly IO. Fly transforms containers into micro VMs that run on their hardware in 30 plus regions on six continents so you can launch your app near your users. Learn more@WAIIO.

Well, our friends over at Speakeasy have the complete platform for API developer experience. They can generate SDKs, Terraform providers, API testing docs and more. And they just released a new version of their Python SDK generation that's optimized for anyone building an AI API. Every Python SDK comes with pydantic models for request and response objects and and HTTP X client for async and synchronous method calls and support for server sent events as well. Speakeasy is everything you need to give your Python users an amazing experience integrating with your API. Learn more at speakeasy.com Python again speakeasy.com Python foreign.

Welcome to another episode of Practical AI. This is Daniel Whitenack. I am the founder and CEO at Prediction Guard and I am joined as always by my co host Chris Benson, who is a principal AI research engineer at Lockheed Martin. It's good to record one together, Chris. There have been a couple where we've been apart so.

That's right.

Yeah.

Yeah. It's good to have the band back together I guess.

Absolutely. I took a brief respite. I had some lower back surgery and since I'm always thinking about technology and AI, I was trying to imagine what the surgery might be like today versus if I had done this, if we were in a slight time warp and gone to the future.

Did they have a robot arm?

Exactly. If they did, reinforcement learning combined with kind of a chat interface and you know, the whole thing. So it was kind of a novel, a novel mind experiment there that I had about what would it be if I was, you know, this was a little bit different time, but yep, all, all good for here.

Yeah.

Well I would hope that in our, in all of our futures as we have medical procedures done increasingly in, by, by AI or AI assisted and other things like that, that those things are very secure as, as they happen. That that would be very important, I think.

Yes.

And on that front I'm really excited because Chris, on this show actually a couple times I referenced the OWASP top 10 gen AI top 10 sort of risk white paper breaks down security and privacy related risks into sort of different categories and helps people think about it. This is a collaborative thing with multiple organizations involved. But to today we've got with us Dennis Cruz, who is a founder at the Cyber Boardroom, but also has been involved in OWASP in various capacities over the years and is aware of all of this that's going on and contributing to it. So we're just super excited to have you with us, Dennis. This is one I've been really looking forward to.

Thanks for inviting me. And this is a topic that I'm very interested at the moment because I think there's a lot of potential. There's a lot of dangers too. But I think it's very exciting times for us as an industry in all sorts of levels.

Yeah, well, like I mentioned, you've been involved in OWASP in different capacities. How did the kind of, I guess give us a little bit of background and how that first off, what OWASP is and what that means for those that aren't familiar. But also, as you kind of started seeing this GENAI stuff come about, how did it strike you from that standpoint of being involved in OASP over the years?

Okay, so OWASP is the Open Web Application Security project. And you kind can say that OWASP is the people around the world that kind of cared about application security, that found that sort of intersection of the world, moving to apps versus networks and then the security elements of it. Right. And I think it's definitely one of those organizations has grown from nobody really cared about it to some more people carrying it to now it's been referenced left, right and center right, and has really changed the world in very, very nice ways. And it's always attracted, I think, the people that want to do better. I think it's a lovely community. It's very open by its nature. But it's also, I think it's been a great hotbed of innovation. Right. In terms of the first OAS top 10, the testing guide. A lot of amazing tools come out of it. OAS zap, you know, dependency checker, there's bazillions of tools. I develop a couple too. And it's a great, great community. Right. And I think Genai, it's an interesting evolution on our technology even from a security point of view, because I think for the first time we have a technology that kind of understands intent. Right. And I think that's quite different now, I want to say out of the bag, but I think what Gen AI is doing, ChatGPT and everything is actually making most of what we always talked about in OWASP even more important, right? Because if you think about it, you know, Gen AI is an API, right? It's fundamentally an API that you send some data in, you get some outputs out, and these days even connect internal systems. And in, in a way, it's your most vulnerable and your most dangerous API at the same time. And I would say that the, the most thing that is quite unique with Gen AI is the fact that you're now sending English or Portuguese, right? Or French or whatever language, or Klingon, right? Whatever language you want to talk to that is now code, right? And that's quite different. You know, for the ones who've been doing security for a while, we always talked about securing code and data, right? Like if you can separate code and data, we know what code is, we know what data is, you actually can secure things quite well. Now you're in a world where data is code. So that introduces a huge amount of interesting challenges. And I think it's one of those technologies that in one end he's going to create a lot of problems, but on the other end, I think he has the potential to solve some of the biggest challenges of scale that we always had in security and in engineering. Right. And in development with that technology too. So I think it's going to add up to both sides of the universe there.

It's interesting. I actually want to go back to something you said, right, as you were starting off a second ago, and build on it a little bit that like you kind of alluded to for a long time, you know, over the years, security was always kind of a, you know, the redheaded stepchild of the of application developer. You know, there was a lot of folks that just didn't care too much. And obviously we've progressed along and we're talking about Gen AI and things like that, what we're doing now. But going back, I'm just curious, as we're talking about security in general in App Dev, even before we get to AI being introduced into it, what has motivated people to, to get into the security side, in your view, as you've watched this? Because I too have seen that the security concern go from being kind of a backseat thing to being very front and center and very important in a lot of organizations. And that's been an evolution. And as you've been in the middle of this ecosystem, how have you seen that happen and why and where does that progression come to?

Well, so I would say from personal experience and I think from most of my peers, there is, you know, there's a thing of. It's nice to make the world safer. Right. There's definitely a shack of like it's nice to do something that the outcome is to make people safe. And I've always liked to stay on the sort of defense, you know. Yes. A bit offensive, but on making things systems secure. Right. So I think intellectual is really cool. I think from a technology point is really cool. There is a thing of hacking and breaking into stuff which is also quite, you know, quite interesting. Right. There's a little bit of a James Bond, you know, hacking into the big system and getting there and all that jazz, which I think is part of the evolution, right. So there's, I think it's a really cool industry. It's very open, it's very welcoming, right. It's. It's also very pragmatic, right. It's in the beginning it's like, can you exploit or not, right? Can you do this, can you do that? Right. And I have to say we need to also look, you know, be pragmatic to say for a long time it didn't really matter, right. Like in a way there was a period where you knew who's being attacked because they were the companies who had the best security teams. Literally there was a one to one correlation. That team, that's security very well. Cool. They've been attacked, right. They're on the, on the evolution, right. And I've now been a ciso. I've worked a lot of companies and there's definitely. I tend to break into three different worlds, right. There is the nation state, you know, high level espionage, There is the ideology attacks, right. And there's the commercial attacks. The reality is that this one is a completely different level. If you're dealing with nation state, you know about it, right. You have hell of a teams and we can do it. Right. But it's a very different game. The ideology, it tends to depend on the industry you in, right. Some industries are much more toxic. They tend to attract that. Most of it is to do with the business model of the attackers. So what has happened in the last, you know, 10, 15 years is that the more we move our society into digital, the more the business model of the attackers evolve. So in the past you can get away with a lot of things. You could get away with crazy secure applications out there and the probability of you being attacked was quite low. Right. Now you kind of can't, right? So I think there's been that evolution of in a way, the stakes are much higher right now. You know, even the recent crowdstrike problem, right. You know, which I still think is a security problem. Right. We can talk about if you want, but that just shows how one problem in technologies can bring down, right. Large chunks of society, right? And in the old days, it was the, the name there, right? The ISS worms that brought down a chunk of technology. But at the time, the impact was limited. Now it really makes a difference, right? Like now a cybersecurity problem in organization of a medium size can break the company, can really severe disruption, right. Even a ransomware incident in one of our suppliers can actually cause a lot of problem in your world. So. So I think, you know, the application security and OWASP and everything, we have in a way matured with the kind of evolution of the market. Right. Although I have to say that I still feel that most companies still. It's still a marketing exercise. I'm. It's a controversial view, but I think we still don't have a good way to measure the cybersecurity preparedness of companies. So there's still a lot of companies that kind of roll the dice and go, well, I hope it goes okay, right? I thought much less than before, but I still feel there's a lot of maturity that we need in our own industry. Right. To do that. And just a final point, the thing I also learned a lot was that cybersecurity is not an isolation. Cybersecurity is a side effect of engineering and business practices, right? So there's a moment where if you want to fix cybersecurity problems, you want to see, you know, fix application security problems, you have to fix engineering problems, right? And, and that leads to that, right? If you do really good engineering, if you do really good development, really good practices, half of what we talk about in application security is not needed because you just do it, right? It's just good practices. The problem is a lot of companies, a lot of teams don't do that. So the side effect is the security problems.

I'm wondering when you're talking about this, this sort of business practices, the engineering, where these security vulnerabilities pop up and that sort of thing. Part of what comes to my mind on the Genai side is the fact that it brings the domain experts a lot closer to the actual technology in the sense that often there are tools or there are interfaces that people are using to create sort of chains of reasoning just with prompts and other things, you know, just with natural language. So they're not to your point. I think now this sort of natural language is kind of like the code of programming these models. How do you think that that that shifts the dynamic between, like the engineering side and the business side? Because from my perspective, a lot of these domain experts and business people are coming much closer to the actual kind of core functionality of an application.

And I think that's the insane opportunity, right? So first of all, I don't view that the real power of ChatGPT is that it can create things, right? The real power of ChPT is that it can understand context, it can understand data, it can create mappings and relationships, right? And what is in practice, you know, the way I visualize what you describe is that in the past, everything you want to do had to be coded, right? It had to be solidify in bits of code, which is also where vulnerabilities were created, but also where, in a way, we were locking down the business logic into code, microservices, trucks, et cetera. And then we got bogged down by the complexity of that, right? And then we couldn't understand the side effects. And then things ground to a halt. And then again, security vulnerabilities appear in those gaps, right? It's those gaps. It's those misunderstanding of how APIs work, or an API that was created over here in a secure state that is now plugged to the Internet. You know, maybe not the best move, right? But I think what where we now entering a phase is that you can have a layer where the business logic is now described in prompts, right? And what happens when you describe the prompt? You start to describe the intent of what you want to see happening. And instead of that intent being locked in code, which is really hard to change, it has a huge amount of things, you know, side effects. You can have that in prompts. Now we need to be better understanding the prompts. I'm doing a lot of work on provenance, deterministic AI, to make sure that we actually have AI outputs that are deterministic. But it's very powerful to be able to start to describe that. Right? Even I'll give you an example, like if you go to any organization, like when we do threat modeling. So one of the practices in security is threat modeling, right? Threat modeling is just social engineer the other side to give us an architecture diagram, right? That is up to date. Because once you have that, you'll find vulnerabilities, right? Because you start asking questions, what about this? What about that? What about that? What about that? Right? You know, in a way, the hidden elephant is most organizations have no idea how the App works, right? They have no idea, you know, how actually things connect. The guys who develop it have gone, right, like, you know, now imagine a world where we start to use genai to explain how it works. Imagine, you know, a code commit that goes to a genai layer that I can ask the question, did this change the attack surface, right? Did the attack surface of my organization change because of this? And they say, oh, if you did, then we need to do a review, right? If you don't, cool, go all the way to production, right? Now these questions in the past were impossible to ask or you had ridiculous static analysis that just about didn't work. Now we can start to codify a lot of those business logic questions. A lot of those intents of what I want to do, right? In, you know, in, in basically in simple language instead of code. Because imagine like if you say I want to do a change that allow anybody from the Internet to access every record from this company, right? You probably go, ooh, hold on, I'm not sure that's a good idea, right? Or you know, I want this to allow, you know, an attacker to run and control JavaScript on my clients, right? Ooh, maybe not, right? I want my secrets to be put on the source code, right? Maybe not. But at the moment all those things get locked in the code, right? And it's hard to understand the side effects because nobody arrives one day and says I'm going to create a security vulnerability, right? Unless they're malicious, right? But apart from those, right? It just, a lot of times it's just genuine mistakes, right? So I think what Gen AI gives us is the ability to start thinking a lot more like three dimensional, right? In our questions, but also in our applications where we start to describe the intent of what we want to do and that can be analyzed, in fact that can be analyzed by other gen AIs so we can start to get a much better sense of actually what is happening.

Okay, friends, I'm here in the breaks with Annie Sexton over at Fly and you know, we use Fly here at Changelog. We love Fly. It is such an awesome platform and we love building on it. But for those who don't know much about Fly, what's special about building on Fly?

Fly gives you a lot of flexibility, like a lot of flexibility on multiple fronts and on top of that you get. So I've talked a lot about the networking and that's obviously one thing, but there's various data stores that we partner with that are really easy to use. Actually one of My favorite partners is Tigris. I can't say enough good things about them when it comes to object storage. I never in my life thought I would have so many opinions about object storage, but I do now. Tigris is a partner of Fly, and it's S3 compatible object storage that basically seems like it's a CDN, but is not. It's basically object storage that's globally distributed without needing to actually set up a CDN at all. It's like automatically distributed around the world. And it's also incredibly easy to use and set up. Like creating a bucket is literally one command. So it's partners like that that I think are this sort of extra icing on top of Fly that really makes it sort of the platform that has everything that you. Neat.

So we use Tigress here at Changelog. Are they built on top of Fly? Is this one of those examples of being able to build on Fly?

Yeah. So tigris is built on top of Fly's infrastructure and that's what allows it to be globally distributed. I do have a video on this. But basically the way it works is whenever, like, let's say a user uploads an asset to a particular bucket, well, that gets uploaded directly to the region closest to the user. Whereas with a single cdn, they're sort of like a centralized place where assets need to get copied to and then eventually they get sort of trickled out to all of the different global locations. Whereas with tigris, the moment you upload something, it's available in that region instantly, and then it's eventually cached in all the other regions as well as it's requested. In fact, with tigris, you don't even have to select which regions things are stored in. You just get these regions for free. And then on top of that, it is so much easier to work with. I feel like the way they manage permissions, the way they handle bucket creation, making things public or private is just so much simpler than other solutions. And the good news is that you don't actually need to change your code if you're already using S3, it's S3 compatible. So, like, whatever SDK you're using is probably just fine. And all you gotta do is update the credentials. So it's super easy.

Very cool. Thanks, Annie. So Fly has everything you you need. Over 3 million applications, including ours here at Changelog, multiple applications have launched on fly, boosted by global Anycast, load balancing, zero configuration, private networking, hardware isolation, instant Wireguard, VPN connections, push button deployments that scale to thousands of instances. It's all there for you right now. Deploy your app in five minutes. Go to Fly IO again. Fly IO.

So, Dennis, you said something that was pretty intriguing to me, which I think is maybe a kind of distinction that is maybe interesting to draw out and get your thoughts on, which is this idea that, like, when you think of kind of the intersection of cybersecurity and AI, you could come at it from two perspectives. So you could come at it from, how can we use AI to help us in our cybersecurity tasks or to create new tools for cybersecurity? And then the other side would be, well, how do we operate AI systems in a secure way? So there's probably some interaction between these two things. But could you give us a sense from your perspective as a. As an expert in this field and also seeing a lot of things so far, how do you see the kind of maturity of these two sides of that coin? Anything you'd want to highlight on either side of that in terms of how both. Both things are progressing, at least at the state of where we are now?

So you're saying that, you know, the difference between, you know, using AI to sort of build systems and. And do things and then using one of those outputs is the cybersecurity analysis. Right, of what you have.

Yeah, there's. I could imagine there's ways I could use AI to fight cybercrime, for example, or to prevent malware, or like you just said, to help explain applications. So that's using AI to help you create more secure systems. Whereas there also could be just. Your AI system is insecure in and of itself. Right. And how you've deployed it and run it.

Yeah. And just on the second one, I think if you're not careful, most AI deployments are ridiculously insecure. Right. Like they are. In fact, we have to take into account that we still don't have a good understanding for how the models work. So the reality is, there's nobody today that can tell us that these models don't have ridiculous backdoors in there. They don't even. Not intentional. Right. Even maybe just the way it works. When we started this, people thought that a string copy was okay. People thought that a little catch between a memory copy in the OS was okay, and then realized that you can drive buffer overflows, ridiculous exploits through it. Right. So I think in a nation state at the moment, like in a early days of understanding, everything you can do with a model. So my. My kind of view in this Is that models that you want to use on that you know how to use models secure, should be read only, should not learn. You don't want them to almost bring any content. You want to give you the content, you run them in complete isolation and you assume that whatever you put on it is already exposed and you verify the hell out of what comes out of it. Right? And I think there's a lot of companies who are rushing into pushing models. The problem is that they not take into account that the models themselves are ridiculously powerful. And this is where you want to imagine that somebody can put a payload that is then executed by a model and that model sits now in the middle of your organization, in your cloud, in your environment, who probably has access to APIs or other assets. That is ridiculously dangerous. Right? And that's what we're doing. Right? So I think in one handle, I think we need to be very careful in putting models in line in how we actually validate the inputs and the outputs, which is kind of why I view them in multi tier sort of flows. And on the other hand, when we use them in a safe way, they're ridiculously powerful. Because going to your first point of how to use them for cybersecurity, what I really like is that I always felt that the way the model for cybersecurity is a model based on the attacker making a mistake. It's not about you protecting everything, it's about you almost. You want the attacker to make a mistake. I make a call that was not supposed to happen, make a download, make a connector connection, access the application in ways that no user will access it, call web services that completely out of sequence. In the past, again, it was impossible to model this. We tried squeezing technologies. Even people that we tried created ridiculous, you know, installations of sims and technology, et cetera, they really struggle that. But I think we now have a good chance of doing that. So that means that we can now create much more, I would say hostile environments for attackers because we force them to follow the paths of the users, which by the way, they don't know right what those paths are unless they're ready in your system. Right. So I think we have a chance of using that. But what we need is we need models that are really, really reliable. So Owas has an amazing top 10 for applications, has a really good top 10 for gen AI models. Right. Et cetera, LLMs. What I think about is most of that is trying to deal with the fact that the models can learn and the models can actually be don't have deterministic outputs. And I like the idea of actually turning the tables around and say, hey, I don't want my model to learn. I want the data that my model has access to be completely determined by the session and the state that that request comes in, which is normal appsec, right? And ideally I don't even want the model to have knowledge, right? I want to give the model the knowledge that is going to use. So I control hallucinations, right? Like you know, Chris, you know, you talk about your operation, right? Like you don't want the gen doing the operation on your back to suddenly go off piste, right and start doing an operation on your leg, right? So in, in this theory would be that you want for example the gen AI model that is facilitating your operation to only know about back stuff or maybe to know general things about the body, understand that? But the domain knowledge that it has should be laser sharp focus to the situation that you in and then that's how you control hallucinations, right? So I think the fundamental problem that we actually went backwards in security is that we now don't have a separation between code and data, right? And that's. I don't think we speak enough about this because for me that's a massive problem, right? It's a massive problem because we really need to be able to distinguish what is code, what is data, what's an input, what's the command. So I'm doing a lot of stuff where I go from JSON to JSON and the latest model do this better where I almost want an API coming in. I give that API, which I can form nicely with data validation and stuff to the model and then the model output itself is an API that is completely strongly typed. So I understand the output, if that makes sense.

It does, it does. And just on the side I'll just say I might actually need that operation on my leg too. But I would prefer it was a separate operation that we planned out just to note it. It's funny, as you were, as you were taking us through that, I have a whole bunch of different pages up here relating to things we're talking about, including on the OAuth site that top 10 for LLMs and generative AI apps. And ironically you were going through that, I was like, wow, you know, they already have this amazing list which kind of addresses these things you were talking about and then you referenced it explicitly. I was wondering, could you kind of take us through like how was that generated? What, what's the thinking? Because it looks Based on everything you were saying, it looks like almost a roadmap of the things that one needs to be thinking about when going through the process. Could you take us through that a little bit?

Well, I think you just nailed it, right. I feel that what you have there is, you know, the team who did it, and I wasn't very involved in it. I was a little bit on. On the outskirts of that project because I thought they were doing amazing work, right. Is that they had a used consultation period, right? They talked to a lot of people. They basically they listed a lot of the stuff that goes wrong. What I think is interesting about that is I think that whole list has a bias for the teams are kind of deploying their own solutions, Right. And it kind of covers a lot of those things. Right. I kind of feel that a lot of that needs to be addressed by the people that provide the models. Right. And I think more and more, I almost. You know, it gets to the point where you don't want to build your own cryptography, right? You want to use cryptography models that are very robust. It gets to a point where somebody's like, you shouldn't be building your own model. So we're like, look, unless you have a hell of a team and you really know what you're doing on that area, right. Most organizations, I don't think, should be building models, right? Because the big paradigm shift for me was when the prompts is where the action is, right? And even if you look at things like Claude and the recent now that people are sharing the prompts for those things, you see how much the prompt is actually impacting, right? The stuff. So go back to the top 10. I think it's a great roadmap for people who are deploying their models to go, do I have to care about this? Does this apply to me? Like, how do I answer this in an effective way? Right? Because I think that's very important.

You know, when we started Podcasting back in 2009, an online store is just the furthest thing from our minds. Now we have merch.changelog.com and you can go there right now and order some T shirts. And that's all powered by Shopify. What do we do before Shopify? I'll tell you, we did nothing we couldn't sell. There were other ways, of course, but they were very hard, very difficult. Shopify let us build out an entire front end. Obviously branded, like Changelog is. It's amazing merch. Changelog.com and our favorite feature is we use their API to generate a new coupon code, a personalized coupon code for every guest that comes on our podcast. And they get a free T shirt from our merch store. And that's so cool. They choose the shirt they want, they use the coupon code, it arrives free of charge to them, and life is amazing. But also you can go there right now to merch.changelog.com and buy some threads yourself. And that's awesome as well. So upgrade your business and get the same checkout we use with Shopify. Sign up for your $1 per month trial period at shopify.compracticalai all lowercase go to shopify.compracticalai to upgrade. You're selling today again, shopify.compracticalai.

Well, Dennis, I, I'm really fascinated by this concept that you brought up about separating the model and the data. You phrased that in various ways. It sounds like you've been thinking about this concept a lot. I'm wondering if you could bring that to a practical level, maybe for those out there that are kind of wondering, maybe in both cases. So I'm using a closed model provider like OpenAI or Anthropic or something like that. There's that scenario. There's also people that are hosting their own model or even running it locally on their laptop with a, a local model server. From your perspective, what are the interactions between kind of model and data or like, as you put it, knowledge and model that are relevant in those scenarios to create either goodness or badness in each of those scenarios.

So I think the first very important thing that is very relevant today that wasn't, I would say, six months ago is that we need to move from this idea that you have one module, right? What you have now is you have a, an ecosystem where you have multiple models, right? And they will go from probably some of the most, you know, commercial if that feature model that you want to use to the open source one, but also from the most powerful to the least powerful, right? Because what you want is this mode where you start with I want to do X, right? And with X you want to start figuring out what is the best model and sometimes what is the best combination of models that will give me that output, right? Because in a weird way, the best deterministic way to do something is code or to have the least amount of moving parts in there. So. And because also remember, there's a cost issue here, right? So the more you use the models, you want a situation where you're firing these model analysis all the time. Now if every one of those is hitting an OpenAI endpoint, that will get very expensive very fast. But it's not just that, right? Sometimes model, you don't want that whole package, you don't need all of that. If you just want a summary or you want a validation, you want this. There's now a lot more models and there's models who are specializing in specific things, who have certain buyers that you want to have those buyers, right? In terms of that. And so I think it's important to start thinking not just of one model, but the sequence of models, but also what is the best model that you have. And the open source models, the reason why they're a game changer, right, is because suddenly you can now run models, let's say with ollama, on a desktop cpu with distance speed, right? And distance speed might not be like, you know, the real time. Now we now get to ChatGPT and Cloud 3, et cetera. But maybe even how ChatGPT was a year ago or two years ago. But what it means, it means that if that's on your pipeline, you now have a pipeline that is run off CPU, right? You don't even need GPUs now you can if you can, of course, if you have them and you can afford them, it fits your model. But it's CPU level that can run a model that is complet completely isolated from the Internet. And I think it's very important. I think it's very important you have a design that has those workflows because you start to introduce them as part of your workflows. In a way, the key answer to your question is people need to pick up a use case, right? It doesn't need to be ridiculous, complex. But pick a use case and then try to do that with a gen AI workflow, right? And that workflow, where in the past you had to code, now is a workflow that has multiple LLMs, it has multiple sequence. I have things where sometimes you create same question to three models, then you use a fourth one to analyze it. And on that pipeline, and what I'm trying to do with the cyber boardroom is fundamentally try to address in cybersecurity how to communicate, how to translate cybersecurity knowledge to board members or executives, but also how to get those executives to ask good questions and to translate what they care about. So a really cool use of technology is to think about translation, right? So for example, if in the past as a ciso, I produced reports and briefings for a lot of people, But I didn't customize them because it didn't scale. Right now I have the ability to create a customized version for Daniel, right. And take into account your culture, your language, your context, your level of interest. Do you care? Don't you care? What's your focus? You know, and I have another version for Chris, right? So maybe Daniel, you're a lot more focused on the financial element. Chris might be more focused on the strategic element of it. So I now have the ability to translate a bit of knowledge into very specific domains of one, right? Because I can feed the knowledge, I can feed what the background information, I can feed the audience. And then I can say ChatGPT or Claude or Llama or, you know, Gemini, translate this, right? And they are really good at that. And they don't tend to hallucinate at that level because you, you create the parameters. So that's a good example of a use case, right? Which is very laser sharp, but adds a lot of value because, you know, imagine these are not just execs. Imagine you have the project manager and you have a program manager and you have the lead developer and you have the QA and you have the marketing person and you have the executive. You can now create briefs for every single one of them. That puts into context why they care. Why is this important? Why this cybersecurity stuff means that when the marketing team does a campaign, the website doesn't block your users because you just have 50% more traffic. This is a real story, by the way. It's like we were attacked by our marketing department and run a primetime TV ad. Great. But if you knew about it, we could have planned. So there's all these lack of communication and I think it's really interesting to do in organizations. In the past, this didn't scale, right? So if you now take into account that you now have multiple models with different level of capabilities, you almost want to think, what is the most cost effective? What's the most deterministic way for me to chain this? Where, you know, you maybe use some cheaper models to do some stuff and then maybe use the last more expensive model to actually do some kind of uber analysis and make sure that you still, it all makes sense. Right? And of course this should all be calibrated by the humans who start to calibrate the inputs and the outputs that go into the system. But that's why I feel that it's very, very exciting now that we having these super competitive race between the different models because we now have a huge amount of models to choose from. And you can now start to pick which is better for each capability, right? And that's why I want to see models that have no content, right? I want to see models that just have understanding and logic, right? It's almost like we need to find a way to strip away some of the content so that I can say, this is my policy, right? This is what I care about. This is my world. Because remember now, we now have big context windows, right? So you can now feed quite a lot of data in a prompt that goes into the model.

It's really fascinating with what you're doing at the cyber boardroom in terms of kind of optimizing using the right models in the right way. And then I actually want to reach back a little bit to something you said a few minutes ago where you were saying, you know, from a security concern, you wouldn't want most organizations to, for instance, build their own models from the ground up, you know, use these foundational models for which you have strong security basis in, you can trust, and then optimize in the way that you were just discussing. I think you've really hit on something because I think a lot of organizations are really struggling with how to approach, approach the different workflows to maximize their productivity and that output while staying secure and not exposing themselves. So you seem to have a really good grasp of this workflow that maximizes the productivity and the efficiency for the different audiences while not getting them into trouble by doing something they're probably not well suited to do. Is that fair? Would you say that that's a fair way of.

Yeah, it's nice compliment, thank you. But I. That's what I'm trying to do, right? I kind of call it deterministic gen AI and people go, well, but you know, it's not supposed to be deterministic. Like, I'm going, yeah, but that's the problem, right? Like, there's a cool side of it for creativity. Great, we already have that, right? What I think is interesting is to leverage that ability to understand context and to write in English or writing Portuguese or write whatever language you want and to do that translation layer, but to be very deterministic, which also means that we need to be much better at provenance, which is basically that path of discrete stack, which this. But also it's a way to scale, right? Because if you start to have provenance on good sources of information, then you don't have to do this all the time, right? You. You kind of, you can build your knowledge base, right? You can Build your workflow base and you can build your confidence. And then it's about creating these microservices that do one thing really well. If you think about it like you don't want a microservice that change behavior next week when a new model comes along, right? Like, dude, like it's like we want deterministic stuff, right? Because we build things on top of that, right? So we need to start getting to these building blocks again, components, right, that they do one thing, they do it really well. They are super reliable. Yes, there might be a model in the middle, but that means that the thing has this size versus that size, right? In terms of the capabilities. And more importantly, I think Daniel, you mentioned this before, is that this allows us to go to the business owners and let them be in a driving seat. Because think about it, in the past we had gherking. If you guys know what that is, when you write stuff in quasi language, if when I do this, then I do that, or given this, given that. But that was always a hack because it was like fucking hard coded to the back end. Now we can actually have the business describe the intent, they can describe the workflows and they can describe what the experience they want for the user, for the data, even data transformations. We can start to describe what I want to get from here to there, right? And then it's about how can you lock it down in the most reliable piece of code and system, right? That can do that. Which is why having models that run offline are very important because that allows you to lock in version control, that thing and then know that you will still go in two months, two years, five years, you will still do the same thing. And I think that's very important. So from a security point of view, what I've learned was every time you have a system that behaves like a black box, you have vulnerabilities, right? Like it's literally, you know, so we creating the uber black box, right, for this stuff. So what can it go wrong, right? So in the past I knew that the less the team tested, the less the team had architecture diagrams, the least they understood how a part of the application work. The more vulnerabilities I was going to discover because they couldn't test it. It's almost like how can the developer understand the side effects of what they're doing if they don't see it, right? So I think we have to be very careful by creating these black boxes, right? That we don't understand the behavior and how it works.

Something that is maybe brought into a little bit more clarity for me as you were just describing what you just described with the black boxes. We had a few episodes ago, Chris, you remember we had the episode I chatted with Donato when I was in London at with Secure. And one of the things that he brought up, I'm curious at your perspective on Dennis, is like, these very, very large models, especially the closed ones, have a huge attack surface. Like the vulnerabilities, like, they're so general, there's so much knowledge kind of embedded that it would be sort of impossible to think that you could kind of fully explore the space of behavior and prevent things like jailbreaks or things like prompt injections, that sort of thing. Whereas sort of the smaller model you bring the data to is either going to perform really good if you bring the right data to the table because it's not embedded in the model, or it's going to be complete trash output, which maybe is better because you're operating in a regime where the data distribution isn't what you. I'm curious if that tracks, right?

You can ask, you can verify. Yeah, I think that's spot on, Right. Look, it tells you something that we still. It's almost like somebody had an equal analogy. Like, it feels like we back in the navigation. Like, Portuguese has a great history of discovering the world, right? So we at school, we learned how the Portuguese, right? And the Spanish and everybody else were like, what's out there? Let's stand the boat, right? And then, hey, look, we discovered a country or we discovered, you know, something, right? It feels a lot of these models are like that, which in a weird way is ridiculously scary. Like if.

Imagine, like there be dragons, you know?

Yeah. Imagine you wrote an app, you do a web service, you ship and go, hey, do you know that thing knows chemistry? It's like, well, you know, that thing speaks 20 languages. Well, so I think we. We is the fact that we still have. We talk about emergent properties and the fact that people still talk about what the. What the models do by probing it, right? Like from the outside as a ridiculous black box that shows you how immature we still are in that level, right? Like, yes, software is complex, but we can actually understand kind of what it does, right? Like, we can actually, you know, okay, given the time and money, we could actually reverse engineer even the most complex piece of software and going, yes, this thing is not going to know chemistry, right? This thing is not going to do beyond this. It might do some bugs, but it has a limited operation. Space, Right. I think the models, in a way, it's a good thing in some aspects, but they have all these properties because they build these really big three dimensional or multiple dimensional views of it. But that's not what you want for mission critical systems. And that's what I was talking about, exploits. My prediction is there's going to be a number of exploits, back doors and seekers of instructions, my crazy ASCII characters, crazy X amount of characters, whatever, math, numbers, whatever, that will trigger the models to go into a place that they do crazy stuff. But we don't know that because we don't understand like the models, right? Like people hack a model almost in English. Oh, pretend that you now write in ASCII art and you do this. Like that is, if you think about it from an exploit point of view, that is very basic, right? Like this, this is going to be way more ridiculous, complex, but interesting. Well, I guess from a stereo point of view type of exploits that people will have once you start to understand how it works inside, right? Which is why I think we also need to start measuring almost like what, what is the behavior of models, right? Like how do they arrive at those conclusions and then even have models that can only do those bits. Does that make sense? Right. So, so again, I want deterministic models. I want models that I can start to vouch for how they work and how they do, even if maybe sometimes they're a little bit less efficient, but you earn the explainability.

So I want you to. Actually, as we're starting to wind up here, I would like to ask you to even extend that a little bit in terms of kind of where you think that's going to go, because you've already touched on trying to get to more deterministic models and some of the things you're expecting, if you were to blow that out in a slightly longer time horizon, maybe several years, where do you think all this is going to go and how do you think it might get there? Speculatively, like recognizing that we're just doing the crystal ball and asking you to tell us when you go to bed at night and you're thinking about this, what do you think is going to happen?

Okay, there's multiple areas, right. I think, I feel on the whole fake news, creation of crazy content, using AI for the attack, that's going to grow, right. There's business models around it. I actually think that's going to force us to have deterministic and providence on news. So I think that's a good thing, right? It's going to be bad, but it's going to force us to address that problem, right? There is a level here that we're going to have to control it just like we control nuclear weapons, right? Other things. I think there's a level here that we have to be careful, right, not to create things that go completely out of control. And we might have a couple, but that's, that's a bigger problem, right, that needs to be addressed. Where I think where we're going to have the biggest impact is like, I think Kevin Kelly had a great phrase in one of his books. He talks about AI even before ChatGPT, but he Talk about AI will become like electricity, right? You'll become embedded in all these little things. But it's not a massive thing, it's little bits, right? It's little things that you suddenly start to have that introduce a level of intelligence that we don't have today, right? So if you think of most of our interactions, they don't have a lot of intelligence, right? They don't learn. They don't, you know, but. And I think as again, the models become small to run in your phones. As the models become small to running microservices, I think what would be very powerful is the creation of these little lots and lots and lots of little use cases that really make a difference and then you start to trust it and then you compound them on each other. And my instinct is that anything that relies on a black box eventually will blow up. And when it blows up, people push back and going, whoa, whoa, whoa, we can't have that. It's okay for proof of concepts, but that cannot be doing stuff also, because technology now, I think had got to a point where we would drown in complexity, right? We would drown in all sorts of things that, you know, we don't fully understand. They don't connect the dots. And even application systems, they're so complex and companies are so complex, right? I think understanding them in the smallest way, that will dramatically change. That's how you change society, right? You change society by literally introducing something that becomes really powerful, really useful. And the final point I want to make here is I think there's a ridiculous opportunity for reframing education, right? And to finally create a learning environment that individuals can be learned in the best way for them, right? So I think we can change away how we learn continuous learning, it's a big thing, but also how we change the education from being like a memory kind of exam based stuff, right? To actually be about learning, right? And now it's about creating customized learning paths to the individuals. Right. And but also I guess just the final point is that this means that the human is literally the one that is ridiculous important here. This is a tool to help, you know, the human to be even more productive. The same way that we use the Internet, the same way that we use the hammer, the same way we use electricity is just a different one. Right. That we had before because you can understand language and we never had that before. Right. So I think it's an insane opportunity, you know. But the attack side, yes, you will go. But we didn't need, we didn't need genai for people to create ridiculous attacks. Right. But on the defense side, I think it changed the nature of the game. Right. So I think it's very exciting of that. I hope that answered your question.

It was great answer.

It's awesome. Yeah. Thank you so much for taking time, Dennis. This has been great. I really have been looking forward to, forward to this and love the conversation. Love this idea of kind of thinking about knowledge and data and model and how those are connected or not connected. Please continue your great work. It's, it's a great contribution. And yeah, thank you so much for taking time. It's been a pleasure.

My pleasure. Great talking to you guys.

Thank you for listening to practical AI. You know what's cool? Free stickers. During the month of September, we are mailing out Changelog sticker packs to everyone who leaves us a thoughtful five star review or blog post about our pods. Simply email proof of your review to stickerschangelog.com alongside your address and we'll mail out the goods anywhere in the world. Once again, that's stickerschangelog.com pics or didn't happen only in the month of September. Let's do this. Thanks again to our partners at Fly IO, to our beat freak in residence, the one and only Brakemaster Cylinder, and to our longtime sponsors at Sentry. Use code Changelog when signing up for a new center team plan and save 100 bucks. That's all for now. We'll talk to you again next time.