Reality of Cybersecurity Risks for AI

Welcome to the Cybersecurity Vault. I'm your host, Matthew Rosenquist, CISO, industry cybersecurity strategist and advisor. And today we're gonna talk about the reality of cybersecurity risks for AI. And I get the pleasure to be talking with Iona Pretschi. Iona is the principal manager for cybersecurity risk and free now, and the president and founder of Women in Cybersecurity Germany, and the host of Cyber Star Talk podcast. Iona was also named in the 40 Under 40 in cybersecurity for 2024. Welcome, Iona, and thank you.

Thank you for inviting me to your show, Matthew. It's a real pleasure to be here.

Oh, we're going to have a lot of fun. AI is just the talk of the town. Everybody just can't get enough of AI, and there's lots of interesting aspects when it comes to cybersecurity.

Definitely. And two podcasters in this conversation, it's going to be fun. Definitely.

Yes. Having fun is mandatory. Definitely mandatory.

And I hope I will not forget until the end of this interview that I'm not the podcast host here.

Hey, if you want to close it out, by all means. Yes, you can. Have fun. We can switch roles. It's beautiful.

Why not?

Okay, so AI, right? It is the talk of the town and specifically GenAI, right? Generative AI has become just a whirlwind of activity, of benefits and potential risks. So how is this huge monster of AI, you know, how is that a risk to cybersecurity in your mind?

So first of all, AI is not new, as you pointed out earlier. It started actually in 1950s, which was like, it was fueled by computer advancement and pioneers like Alan Turing and you know. What is new though is actually the sophisticated generative AI, which is built based on a huge amount of data. And we have to accept that by far, this is the most consequential technology that is transforming and sort of reshaping the world as we know it. What is fascinating about generative AI is the ability to augment human capacity. It is becoming increasingly important across a variety of roles and industries and offering both opportunities for growth as well as potential misuse. And no clearer example can be found than in cybersecurity field where This technology is actually redefining the arms race between security enhancements and the emerging threats. But every technology developed, every technological development actually comes with advantages and challenges. So basically, the intention of a new technology in most of the cases, because I cannot say 100%, is for the good of the society. However, what people do with it, it can never be guaranteed. I remember of something that I read like a couple of months ago. Thomas Edison invented the phonograph to help people with visual impairments, and he was horrified when he noticed that most of the people just used it to play music.

Unintended consequences. In this case, a good, a good consequence.

Absolutely. And so cybercriminals are now using generative AI for creating new attack vectors. Some examples: convincing phishing emails. We don't see poorly written emails full of grammar errors anymore. They are now over. Attackers are using generative AI tools such as ChatGPT or Google Gemini or any other tool that is AI-powered to craft emails with high quality, almost indistinguishable from legitimate ones.

And they can do it in any language they choose. All right. Absolutely. And still maintain that level of authenticity and compellingness to get their victims to do what they want, right?

Yeah, that's really powerful because now these generative AI tools are supporting many languages. So obviously they don't need to use translators anymore. And you know, with translators we have other problems because they don't keep the context of translation, obviously. And a native speaker can directly spot it. So another big thing that is happening is deepfakes. Nowadays, Just a 3-second voice sample can train a generative AI model to sound like anyone, literally anyone. And just a couple of days ago, I read the news about the— you probably heard of it as well— the UK Foreign Secretary David Cameron fell victim of a hoax video call with someone who was claiming to be Petro Poroshenko, the former president of Ukraine. And just after the call, he became suspicious and realized that he had been duped. But what I really like about this story is that the UK Foreign Office went public with this incident. And that was a great example to raise public awareness about this topic.

Well, it could happen to anyone, right? And the fact that it's possible to do this, coupled with the fact that these tools are openly available, they're free to use in many cases, and you can download open source and create an engine yourself in, you know, at home.

Exactly.

It really extends these tools out to anybody that potentially wants to use them, both for good or maliciously. So it's not just up for nation states, one attacking another. It can be for any kind of cybercriminal or data harvester out there that wants to use them.

Absolutely. And it was another example earlier this year. It was a huge deepfake attack in Hong Kong, and you surely heard about this. A finance employee of a multinational firm was tricked into paying $25 million. To a hacker who was using a deepfake technology pretending to be the CEO of the company. So beware of your digital twin. It can happen to anyone at any time. Another—

And these GenAI systems, right? They're now multimodal. So it isn't just text, it's sounding like the CEO in an email. It's also voice. It's also now video. And so we're seeing some of these attacks when you're on a Zoom call or some other video call. That multimodal capability can be pretty convincing when combined together.

Absolutely. And especially if you combine it with phishing, then you see that it's a huge harm. Another topic regarding how the AI is being misused is the code generation for malware. So there are many attackers who are using now generative AI tools to generate malwares. So AI has basically lowered the entry barriers for non-tech people to get involved in the field. You don't need to be a programmer anymore to write a piece of code for malware generation. And there are some, sophisticated types of malwares that are being empowered nowadays by AI, and those are known as polymorphic malwares. They are very dangerous. Such malwares can modify its behavior in order to evade detection by traditional security measures. We're talking a lot about data when it comes to AI, right? Because AI is basically— operates basically on data. So another attack that is happening quite often is the data poisoning. Mm-hmm. Cybercriminals can use various tactics such as backdoor tampering or flooding or API targeting to introduce errors or false information into AI models. And as AI models rely on large datasets to to learn patterns and make decisions. By introducing this manipulated or malicious data into these training datasets, attackers can basically influence the model's learning process, or, you know, once the model is trained based on poisoned data, it can then exhibit unexpected or incorrect behavior.

Hallucinations even. Right? If you can help spoil that foundation of its perception of the world or a topic, yeah, you can get some very incorrect data. But you bring up some interesting points, right? You're saying AI can be used to create malicious code, but on the other hand, it can also be used to create very good code. We're seeing programmers use it to develop programs with high fidelity very, very quickly. In fact, you know, I'm talking with certain industries and they're saying, yeah, programming, a lot of it in the future is actually going to be done by AI because it's faster, it's better, it can self-check, it can do all these great things.

And they have the security by design in mind, right?

Yes, yes. So it's a powerful tool. Can be used for these wonderful, great things, in this case, creating or developing code and programs. But that same exact tool can be used in malicious hands to create malicious software, things that attack and undermine. And, you know, we see that as well for code checking. You can use generative AI tools to check to see if there's any vulnerabilities in the code that, let's say, you have. But on the other hand, the malicious actors can use those same exact GenAI tools to see if there are vulnerabilities for them to exploit.

Absolutely.

Using it good to patch, but also using it bad to exploit. So, you know, AI is just not cool.

I would also highlight this, this point, the automated vulnerability discovery. So attackers are using AI quite a lot to identify the vulnerabilities in codes. So they can scan vulnerabilities, launch attacks, and adopt strategies in real time. It can help them identify zero-day vulnerabilities by analyzing these large code bases and network traffic patterns and exploit these weaknesses before they are patched. And what is the normal timeline for patching a vulnerability? It's like between 20 to 45 days. And imagine now if someone is using a powerful tool to identify these vulnerabilities within a day. Like—

And using that tool to generate an exploit, right? To automatically, so again, just pounds and shrinks that window. And you'd mentioned, you know, and we're talking a lot about technology, right? Code. These same GenAI tools can be used in phishing and social engineering attacks with that same kind of learning process, right? It can send out very customized phishing or social engineering attacks and learn from what succeeds or what fails. So it isn't just the technical vulnerabilities that it can learn and iterate on. It can learn and get better even with the social engineering attacks, attacks against the wetware, people. Right? So, there's concerns there too, wouldn't you say?

When there is data, there is something to learn. Absolutely.

Yep.

And yeah, then we come to a very important topic, which are, which is biometrics. So, AI is able to mimic behavioral patterns to bypass biometric security systems. So, hackers are trying to basically train AI systems on large datasets of biometric data. And by analyzing these datasets, AI can learn the unique patterns that biometric systems use to authenticate individuals and then generate synthetic biometric data that closely resemble the patterns of a legitimate user. And these techniques are known as generative adversarial networks, can really create highly realistic samples of voices, of facial features, or other biometric traits. And these synthetic samples can then be used to spoof biometric security systems everywhere. I think this is one of the most dangerous usage of AI for bad.

Oh, absolutely. And, and And if we take it even a step further, right, we've got AI that's creating entire synthetic personalities, right? We see them on social media and—

Oh yeah.

Yeah. I think there's gonna be a day where these systems get so in-depth, they're also creating synthetic biometrics and everything else. That becomes another problem, especially for social engineering or fraud, things of that sort. So, it's an interesting rat hole that we start to go down. But, you know, we've talked about there can be a lot of good, which is why GenAI is being adopted. Tremendous good benefits, you know, efficiencies, new capabilities. So, the world is rapidly moving and embracing it. But we've also, you know, you've highlighted a whole bunch of potential harms that these same tools can be used in maliciousness. And we see those, right? We see those in the news and it's a little scary. Where do you see the hype, right? If there's a GenAI attack against somebody, it's newsworthy, right? And we see it in the news. Is that creating this hype that it's worse than it is, or is it really that bad? I mean, what is that ratio of hype to reality that you see right now?

There's a lot to unpack here. It actually depends on the lens through which you look at it and the targets that we have set as a society. So in cybersecurity, I would say AI is a now a tangible reality. So organizations are already using AI for incident management, for threat intelligence, for code review, for customized information security awareness trainings, adaptive reasoning and protection. And recently also I've seen some quite good usage of AI in real-time risk management and auditing and third-party risk management. Which takes a lot of resources from organizations. So there are a lot of, there is a lot of use, good use of AI in cybersecurity. I don't think in this regard it is hyped. It's just reflecting the reality. But if we envision AI manifested in human-like robots being our absolute best personal assistant that reads up our mind, or our emotions just by looking into our eyes and then go to supermarket to buy us a chocolate because we are having a bad day, we are not there yet.

Okay. So we don't have to worry about that general AI, right? The Terminator scenario just yet. But, you know, I mean, we see companies like, you know, Google and Microsoft and so forth, they're using those technologies to mine a lot of that data that they have, right? That unstructured data, what we search and what we click on and what we like. Are there privacy or ethical risks with such powerful analytical tools potentially mining that data and undermining our, again, our privacy, you know, expectations Do you think there's ethical issues that we have to contend with?

There's a lot of data and privacy risk related to AI, definitely. But the question is, have organizations already started doing this risk assessment or are they conducting an effective risk assessment on AI? I don't think so because it's still in the infancy phase. So most of the AI tools are still in the experimental phase. And I, I don't think we are there yet. So, but what, what I wanted to point out is that this AI hype cycle is nothing new. If we look back at history, every new technology has generated inflated expectations and overly optimistic timelines. There always has been a tendency to overstate both positive and negative impacts. Mm-hmm. So the core appeal of AI, of generative AI, is the potential to automate and augment a wide range of human tasks. Generative AI is definitely a transformational moment for us. Everyone gets that. But is it really going to destroy democracy, create mass unemployment, or level the entire industries overnight? I don't think so. The potential to go dystopian is always there, and we need to take care as a society that this is not happening. But what I found interesting is a recent research from McKinsey, which suggests that generative AI could add between $2.6 to $4.4 trillion annually to the global economy. And that's quite promising.

Very much so. And you you're talking about something that's very transformative, right? When we look at electricity or we look at the internet or we look at, you know, even cloud computing, things like that.

Critical infrastructure, yeah.

Yeah, I mean, it's disruptive, absolutely disruptive. But at the end of the day, we can't even imagine our world without those kinds of capabilities in place, right? So, but you know, the invention of the car or the automobile, tremendously helped logistics, transportation, communication. It helped shrink the world, but it also created, you know, smog, or, or contributed to smog and auto accidents. And, uh, you know, we can— if we only look at that lens, right? You said we look at it through different lenses and only look at the, the number of fatalities that automobiles create around the world, we might think, hey, this is horrible. If we look at the economic impacts and the social impacts, we may say this is vital just to society. Where do you think AI fits kind of in that scale? Do you think it will be more, much more beneficial, or we're gonna realize that this has just too great of harm and we need to pull back on it? Where do we think?

I definitely think that it's gonna be beneficial for the society. But we have to be careful and we have to manage the risk so that it doesn't go uncontrolled. The reality check tells that 80% of AI projects are failing according to the Harvard Business School. Hmm. And Google and Amazon have reportedly been damping down expectations about the AI impact on revenues. And I remember in the recent earnings call from Salesforce, they said that the AI projects that they are managing won't make a material contribution in the financial statements of 2025. Wow. So all in all—

Fascinating. Really? All the way out to 2025. Okay.

So all in all, this macro effects of AI have not been felt so far. As its adoption remains largely experimental, I would say. However, on the other hand, AI is perceived as a self-sustainable technology as it can generate savings by reducing repetitive tasks or even full-time employment. As you pointed out earlier, we don't need that much, like a high number of employees anymore, like especially when it comes to content management or developers. Because we are— companies are now using AI tools to assist with those tasks. So I think this needs to be reassessed on a periodic basis to do a reality check and see whether our expectations are actually met or we need to readjust that. Another interesting point to discuss is that AI initiatives are actually stealing budget from other sectors.

It's all about competition. Budgets are finite. It's a zero-sum game. Everybody fights for it. And maybe the most sexy tool gets to be the favorite. And it seems like AI really is that favored tool to throw money in.

So 67% of global 2,000 companies stated that they are not using new dollars for AI evaluation. They are actually taking it from other areas wherever they can. And only 33% are actually introducing new money to fund generative AI. So I mean, these are just some facts so that we can compare whether those promises are really being met or not. Companies are usually getting whipped up in the excitement and the buzz of generative AI at rates that we have never seen before around any technology.

It has to be in every marketing deck, every security or productivity or IT tool tech. You have to have AI in the marketing deck. It's, it's, it's shame from us.

So another contributing factor to AI hype is that we as humans, are mainly driven by immediate rewards. We want things to meet our expectations now.

True.

AI is not a spontaneous phenomenon, and it was not born with ChatGPT. The latest AI breakthroughs are built upon decades of relentless progress in fields of data science and machine learning. So to cut through the hype, it is important to have a clear understanding of the previous, but also the current state and also the near-term trajectory of AI. Otherwise we will be biased and we will have the wrong expectations.

It's a great point, right? I mean, AI has been around, you mentioned Alan Turing and you know, it's been around for half a century, right? Over 50 years it's been around. And, but this latest incarnation, which is, You know, if you look at the umbrella of AI, you've got machine learning, which is one piece of that. And one piece of that is deep learning. And then from deep learning, we're starting to see things like ChatGPT, LLMs, large language models, and so forth, multimodal models come out. But that's just one sliver of a world that has been around for a while. And other areas are also being explored. So this is not the end-all be-all. Of artificial intelligence. It's just the piece we're seeing now. I think more will emerge just like the, you know, an iceberg. We're only seeing the tip.

Spot on.

There'll be a lot more coming. So you had mentioned that we need to proactively manage the risks. We wanna embrace all the benefits, but we need to manage the risks as well. So, you know, what are the top areas that you think from a cybersecurity perspective that we need better understanding so that we can manage those risks, so we can develop effective controls?

That's a great question. Absolutely. It's a very fundamental, So we have to understand the architecture and data flow to implement security by design and security by default in AI development.

Okay.

So I think this is a crucial component. So we need to understand all components of the AI system, including data sources, data storage, processing units, any communication channel interfaces between those systems and map out how this data moves throughout the system from ingestion to processing to storage and to the output eventually.

So transparency of the architecture to understand all the components and even where it's fitting into existing systems, right?

Exactly.

I can imagine if budget is coming from certain departments, they want to integrate these tools into whatever they currently have, right, for the benefit. They're not giving up budget for no reason. They're expecting a benefit. So understanding that architecture has got to be the first step, right? What data, what access, what controls, all these kinds of things. Okay, so that's number 1. What would another one be? I'm looking for your top 3 here, right?

I'm saying again—

Transparent architecture. Okay.

I want to add another interesting component there, which is basically identify which data is sensitive in this ecosystem. So we need to take care of PI and PII data. We need to take care of confidential or secret information, even internal information, of course, depending on the classification structure that every organization has. And basically label this data if possible or if feasible, because I'm not a fan of labeling data whatever it takes. I need to—

That's a tough one. Even before AI, we've had problems in identifying sensitive data, labeling it, tracking it, managing it. That's a whole different discussion. Do you think GenAI will help in identifying sensitive data and automatically labeling it? Do you think that's a fair use of GenAI?

I think it can help, definitely. The question is how good, like what is the quality of the instructions we are giving to the language model? So it will eventually depend on the prompt that we are giving to the language models. How to identify this data? What are the requirements? What is considered a secret data? What is considered like confidential or internal information? We need to give these guidelines to the AI model in order to train it. And eventually, I think there will be more and more tools that will automatically identify and label data, but we need to take care of the quality check of this label because mislabeling can lead to a catastrophic impact for the company.

Very much so. It does seem like, right, we talked about some of the attributes of what AI does well. It can do repetitive data, it can do analysis very well, it can learn, right, and optimize. These are all problems that we've seen when it comes to data protection, especially in data classification, which is necessary to protect it at the right levels. It seems to me like AI, these models, especially if they're learning and getting better, could very rapidly displace some of the biggest challenges when it comes to data protection, specifically data classification. It seems like this could be a huge win if we get it right. What do you think?

Absolutely. Absolutely. I think it will be augmenting the capacity of our data protection teams and also information security, because, you know, this is something that we share as a responsibility. It's not either on our plate or on data protection plate. It's something that we work together cross-functionally to ensure the safety and security of our data. So PII data is more specific because we do have data protection laws. And I hate to say that, but we need to protect every information that is critical to the company and not protect only PI and PII data just for the sake of not getting a fine from the authorities.

Yeah, there's more data, more sensitive data than just PII, especially intellectual property and trade secrets, all sorts of different things that we have to fall under this umbrella. But I'm glad, I am glad we have privacy laws because for the longest time we weren't protecting PII or private health information as well.

So that's true. And another area where we should focus is definitely identifying AI vulnerabilities and adversarial machine learning attacks.

Okay. Having AI look or check other AI systems? Is that what you're talking about?

Yeah, absolutely. So we need to inspect the systems to understand how do they work. Like attackers can introduce subtle changes to input data that cause AI models to make incorrect predictions or classifications.

Right. Or act in unprofessional ways. We've seen that where companies have released a chatbot, and within a day it's, you know, spouting obscenities and racial remarks and, and had, you know, had to be pulled out, um, simply because it could be too influenced by those providing input. And no company wants that embarrassment, right?

Right. But we cannot, we cannot protect that if we don't know how it works. So we need to study how the attackers can poison training data to manipulate the AI learning model process.. And then try to identify any control that could mitigate this risk. So, and the third point for me is the ethical use of AI.

Ooh.

Okay.

It's a hugely important topic that rarely gets discussed. So tell me about what you mean ethical use of AI. Ethical use of AI and why it's important?

So we talked about this earlier when we said that technology can be used for good and for bad, but the intention, usually the initial intention is for good.

Yes.

So we need to make sure that this AI system is actually fitting to, you know, to the strategy that the system was created for. To the purpose. And therefore, we need to understand how biases can be introduced into AI systems and understand the impact that they have on decision-making process. Because, you know, one of the biggest promises of AI is also assisting in decision-making. And if we take those data for granted, that output from AI, if we say that like this output is correct because, you know, our data is— training data is correct, then you have a trust in this model. But what happens if the model has been biased? If the attackers have actually poisoned the data? So we need to make sure that we have quality checks for detecting and mitigating biases in, the AI models?

I mean, bias can be one of the biggest problems because nobody wants to admit it, but we're all biased in some way.

That's true.

The engineers, I mean, they're people, they're looking at data and we've seen incomplete data or biased data or weighting within GenAI systems that have excluded people unintentionally. But they've excluded people or organizations, or the output has been biased and it can be amplified, right? A little bias in an AI system can quickly grow to be something very, very significant over time. And if we just blindly believe that it's telling the truth, it's including everybody, big mistakes can happen, right?

That's absolutely true that we as humans are biased, but we want AI not to be biased. AI should be better than us, right? Yes. So we need to learn the importance of making AI models interpretable and transparent. I think it's very crucial to be transparent, like for the service providers to be transparent on the data that they have ingested in the language model.

Here is the start. Do you see that happening? I mean, I see things in the news, right? OpenAI just got in trouble a little bit with actress Johansson, right? There may not be transparency in how they're training their models and they may be grabbing images that they don't own. They may be grabbing voices that have not been authorized and training their models to mimic that or reproduce that for their own gain. I mean, what are you seeing? Is there a lot of, is there just, is that overhyped? Is that happening? Is that something we should be concerned with in regards to ethics?

I think we should really be concerned about the copyrights thing because You know, we see copyrights being violated not only when it comes to, you know, articles or books, but also artists are complaining about their artwork. So I don't want to mention any name of the tool, but definitely there are some image generation tools that are— that is clearly that they have abused with scraping the internet and getting the artwork from many artists and using that data to generate new images.

Or video or sound, right? Or video. It's all multimodal. And okay, so back to ethics. You know, when these large language models started coming out and image models, we knew that it was academia that, that drove a lot of this, right? And some of the— and you have to have base test model, right? Because you wanna test your model against somebody else. So there needs to be these large databases of data to be able to use and grade. And in the very beginning, we saw these academic leaders go out and they need a massive amount of data and they were scraping it off social media sites. And there was no permission. In many cases, the social media site didn't even give permission, but you would create these massive databases that other organizations would then use. And, you know, there were images of children, there were, um, you know, unapproved, uh, images, uh, medical images, health images, things like that that were included in these datasets. And there was a little bit of a backlash and Are we seeing the industry kind of shift now to try and get more ethical datasets? Or do you think that's still a pervasive problem that they're just grabbing whatever data they can get their hands on and not worrying about the legality or the ethics of its origins or approvals of it? Where are we?

Yeah, we're actually seeing an increasing number of regulations such as EU AI Act, a US executive order on AI, Artificial Intelligence Data Act, NIST AI Risk Management Framework, and lately also the ISO 42001, which is the standard for AI. So there is a lot of good work being developed, but still Still, I don't see any implementation of these standards in practice. So we are, uh, it's too early to speak about the implementation. I mean, uh, only 20 days ago on March 21st, um, the EU AI Act was approved.

Yes.

And it enters into effect.

That was groundbreaking. I mean, that, that, that's one of the biggest pieces of legislation that have, that has really tried to address some of these issues, right?

And I really like it because it's risk-based. And you have like different categories of AI systems, AI risks. So you have like systems which with no risk and then limited risk and then high risk and then prohibited systems. And for each level, there are clear guidelines on what to do, like what information security measures to put in place. Rules about transparency and also some models to fill out for the registration of the AI tools and rules for service providers and for deployers. And it's really a great initiative to regulate AI. So we are seeing more and more regulations coming up. But implementation will take time. So there is a stage between— a timeline between 6 to 36 months for EU AI Act to be implemented. And there are even some existing AI tools which were developed and put into operations like 1 year before the EU AI Act was developed. And those have time until 2030 to implement these measures.

What should cybersecurity leaders do now to manage the current and future risks?

So yeah, more and more companies are tapping into the potential of generative AI. That's no question about this anymore. What does this mean for cybersecurity? We as defenders need to surf the wave in order to protect our data. We need to change, and these legacy tools and approaches won't help anymore. I think cybersecurity leaders must adopt new skills. And what are these new skills? In my opinion, it's about managing cybersecurity with AI. We should actually use AI natively in our cyber defenses in order to keep pace with, with the attackers. So this is about augmentation, empowering cybersecurity teams and augmenting their expertise. Use AI for threat intelligence, for incident response automation, for vulnerability management, for whatever task that AI can do better than us. And we need to foster a better collaboration cross-functionally between different departments like IT developers, data science, data protection, cybersecurity, procurement, and I don't know if I missed any of the important teams, compliance as well. So we need to align together especially when it comes to evaluating AI systems because we need to see 360 degrees and we as cybersecurity professionals cannot do it alone. So we need the stakeholders on board.

Is AI undermining our current security controls?

I think it is already undermining our security measures, especially when it comes to shadow AI. So we are seeing more and more business units inside the organization trying to purchase AI tools right away, you know, because they have their own cost center and they can do it, but they shouldn't because they are sort of mirroring the shadow IT term that we know. So sort of not involving IT or cybersecurity or data protection team in the procurement process at all. That means that we as defenders have no visibility on what is going on when it comes to using third-party tools. Because if it is a system that needs to be purchased and configured internally, they would need IT, right? But if it is something like, you know, ChatGPT or Google Gemini or Microsoft Copilot, whatever, they can just go and purchase a license and use it without actually registering this tool into the company third-party register. So this is a huge risk in my opinion that needs to be mitigated. And what can we do about it is actually raise awareness, like prepare AI guidelines for both users, like end users in the organization, and custodians, which are like, who are basically consisting on developers and IT teams who are basically in charge for setting up these tools or developing these tools internally.

Is AI a new type of problem to manage for cybersecurity?

Well, I think AI is just another technology. Although it is powerful, more powerful than what we have seen before. But AI shouldn't drive the need for the creation of, you know, an AI department in the organization or, you know, a Chief AI Officer role. I mean, we are seeing this in some of the organizations, but it is also introducing friction to the business because that's a huge cost to setting up a whole department to be in charge of AI. And as we are seeing AI being integrated even in the existing tools that we are using, you know, for example, conference tools, almost all of them, like they are integrating AI as a component, as an assistant to help with meetings, minutes of meetings. So every tool that we are using almost has AI features. So this is sort of coming as an integral part of the tools and the systems nowadays, and I don't think that there is a need to separate AI from, you know, the technology that we are currently using.

Do you think new training is required to help personnel to understand how AI applies to the current policy?

I think that's crucial, and companies have to allocate budget for education to include all team members in this. And not only education, but also innovation. A lot of ideas come, uh, come from the team members basically, and we have to nurture that. We need to adopt the leader mindset in cybersecurity especially, and walk away, walk away from the classical manager approach, because this is a trap. While some cybersecurity leaders are trying to be the smartest in the room, they are failing to make the room smarter. So I cannot highlight this enough, we need to empower the team. And provide them resources for education, especially when it comes to AI. I see a lot of teams nowadays in different companies who have no budget about, like, who have no budget for AI at all. So how can they know about the latest threats and vulnerabilities if they are not able to test anything, if they have no visibility at all on what is going on? So education, definitely information security awareness trainings on AI is a must nowadays. And the companies that are not doing that now, they are definitely in a very bad position and they are vulnerable, I would say, to many sophisticated attacks.

How should cybersecurity professionals act to address the emerging challenges.

AI requires transformation of cybersecurity how we know it. It doesn't need to be refined, it needs to be reimagined. And where do we start? I think a good starting point is to create the blueprint. So we need to understand what are the generative AI applications that are being used in the company? Like who is using them, how much are they being used, and what is the data flow in and out. So then we need to map this blueprint with risk. I am very much fan of risk-based approach in cybersecurity. So application risk, data risk, AI governance risk, we need to map the blueprint with those risk domains. You know, data is the fuel of AI. So we have to make sure that we understand our data. We know where the data is and have the right classification and labeling for it where possible. And third is mitigate the risk by implementing zero trust architecture, implementing XDRs, like the extended detection and response mechanism, to collect and correlate data across multiple security layers. And do threat modeling, expand threat protection to AI tools because our traditional techniques are not going to work here. We need to adapt inclusive security tools to AI risks and controls. And there are organizations who are afraid of AI. They're not using it because they don't know how to manage the risk. And in my opinion, this is not a risk avoidance. This is risk ignorance, as the employees will still use it for their daily tasks and expose company information in the wild without any control. So banning or ignoring AI is not an option. With AI, we can unlock massive potential, and I think companies who are not— which are not considering AI are definitely in a competitive disadvantage. And we have to extend our focus not only on confidentiality, integrity, and availability, you know, the CIA triad, but to transparency, to fairness, to reliability, to safety and inclusiveness. I think we have a lot more on our plate than we ever had before.

Iona, thank you for all of your insights. Very much appreciate you coming and talking today.

Thank you so much for making space for these topics and discussions, Matthew Rosenquist. It's an absolute pleasure chatting with you today.

And thank you all for watching. Be sure to subscribe and catch all the Cybersecurity Vault episodes where we chat with industry leaders like Aayona to dive into the most relevant and interesting cybersecurity challenges, perspectives, and best practices. Thank you all. See you next time.