Machine Learning, AI & Cyber Security Part 1: Used for Good

Welcome to the RazorWire Podcast, where we discuss all things in the information security and cybersecurity world, from current events and trends through to commentary from experts in the field providing vital advisory on what it is to work in the information security and cybersecurity space. Hello and welcome to another edition of the Razor Wire Podcast. Today I have 2 fantastic people, one of which I've wanted to get on the podcast for a while. We have a recurring guest, Oliver. Oliver, do you want to say hi?

Oh, hey everyone, um, good to be here again.

And an old mentor of mine, somebody, as I said, I wanted to get on a number of times in the past, and, uh, we finally cornered him.

Hi, James.

Hi, Oliver.

Pleased to be here. Thank you for inviting me.

Fantastic. So today we're here on the first part of a 2-parter for this particular podcast. What we're looking into today is the application of machine learning and artificial intelligence for the information security sector, what it means, what, what we could be looking at, what we could be going with, what we could be seeing in the future. There's been a lot of development in this space over a number of years. We'll go into that a little bit more in depth in a minute. The second part of this, which will be in a separate podcast obviously after this, is what would feasibly happen and what could happen if the malicious actors that we're quite often up against get hold of that same selfsame technology. It's very important to remember that a lot of the technology that we use, they also have access to. And with things like artificial intelligence and machine learning, that could have some quite serious and quite dangerous implications. But we'll leave that one for that second, uh, that second part. So for the first part, here we are. So artificial intelligence for many years was an application for really sci-fi. We've all seen the robots coming back in time to murder people's grandparents or parents. We've seen, you know, plenty of time travel movies. We've seen AI taking over the universe and various different things. It was very much a staple of sci-fi. for many, many years and indeed still is in many respects. But as our kind of grip on technology and our development of technology has moved on so significantly, especially in the last 15 to 20 years, we are facing with the real-life possibility that artificial intelligence, machine learning is going to become quite a significant part of our technological stack going forward, both in business, be it in government, wherever it may well be. I mean, we've all seen recently one of the Google engineers who was fired turn around and say that Google actually do have an AI about at the age of 7 or 8 on the intelligence scale internally. But obviously nobody can necessarily corroborate that unless any either of you guys got more information that you could, you could add into that. But we are seriously looking at a significant shift. You know, some quantum computing is starting to get easier. I know that the UK have government have just gotten hold of their first quantum computer, so they say. Obviously, I haven't seen it. And there's a lot of vendors out there who already use elements of machine learning to help facilitate some of the services that they perform. So who should we start with? Who wants to go first, Oliver or John? Open season.

Well, I can throw out some ideas to start with. I think where AI is a force for good, AI being a very wide term, what it actually is, is, in my view, a set of algorithms that are very, very good at identifying patterns. To an AI system, anything looks like some clusters of similar things and outliers that are not like those things, which means it's very good at spotting unusual behaviors or actions. In, obviously, the cybersecurity world, That's good for us. One of the things it also does is, as humans, we tend to fix the last bad thing. So if we've just been popped by a, I don't know, an SQL injection, then we tend to really think very hard about making sure that, you know, an SQL injection doesn't come along again. If we've just been hit by ransomware, then as an organization, as defenders, we tend to be very sensitized to any indicators of another ransomware attack. And of course, AI isn't necessarily sensitized to the last bad thing. It is again looking at those clusters of unusual things or things that look like they cluster together. And of course, the other thing it also means is that if we have a machine learning system that's been trained by a group of experts, then we can leverage that expertise from the vendors in order to bolster our, again, our human detection and response efforts. Certainly, one of the things I think is really interesting, AI has been in fraud detection, one of my areas of interest for many, many years. You see it all through the payment space, including obviously, say, the payment process itself, authentication, or anything that's high impact on the payment process tends to have an AI saying, this is the risk of the transaction at this point of time. I think that's quite exciting. It means that when we make a payment with our card online, then it's not just, is it the card? Has it got the right CVV and all the other things in the dataset? But a whole myriad of factors around the behavior of me as the payer, the behavior of my machine, all of these get fed in and analyzed and then get fed into, is this transaction within a merchant's risk identity or not? I think that's quite exciting.

I started working closely with data science teams maybe about 4 or 5 years ago. Coming at it as a security person, it was hard. You're used to being an expert in the field, you come into a new domain, and all of a sudden, you're not the expert. Most of the times, I've worked with 2 different data science teams now where I've done knowledge transfer as a subject matter expert to the data scientists. I've had at least some hands-on experience. One thing which I noticed that in the industry, there are some people who are just embracing it, there are other people who are absolutely resistant to it. I call them neo-Luddites. I think part of it, the argument is always that it's not understandable, it's a black box. Of course, that's not true. Deep learning, to an extent, is a black box, but there's explainability tooling which can actually help you with that. Overall, I would say that as an industry, we're going to have to embrace the black box. That's my first takeaway. Even if you don't understand it, that failing is not with the technology, it's usually with you. There are people who understand it. If you don't understand it, you need to go and learn about it. It's not something that's easy when you're used to already being an expert. In fact, it disincentivizes you of learning something if you're basically starting as a novice. I think that's a part of the problem with a lot of security practitioners. At the same time, it's vastly powerful. No, it's not AI in the sci-fi sense, but it is vastly powerful in certain use cases. Classification is a good example as a decision-making tool. I think as an autonomous decision-making system, we're not quite there yet, and that's a pertinent point. There are different levels of maturity across different AI fields. And my favorite example, you mentioned the Google engineer. It is easier to create a chatbot that can fool a human into believing that it's sentient than it is to detect hackers. That, that's really crazy if you think about it, because you'd think that language is actually a more, more challenging problem, but it's not really because we've been working on it longer. It's fairly structured. And more importantly, it's something where you can train an engine to be able to do it fairly easily just by ingesting lots of speech. If we look at trying to detect hackers, you have a human adversary who is actively engaged in trying to evade what you're trying to do. I think that's a fundamental difference between playing a game or something like Go or chess, where it's quite a small set of rules or moves, to actually having a human who can take any attack vector in. Yeah, but it's not going to go away, but we're still in the early stage. My favorite example of this is just the fact that There's this great anecdote about Henry Ford that he was asked what he would have built if he'd have asked his customers, and he said a faster horse. I think we're at the stage right now where we're using AI to build faster horses. I've seen vendors using AI to create detections. That's insane. The whole idea is that you don't need detections when you're using AI. We still have to make that thought jump of doing something different, new with AI, but we've only just hit the point where it's available, so we're still trying to build that faster horse.

I think the other thing as well is we say that when we are if you like, selling these things to CISOs out in the world. We're selling to people, as you say, who are expecting horses. It's very difficult sometimes because I think we become very enamored of the technology in itself. I've certainly had end-user CISOs say to me, well, I really don't want to hear any more about the magic of AI. I don't want to hear AI. I want to know how it's going to solve this business problem. So I think, first of all, I think the idea of, yes, we are still, as you say, building faster horses because that's what the customer demands. We're certainly in a situation where we need to be responsive to that. But I think the other thing is that we need to get to the point where we are, if you like, taking a leap forward and saying, right, well, when you say you actually want a faster horse, what you really want, as you say, is to be able to get home 10 minutes earlier. or to wherever you're trying to get to 10 minutes quicker than you would be by your existing method. So again, what are they trying to do? Yeah, they're trying to reduce risk to the business. They're trying to stop attacks, say shift as far left of the boom as we can. And so again, there's a lot of things we can do. And I'm seeing some interesting pieces where AI analytics are being used in the DevSecOps process. So it becomes part of the CI/CD, it becomes part of the code assurance process. And so we're shifting left there. In terms of the detection phase, as you say, we're getting to the point where we're able to identify early ripples in the water and know that means there's an attack coming. And I think that gets quite interesting, especially when you start looking not only at the panoply of external attacks, but also when you start looking at internal risks to the business as well. Again, do you see any evidence of errors that could themselves be harmful to the business, whether they be malicious or non-malicious? I think something that I see is very interesting, something I've picked up. There's a company out in Vegas doing some research on non-obvious relationship analysis, NORA they call it. I think, yeah, Ollie and I were talking about this a few days ago, this idea that you can say Spot, employees in a casino. They would appear to have no obvious relationship between them, but in actual fact, what there is, is they are working in collusion for a particular, in this case, malicious end. And I think, yeah, the casinos, for example, are very interested in this, and I think some government agencies are also seeing this as exciting technology as well.

And it is exciting. I mean, it is. It's an area of technology I personally have been looking forward to for some time now. I mean, One of the biggest problems for, say, cybersecurity professionals, you know, as well as governance, I suppose, you know, especially if you're in instant response, is trying to disseminate what is going on when something's kicking off in any decent timescale. Because as we all know, there's a million logs being generated by firewall infrastructure, by IDS/IPS, you know, they're all being pushed into data lakes and various other places as well. There's endpoint security and— Trying to figure out really what's going on within any reasonable timescale is very, very difficult. I think this is why you quite often hear in the media about, you know, a breach, and then it comes out that the breach actually occurred, say, 9 months beforehand, but it was only detected sometimes by accident, sometimes whatever it may well be, many, many sort of like months down the line. And Jonathan, you mentioned car payments. Again, that's quite often the case with car payments because the data that they use, you analytics they use requires a certain amount of data before they can determine whether or not some kind of fraud has occurred with that card. It's not like one company gets, uh, done, uh, the first transaction, they go, ah, right, there's a compromised card straight away. Sometimes it's— it takes a little while for them to establish it. And I think when we do get almost like a companion AI that can sit next to that information security professional that you can just say, look, what is going on? You know, or it can alert you and say, I think there's something going on over in this, this corner of the network, or this individual has started logging on at funny times of the night. You know, you can start correlating events into more of a story, and then it's obviously for you as the human to determine what action needs to take place. Maybe the AI can have a certain level of autonomy to kind of head things off at the pass, but I think we're quite far away from anybody. And it's something that both of you mentioned before, you said before, there's still a certain level of distrust by allowing AI to just take control. Oh, we've got AI. Yeah, but I don't want AI. I want it to tell me what's going on. And that's what I'm seeing from a lot of people. And Oliver, you mentioned something similar. Do you want to carry on?

Yeah. I mean, so it's interesting. I think a lot of this comes down to what you're using. AI for, right? We can use it to improve our processes. So as you mentioned, we can use it to aggregate different alerts, try to validate whether they're actually belonging to one, a single kind of attack narrative. Um, but the other thing around detection and automation is really that it comes down to level of confidence. If you have a high level of confidence, you can automate. And in fact, it's desirable. You don't want to keep a human in the loop in a speedy process that The best example would be stopping ransomware. That human in the loop is just friction.

Yeah.

That will actually stop you from responding quickly enough, but you need to be sure that it is what it purports to be, or the risk has to be so high that falsely blocking it is worth it. We have better ability to do that using machine learning because we're able to assign a probability, we're able to train the model to improve, which we weren't able to with very one-dimensional signal signature-based approaches.

Mm-hmm.

And, and so, so autonomous response is something which we're gonna see growing over the next couple of years. But the other thing to think about is the fact that we are using it to detect things which would be almost impossible to detect any other way. And I'll give you an example, living off the land. So you have no indicators of compromise, you don't have a file hash, you don't have this known list of, of bad software. What you have to look at is who is doing what, compared to everybody else. And that's a typical, you know, user entity behavior analytics problem. It's a typical behavior problem really where you're, as Jonathan mentioned earlier around the clustering, you take a peer group, you say, these are salespeople, they're all accessing Salesforce, they're all accessing this file server because that's where the contracts are and so on. And all of a sudden somebody deviates from that.

Hmm.

The problem is that it's not necessarily good or bad, right? It's suspicious. And at that point you need to bring the human in the loop. But it does allow us to detect attacks which before we were not able to. And that's, that's just catering to this, um, to the adaptive defense-offensive cycle that we're in, essentially that arms race against attackers. So it has various applications. And I think the problem is that when it's missold, when it's not clear which problem it's solving, and if the problem isn't painful enough to that individual business user or whatever, there's a huge mismatch in the market there because we're just saying AI. But, you know, it all comes down to the detail, when you're using it, why you're using it, and so on.

I see a lot of vendor materials mentioning powered by AI, this, that, and the other, and it's like, yeah, but which part of it is the machine learning component? You know, whereabouts in your product does it sit? You know, John, you've probably come across this quite a bit.

Yeah, and I think there's a, as you say, I think the challenge we have is that it's very easy to say, powered by AI is a magic pixie dust, you know, because it's powered by AI, it's automatically better. But as you say, the question is, well, what are you doing with it?

Hmm.

What are you actually making? What business problem or what security problem are you solving with AI? Is it faster to, yeah, faster to detect? Is it more accurate in detection? Is it a quicker Is it a cost and harm reduction benefit that you're providing? And we're not quite there. I think a lot of vendors have, you say, you know, fallen for this. You know, if we go in and we shout how great our AI system is, our end users, you know, our prospects and our customers will, you know, embrace it with open arms. But of course, what we're seeing is actually quite the reverse. And I, you know, I say I've had end users say to me, I really don't want to hear anymore about these blessed AI solutions. I don't want to hear about the blessed magic of AI. You know, what I really want is, what is this actually going to do for me? And what are we actually saying there? Well, it's frankly, it's back to the good old message of we need to be selling and presenting on benefits, not features.

Yeah, solve problems. And, and, and it's a, it's a funny thing because like back in our days as industry analysts, One of the things we always looked for was gratuitous AI. My favorite example is land speed violation. So you have a user logging in from Rome and from New York within 2 hours. Obviously, they can't travel the distance between it, right? You don't need AI or machine learning to be able to solve that. You can do basic statistical analysis. And indeed, the team I'm in, we're considered a data science team. Sure, we do some AI, but that's probably maybe 10% of the entire thing that we do. The rest is just algorithms, Better maths.

I must admit, actually, I'm slightly proud of this. I just got myself a new Jag in my younger and wilder days, and I'm based in the UK, and I was asked to do a forensic investigation in Belgium, in Brussels. And so I thought, well, I'll stick my gear in the boot and I'll drive my Jag. And I actually got stopped. My card got pinged at a French petrol station. And when I rang my bank and I said why, they said we didn't believe it was possible for you to get there that fast. I was like, good.

So you actually triggered the land speed violation.

Any traffic police listening, that was, you know, purely an accident.

But I mean, you know, again, it's, it's, it's a funny thing when it comes to— so we talk about the land speed thing now, you know, we're in this era where comms is, is getting faster. The ability to access comms is pretty much getting to the point where You could access it anywhere you want, you know. And, and if Elon has his way when he kind of opens up, uh, Starlink, you know, you could be in the middle of the ocean, you know, as long as there's a satellite floating near past, you know, past where you are, then you'll have— we consider broadband level of access as opposed to what it was previously with satellites where, yeah, let's face it, it was pretty horrible if you ever tried to, to do anything over a satlink. You could be there for a long time. And organizations are getting bigger, and that all the information coming in and piling into the infrastructure is becoming like really difficult to handle. You know, a lot of SIEM providers, a lot of SOC providers are taking away, you know, elements of that pain. But now we're getting to the point where we've got multiple third parties involved as well. They're monitoring their own kit. It's not being monitored by you as the organization, you know, platform as a service. This is where I see the application of artificial intelligence. And I'm not talking about true AI where it's actually a living being, you know, but maybe sort of like a couple of steps beforehand where it's just sitting there on your infrastructure looking at potential vectors that are either being affected or could be affected. As well as looking at information coming from those third parties, because we're in this really awkward situation at the moment, and it's and it is getting harder where we are relying on these third parties to be doing what they should be doing. But if you look at a lot of breaches over the years, a lot of them were instigated usually through third parties or because of some kind of third-party involvement. And with AI, you know, this is why I get frustrated sometimes with vendors that say we're powered by AI and machine learning, and it's like yeah. It's one small corner of my problem that it handles, and I do vision a relatively sort of interesting future. I don't know; I wouldn't want to give any time scales on it, where we will have this kind of AI companion, as I like to you know term it, advising us as infosec individuals and saying there's a problem with the IDS. I'm just starting to see some issues from the IDS. I'm also starting to see some. Funny activity from an account out, you know, with a, with a vector coming in from China, only I know the individual was in, I don't know, the UK, Reading, 5 minutes ago. And correlating all that information into a story, we're seeing a lot of kind of, um, I've got a lot of vendors who come to me with kind of like their SIEM products, that kind of thing, and they're all starting to go down that story methodology now. And it was something I think, I can't remember if it was Jonathan or whether it was you, Oliver, that mentioned it earlier on, I think it was Oliver actually, where it's like every action that the AI doesn't like, it dings a little. Oh, there's a slight risk. There's another slight risk. Oh, I'm seeing another piece of activity I don't like from that same vector. And then it'll hit a hard level where it goes, no, I really want to have a look at that. And I'm quite excited to be able to have something that can actually help me do that. Because at the moment, I'm reliant on about 15 dashboards, 35 feeds, how can you do that?

You say AI, but here's the weirdest thing. You can do that visually as well. If you take the MITRE ATT&CK framework and you map alerts to it, you can actually— the idea is that if you have a low confidence alert, that's ambiguous. But if you have 3 or 4 low confidence alerts, which all map to a certain pattern, you have a higher degree of confidence. Machine learning indeed takes a similar approach. I know we're working on something similar to that, where you're basically looking at this larger data amount. It's definitely something which is coming, but there's already some basically available in products. But the true limiter still is the underlying data. That's something we don't talk about a lot, the fact that your AI is only as good as the data that goes into it.

Yeah.

We have a data problem in security. Our data, whenever I speak to our data scientists who come in from finance or healthcare, they're always quite appalled at how bad security data is.

Yeah, well, nobody wants to release it. I think, you know, with, with, with a lot of other types of information, you know, there's a wealth of it as long as people have signed off that they're willing to allow it to be used. Whereas with security, you know, we only see not even a tenth of the, the issues and breaches that have actually occurred. But John, you wanted to say something?

No, I was going to say, I think it's— I think one of the things I find interesting about the AI piece is, if you like, the concept of signal-to-noise.

Yes.

As you say, we only— and you just made that point that we only see or we only detect a fraction of what activity is actually going on. That's our, if you like, our human-based or traditional-based signal-to-noise ratio. One of the things I think that AI does very well, because, as you say, it ingests everything, it can look below that noise floor. It can detect activity that we would not be able to detect. And as I say, we just cited a few. We talked just about the land speed violation. Well, actually, that's something that we as humans can detect. That actually comes out above the noise floor. And as I say, we all just explained how it works. So that's us being humans. That's us using deterministic methods. Where AI gets interesting is it picks out these multiple, otherwise unobservable signals and correlates a picture that says, actually, here's something happening that you may not have seen. That, I think, does get exciting. Now, of course, because it's not necessarily deterministic, it's not responding to, this has happened, this has happened, therefore do this. Non-deterministic systems have a probability of reliability. How confident is the AI in what it's detected? How confident are we? This is why we talk about risk detection rather than a definite detection, which means that the way that we respond to these signals has to be different. We get, as they say, a shift left in our response time. But, we also get— the price of that is that we have to respond differently. We have to be more investigative rather than saying, right, this is definitely something bad that we must jump on and squash. The activities of the operations teams, of the response teams, are going to change.

I have an interesting anecdote around the probability. We've been debating whether we start putting basically an indicator on each machine learning detection of what the probability is that it's a true positive. The downside to that is that the way that security people are, they would just start tuning away the ones which don't have a high level of confidence. But the level of confidence does not correlate to the importance of the alert. Just because something has a low confidence of being detected doesn't mean to say it's unimportant. It's just harder to detect. The way that people have been, I would say, conditioned based on intrusion detection systems is just to try to tune away anything that has a low level of confidence. Even there, you notice us having to learn how to read machine learning and AI is something we're going to have to evolve, but people aren't very comfortable dealing with probabilities, especially not most people. If you don't have that mindset, It's a hard concept to get your head around. Like, go to your executive, you are the executive, James, but in our case, if I went to my boss and said, well, we have a 0.67% chance, you know, what's it called, a 67% chance of being successful, he'd tell me to go to hell. He'd want to know yes or no.

Mm-hmm.

And so it's interesting that it's not that straightforward. You'd think it's easy, we just give them the probability, but actually that doesn't mean to say that they're going to interpret that the way that we'd like them to.

Do you guys think we're going to get to a point where we will, or the industry, you know, the security people who tend to be kind of a suspicious lot as a general rule, you know, let's be honest, allow AI to take actions without first authorizing that action from a human? Do you reckon we are going to get to that point? Because I think it's interesting.

It's in deployment. There are companies actually doing this right now. I like Darktrace as a vendor where they have users doing that. SentinelOne have it, it's— but it's selective. It's based on policies. You don't allow it as a general blanket. The other thing is that this is a generational thing. As I mentioned earlier, you know, if you've been doing this for 20 years, we have this learned helplessness around automation because we've been burnt by anti-spam intrusion detection protection systems. But younger people, and even us as consumers, are getting used to this more and more. But the big driver is just external. If the adversaries start getting more aggressive, We're going to have to automate it. It's not even a choice thing.

It's— I mean, I could just envision it being a much easier thing to turn around to your, you know, your companion AI and say, once it's delivered some details, you say, go and disable that account, and then go to the firewall and do verbose logging on that IP range, you know, that you've been seeing some of that from. Because at the moment, you'd have to do that manually. You'd have to speak to a team if you weren't the one who had control of it. They would then have to go off and make those changes. Then you'd have to make sure that those changes were correct. Whereas that AI, if you had the correct AI, could literally just go, boom, done.

But machine-speed attacks require machine-speed defense. That's essentially what it comes down to. If the adversaries start using— and there's this whole discussion around this in warfare around autonomous drones and stuff. There's this fantastic book called I, Warbot, which delves into the philosophy and the strategy around it. And in reality, this is something where sometimes it's highly desirable to be autonomous. I'll give you a perfect example. If you don't have a lot of people working on the weekend, why would you not want an attack stopped autonomously? So that's a very good policy-based example where it depends on the circumstances, where you're doing it, when you're doing it.

Kind of like a thresholding thing then?

Yeah, that's another aspect to it. Which threshold do you do it, or at which point in the kill chain? If you're detecting data exfiltration, That's the thing. It comes down to this simple discussion around what's worse, a false positive or a false negative. However much people complain about false positives, it is a false negative. In the same sense, if you have something that's going towards the end of the kill stage phase, so data exfiltration, data encryption for ransomware, you want to respond quickly because you don't have the time to go through a manual process.

Do you think the business would you know, support that? Do you think they would, they would feel confidence with that kind of— because it's like, it's, it's almost like turning around to people and saying, let the AI take control of your, your, you know, your security and your network. I can see a lot of IT people thinking, well, that's kind of cool, because, you know, they're techies. Every— I don't know any techie that doesn't like the idea of AI. But—

I, I think with damage, cyber insurance, and so on, I think that mentality is changing. to the point where there are companies already doing it. Actually, and I've spoken to organizations, large organizations who are already automating aspects of response. It depends on what your risk profile is, what you're trying to automate against. Yeah, so that is changing, but more importantly, once again, it comes down to the adversaries. It comes down to potential damage if you don't automate.

I think there's also that factor which, funnily enough, we're going to be talking about next time, for part 2 of this is it, in essence, it is a piece of software, isn't it? And pieces of software can be used to either attack themselves or—

Any piece of sufficiently complex software has bugs. Some bugs have a security impact. And yeah, that's a— that is a fact of life. And we see the impact of this actually in, you know, Frequently in the software contract space, people execute contracts in code, they proclaim that code is law, and people find loopholes in the code.

The counterexample would be payment. Again, credit card companies are perfectly happy to automate the blocking. That's the point. You get it where you have to call them, you have to unblock it, and so on. I think we already have a precedent there, but it really depends on I don't think companies are just going to have a blanket policy of enabling automation. But if you have certain critical assets, you have certain critical processes that you're trying to protect against very distinct threats, that's where it's going to come down to it. So it's going to be quite selective. But increasingly, we have to adopt it purely and simply because the attacks are getting that fast as well.

Do you guys think that security AIs are going to replace us ultimately, eventually? Because if we're going down the AI route, and let's face it, as a species, that's really where we're going. We all want it. I think there's a few people who probably don't, but—

I counter that. I don't believe that's the case at all, but that's a bigger discussion. Because you're implying that strong AI is even feasible, and that's a whole philosophical discussion that's really deep going down onto the quantum states and everything. But the bigger question is, If that happens, it won't be first in security. We don't invest enough. We're not the— so you're going to— that kind of capability, you're not going to find coming from a security vendor. We don't even invest that kind of R&D money. More importantly, there are other better-funded industries like finance, healthcare, where you're more likely to see that kind of a capability. But at that point, it's kind of a moot point because you've also replaced the attacker. If you're talking about replacing us, you've replaced almost every human in every other job. And that's a completely different world. At that point, cybersecurity is probably not going to be the biggest concern. Yeah.

Unless you own those systems.

At that point, it'll be machine against machine learning in a lab how to bypass one another. Humans will be out of the equation except for some kind of a god developer. But I said that is so far in the future, man.

That's I've read Ray Kurzweil and The Singularity Is Near, and there's some interesting concepts in there. And funnily enough, he does kind of touch upon security in that. But I mean, that's a, as you say, that's a whole other, whole other discussion. I think, you know, I don't think they're ever going to replace a human. I think people, whilst humans are still here, I think people are going to be a bit worried about leaving everything to some automated response or to almost thinking creation to, to protect them. They're going to want some kind of level of validation. But I think it's going to definitely change and speed up our response times. As, as you guys have said, you know, it's, it's a key tool. And not just for our profession, but say for the IT profession, you know, if something's going down or something looks like it's going to fall over, you know, AI is going to detect that far quicker than feasibly somebody, you know, sitting in the offices, especially if they're going to get a coffee and they're not looking at the not looking at the screen as it goes red. I don't know. I like the idea of AI. I'm all for it. I'm interested to see where it comes. And I just hope that it's not all individual AIs for individual solutions. You know what I mean? Because that's just going to be another dashboard hell, but without the dashboards.

I think we'll certainly see, as Oliver says, I think the dawn of a universally strong AI that is good for anything. is some way off. What we are seeing is the development of several narrowband AIs. For example, there are companies like a company I work with, Datavizor, sell an AI which is very, very good at fraud detection.

Hmm.

As Oliver said, there are companies like Oliver's employer, Chikaronics. They are working on AI systems that are very, very good at aspects of cybersecurity, behavior analytics, and so on. We are seeing the growth of these very narrowband systems. Where I think it becomes interesting is if you then say, well, what happens when we find a way of linking those capabilities together? Since we are seeing that attackers are not shy about using whatever means is necessary to achieve their ends, which are mostly now monetary and organized. why should we not as defenders try and do the same thing? If an attacker will say, well, look, I can defraud this system, I can submit false transactions, I can take over accounts, I can coerce people who are account holders. Oh, and by the way, there's a loophole there I can slip in some ransomware. Why should we not try and get our AI systems to join up in a similar way?

Yeah, you know, there was a study released last week from— I can't remember what, so I could probably dig it out for you, but it was basically this mathematical mathematical model by a study group which showed that human-machine teaming is the most effective way of doing it, and for the pure and simple reason that there are things we excel at and there are things that AI machine learning excels at.

Yeah.

I think that's a more realistic way of looking at it. We're going to have processes where some processes are going to be very heavy on the AI side, some processes very heavy on the human side, even to the point where you can delegate some things completely to a machine, but the majority of real-life processes will require, to different degrees, combination of the two. This human-machine teaming is really the next epoch for the next, I don't know, 10, 20, who knows how many years. Um, it's just a— it's just the fact that depending on what problem you're solving, there's going to be different degrees of how much the human is involved or not.

It's, it's certainly interesting. And I think John, John touched on a very good point there of how do we join it all together, because these are all disparate products with disparate versions of AI, machine learning, and all the rest of it. Is there going to be kind of something which is, you know, almost like the One Ring? It binds it all together so we can actually figure out what's going on. Because, you know, if you've got— you hear quite often when I speak to customers, you know, oh God, do I have to deal with another dashboard? Is something that we get time and time again. And you've got the endpoint dashboards, you've got the firewall dashboards, and everybody wants to create this product or find this product, which is— and I hate this term— a single pane of glass. My God, if I hear that too often, I get hives. But it's kind of the same thing with the AI because you don't necessarily want to dig deep into the configurations of that particular solution or software. What you want to be able to do is to react. AI gives you the information. It will then take the appropriate action with the multiple products that are available to it in maybe a more modular fashion.

And this, I think, is the moment that we as humans are in that loop. We're in that position of being that supervisory responsive control layer, if you like. We're in the situation where they say we can prescribe, and Oliver gave a great example earlier. We see some unexplained activity in an unusual time of the day or week, and so it's very simple to say, well, actually, we should lock that account as a precautionary measure, perhaps change some firewall rules so that access to the potentially vulnerable systems or data is limited, and then there's nothing there that is actually not undoable on Monday morning if needed. I think that those sort of simple responses we're seeing are automated. But, as I say, where we're looking at more complex responses, indeed, to complex attacks, it still requires human agency.

Yeah. There was this great anecdote around Facebook having developed 2 different AIs who were communicating. We had to shut it off. I think that's— we were mentioning earlier the fact that we're trying to build faster horses. Well, the truth is that machine learning algorithms, what they need is to be able to exchange data.

Yeah.

This idea of being able to interconnect the way that humans view build a solution, that's building a horse, not a car. The car approach is just to basically let them hash it out to have a reward algorithm, which if they complete a certain problem increases the fitness of that generation, and then just have them hash it out. to develop their own way of doing it. What's required most of all is an ability for them to exchange data in a meaningful manner. And as I mentioned earlier with security, our data being immature, we still don't have joint standards. Between different solutions, that's what I meant with dirty data, the fact that you will get an intrusion detection alert from 10 different companies and they will all be completely different and incompatible. So you have to basically parse it. Of course, machine learning is going to be able to do that much quicker than we do, but Once they can start exchanging data, because there's never going to be a universal AI algorithm that does everything, that will be a collection of different algorithms doing different bits, much like we are to a certain degree. But it's interesting, because of course, the AIs are going to have to intercommunicate the dangers that they're going to do in a manner that we can't actually understand it. The explainability may possibly be lost doing that.

You're right. I do remember that piece of where they got the 2 chatbots chatting to one another, and they developed their own language eventually. So yeah, you know, but again, that's, that's AI for you. You know, again, if you want to go all the way down it, the moment somebody does create a true free-thinking AI, it'll be able to improve itself to the point where we won't be able to cope anyway. So I think by that point it's a very different world and read Ray Kurzweil for that one. But it's been really interesting. Shall we have—

I mean, have you guys got any kind of closing statements you want to give or anything you want to plug with what you guys are doing before In terms of looking at this, how this kind of technology evolves, right now, we're still very much at this point where people have these very high ideals and expectations of what AI is. And in reality, we are still right at the beginning. What we're seeing now are just the beginning steps of what we're able to do. And so I think people are expecting way too much of this. At this early stage, this is not going to happen within a couple of years. Most of the algorithms we're using have been around for 20, 30, 40 years. What hasn't been around is the ability to compute and work with data at this scale to actually be able to make a lot out of it. But we're nowhere near the point where some people believe we are. Working with this hands-on, don't be afraid of how smart this stuff is, be afraid of how dumb this is. My favorite example is always something like Netflix. Netflix AI algorithm is based on the fact that if I watch something, it shows me something similar. Amazon's AI algorithm is based on the fact that if, if I bought something and someone else bought something, then show me what they bought together with it. That's not very smart.

There is a limit to the number of toilet seats I need in my home.

Well, one of my favorite examples, I buy a laptop and Amazon says, would you like to buy this laptop too? So how smart is the machine learning really? It's bumped up sales, but it's not smart at all. And, and, and so, and so people have this TV idea of where we are in the moment because of programs like NCIS and stuff. But in reality, we are far away from that. We are just at the beginning. Um, and so I would just say, keep those expectations a little bit lower. Otherwise, we're just going to run into this wall where people are disappointed and we'll have the 6th AI winter.

I mean, we're at the stage, as you say, where people— we are learning what this can do. Certainly, the companies I'm working with, they are learning sometimes the hard way that it's really important to train these systems right. There is a theory that you say, any data is good data. Feed it to the AI. But we know this is actually a bitter experience of totally just And obviously we'll come on to that in the next podcast. So we're learning we need to teach these systems how to give us results that are useful. Certainly this is an interesting area and it will— it is a powerful tool that can be of great use.

Now it's going to be interesting to see how it pans out for, for the force of the good, the light side of security. I agree with both of you. I, I also think we're I think the, the last 20 years of the rate of the increase in technological capability that we've had, the exponential rise in computing power, the exponential rise in the ability to communicate at faster and faster levels, is gonna, it's gonna push that development quickly, very, very quickly, far quicker than I think a lot of people are going to be able to cope with. I agree with you, at the moment it's very, it's very low level, you know. Everybody loves the AI buzzword when you're when you're, you know, they're trying to sell their product. But I do think that, you know, we are, notwithstanding some of the fun and games that's going on in the world at the moment preventing that, you know, the supply chain issues, for instance, good example. I think once, once those items have been resolved, I think our, our level of technological development is going to carry on exponentially increasing. And I do think that, you know, once we've, once we've opened up the box with, with even some rudimentary, very rudimentary AIs making some of the decisions for us. I mean, we've already got some of that, but I mean like, you know, deliberating and making its own decision. I think we're going to suddenly see a massive explosion, and I do think it's for the better. But we shall see, we shall see. So that's the end of this first part. The second part, uh, will be about what happens when the bad guys get it. I think that's going to be a very interesting conversation because Whereas us on the light side of the force have morals and ethics and don't like to do much in the way of harm unless we absolutely have to. We're going to start talking about what happens when the bad guys who have no qualms whatsoever utilizing this type of technology for their own ends. So thank you to everybody outside listening in on this particular podcast. We look forward to speaking to you soon. Keep an eye out for the second part of this, and if you've got any questions, please feel free to get in touch with us. Thank you very much, and we'll all be seeing you soon. Thank you again, guys. It's been absolutely fantastic, and we'll see you again for part two if you ever want to come back.

Thank you. Cheers.

Thank you for listening to the Rosewire podcast. If you like the podcast, if you love the podcast, please feel free to. Subscribe, and if you have any questions, please get in touch. Thank you very much and have a great day.