EP190 Unraveling the Security Data Fabric: Need, Benefits, and Futures

Security.

Hi there. Welcome to the Cloud Security Podcast by Google. Thanks for joining us today. Your hosts here are myself, Timothy Peacock, the Senior PM for Risk Analytics and Cloud Threat in Google SecOps, and Anton Chuvakin, a reformed analyst and senior staff in Google Cloud's Office of the CISO. You can find and subscribe to this podcast wherever you get your podcasts, as well as at our website, cloud.google.com/podcasts. If you enjoy our content and want it delivered to you piping hot every Monday, please do hit that subscribe button. You can follow the show, argue with us and the rest of the Cloud Security Podcast listeners on our Google Cloud Security Community page. Without any further ado, let's turn things over to today's guest.

Hello, and our guest today is Josh Libourdi, Staff Security Engineer at Brex, and the topic is Security Fabric. It's a big mystery. Welcome to the show, Josh.

Thanks. Happy to be here.

So, this podcast episode was born out of a LinkedIn discussion that came to the subject of security data fabric. So, this is very fascinating. We sort of hear people talk about pipelines and about channeling signals to SIEM, to detection response, but I haven't heard security fabric, security data fabric crisply defined. So what is this thing? Can you explain this technology and perhaps is there a market for it?

Yeah. So you use a lot of phrases there. I think my opinion is security data fabric. People, mostly vendors, are using that to kind of pitch or sell platforms built on top of security data pipelines. So a pipeline would be the technical implementation. The fabric part of it would be all the features you're adding on top of the data pipeline, which would just be, you know, moving the data, enriching the data. normalization, standardization, things like that. From a technology point of view, there's a lot of like movement here in the open source space. A lot of these systems are built on like Apache software, things like that. But the value add from a vendor would be the unique things they bring to the tech.

But if you go back to 1990s and think about people using syslog, the need to shovel telemetry from sources into some kind of product or an open-source project was there, but we didn't talk about security data fabric or pipelines. So, what is the difference between the modern era where we do need it seemingly and the classic era where we just, you know, syslog stuff from sources into the SIEM?

Yeah. I mean, that's the million-dollar question for a lot of these companies, I think, because syslog to SIEM or any proprietary protocol to SIEM, Mm-hmm. Is the same thing effectively. It is taking data from someplace and putting it somewhere else, and that's not different here. I think the difference here is now there is another component, which is the fabric or the data pipeline, whatever you want to call it, that sits in between the two. And so the question is, is there value there, right? Is there 10x, 100x value in like security use cases by having that sit there? And I've built my own systems that do this. I've looked at a lot of vendors that do this. I would say there are some that do it better than others.

Mm-hmm.

But I think this is coming up a lot today because of all this interest around decoupled SIEM or SIEM as individual components, right? So yeah, like any customer who is happy with their SIEM, in my opinion, probably doesn't need this because the SIEM usually just gives you a way that ingest all of your data. There's maybe more advanced use cases you could have if you have one of these systems in place. Maybe if you don't like your SIEM provider, this would be great for you. But it is kind of solving a problem that was already solved by a lot of SIEM vendors in a new way. Hmm.

Well, I guess filtering does come up even if you do use and like your traditional, like, monolithic SIEM, but you want some kind of extra degree of filtering before you hit the SIEM. Especially for SIMs that charge per gigabyte, right? Like presumably you want to not hit the cash machine.

Right.

You want some data to not be counted so you don't get charged for it. Is that a big part of the story? Or I think with some of the vendors you mentioned in the notes, it seemed to be a reasonably large part of the story, like preventing customers from overspending on a certain SIM.

Yeah. Yeah, I think there's a big part of this here where In my experience, some SIEM have filtering capabilities built into them. It's like a filter that's usually added pre-index that should save you some money. But a lot of vendors in this space, I think, have really made a name for themselves by saying, you know, we're going to save you money on your Splunk bill.

Yeah.

And to them, it makes a lot of sense because let's say I pay a company like Splunk $1 million a year, but I could buy this product that reduces my bill by $200K. Right? And even if I'm paying a new vendor some of that money, I'm still reducing my overall spend. So this is like a— I would say filtering is like a table stakes kind of use case for these systems where customers, people that use them, are just going to expect that it is reducing the noise in your event logs and in theory giving you more signal, more useful security events. But then also the side effect is like, Lowering the bill for your SIEM. But, you know, I personally, as like a security practitioner, think that is the most boring use case. Like everyone likes to save money, like our CFO loves it, right? But like, it doesn't actually help me do security better.

Yes.

To just filter logs. And so I'm personally not super interested in it as someone who does security every day.

Yeah, I have exactly the same take. It's like from architectural purity point of view, it's just silly. Like, I don't want the middleware between my middleware and logs. I mean, like, maybe it's an excessively cynical take, but ultimately, if I'm architecting collection from sources to analytics, I don't envision a separate box in there that says, reduce my bill.

Yeah.

I know that practically it's there, and of course, people make serious money doing that. But from an architectural purity point of view, that's just a little silly. That you have to pay for this box to reduce the cost of the other box.

Exactly. Yeah, it is silly. You know, my personal take on it, having been in the industry for a while, is that, you know, SIM vendors, they are not really incentivized to give you a lot of, you know, knobs to turn in this area. Like, they are incentivized to collect all your data and store it at sometimes an unreasonable cost to you. So that's kind of like the wedge some of these vendors have when they kind of enter this space, is they wedge in on the cost aspect and then sell you on the value add. Another aspect of filtering is usually discussed in the same breath as routing. So, ah, that's right. Don't remove all of your data, but maybe send some of the data that is less useful to a more affordable storage tier. Some SIEM vendors can do that. A lot can't.

And some can do it for some data only. That's also a tricky point. Like nobody can, nobody can route logs they picked from a database by doing a select star.

Absolutely.

Like if you select star logs from Oracle, then you can't route them as syslog. That's Like a lot of work, and it also would be very proprietary.

Exactly. It's really funny because when you kind of talk about these topics at a high level, it's like, yeah, everything is kind of the same, it supports all the same features. But then when you dig into the details, you're like, oh wait, you limit me on doing what? Like, okay, you charge me extra to do this? Like, it's— it is really kind of silly. And I think that's another area where like these companies come in and they're like, hey, don't worry about what your SIM vendor is charging you, we're gonna come in and at a much more reasonable cost and give you more control over your data. That's kind of the story there.

But there is, apart from filtering routing, there is also a degree of normalizing and perhaps even enrichment with some of them. So to me, if I, you know, as I sometimes joke about 3:00 AM, if somebody wakes me up at 3:00 AM and says, why buy these products? I would think about your boring cost reduction, volume reduction argument. But for a more nuanced take, then routing comes up. Routing by log type, routing by log type and source. I mean, all of this is fairly elaborate, especially with some products. But isn't there also a third pillar of this where the data becomes better? There is a degree of normalizing or enrichment that these types of security fabric, security data fabric tools provide, right?

Yeah.

Is there quality improvement, too?

There is. There is. This is where I've spent a lot of my effort over the past few years. is in this area. Like I said, if you focus on making the quality of the logs better, which produces faster threat hunting exercises, more efficient incident response investigations, detection quality should go up dramatically because it just becomes easier to write these things.

Rules writing becomes way easier. Like, talk about weeks to hours kind of easy, right?

Yeah. Yeah. I mean, it, it is really shocking when you see it. working. It's almost— I usually use the phrase like indistinguishable from magic when you see really good normalized data.

Yep.

And the funny thing is, like, you know, mentioned earlier, SIEM vendors aren't incentivized to give you many knobs to turn in your control of your data. It turns out that like the security vendors and system vendors that produce this data, they aren't really incentivized to give you good data either.

Yeah.

They just kind of like give you data and then let you figure it out in your SIEM or analytics platform. And this solves that problem. But yeah, I mean, there's like a few things that you can, I think, use as like a criteria to understand, like, am I doing this well? And so that would be like unified data modeling.

Yep.

I would be really scared of any product that doesn't use like a standardized, what I would call a standard unified data model.

Yes.

Like, yes, every SIEM product seems to have their own. with varying levels of success. And that, to me as a practitioner, that's kind of scary. The industry has recently settled on a few that I think are more popular than others. I would just urge anyone who's thinking about doing your own security analytics, or if you were thinking of starting your own company, just choose a standard that already exists or build on an existing standard. Everyone will be much happier. But I do want to say that unified data modeling is a core pillar of this. If you— that's the starting point to really good normalized data.

So, I have this— when this discussion gets to this point, which is, of course, very interesting, I get this depressing thought how we have devolved from the age of ArcSight and IntelliTactics and NetForensics in the early 2000s when everybody had a standard schema and there was even an attempt to make a standard back in the CEF days. And then somebody— everybody was indexing logs. in bulk. And it's almost like sometimes feels like you're doing archaeology and you discover the forgotten technology from a bygone era. And you say, whoa, what do you mean they had this in early 2000s? How can it be? And then you realize, yeah, they actually had single schema logs in early 2000s. And then we didn't for a while.

Yeah.

When search indexing-based vendors became popular. And then suddenly people are rediscovering this and digging in the dirt and saying, holy cow, they had this tech 20 years ago. We want this. Like, that's the vibe I'm getting sometimes.

It's really funny to me. I agree 100%. And I think the SIEM space, like any security analytics space, is just perpetually stuck in a cycle of doing the same thing every 5 years. And I don't know why, but I do agree. I think that the difference today is the volume of data, right? Like, that's the big difference. Like, now we're In the cloud, now we are dealing with, you know, tens, hundreds of SaaS applications in your large enterprises.

Mm-hmm.

It's no longer— we're like, we're collecting just a lot more data. So this problem is exacerbated by a significant amount. And so what that produces is like a need for simpler logs and the vendors aren't providing it. Like no vendor is going to reach out to 100 other vendors and say, hey, we should all like standardize on this one log format. That's why some of the standard schema frameworks have appeared over the past few years. And also why these data pipeline companies or data fabric companies have sprung up over the past few years as well.

So, is the security data fabric kind of a sign, maybe, I'm thinking while I'm talking, that we cannot all agree on the same schema, but we want the benefits of that? So, why don't I buy or deploy this tool that makes my, all my data look similar, even though we don't have a standard. Is that, I think it's not entirely fair, but is that close?

It's kind of like, yeah. I mean, it's a solution to that problem for sure. I mean, the reality is like any security practitioner will tell you like log quality varies across vendors, even vendors who do it well, the quality is not as good as it could be and logs are different. So I don't even mean things like where you put the IP address, the field you put it in is different. Like obviously that's different, but some logs don't have an IP address. Some logs don't have like a username.

Yep.

You can't do any identity attribution. So it's not that we should expect maybe vendors to all have like the same fit to the same schema, but like I'm a fan of the model where your schema has like normalized values, like IP, user, things like that, but also a place for the original event. That's really important because none of these models ever have enough place for context in them to give you the full picture.

Yes.

They're just like Shortcuts to draw conclusions. That's how I see it.

Right. Even in the game, to pick another example that possibly dates us, when you had the Windows event ID in the logs, you know, human, you know, maybe a dozen humans I know would immediately tell you, oh yeah, that one, sure, 512, I know what it is. But like, that's a tiny minority. Most other people want to see what the event ID is. So even that type of ID dereference is required, right, before you can make use of the logs. And that's, that's smart.

Absolutely. Yeah, and I think there's like— that's a great example because like your Windows forensics experts might memorize all that stuff.

Yeah.

But like not every SOC or security team is full of Windows experts, and I'm a big fan of using this kind of technology to lower the barrier to entry to do good analysis. If I could just read a description of the event in the log, like this Windows event is for something something something, instead of memorizing the number, It's a very simple improvement, but it is very helpful for, I'd say, diverse teams and skill sets.

Right. And then we also picked Windows. We didn't pick SAP or Salesforce or something else. Windows ultimately is, in many minds, for obvious reasons, it's used everywhere, but you would have other types of even more cryptic logs. Don't even get me started on Log4j, not for the reason of the incident, but for the reasons of the logs being excessively messy if they're produced by Java-based apps. I mean, Sorry to reminisce about the old times, but my first job, I think, was at an integration lab of a SIEM vendor. And this whole, how do I deal with Log4j, was just a nightmare. Again, how do you turn this into something useful? Well, you don't. You kind of parse and pray.

Yeah, yeah, I agree. I think there's something to be said about this is that because of how much data there is, not just in like volume, but also like diversity of data, all the sources, It is a significant amount of work to make all of this data make sense if you want to go down this path. And I think that's probably the value add of a lot of the emerging vendors in this space is it's like, why would you spend hours modeling your data, normalizing your data, enriching it even when you could just, you know, buy us and we'll do it for you?

I think that you've made it much more clear to me that the Fabric is kind of the crutch or a shim for the world that wants the single schema but can't have it.

In a sense, yeah. I think because— yeah, yeah, yeah, I agree. I mean, I think like, again, like there's 2 like core use cases. It's reduce costs and then it's normalize all your data. The 3rd use case, which is the most powerful one, is one I haven't really seen any vendor do exceptionally well, which is enrichment. So you have all of this data. Your data does not exist in a vacuum, right? Like you have EDR logs, you have network logs, you have SaaS application logs. Like these things don't exist in isolation. They all relate to one another. Enrichment brings them together. That's a big area of opportunity, I think, with these kinds of systems.

So in essence, I think that's— well, we can make, you know, kind of vague jokes about some of the vendors, but ultimately, if you— if a vendor shows up and says, You can shovel quite debatable logs that are hard to use for detection and response, but then the output, you'd get something your team could use.

Right.

I mean, I'd pay for it. I think that's a much cleaner, much crisper value proposition, and it would make the tech much more glamorous than the Splunk build reducer.

I agree. I think, like, this is the thing security practitioners would want. I mean, so, like, whenever you buy a product, right, there's, like, the product your CISO buys, then there's the product the practitioners want. Hopefully you're in line with one another. That's not true of every security team, but like, like the CISO wants to reduce and the CFO want to reduce the bill, but the security practitioners want data that looks better and makes their job easier to do. Right. 'Cause we're all very busy. So I agree 100%.

So I want to touch on 2 things as well. And I think one of them is kind of, if I am, if I have a team that's in alignment and they want to buy one of those or buy or procure, because some of it is open source, how would you think about the factors like what would they use to choose which one is right? You mentioned a few vendors, you mentioned a few open source projects. What key factors I would use to decide on the right one for me?

Yeah, I mean, when it comes to, in the case of either a vendor or an open source solution, you know, what are your data sources and where are you putting the data is important. So obviously, like, we want your, you probably want your EDR, your identity platform, your cloud service provider logs. You want them all in this kind of system. So you want that to be supported. That's a really easy one. It's like a checkbox kind of criteria. Yeah, that's easy. Harder criteria to actually determine just reading documentation from a vendor or an open source project is how easy is it to deploy and manage? This is essentially like, how much time am I going to have to spend on making sure it doesn't fall over one night on a weekend? Because This is like a very critical part of your security operations team, right? Like you want this to be a very reliable system because otherwise all the other processes kind of die off, unfortunately. And then it's also customization. Like this is maybe more for power users, but like, and this is probably where the divide between a vendor and an open source project comes in.

Mm-hmm.

Like if you have lots of custom applications, Like your vendor is probably less likely to support modeling that because it's a log format that they've never seen before and no one except you has. There's also customization from the point of view of maybe you don't like their data model and you want your own data model or maybe they don't.

That's a bad battle though. Like if you prefer the middleware slash Fabric data model to your SIEMs, but you still have your SIEM, is that really, like, is that battle worth fighting? I think that I'd rather jump off the window than fight that one. Or replace my state.

It's a challenge. I mean, like your SIEM. Yeah, we've had this. I've actually seen this challenge personally where like your SIEM expects data in their own custom format, but you want, maybe want one of the more standardized ones. Like I would walk away, like if I had the opportunity to walk away from a vendor, I probably would at that point. 'Cause I would rather just not even think about that problem.

Yes.

But I think like it really speaks to the level of customization. That's available. And then with open source, you know, the differentiator between a vendor and open source is the classic problem of like, with a vendor, someone is accountable to making sure it works and support you. With open source, you are dedicating some internal resources to making that happen. I would say, you know, what I always use to judge like open source projects is like 2 things. Like one is, is it giving me a capability that I can't buy? That's a great place to start.

Yeah.

Because like you just can't buy it, so you should. Start there. But also, if I could buy it, is it giving me like 10x value? That could be cost savings, that could be capabilities. I usually look at that and if it's like, yeah, we save like a dollar a day, you know, using open source. Well, that's probably not worth it compared to a vendor.

No.

But if you're saving, you know, $100,000, $1 million a year, that's a little different. And that's kind of how I would evaluate those 2 kinds of scenarios of who you're your solution provider is.

I think I want— well, I mean, to me, this is gold, and this is worth the price of a podcast on its own. But I think that one point I want to emphasize from what you said is if your security data fabric is less reliable than the SIEM, or the SIEM in this case, or whatever tooling, then you are in deep trouble because you're sort of paying to make your life worse.

Yeah.

And that's like, don't do that. I think this is also a little tricky to test because I mean, not that I was scarred for life by dealing with unreliable SIM collectors in my childhood, but yes, it has happened. Then SIM collectors would be unreliable, and then SIM vendors spent 20 years making them reliable. And then I put something in front of them and say, hey, this thing I need for these reasons.

Exactly.

But is this as reliable as a 20-year-old— well, maybe not 20 in case of some vendors, 10 years of improvement collector from a SIEM vendor versus something that a startup just wrote or an open source solution, right? So reliability to me is an interesting, tricky question.

Yeah. Yeah, I agree. It's really funny when you think about it, because like when you know enough technical detail to really evaluate these solutions, you realize how complex they are and how much risk you introduce by adding more.

Yes.

I mean, usually like risk doesn't get low, like risk doesn't go down when you add more to something, right? It always goes up.

Yeah, that's exactly right. Yes.

So that's true here where like if you had just like a source, your source was set up in your SIEM, that's like a one-to-one kind of connection.

Mm-hmm.

But then you're now here, you're introducing something in the middle that could have its own reliability issues. It could be misconfigured. So you're losing data. There's a lot of monitoring. So like we run our own open source projects where I work on this. And like a tier 1 feature from day 1 of that was reliability and monitoring. We want to know exactly what it's doing all the time. Because it's so important.

Yes. Which of course means telemetry from that system itself. And basically, if you dwell on this, which I think we don't have time to, you realize that there's a monitoring of that tool itself, performance. So you don't lose data, you don't out-of-sequence data, you don't corrupt data. Like, it's a very long list of requirements that stems from this thing better work.

Exactly. Yeah. And that's why most people would probably just go to a vendor who can provide that reliability guarantee. But as we've seen recently, it's It's not always true.

So I want to explore 2 things before we close. And one of them is there's this interesting connection between this technology, security data fabric, and this whole emerging federated data search thing that some people are trying to promote. I think some of the vendors, specifically Kribble in this case, offers both security data fabric and federated data search. So why are they connected? And are they connected? Like, I'm sure you have an opinion on this and I'd love to hear it.

I do. And I have to say, I don't fully understand the federated data search use case. Like, I understand it at a high level. Like, the idea is that you have data in multiple locations. It might be cheaper to store it in one place than another, and you have a single, like, search interface. Like, that's really easy to understand.

Mm-hmm.

What I don't understand are, like, the more tech— like, deeper questions. Like, I have data in multiple places, maybe even data I don't control. So, what happens if Like someone who controls that data just deletes it and I needed it.

Yeah.

Like what happens if your federated search tool is actually searching a SaaS API that only stores data for 7 days and I need it longer than 7 days? There's a lot of like nuance here that I think I'm— makes me concerned about this. And I would personally, as a practitioner, probably just focus on collecting data and storing it like affordably on my own. And there's also like, A big question about compliance, like when, who, who is the owner of this data? There's, there's so many questions, but I do think they're related because they are like the, if you want to call it like the core tenets of a decoupled SIEM, right? Is a pipeline or fabric and then a federated search capability. Yep.

I think that this, I mean, in my more cynical moment, I felt that federated search is trying to trick other people into paying for your logs. But it's ultimately kind of what you just said, but in a phrase in a more humorous way, like you can trick them into staying, saving logs, but they save logs for 7 days and you need them in 90 days. So, there's only so much tricking you can do and you also can't control people's budgets. You can call an API and say, increase the retention time from 7 to 90, but then what? Who pays for increased storage at the other end? I don't know. Calling API is easy, paying for 90 days versus 7 days is hard. Okay. In any case, I think that it's a fun thing to discuss over drinks and even if it's coffee and not beer, but I'm with you in terms of if you really need it, you're storing it. Like, that's kind of like, I don't think there's a way around.

Agreed.

That one. Yeah. So any fun futures for this sort of like security data fabric? Anything exciting you expect to happen? Is open source going to win? Is commercial going to win? Or will SIEM vendors eat it? Like any pointers?

Wow. There's a lot to talk about here. So like, I think, I think all of it. So I think like eventually the vendors will win over open source in this area because Oh. That is, this is a personal opinion. I've seen a lot less interest in open source over the past maybe 5 years. And that's probably because there's just a security startup for every problem now. They identify a problem and then they get a couple million dollars to fix it. So there's that. But I think like this can go in 2 areas. Like one, you've seen Cribble, their initial use case is save money on your Splunk bill. They are probably selling to a lot of security practitioners, but now they're I think moving into the IT space along with their search capability, I think like you're going to see a split toward more broader use cases, but also just kind of selling you the same thing, which is a SIEM experience, but it's— they call it something different. So you have a data pipeline, you have a federated search. It's actually basically just the same as a SIEM. I think— I really don't know if people will like this.

Mm-hmm.

That's the thing is like people like kind of like the SIM, although they like to hate it as well.

Yes.

Yeah. So I don't really— don't know if this whole approach will win out. Maybe we'll know in like 10 years, right? I mean, I think Cribble and companies of that ilk will probably see success, but will a decoupled SIM be the future? I really don't know. If it is, this is like a very important part of it.

Yes. The profound insight here is that If decoupled SIEM is the future, then it's a critical part of the future. But if decoupled SIEM is not the future, then this may be a side branch of technology. Like, that's the dilemma, right?

Exactly. Yeah.

Perfect. So, our traditional closing question is anything on recommended reading and maybe one tip on dealing with security data fabrics?

Yeah. So, recommended reading. There is a company who works in this space called Tarsal. I'm not You know, advocating for their product. I've never used it, but they do have a series of blog posts that kind of go into this topic. So if you would like to learn more about security data pipelines, that's probably the best place to start. I'm always happy to talk about it if you find me on LinkedIn. I love this topic. And one tip, I mean, I really just think that we have accepted bad data for too long, and it's time to just rethink like what your data could be. So for that reason, I would, you know, take a closer look at unified data modeling, take a closer look at enriching data before it hits your SIEM. And if you can't do that today, maybe you should start talking to either vendors or people in the industry about how they do it, because we've seen so much value add from tackling this problem where I work that I would recommend it to everyone.

Yes, that actually makes sense. Thank you very much for this. This is a fun discussion. And again, it's one of those podcasts when I feel like for another hour would do it justice. So thank you for being on the show.

Happy to be here.

And now we are at time. Thank you very much for listening, and of course, for subscribing. You can find this podcast at Google Podcasts, Apple Podcasts, Spotify, or wherever else you get your podcasts. Also, you can find us at our website, cloud.withgoogle.com/cloudsecurity, Please subscribe so that you don't miss episodes. You can follow us on Twitter, twitter.com/cloudsecpodcast. Your hosts are also on Twitter, @Anton_Chuvakin and @_TimPico. Tweet at us, email us, argue with us, and if you like or hate what we hear, we can invite you to the next episode. See you on the next Cloud Security Podcast episode.

Bye-bye.

Security.

Security.