Sharing in Cybersecurity: Simplifying Cyber with Grace Chi

Thanks for tuning in to Simplifying Cyber. I'm Aaron Pritz.

And I'm Cody Rivers.

And today we're here with Grace Qi, co-founder and COO of PulseDive. Welcome to the show, Grace.

Thanks for having me.

Awesome. Well, we were lucky enough to meet each other at Cyber Marketing Con in, uh, where were we? Philadelphia? Pittsburgh? Philadelphia. Yep. Philadelphia. You know, in, in December, which is the best time to be in Philadelphia. Just kidding.

Beautiful.

But, or not right after the Super Bowl because that got a little crazy up there. But anyway, we, we were really excited to talk to you because you're doing so many things for the cybersecurity community and specifically threat intel. But we wanted to focus our topic today on the state of sharing. And sharing is so important within cybersecurity and what you do at PulseDive obviously opens up a lot of that sharing. But I want to cover people, process, technology sides of sharing and tips for our listeners of how they can get more plugged into the community and most importantly, share information, receive information, and use it to help reduce the risks that we're all focused on. So let me let you give a little bit of a backstory on yourself and then we'll dive right into that topic.

Yeah, we'll keep it sweet and short. So PulseDive is a vendor in the space. We offer community-driven threat intelligence and everything we do is focused around creating frictionless threat intelligence solutions for growing teams. So that means whether you're the single IT person that somehow is also responsible for all of security in a small org, or you have an established large flourishing budget and group of people and processes, there is something that fits for you. And the focus, our keyword is frictionless because we think that in security there's a lot of technology, there's a lot of talent, there's a lot of processes, but it becomes, very burdensome in a lot of ways. So personally for me, I came in from a non-security background. I had some experience with security startups before, but as I jumped into the space of threat intelligence, which was not my domain expertise at all, and still today, right, if you ask for specific threat intelligence expertise, I'd pass you to a thousand people before I'd recommend myself. I work a lot on the business side, but I love the threat intel community so much. And I saw as I was getting in and there's all these shiny tools and cool, you know, exploits and lots of news about threats, at the end of the day, digging in deeper in my first couple years with PulseDive, I realized a lot of this was still happening on a very peer-to-peer, one-to-one basis. And so, as you mentioned before, the state of sharing, that ended up driving some of my research, which was completely not vendor-driven. It doesn't lead to any deals for me or anyone else I know about what is happening within the CTI space when it comes to networking. And by networking, not technical networking, and also not networking for a new job or to close a deal or something. Networking in terms of, I'm a threat intelligence practitioner or adjacent practitioner. I need to do better for my company and my role. How do I connect with people that way? And so that led to a huge report, and that's one of my passion topics that I do in conjunction related to Pulsedive, but really not a direct output of, you know, driving the Pulsedive business.

That's really cool. So for those that haven't seen that report, give us the high-level synopsis of what you cover and what, what they, You know, if they go download it now, what, what, what, what can they do with it?

Yeah. So these reports are fully ungated. I did 2 years of this because I wanted a quasi-lateral study over time of how there were trends that were changing. But my hypothesis going in was that while this quote unquote networking, CTI networking phrase was extremely valuable, it was underrepresented, under-acknowledged in the space. So people put a lot of emphasis on what tools you have and, you know, potentially what you've implemented, what your program looks like. A lot of the value that was driven was somebody who's giving you a just-in-time warning because you met them at an event or you connected with them through an ISACA sharing group, being able to get you ahead of a threat or just in time for a threat. But that was never really attributed to. So, I went in and I said, as I'm observing this field as a little bit of an outsider, how much of this is taking place in these one-to-one DMs or in these trust groups? Let's demystify that for everyone else who says they're interested in CTI, trying to build up a CTI program. Maybe their security manager who doesn't really understand or acknowledge that at all, to have awareness that this is happening and it can be a leverage strength and skill versus a liability.

Awesome. So Grace, this is very interesting to me. I think, you know, here in Indiana, myself and Aaron are involved with the ISACA group, but trying to get, get this started, and we learned of it from the national stage with like the, the Health ISACA, and they do a great job, but to your point earlier, they don't have the budget to get there for travel or the cost of membership. So what do you think you're doing? Like, that's, that's kind of new. And give us a kind of some insight of like connecting folks to your point of like, if they can't afford to get to an ISAC, or you've got Health ISAC and Financial ISAC. So how do you kind of take these little different communities and kind of distill them down and open the doors?

Yeah, that's a great point. And I'll step back into the research I had done, because one of the key points was what channels are are people connecting? ISACs and industry formal groups was one. There's social media, which for all its toxic junk, there's also still very much so value there. Reddit is obviously a fan favorite. And then there's also one-to-one and industry events and so forth. So I think there were 7 channels that I looked into, and you can find all this on blog.pulsedive.com if you look for the CTI networking report. And ISACs are really valuable because they're established, they have key members, membership players, they have funding, they have tooling. The truth of the matter is a lot of the companies out there that need to be plugged into the threat landscape and what's happening don't have the capability. And even if they had the money, right, they might not have the time because they're stretched very thin. And so what we saw in the research was that one-to-one and peer-to-peer relationships, so like unstructured, let's call it free and ungated in terms of who you're affiliated with, were always the most valuable, the most timely, the most, uh, actionable intelligence you could get. But outlets like ISACs, outlets like industry events, so a BSides conference, which is what, $5 to $50 max, being available and active online on social media within your comfort, those were all feeders to get you connected to the people that could then think about you when they see a piece of a puzzle that might impact you. And so ISACs are really valuable, but they are not one and only. I think they just come up as the first kind of obvious structured component, right, channel that somebody can get involved with. But I also hear a lot of times that it's great that these exist, but they are a gateway for then the practitioners that are member organizations to then talk to each other more directly too, versus just in the formal meetings and the formal settings.

Yeah, that's— so that kind of goes to my next question too, which I think is key. One thing we get, you know, you see a lot of threat intel like vulnerability —topics or tactics from, from bad actors. One thing that we get from virtually every client and potential client and just, just kind of colleagues in the industry are benchmark data, and particularly around like NIST assessments and that we get a lot of. And we have a lot of internals that we've developed because we have our clientele growing and benchmarking for, call it, small financial regional bank or something, or global pharma company under 40,000 employees. But that's one thing I see as a gap right now is that people are sometimes scared to share or don't have the avenue to share that data. And then also you don't want to be public because it's— I don't want to say, hey, I'm a 1.5 in recovery or something, so, uh, or recover. So I think, do you see anything like changing there or any ideas on like how that improves or how we kind of get access? Because benchmarking I think is also valid when you're looking to go fight for budget of like, this is where we stand against our peers or our co— you know, that data right now, at least I've not seen yet. And you could chime in, you know, that we've not seen freely available outside of things we have ourselves. Yeah.

And I think benchmarking, so that leads to some of the challenges that we looked at and some of the top challenges were legal liability, right? Your legal team does not want you sharing anything you've seen because that can cause all sorts of issues, even if that might have positive benefit of you sharing that data. Two, there's also TLP, right? So can you share something even within a trusted group? Obviously publicly is not always the best. And then also like fear, like reputation and just competitive advantage, right? Those types of external factors that have these security practitioners being like, I'm not allowed, I want to, I'm not allowed to. Unfortunately, I think the truth about benchmarking is with any sample size, a lot of data is not there. And where I'm witnessing in my research, it's more of an intuitive, like, have you guys seen this? I'm witnessing this issue. Are you also getting this? And I will not name any names, but recently I had a lot of people reaching out to me because I do love connecting people for the sake of them helping each other in their roles, being like, we got an alert from this vendor that, you know, one of our clients are compromised and we have no idea. There's no details. Help me because this is becoming an incident for our client, but we don't even know what the validity is, right? And so I can bring them saying, oh, I know that you have a shared vendor, or hey, I know somebody who used to work there. Help paint a picture. And so unfortunately, this is a very anecdotal point by point and not a large benchmark. But when it comes to these moments where there is a fire, how much you know somebody or can get connected to someone to give you pieces of the picture so you're not scrambling to write a full out RFI and doing research or using ChatGPT to make up some sort of essay for you is really important because timeliness is so key in all of these events.

Awesome. Grace, do you have any— obviously being so close to threat intel, even though I think you described yourself, you're not, you're, you're not a threat analyst and you came from more the business side, but you probably have come across a lot of stories of like how threat intel saved the day or prevented something. So do you have like a marquee story that you like to tell about that would maybe encourage those that are maybe starting to build their threat intel program or they're not quite there yet?

We had a workshop, so there's a few quotes and I like quoting the, the customers and the users versus like our own stories. Sure. Because I just feel like it's, way more interesting, right? There was a workshop that we had delivered. It was about, and you would love this, connecting threat intelligence and risk because the different audiences, same type of shared outcomes, a lot of overlap, but not in how things are executed. And I found this really great example of threat intelligence people being successful in a Reddit thread. And they were saying, when risk was coming to ask me about, you know, what policies, what processes, what structures, how to react to certain threats on the risk register that are coming, they were helping define that. So at a process level, having threat intelligence become this trusted advisor to help translate, right? Because threat researchers, threat intel analysts are very analytical, they can process, they can communicate well, is great from the program level. On a very discrete granular level, there are times where somebody who's working in a threat intel hat, maybe they're in a SOC, maybe they're dedicated threat intel on a team. I have some of these vignettes as well in my report. They'll say, "I got an alert over the weekend that some compromised network was related to our environment, and so I sent that over and I became the conduit of all these other sources that were providing granular raw data to the incident response team." And so this is a way in which you have this awareness, this almost like presence in the market of peers that they can reach out to you as an intel analyst, intel practitioner to then help be a channel to communicate information to the right team members at critical moments in time. Great.

All right, Grace, I'd like to pivot to, you know, at Reveal Risk, we do a lot of process-oriented work and I feel like we're big supporters of tech, but tech without process sometimes can result in more risk and more complications. So we spend a lot of time building and improving processes that make tech and cyber work better. So would like to talk about that specific to threat intel and threat intel programs. How does the role of process come into play and where have you seen examples where lack of process can make for not a good program result and maybe those that have invested in it, how do you invest in it if you're an early program adopter?

Yeah, with threat intelligence in particular, this was one of the biggest headaches that I witnessed working with our clients and users. And as an awful salesperson, sometimes I would say, "Hey, you don't need the thing that you came to us for." And I ended up writing a blog about this, about tips for tips, so tips for threat intelligence platforms. And it was about building a process around the procurement piece of getting a threat intel platform. One of the biggest mistakes, because let's be honest, products, threat intel, all security products, there's a lot of noise, a lot of new adjectives, a lot of new abbreviations that can be very confusing for customers. Sometimes it leads to somebody maybe outside of the organization or program itself demanding that, you know, you get a new tool or saying, hey, we need this, or Sometimes it goes bottom up where somebody says we need it, but then at the point of successful implementation, it's not resourced properly, right, to succeed. And so I had kind of 3 key tips, and one was building requirements. Threat intel folks are very good at requirements overall, but not always necessarily in procuring a technology like a platform for themselves. And so we talk about and we introduce like what a Moscow method is and that the must-have, nice-to-have, right, will-not-have type of tiering. As well as applying that to like the NIST Cybersecurity Framework, depending on how deep you want to go, and then assigning a point person. It is unbelievable how often people are like, let's buy a platform and then we just like turn it on and then we're going to walk away. And yeah, I say all the time, if you're not curating the data, if you don't have some sort of automation feeding unique internal telemetry or feeds, and then you're commenting and manipulating that data for your benefit and pulling that out, you don't need a platform. I would love the paycheck if you need, you know, I would love it if you wanted to procure either way to check a box, but you don't need this. You could do just as well with feeds and APIs, automation and other ways. And so what we hear a lot is that process-wise, they're not actually thinking about one, the requirements of what does it need to do day 1, day 30, day 60 versus shiny bells and whistles of what it could do in the future for undisclosed price point. And then two, do you have an engineer who's building the integrations? Do you have an analyst who's going in? Is it not just supporting a POC, but also supporting the maintenance of this product over time, which sounds so obvious, but it is something that I constantly am reminding and pushing our clients who are in the procurement cycle to think about and to really address early on. And then the last piece I had was confronting pricing very early and often. And there's no vendor bashing. It can be very confusing, but when you go in thinking positively or not fully understanding pricing models, scaling models, what integrations or what seats will cause more pricing, that can cause a lot of pain on both sides and end up being churn for, unfortunately for both sides as well, one year later during renewal.

Yeah. What, what, what you said really resonates on the buying tools and then walking away. And I've had conversations over the years of what drives that mentality. I think there's something in the reward system within an IT of how IT projects are awarded. And the word project doesn't necessarily imply the ongoing care and feeding and support and building it to scale. So I think part of it is a lot of us came up in IT and delivery and project execution, and there was always a new initiative each year, and it was easy to neglect last year's initiative. So I think same thing with threat intel programs, GRC programs, you got to plan past the launch because usually that's not— the launch isn't where the value is. That's usually like base MVP capabilities. And like you mentioned, integrations, use cases, you Who's going to consume the data? All of that is process work that's not going to come from any vendor out of a box, that there's no magic. And you can find people that have done it before and get those people. But on the flip side, like, don't expect, because it's going to be a big letdown, that the tool is going to automatically do everything that a 3-year-old program or a 5-year-old program would have been used out of that tool.

Yeah, exactly. And I think there's a lot of like the, what is it called, eating with your eyes and not with your stomach necessarily, where you have the budget but maybe not the time and energy to support. So at the end of the year, you don't have ROI and fingers are being pointed. So you're in a defensive position and that's the last thing we would want for any of our customers. And the other piece as well is being able to see, like being able to say that this is the product that we need that's going to deliver XYZ immediately and over time too. So it is a big sticking point. It is something we repeat to our customers over time and we almost challenge them in some ways. Like, can you answer these questions? Do you have a requirements sheet that you're allowed to share with us? That's like, you know, Cross out so we can tell you what we can and cannot do right now in terms of your use case. And if you don't have a use case, you know, once again, I will help them as much as I can, but there's no vendor. And if there's a vendor giving you these answers, they're just giving you the answers that they're gonna score 100 on. Yep.

Yeah. Grace, thank you too. So I love to give out tips to our listeners and kind of like, if I have nothing now, what's the next, my next 3 steps? And so thinking from like a, break into 3 phases of like gather, review, and action, right? I'm new program and new CISO, new person in security, we don't have a threat intel program yet, what are some like, here's 3 steps or kind of, you know, the first like 90 days I'd look into of setting up a threat intel program or even just to your point, I've got all these feeds, how do I gather, review, and action threat intel?

Yeah, and this is a challenge that even as a vendor I faced where I've gone to clients being like, I will give you a year of feed for free. Can you just one-to-one benchmark? We talked about benchmarking. Tell me what we give us, tell us what our false positives, tell us what our false negatives are. And there's just no capacity. You know, we've gone out and said, we'll give it to you. Can you do this? Hey, can you help us with this research? And the answers have still been very anecdotal, right? Piece by piece, we got some feedback. It helped us improve our algorithms and our processes, but it was never at that top level that we'd hope for, that academic level of research. But I would say if you have a security environment and you're considering threat intelligence, call me old-fashioned, but I'm like, where are your current pain points right now? Where are you wasting too much time? Where do you need capabilities that a single resource that you have could peel off some hours, or there is, talent in your team that is interested, right? Being, being able to nurture a SOC analyst or, you know, detection analyst or somebody, malware reverse engineer who likes intelligence, to then think about it from that level and then do research and context and look at what the capabilities are is really important versus thinking, well, we first need the tool and then all these crazy commercial feeds and then we need to join an ISAC all at the same time because you won't have the structures to support it underneath to get your ROI, which is the most important thing, right? The ROI that then reduces your risk.

Yeah, I think from like the— if I'm thinking of like strategic, operational, and tactical, kind of like those, those different tiers at the tactical level. Yeah, we see an increase in Citrix vulnerabilities or RDP and manufacturing or, you know, some like legacy tech or OT stuff. And then I think at like strategic level of like the same threat intel to two different people yields two different outcomes. And so how do you kind of delineate what goes to my tech tactical team for the firefighting, and then what goes to my executives for additional budget or capacity planning and those kind of things. And I think that's where I think a lot of our, at least some of our clientele have struggled of like they get a feed from their MDR provider or, you know, or maybe they've got Recorded Future or one of those type, type of services. But then it's like, I don't know how to change the narrative and push it to the right person to then warrant action. I think to your point earlier, that's kind of where it's hard to then at the end of the year, assign an ROI to it. Did we get value from it? Do we just not use it? Do we need to action it better? And I think that's a lot of our findings we see in our NIST assessments with clients is like, I've got threat intel, but I need to action it better, but I don't know how to action it better. Right.

And the, the most granular is like, we need threat intel, and then they get an abuse.chv and they're like, okay, we have threat intel now. Right? Like, and that's fine. That's a starting point. It is a decent high fidelity open source feed. But where I do see a huge gap, and it really takes a special type of advocate in the organization and leadership support is being able to speak to that risk and executive team. I think the tactical support and the even operational level intelligence, there is good groundwork there. But when it comes to the even the intel cycle, right, of collection and processing and analysis, strong communication, I have my thoughts about it. I think it falls apart in a lot of ways and then getting the feedback right. So communicating findings in the right format to these different levels of teams and then actually pulling in the feedback from the board, from the CISO, from the CEO, right, about risks, what they're looking for, and then how to bring that back through the cycle again is something that's come through a lot of maturation, I would say, as just an area of focus even in the last 5 years. Yeah, I love to—

I'm going to ask you a question about dos and don'ts as you're building a threat intel program. And I mean, this is— we talk about sharing and benchmarking, but if I can help a client or when I was on the corporate side, if I could talk to somebody at an ISAC that has done something that I'm about to do and they have like, tell me your pitfalls, what would you do again differently? Or tell me what worked. Give me your thoughts on building a threat intel program and the dos and don'ts for someone just getting started.

That's a big, that's a tough one. And I don't wanna be the security person that says it depends on what you have, but obviously it's true, right? Depends on who's on your team, what kind of buy-in you have from leadership, what kind of support you have, right within your existing talent and technology stack. But I would say the one big miss that I see is like industry-level and peer-level research. I think what we hear in the news cycle is, oh my God, ransomware, why? Oh, there's a war in Ukraine. And then everyone and all the leadership is like, tell me about this when it may or may not even be relevant. So Scott Small over at Tidal Intelligence, and I think Simone over at Orange Cyber Defense, also delivered a workshop about how to do like threat profiling for your organization in a realistic way versus drowning in spreadsheets with data that you're just making up, right? So being able to look at what your threat profile looks like, as light as you have it, but definitely creating that kind of dossier for yourself is important. The other piece of don'ts is don't just try to configure every feed. It's going to be noisy. It's gonna cause a lot of headache. It may cost a lot of money that your leadership isn't really gonna understand the quality of. So it's much more of a be selective and implement over time. So you can actually begin to see what feeds are providing valuable enrichment for you, what types of data, what type of IoCs are important, as well as like what types of potential like tooling or commercial vendors you're purchasing from that have the capabilities that you need in smaller batches. Because we do know that there are teams that are like, oh, we have an open source platform, like a Mistral or an OpenSea, just add everything. And then they're spending their time culling or just drowning in alerts, not really thinking at that strategic level of what is providing value. And then of course, selfishly, I'm going to say getting engaged in the network, whether that's a trust group, whether that's just people that you've known over time that are working on similar projects or that can offer you insight into new tools and new techniques that are helpful to keep you on track of your news and just be able to think about you, that has compounding value over time as well, even though it does take time and effort right from day one to build up.

That's awesome. I'm curious your thoughts on threat modeling and what program CTI teams or cyber teams in general should be doing with threat modeling. And when I was a Six Sigma black belt on the corporate side, we had a tool called FMEA, Failure Mode and Effects Analysis, and it was kind of very similar. That was for analyzing a process and anticipating what could go wrong, but threat modeling and then kind of using intel to validate or to to focus you in, I think is super powerful. Give me your thoughts on that. And is that— is threat modeling something you advocate for?

So I will say that is it in theory something I advocate for? Obviously there are threat models out there, but I think somewhere where I do see the pitfall is I see assessments from certain vendors being like, oh, MITRE ATT&CK, right? We prevented 100%. Therefore, you know, we are winning as a vendor. And I think that's a mistake where let's say you've modeled something out and you have detections that fit what is already known, unfortunately it gives you a false sense of security in a way, and it becomes a sales tactic. So I would pass it off to someone much smarter to myself to say, hey, use this specific toolset or this specific GitHub repo to guide you. But I'm not anti, it just, I think that in a lot of ways it becomes convoluted with sales processes, like a lot of, uh, a lot.

And I think threat model is to model something, you have to know it. Whereas threat intel, you could be learning something that's a zero-day or an unknown. So I think maybe there's a place for it, but not being overly complacent that, hey, we've modeled our threats, so we're good. Or whether it's a corporate practitioner saying that or thinking that, or if it's a vendor trying to claim turf in their sales pitch.

Yeah. And I think the other piece that this reminds me of is like the dos and don'ts. When I talk to or just witness, right, conversations, discourse online, a lot of threat intelligence conversations used to be about external stuff. Like, what can I buy externally? What can I, get extra, what tool can I add? And not enough time and effort was focused on internally, how much data can you track? What can you learn? What have you observed? How do you structure that into intelligence in its own right that is then enriched by, right, external sources? So I do think that that's another piece where having a threat profile, being able to understand your threat model, being able to understand, you know, your risk appetite related to threat intelligence is important where we need to not only look outwards, but make sure that the internal stuff, which is really valuable, that no one, right, will be able to just sell to you in a feed in a CSO SV or STIX/TAXII format will be able to compound, right? Force multiply any tool, any process, any person you hire on top. Awesome.

Yeah, I think so. Grace, this has been phenomenal conversation. We like to always kind of at the end, kind of end with like a fun fact or something about a fun story. So for those out there who never met, I'm certain most people have never met before on our podcast until today. Um, what is a fun Grace fact that someone may not know off the street, you know, it could be history, it could be a fun talent, but is this a reveal?

Are we, is this going to be some intel on Grace that the community doesn't know? My goodness.

There's some intel that's more I've shared before in the past, even if they don't know me, but this one is fun. I don't talk about that much. I do have some watercolor illustrations published in an art history textbook. Oh, there. That's pretty neat.

How long have you been painting?

My whole life. I think back in the day I was learning piano and I hated it. And my mom being so smart, she was like, that's okay, you don't have to do this because I was dragging my feet every day. And she said, then pick something else you wanna do. And I said art. And then I think because as a child I picked it, I had, I kept with it much better than being told, right? Yeah. That idea of choosing to do something versus being forced to do something. And so ever since then I would just take so many art courses. I still practice, I take commissions once in a while, but it's more just for fun. It's just a way to get away from the screen. That's awesome.

Well, very, very cool fact.

Now we're gonna have to, uh, do recon to find a piece and maybe we'll land that in the, the post cover of this podcast.

So if we can find it, maybe I'll share something with you.

We'll see. Oh, there we go. Well, interesting.

First, see if we can find it. That would be the security answer. And then if, if we fail, if, if our pen test does not work on whatever the public domain this is, then we'll ask it for you.

Okay, perfect. Awesome.

Well, Grace, this has been an awesome conversation. Really appreciate you coming on to share with our community and, and talking about sharing and intelligence. So I really wish you best of luck and appreciate your time.

Yeah, thanks so much for having me again. Have a good week.