EP286 Building an AI-pilled, solo vibe-coded, Clickhouse-based SIEM with Dan Lussier

Security.

Hi there. Welcome to the Cloud Security Podcast by Google. Thanks for joining us today. Your hosts here today are myself, Tim Peacock, Group PM on Google SecOps, and of course, the one and only Kyle Champlin, another PM on Google SecOps and good friend of both me and the show. Kyle, thank you for joining us as a special co-host today. Listeners, you can get this podcast wherever you get your podcasts, as well as at our website.

youtube.com/@CloudSecPodcast.

If you enjoy our content and want it delivered to you piping hot, please do hit that subscribe button. You can follow the show, argue with us and the rest of all our Cloud Security Podcast listeners on our LinkedIn page. Kyle, what did we talk about today? It was like AI, ClickHouse. What are we doing here?

Oh yeah. Well, so definitely one of my favorite topics of all time is just how do you deal with large amounts of operational data? And specifically we talked about Nano, which is a SIEM somebody literally built apparently in 2 weeks at the end of the year.

By themselves. With AI.

Yeah, I guess in conjunction with AI, but certainly I would say there was a singular vision behind the problems to be solved, which was pretty amazing. I think taking the angle of must be fast at scale and must be cost-effective was kind of an interesting take. It really strips down what I thought sim was at this juncture in 2026 after multiple decades of different permutations, architectures, etc. So I could have gone on for hours.

Kyle, how long have you been doing sim? When did you first work on a SIEM?

That's a great question. So on the product side, my first time working on a SIEM directly and sort of owning it was at Splunk and was specifically Enterprise Security, or ES as we like to call it. And that was back in like 2015, 2016 timeframe. So it's been over a decade and things have changed.

Yeah, new world, isn't it?

It is, it is totally a new world. And Dan in particular, you know, talked a little bit about having, I think, lived through each wave. I think All the way back from, if you count ArcSight as kind of wave 1.

I'd say that's wave 1.

Splunk would be wave 2. And you could argue SecOps is maybe in between waves right now, but Panther would be wave 3. The themes, the core ideas of what we were trying to solve were a little bit different each time, or maybe a lot different.

Maybe a lot different. Well, listeners, without any further ado, let's turn things over to today's guest. Today we're joined by longtime collaborator, good friend, Dan Lussier, the founder of Nano. Dan, thank you so much for joining us. You've been in the security operations world for a long time, and you've done something I'm blown away by, which— spoiler alert, listeners— Dan has vibe-coded an entire sim.

Yeah.

What led to this? What was the moment where you said, you know what, I've had it, I'm just gonna build what I want?

I would say probably for the last 2 years or so, I've been just kind of building anything and everything that would come to mind. Like, some of my earlier projects were building a C2 platform in Python just to kind of see what it would be like if I could bypass EDRs and stuff. And it did. So I knew kind of then that there was power to be had there. And in those days, it was still like copy and paste the code out of ChatGPT, or there was no harnesses, no agents. But yeah, I mean, that was like the ticket for me was I've been doing this for a while. And then around December of this past year, I was laying in bed reading my Google feed and it said something about Postgres introducing a new extension that would be really, really good for keyword searches. And I was like, you know what? I bet I could build a SIEM. And I bet that I could do it on Postgres. Now, I'm very familiar with Postgres. I kind of knew when I was thinking that, oh, it'll never happen because it just doesn't scale well for SIEM, generally speaking, large data. So I went for it anyway, though. So I started building and built, built, built. And then, yeah, eventually got to the point where I was like, oh, wait a minute, Postgres really isn't the ticket for this. Once I started blasting like, you know, 50 to 100 terabytes of data into it. It didn't like that. So that's when I started poking around, looked at DuckDB, looked at ClickHouse, looked at Apache solutions that are out there. That was kind of like the pivot point for me. And I spent like the— I had 2 weeks off from work at the end of December, and I spent that entire time pretty much architecting how I wanted it to look, what I wanted the feel to be, how fast I wanted it, how little resources you could use while still having, you know, like everything you want, right? Like, To me, one of the biggest things is like, how much can I deploy a full SIEM in a package on like 2 vCPU with 4 gigs of memory and still ingest 10 to 20 gigs of data per day and not keel over?

I just wanna touch on one thing you said there, which is you had 2 weeks off at the end of the year and you decided to spend it building a SIEM.

Oh yeah, totally.

There might be something wrong with you.

Yeah, no, there is, there is many things I would say.

All the right things are wrong with Dan, for sure. And I'm curious, so you, you've— I think the 3 of us actually largely have been in like the broader sim space for a while now. So to kind of land on sim as the thing, I want to unpack that if I could with you. Like, why not just try to fix something? Like, maybe if there's open source stuff, there's a lot of open source sims out there. Why not start there? Like, why build it literally from scratch, which is amazing, by the way, and on hard mode, by the way? So you're using Rust and ClickHouse.

So yeah, so There's 2 things you need to know about me. One, I'm absolutely insane when it comes to wanting to just build new things and see what it looks like and just beating myself up when it's not working and trying to argue with AI agents for some reason. And the second is I really, in my head, always kind of have a design of exactly what I want and what I would love to see in a product, right? So I've often— Tim, we did a lot of meetings in the SecOps days where we would talk about You know, here's feedback, feedback, feedback. So those were all, like, things that I really wanted to see in a platform. Some made it, some didn't. And in my head, I was like, I know exactly what I want to see kind of get delivered into a product. And that's really what helped steer that direction for, like, going down the SIEM path. In addition to that, I absolutely love dealing with logging, dealing with telemetry, being able to write detection rules, and then also kind of bolting on all of the, you know, bleeding-edge AI stuff that's coming out right now, obviously, is very impactful to these type of tools. So, I saw what a lot of the mainstays are doing right now with it, and I was like, cool, that's what they're going to do. I'm going to do it the way that would help me in a platform. It's a little different than kind of the rest, but it's achieving what I wanted.

So cool.

So you did this all by yourself and you didn't raise money. How much did you spend on the tokens for this? How big is the hole on your credit card?

So you got to figure, and I wrote these stats down, hold on.

Oh, yeah. Nice.

So, Claude Code Max at $200 a month, Codex Pro at $100 a month. So, do that over the past 6 months or so and still paying today, obviously have that. And then, in addition to that, infrastructure costs, right? So, like, you can't build a product and not stand it up in Kubernetes and actually see what it's gonna look like and scale it. You just throw it on a VM, it's not gonna be realistic to the real world. So, Yeah, I would say I'm probably all in around $7,000 out of pocket.

Okay, so hang on. I've been in meetings at Google that easily cost that much in the span of about 30 minutes, but you've gone from zero to a working sim for 7 grand.

Yeah.

Are you, are you still connected to reality or are you just like totally AI-filled now?

There's definitely a mix there. Not, not just this project, even in my normal 9 to 5 is still very AI is just a lot of the things. It just speeds up stuff so much, right? I was already kind of ahead. I've been principal engineer. I've already been kind of on that higher level. And then when you throw AI into the mix, you can take that and say, all right, well, things that I was already, you know, take me 4 hours now take me 10 minutes. So like now I, and then I can automate it and replicate it. And yeah, you could just, so the AI pill is definitely being swallowed, I would say. But with that said, I still like to get outdoors and enjoy life outside of AI.

That's a relief. I'm glad to hear that.

Yeah, glad you're touching grass, as they say. So, you know, let's actually continue on the speed theme. So one of the things that's kind of a marquee part of Nano is just what you would refer to as sub-second search speed. But on balance, you actually put this out there, you know, you're looking for a cost efficiency thing. So those things have to kind of be both in tension and also in balance. Is that where you think the big sort of innovation and value lies for customers? Is it in Pivot, the AI assistant that you've built? Sort of like, how much is this like a reaction to what I think we can all agree are largely failings of the SIEM category to deliver like a really amazing user experience? And like, and maybe a more interesting question, I'll give full credit to Tim, is like, how does writing stuff in plain English change the game? Does it really for analysts who have spent years in like domain-specific languages? Like, how does that all kind of hold together for you?

So for the first part of there around speed and reliability, right? So those are like the two things that I've been doing this for a long time. You guys been doing this for a long time. I have POC'd and POV'd almost every sim on the market. How many times that you get into one of those tenants and like day one, it's amazing. Like, wow, it's beautiful. It's got all the speed, everything. And then, you know, a month later, it's like, oh, well, that sub-second search is now 30 seconds or 20, something like that. And that's just so common, right? I haven't seen really a platform that has gotten away with— it's just the realistic feel of it, right? The databases get larger, everything gets bigger, and you need to scale for it properly, right? And I know users themselves will be the bad guys in this situation where they're like, hey, I'm only gonna send 5 gigs per day, and then they send 5 terabytes, you know, per hour.

And 4 of those are, you know, VPC flow logs.

Right. Yeah. Which they'll never look at. Yeah. So no, I like that. The speed part, and that's kind of why I went with the ClickHouse thing, I found it to be very fast. And even after dumping in 100 terabytes of data over 3 to 4 months, 5 months into the same tenant, even with— you still have to obviously put the resources up at that point. It's no longer like a 2x4, but it can handle it and still come back with fast searches. And to me, that was like the selling point for me. Like, wow, I can really have a quick, quick sim. It's always running, it's always fast, it's not choking out all the time. I would say the most choke that I've seen so far is when I push an update to a new query that AI wrote, and for some reason it decides to like rebuild all the materialized views in ClickHouse, and suddenly we're out of memory on everything. So, we don't do that anymore, but in the early days it was happening quite a bit as I was getting familiar with ClickHouse. So, the second part of your question, what was it?

Oh, I actually wanted to drill in a little bit on Pivot. So, there's kind of like, so Tim said, hey, AI build, like there's a lot of value, I think, to still be discovered in the use of AI. But I do think it has to be paired with, as you alluded to, like a user experience that scales for sure. But let's drill into maybe Pivot and like natural language. By the way, as you just alluded to, there's a sort of DevOps moment waiting for every single, like, you know, machine-generated query that you're like, oh, wait a minute.

Yeah.

This just rewrote my entire database or, you know, everything to dev null or, yeah. So I'm curious, just sort of like maybe even the balance of like, the dangerous output, potentially dangerous output, operationally speaking, and the value it can bring to folks that don't want to learn SQL or don't want to learn, you know, a specific dialect of a DSL.

Yep. So I would say, like, and I've done a lot of migrations over the years, the biggest challenge is always like day 1 with many SOC analysts and many incident responders, right? If you don't knock it out of the park on day 1 when they get into that platform, they pretty much have a bad taste in their mouth for the remainder. Like, they're not going to like the product. They're not going to want to convert with you because often you're bringing somebody away from a product that they may like, right? Like, oh, I really love that solution, and now you're trying to put me in something else. I'm already upset there. So how do you help that, right? And I think that the natural language is a huge one. I know with SecOps, it made a big difference, right? Like, having that natural language in there because a lot of people didn't really fundamentally understand the query capability. You know, on day one, whereas, you know, natural language is like, here you go, just say what you want.

A lot of people didn't understand the query capability. That's one of the nicest things anybody's ever said about using YL2 for search.

I'm trying to be very safe and political here.

I think the heart of it though, as you pointed to, is it's kind of, especially this day and age, 2026 and SIEM, I mean, it's kind of, you're always coming from something else, right?

Yes.

Why it was kind of mind-blowing that when Tim shared the article, your sort of launch site, and it's like, wait, somebody built a sim in 2026? Like, who's this crazy person? We gotta talk to him immediately. So, that's super helpful, for sure.

So, I wanna pick on your architecture because we've had so many architecture debates on this podcast over the years, which I never thought my podcast would be a podcast for arguing about sim architecture. That was not on my agenda. But you went with a pretty tightly coupled design here. It's like full stack. Why'd you do that instead of, say, you know, not that? Why did you go with the coupled as opposed to the decoupled?

It's not to say that you can't decouple parts of it. Shortly after I launched, I was immediately meeting with potential customers that were like, hey, we just want to have our ClickHouse in our ingestion pipeline. Can you do that? And I'm like, well, yeah, I mean, it's just the SIEM is just something that can sit in the middle. So detaching some of those components is not the end of the world. So it's already been done for me. But the core of it was, I often look at these solutions and see, you know, like SecOps and stuff, they're fully together. And it's something that I always liked, right? Be it Splunk or others, they all came to that market of like, you're going to have this centralized location, you're going to have this schema, you're going to follow this. And then down the road, things benefit from it, right? The AI benefits from it. Everything kind of benefits from a structured setup. So that was kind of how I landed on like—

Right on. I do want to, just to continue the debate, so I'm a data nerd for sure.

That's why I chose Kyle as host for this episode, by the way. I was like, who can really get into it with Dan about this?

One name.

It was always going to be Kyle for this.

I appreciate that. It's kind of a dubious honor, but I certainly appreciate it. Being the data nerd, not being on the podcast. Podcast, it's wonderful to be here. So ClickHouse is interesting to me. So can I just, full disclosure, I worked at a startup before coming to Google that actually built on top of Snowflake. So we had this sort of very abstracted, I'd argue, very nice, elegant UI and set of tools to interface with. But under the hood, the engine was Snowflake, but we fully managed that. It was completely opaque to the user that that was on the backend. And there's a lot of benefits that I found to doing that. But why ClickHouse specifically? And I'm curious sort of what your learnings are there. Maybe we'll go toe-to-toe on some of the finer points. I don't know how much time we'll be able to do that.

So ClickHouse in particular, What I wanted to do was similar to kind of the SecOps did with the— you're gonna have like these highly indexed fields, right? And then the rest of it is gonna be in like a JSON store, right? Where you'd be able to access it. And ClickHouse in particular has really good means of like the Bloom filters and whatnot across those JSON filters to still have really fast reads. So when you're doing your searches and it's going through and going through the granulars to find, you know, like exactly, you know, that hit or mark for keyword searching, because that's like, to me, Keyword searching is like a core component to a SIEM, and some don't have it, some do. But like, there's so many times the analyst goes in and is like, I just have this word, or I have this thing I want to look up. I don't want to know the before and after. And ClickHouse, with the way I implemented it at least, just really works well in that regard, right? Like having the— I'll like the— your core fields like IP addresses and things like that are mapped perfectly. I have UDM, and then I have OCSF is also supported now. So like having kind of both of them Because I got tons of feedback right after release where everybody's like, why didn't you just use OCSF? So yeah, I did that. But yeah, being able to kind of— that's why I like ClickHouse was, and you can run it relatively lean and still have a pretty good backend operational.

Yeah, that makes sense. And I think maybe to unpack that a bit more, I try to think of them as kind of 2 buckets. You alluded to this. There's kind of a world in which you could bring your own. So if the customer wanted to repatriate and run ClickHouse on-prem, they could do that, right? Was that in the calculus or that was more an afterthought or just kind of like, because you seem to be very into the architecture, which thank goodness, but it's, some of those decisions are also as much like where it actually runs and it gets into other discussions, like not just compliance, but sovereignty and all that. Was that part of it?

Sovereignty was a big one.

Oh, interesting.

Yeah. So, sovereignty, it was, so when I was building it, it wasn't on my radar. I was more focused on like, here's this bundled solution, deploy it how you will, right? Or I can deploy it for you through my deployment scripts. Like you choose. You put it in whatever cloud environment you want, the full stack will be there for you. And then when I started meeting with some potential customers, they were like, oh, well, you know, I want to be able to like air gap this or potentially put it into the data sovereignty and having it, you know, localized to just my region. So that's when I was like, all right, cool, we can totally do that. Because the SIEM had already been— it's not like it was decoupled, but it wasn't hard to decouple it, right? Like the vector part was here, the, you know, ClickHouse is here. Hey, you want to bring your own? Okay, we just switch it up and you just use an environment variable or whatever to get your data to the proper location.

Right on. So kind of like a hidden bonus feature.

Yeah, yeah.

Luckily, right?

Because I asked AI, I was like, well, I was just looking for converting this and adding this. And it was like, oh, no problem. We built it so it could handle that.

Yeah, yeah. So you got to take credit for the AI answer.

No, I mean, Dan is not the first CloudSec Podcast guy to sing the virtues of ClickHouse. We had Alan from RunReveal like 50 episodes ago, like a whole year ago. We already talked about ClickHouse on the show, so that seems like a pretty solid choice these days. I want to talk a little bit about the kind of analyst experience side of things where, you know, the mapping of IPs to owners, the looking up asset criticality, this takes a ton of time for a lot of people. Is your AI assistant doing all that as well? Because, you know, we've got Tina in SecOps doing, you know, triage investigation agent. We named her Tina.

Oh, is that what she's named now?

It's Tina? Yeah, yeah, internally we call her Tina. Yeah, yeah, yeah. And then, you know, she's got siblings that are still cooking.

That's true.

But we do a lot of that stuff. And so I'm wondering, how much of that have you imbued into your AI and how are you thinking about the line between helpful context gathering versus maybe actually making changes and doing things that might be risky?

Yeah. So Pivot itself will take, because the core build of the highly indexed fields and whatnot and kind of logic behind it is I have something called identity resolve, which will map either against DHCP, but I've seen just throughout my years that DHCP can be a great challenge in actually getting that telemetry from most organizations. Especially smaller, you're pretty much, you know, out of the water for there. So I map it primarily to EDR events. So like a Sysmon, Defender, CrowdStrike, whatever, you know, that's where you're going to get kind of that matching. It looks at DHCP as like a first class, and then, you know, EDR is your second. So if you have DHCP, it'll pull that first. And then Pivot can utilize that, right? So when it gets, you know, essentially you'll have detections are firing, detections eventually bubble up to an alert, alert turns into a case, right?

Yeah.

And then you have your cases kind of just sitting out there and they're beginning their triage. I use cheap models for the initial triage because the cost for AI is astronomical, right? In these scenarios, when you're looking at, I want to run AI analysis across my 500 cases, 1,000 cases per day, the top-tier models are going to just blow through an AI budget within seconds. So the focus is Pivot goes through and says, all right, I got this case, I'm going to do my analysis. Another case comes in associated to it, another alert comes in associated to it, whatever. It continues to go through those. It looks across your historical cases, as well, to see if in the last 30 days any false positives, true positives, anything looks similar. So it's doing all of that analysis. It's branching out and doing searches automatically, as well. So like, hey, we're going to go back, you know, 30 minutes here, ahead 30 minutes from the time. So it does like a plus and minus there.

Yeah.

But all of that said, and I'm sure you guys feel the same way, AI is not at the point where it's going to say case closed. This alert is done, case closed. Like, I don't know how you would ever say to, you know, a regulator or something that says, hey, how did this get missed? Oh, AI said, right? And that's a huge challenge, right? And even when AI becomes fantastic, Fable, Mythos, everybody else, right? Like, it's still going to be a challenge in that if you miss something and then somebody comes and says, hey, how did you miss that? So what do I now go and sue Google because Gemini decided to, you know, like, or, you know, Tina's decided to tell me something. So like, you know, that is the challenge I think is just, yeah. So I have options for it. You can toggle it on if you want it to answer completely for you, but the main toggle by default just kind of puts it higher and changes criticality based on what its analysis is. So yeah.

Yeah.

This is a surprisingly cautious AI triage answer. for somebody who is as AI-pilled as you seem to be. So this is unexpected and fascinating. I love it.

Yeah, for sure. You could almost just say there's just not enough case law in terms of how these things might be.

Boo!

That's a great pun. I'm jealous.

I appreciate that. I'll have a few more for the rest of the show.

Please, please.

So I want to hearken back to— it's hard to categorize these as waves. You could argue maybe, well, what is now SecOps or Chronicle at the time, even Panthers or some of these other folks were maybe like a wave 3 of SIEMs. So I'll call them wave 3. People will argue in the comments, I'm sure. Apologies ahead of time. And shout out to the friend of the show, Jack, I believe. When we think about Panther specifically, they had a really strong point of view of like, hey, detections as code is like the only way to make this work at scale, and I think sort of to handle a bunch of emerging kind of what we'll call themes in the SIEM space. So it was like, you write some Python, gets deployed like normal software through CI/CD pipelines, Jenkins, whatever. And I do think like that works if you do— if you're sort of you have a lot of engineers. It basically is a simple way to do it, but not necessarily a lot of analysts. And so back to this simplicity of that first experience, I guess you could say an analyst never forgets. There's your second one, Tim. How do you think about optimizing between— that's a very powerful expressive thing. Python's very expressive as a language. Usually DSLs are not as expressive. And balance that with ease of use, visual editors, or maybe in this world where we're all using MCP servers to live our lives, Sort of where are you in that spectrum? How are you thinking about it in terms of Nano and how you're sort of implementing it from a product perspective?

Sure.

So first, for the Panther side of things, those guys are great, right? Like they have like one of the best AI implementations I've seen in a product and like their detection as code is like top-notch. So a very cool platform over there. From my perspective, I love detection as code. It was like so much of what I put into when I was kind of like working back and forth with David French and whatnot on the Google side when I was doing that.

Shout out to Frenchy.

Yeah, Frenchy, that's right. So yeah, detections code to me is like one of those— I hold it very, very close to my heart and something that I love to do, but I fully understand who can do it and who can't. I would say compared to like a year ago, we're much closer to allowing smaller teams to be like, go at it, right?

Sure.

If they have access to a Claude code or an Anti-Gravity or something on their endpoint, they can hook it to an MCP server and be very successful. Right? Like, if the MCP server, which, you know, for Nano, that's what I did with mine, it calls all the prevalent APIs and it can just, like, do the whole rule for you, including the points of, like, getting it into GitHub, you know, having your peer analysis and all those things happen. And I think that that is what's going to be powerful over the coming, you know, like, upcoming months is that people are more and more so getting hands-on with these agents and they can start to get familiar with how to use them. And 100%, this ability to go in and just say, hey, I got to detect Kerberoasting, you know, and can you help me write that rule? So instead of them having to go on Google search, what are the event IDs? What's this? What's that? They can just say that, and it'll kind of pull through and write it for them, right, based off of whatever their SIEM platform is. And I think most SIEMs right now are kind of really heading in this direction. I've seen it with SecOps Panther. They're all kind of, hey, you know, here's your MCP server, go and build. And I think smaller teams can take advantage of that. That said, there are still many people that just wanna go in, utilize a nice UI, and that's their, you know, I just wanna write my queries, leave me alone. You build both, right? So, that's that.

For sure. I think it is a yes and, for sure.

Yeah.

So, I hate to do this because this has been such a shockingly fun dive into SIEM architecture. Again, things I thought I would never say. But I have to ask our traditional closing questions. And so, one, do you have a tip to security leaders who are thinking about what their 4th wave SIEM will look like? And then 2, recommended reading, which could be anything from, of course, your blog post, which we'll put in the show notes, to really anything, Mary Shelley to Mary Oliver.

Yeah, I've done a ton of these. Even like 6 months ago or so, I did a full analysis or a full SIEM evaluation across the current market. So I would say the biggest things we were kind of touching on this earlier was, as you do your evaluations, really figure out what your team is looking for, right? Is it the AI front of things? Is it we really want stability and fast searches? Like, what is most important? And then, like, get at least a month-long POV/POC going so you can see and send lots of data or ask the vendor, hey, can you give me a tenant that has 3 months of data in it?

Mm-hmm.

Just so they can get a feel for, like, what does that look like realistically, right? Because it's hard if you do like a 2-week POC, you're just not gonna know. You don't know what's gonna— like, it looks good with, you know, 5 KB per day, right? Like, yeah, but it doesn't look good when you start sending 20 gigs per day or the price becomes astronomical, right? So I think from that regard, those would be the biggest hits that I would have when you go through that process. And then just test, test, test, find what works best for your teams and, and go that path. So on the reading side of things, I primarily read technical articles nonstop. So like, I would say keep up with any articles related to AI and security and SIEM and things like that. You know, they're going to keep you abreast of like everything that's going on right now. So, yep. Right on.

Amazing. Well, Dan, thank you so much, not just for joining us on the show, but also for contributing Nano to the world and for all of the wonderful years of collaboration between you, your teams, and our teams here.

Yeah, appreciate it.

Thank you.

Thank you.

And now we are at time. Thank you very much for listening and of course for subscribing. Please subscribe so you can get new episodes piping hot And if you love our content, please drop us a review on your platform of choice.

You can find this podcast on YouTube, Apple Podcasts, Spotify, or wherever you get your podcasts. Also, you can find us on our website, cloud.withgoogle.com/cloudsecurity/podcasts. You can argue with us on the Google Cloud Security Community site, googlecloudcommunity.com.

You can follow us on X, x.com/cloudsecpodcast. Tweet at us, email us, argue with us. And if we like or hate what we hear, We can invite you to the next episode.

See you on the next episode of the Cloud Security Podcast by Google.

Security.