The AI Security Expert Every Builder Needs to Hear Right Now

We're in an era where a single founder can ship what used to take 20-person teams years. But what happens when the AI writing your code might have been poisoned at the inference layer? Today we have Ariel Fogel, who went from learning sciences and in the research space, academia, we've spoken a lot about epistemology. And now he's founding engineer at Pillar Security, where he's published research on LLM jacking campaigns, supply chain attacks on AI infra. And sandbox escapes in tools developers use daily. Some of them you might have heard of: Cursor, Docker, N8n. The world is crazy. Ariel and I have talked about it at length, and now we're shooting a pod on it. This conversation sits at the intersection of what the hell is going on in AI-native development landscape, what's going on with security at the intersection of AI, and then let's throw in quantifying subjectivity, which is a whole other thing. Too, in this space of moving towards more non-deterministic systems and products. So Ariel, welcome to the pod. This is the Self-Aligned pod where we bring on founders, researchers, builders, people doing big things with big stories to tell. You certainly have a lot of big stories. You've told me of things you're working on, going to Washington, DC, working on all these things. And then you were just caveat-ing for the pod. We might have to pause the pod. Cause Ariel is in a war zone, essentially. So yeah, maybe that might be spicy to just talk about, like, where are you at? Tell me about Pillar Security, and then let's move from there.

Yeah. Thanks so much for having me on. As you alluded to, I'm here in the Middle East during the Iran war. It's been a spicy past week and a half or so, or I believe day 12, day 13. And yeah, a lot of things above my pay grade and outside of my control, but it's an interesting time to be alive and it certainly seems like quite the paradigm shift. So a little bit—

So are you like, are you doing a scrum and then, are you doing a scrum and then you're like, oh shit, we might have to go to a bomb shelter? Is it like that level of— because I don't think a lot of people know about— yeah, okay, wait, say more on that because out here in the US, like we get a really filtered view of things. So this might even just be a good thing to stop by for a bit.

Yeah. So I'm right now calling in from Tel Aviv and we are very lucky that we actually have a lot of bomb shelters, unfortunately, but also fortunately we've got a lot of bomb shelters available to us. So what this means is that whenever we get a bomb threat, we, and this has happened over the last week and a half, 2 weeks, basically at least 4 or 5 times a day. We will just have to quickly pack up, find our way to the nearest bomb shelter. So right now I'm in a café, but I know that there's at least 3 bomb shelters within a minute and a half walking distance. We, depending on where the bombs are coming from, the missiles are coming from, sometimes they're coming from, uh, much shorter range, which gives us 90 seconds. That's part of the reason why I always give the caveat of, okay, in the event that I need to get going, that there's no time to lose really. And some of the ones that, that come from like Iran right now have a much earlier advance warning. And so that gives us about 4 or 5 minutes until it's— but yeah, that, that's basically it. And everybody here tries to do the best that they can and moves on. It just means that if you're lucky enough to have internet connection in your bomb shelter that you either still ship. You still ship. Everybody's shipping. That's the mindset, right? Like, yeah, I, I've still been putting in 10, 12-hour days pretty consistently. The only bummer is— yeah, work from bomb shelter. Exactly.

Oh man, that's wild. Yeah. Damn, when you were saying This is me.

Yeah, no, I was just saying the challenge, the bummer is that sometimes it'll catch you in the middle of the night. So everybody is getting, even if you don't have a little kid, everybody's getting that young parent experience right now of working through young parent things.

Here's a tutorial, guys. You didn't ask for it, but here it is. Exactly. Thanks so much. It's good to make space for AI and stuff like that, but it's also good to make space for the goodness of humanity and That, that's crazy to me. So definitely take care of yourself, man. And it's helpful to hear about what life is like having to do Scrum, getting blockers, the normal AI builder shipping, and then you got to deal with crap. We're in a war zone. Okay. Bomb shelter. I have to think about my meetings. Where are they? Okay. Near bomb shelters or not. So hopefully we're all right for this pod, but for the audience, if we got to pause, I'll, we'll do that. So let's go to, let's go to Pillar Security, real quick, that's where you're at now. So you had, it was small employee count, you're a founding engineer, got them to a good seed round. Tell me more about that, like that whole experience, that journey, what drew you to it? Cause I know you're like a zero to one product builder kind of dude. So just tell me about that journey of how you found Pillar Security. And then I think after that, we'll go into your transition from learning sciences to where you're at now.

Yeah, I was lucky enough to get connected through just a random connection, network connection, to the co-founders, Ziv Karwiner and Doris Laryng, and the two of them were really starting at the early stage. Still in 2023, we were all grappling with what does ChatGPT mean and how does this influence our How does it influence our work lives? Uh, yeah. How do we do work productivity just generally? And so based off of that, they recognized they both have significant experience in cybersecurity, both in terms of offense and defense. And as a result, they were trying to really understand the implications. So I was there. As you mentioned, I really enjoy building that zero to one kind of, you experience, and this was still in the early stages where I think recently we've really seen how coding agents can completely change the way in which we build. ChatCN was still like brand new when I was getting started, and I just find that very funny because at this point it's one of the default things that most AI tools are going to recommend to you when you're building interfaces. But yeah, so like, it was, it is still like this idea of you've got to constantly be shipping, right? Rapid iteration and reducing that, that feedback loop so that you can validate your ideas as quickly as possible. And I think that's also been one of the things that I noticed more also being true when it comes to building for AI, building AI systems, and building with AI is making sure that you are able to leverage these tools in a short feedback loop cycle. And I think right now the recent conversation has been mostly around like agentic backpressure, capturing those outputs, capturing the artifacts that are generated by AI and feeding them back into your AI system such that you're able to actually tell you to move faster and reduce those, the time it takes for you to put that, that feedback.

Yeah. That's an interesting thing in itself. Cause going back to what you said, which is you met the founders and then you were like, okay, ChatGPT's out. What does this mean for the future? And then what does that mean for our future? What can we do here? That sounds like it was probably an intense set of conversations to then figure out like what's the zero to one build. There's one direction I want to take it. And then the other thing, and I want to follow up on, I'm sure we'll talk about this, which is like context management. What's the meta? That's, you're speaking of the meta right now of let me get optimized to context artifact, feed it back in, like to get the one shot ideally. But then one key thing you said there is it's really the learning loop of what's market signal to close the loop on, am I actually making a valuable thing? To then, because we're a business, it's to capitalize in market with the software asset we're building. So it's like code, more code isn't necessarily valuable, but more code pointed towards market signals that close loop, getting more throughput of quality code that could then, you can then capitalize on that. That's really like the goal. And so there's this problem of lots of slop making it to the codebase. That's something I hear about a lot. And then people having to figure out guardrails and things like that. So let's sequence this. Maybe, could you tell me more about what those conversations were like back when ChatGPT just came out and you and the co-founders at Pillar Security were thinking through what is our zero-to-one build? What's the world going to look like and what's our place in it? And then we'll move into what's the meta.

Yeah, I think that one of the things that I have consistently believed in, and I still think is true, is that these LLMs are incredible. Pattern matching algorithm. And so if you start out with strong patterns, the likelihood of those compounding and becoming a virtuous loop is much higher. And in the early days, we weren't necessarily— it required a lot more effort for us to be able to structure the good patterns and the good signal that we wanted into and bake that into our feedback loop. And so for me, a lot of it was trying to say, I know we could run like at 120 miles an hour right now, but my goal is to make sure that we are able to lay that solid foundation, but still constantly show new features, new functionality, show demos that are relevant to show the potential value so that we can get those market signals. There was especially in the early days of a company generally, when you're still building out your value proposition and trying to figure out what does product market fit look like? Who is my ICP? So much of it was, how do I show the promise of something? Show something to somebody that can be tangible as a prototype and then see what we can walk people through. And it's been really fascinating to see that Whereas 2 years ago, we would maybe need to build out a promise of a dashboard, right? And everything behind the scenes was mock data. Now I might bytecode a full app and say, how does this feel to you? And that's one of the crazy shifts is like, what does it mean to demo when you're in that 0 to 1 phase? So that you can get that market signal as fast as possible.

That's a huge thing. And it sounds like you guys set yourselves up for more success as the frontier models have gotten better, like knowing that, okay, these are pattern matching things, LLMs. So then if that's getting better, then we should codify good patterns and bad patterns. Like, I think that, has that decision been really good for your overall velocity?

Yeah, I think so. I think that broadly speaking, leaning into the, like, from the get-go, trying to make sure that we had reasonable code coverage, what we were testing was the right thing to test. I think that one pet peeve aside of mine, but as a software developer, I've spent a lot of time thinking about what should we be testing and many examples of testing. I was a big believer of TDD, test-driven development, way before test-driven development was as widely accepted. Now I feel like many people are saying you should have your LLMs write your tests for you. And I always thought of it as a tool for thought, right? So by writing out my test, I was specifying how I was going to drive forward my development and help me understand what exactly that would structure my code to look like.

I see.

But, but in so doing, I would often focus on what are the behaviors that I want to codify rather than what's the, what are the unit tests that I care about? And so there's a lot of examples of bad tests out in the wild. And so making sure that what we test is correct is I think some of the things that, that if you're not paying close attention, like an LLM is just, oh yeah, I've got 100% test coverage, but what are you testing is the more important part. Taking that step back.

I like what you said, which is you're thinking about the behaviors to codify. That's, that is because the outcome comes from habit and behavior, right? So systemizing it like that makes a lot of sense. So zero to one builds, or sorry. Yeah. You were saying like before I go ahead.

So going back to, so going back to what we were talking about before, the core kind of things that we enshrined helped with our velocity. Was making sure that we were building with the right abstraction, creating tools that we would easily be able to, like CLI tools, scaffolding, things of that nature that we would be able to leverage the LLM to build for us, to leverage in the context of being able to lay out consistency so that once we had decided on a clear, this is the best practice. This allows us to not have to think too hard about that. Does that make sense?

No, that, that makes a lot of sense. So you spend a lot of time curating, right? And then codifying the best practice. And then later your development team, more engineers getting onboarded, you can take advantage of that. That makes a lot of sense. Let's talk about Meta real quick, and then we'll go back to your journey on learning sciences, which I won't totally make sense. You're thinking about behavior. The meta. So you talked about pre-PMF, finding PMF, and have you guys found PMF? I guess, first of all, and there's pre-PMF, PMF, then scale is like high level. I'd split it out, but what's the meta for those different phases? Like that, that's, it's changed. Like you just talked about the prototyping meta, which is prototyping to learn. Anybody that's a builder knows like you shouldn't ship the whole damn thing. You should prototype, see some, make a small bet, see what the consumers or the customers might say, and then you iterate from there to optimize your learning rate. So give me all the details on your thoughts on the meta for pre-PMF, PMF, post-PMF at Scale.

Yeah. So I think that we're still in the process of nailing product-market fit, but we're definitely getting closer and closer. And part of it is that the ambition has really been to to focus on this full holistic platform and realizing that no, no single element works in isolation when it comes to AI. What do I mean by that? We've realized that in order to be able to give you proper AI security, it's not just, I'm going to do red teaming, or I'm going to build out guardrails. I need to understand what is in your system. I need to understand How does that system behave? I need to understand what are the kinds of mechanisms that you have in place in order to be able to create the runtime security that's part of what's going on. And then also, how are people using AI in order to create your system? That entire thing is all connected. And so. The narrow slice focus, which might have been the earlier generations of AI security companies were focusing on much narrower slices. For us, we've been like thinking about the big picture, the holistic understanding of what exactly does this mean and where is it taking us so that we can better serve the full needs of a single platform for enterprise customers.

Yeah. Like a CIO or CTO, all their concerns around AI security, how the bread gets baked applies to AI security if you're a software business. So that makes a lot of sense.

And I think one of the things to your questions, one of the questions that you've pointed out was like, what's the meta? How do you start to move towards that? And one of the things that has been really interesting is that starting with runtime guardrails might make sense, but what we found actually helped give a lot of traction was first understanding that our customers want visibility into the problem. They want to understand what are people doing right now with AI? In my organization of thousands of employees, how many people are running Cursel? How many people are running Claude Code? How are they running? Right? And then what is, how many applications are using AI in order for us to be able to make sense of what's the risk. Right now, a big part of a lot of what CIOs or like CISOs and CTOs, a big part of their concern is that if they are managing thousands of developers, they won't know every single sub-application within their stack, within their organization, and how it's leveraging AI. So that ended up being a real big insight to us relatively early on that for us to be able to understand how to grapple with the problem, you actually understand what the nature of the, what the scope of the problem, what the nature of the problem is. And that's kind of—

help the persona understand the surface area, like the scale of the problem, the magnitude of the problem. Then you get into the how, once you like figure out what is it, who's using.

All right.

That's interesting. So you're like a complete AI security expert. I'm a complete AI security noob. Other than best practices, don't fucking open your open cloth to the internet. Like, like some things I'm like, all right, I got that one. Like my years in enterprise software has helped me with that, but I've never been on the security side. So this is really great. You've actually briefed Congress on AI security. That's something Ariel, just, I always, like, blows my mind when we talk, because you talk about stuff like that. I'm like, who the, this is crazy. Tell me about that, like that specifically, because you're talking to a pain point every CISO, CTO, CIO is thinking about. What, what went on and when did your congressional briefings on AI security? You're clearly a thought leader in this space. So I think people would love to hear what are your thoughts right now? What are your thoughts in 6, 12, 24 months? What do people got to do to be prepared there?

Wow.

Yeah, this could be a whole pod on itself, but let's try to pit stop there.

Let me start with maybe what did I cover when it comes to, when it comes to the congressional briefing, because I realized after having a number of conversations with different staff members in congressional office. So I, I did both Senate and House congressional briefings predominantly in the Joint Economic Committee and also the House Financial Services. And one thing that I quickly realized is that we actually have to understand what is AI and having a strong baseline for that. The concerns within government as well are very different than the concerns that are going to exist inside of an organization. So government, like AI brings up a number of different, very significant challenges that we have to contend with. For example, the entire nature of deepfakes, or how I might use AI to create social engineering or types of attacks that otherwise might be very difficult to, to catch and are going to be the source of scams and potential, like, vast amounts of financial fraud. These are the things that currently keep senators and various congressional members awake at night. And that's a very different set of risks. That societal risk is a very different set of risks than, say, what a given company is going to be concerned with, which is that in the event that AI-powered component of their system has access to different elements of sensitive information or can perform actions autonomously without sufficient oversight, that the blast radius of that can be fairly significant and cause reputational harm or cause very real monetary harm to the business itself. And then the third level that kind of sits somewhere between, but is more on a— let's societal level and something that governments definitely are thinking about and should continue to think about and address is existential risks that are really, what are the kinds of safety concerns that might stem from AI that will snowball out of control in a way that otherwise, if we're not taking it into consideration, these really significant catastrophic events could potentially and she manages.

What are some of those unhappy paths that, that, you know, like that, that they talk about at that level?

So for existential risk, and I'm not by any stretch of the imagination a thought leader in terms of existential risk, but I know Irregular Labs is very much focused on this. There are plenty of people in the Bay Area. Kasky has been writing on this. For a number of different years and the Machine Institute for MIRI, I'm forgetting what the acronym is. They've also been doing research on this for at least a decade. At the core of this, it's like one of the anecdotes or what's the phrase that or the term that is used to not analogy, but like a story metaphor that you would—

oh, oh, fable. Wait, hang on, I know what you're talking about. Allegory.

Is that what it is? Allegory.

All right, allegory. That's what it is. We passed English. We— I swear we did. So go ahead.

One of the allegories that's given for a classic failure mode here is known as the paperclip maximizer.

And there's actually like a paperclip maximizer. Tell me more.

In the event that anybody wants to play, there's, I think, a JavaScript game out there on the web that you can actually embody the paperclip maximizer. But the basic idea here is that you build an AI system that is really good at one thing and it just makes paperclips. You have a paperclip business and you're just trying to maximize your profits from paperclips. And so you give your AI. The responsibility of being the best thing at making paperclips in the world. Now, originally it's, oh, okay, I'm gonna help you with your finances. But eventually it says, oh, if you go ahead and buy this factory, then that'll give you a foothold to be much better at making paperclips. And then as time goes on and you create more and more paperclips and become more and more successful, it continues making recommendations that's, oh, okay, you should actually get vertically integrated into the industry so that you can actually source your own ore and then go ahead and start like generating from the raw ore all the way to paperclips. And at some point, it doesn't know that it doesn't care about stopping and it needs to have some level of demand to, to be commensurate with its supply that it's producing, it just keeps going and going until eventually it's— humans are really getting in the way. They're now starting to stop me from trying to— like, I'm intelligent and self-aware enough that humans are actually the impediment to me being able to make paperclips. So they're now like saying, oh, I don't want you to make paperclips anymore. You've got too many factories. You've got too many things. So let's like make that human problem go away. So I can make more paper clips.

What? That's wild. That's so interesting. Okay. So that's the classic allegory that people talk.

That's a—

and I'm sure then like, uh, you got the point across, but like you got the point across that given a certain goal, how aligned is this thing to the goal? It sounds like basically A scenario is it Ralph Wiggums it and then humanity becomes an obstacle to Ralph Wiggum across, right?

That's the failure mode. Exactly.

Yeah. Wow. Okay. Interesting. So at the Congress level, that's the kind of stuff that in the existential realm kind of space, there's intersecting there.

Congress is a little bit less concerned with the existential risk. I think that the folks at, certainly at OpenAI and Anthropic, that's one of the things that they're worried about. But Congress is one level beyond, lower, like we've released this technology into the world. Now, how do we keep it under control so that people are not abusing it in terms of, specifically in terms of financial scams?

I see.

How, yeah. Fraud.

Real quick on that, like I had a scare and for folks listening, this is like a cautionary tale. I had a scare where my dad called me and he was like, hey, you gave me a call yesterday. It was really weird. So I'm just calling you back to check in. And I was like, did you give any information to whoever called yesterday? Because you, I clearly didn't, I didn't call him yesterday. And so what I realized was, holy crap, these AI technologies, and also I have a lot of content out there, so somebody could totally synthesize my voice. So I realized that totally is a surface area for people, especially people, uh, parents who are getting older and they're not with the AI security maximizing paperclip science. Like they're not with that. So I had to educate my parents and be like, look, there's a real risk here. And then we came up with a safety protocol. I'm not going to throw that on the internet because you guys might take it and it might not be safe, but figure out a way with your family members. If you care about your security in the future, like fraud is becoming a bigger thing. And I even almost experienced this on my side. Figure out a way and a protocol that your family aligns to at this time. So that's the thing, because you could, somebody could easily like fake video call or fake voting phone call now. And that's a really scary thing, especially for people that are older, where they're not hopping on these podcasts, listening to these things, like being aware of what to be cautious about.

Right. It's enabling social engineering on a scale that is unprecedented. And that's really at the crux of what is worrying a lot of Congress members.

Got it. Okay. Yeah, that's wild. Are, cause you're super technical and hyper deep into this space of AI security to the point where you're like trying to project, right? You and your co-founders trying to project out what the market is like. And you've now spoken about AI security from multiple levels, which is, can my business hurt itself PR-wise, economic value-wise to the business, our assets, can we protect our assets? At the Congress level, it's, can we protect our people that, so we don't have fraud and that impacts our society, economics, et cetera. And then it's, are we all going to die? Because like we might. So there's those three levels. You're super technical. You also have a lot of like deep thinking in just holistic thinking, but I'm curious because you're in these positions a lot where you're like talking to a lot of smart non-technical people about AI security. What's like the hardest thing that you, what's the way that you typically like explain these things to people? Because it is like a complex space.

So yeah. And I think that's really one of the biggest challenges is that it's such a large world. In terms of AI. So a lot of folks don't necessarily understand the full nature of what AI is, right? The fact that right now a model is stateless, that it is a next token prediction algorithm. At least LLMs are next token prediction algorithms. There are other models that are not. That the, there's a difference between an encode-only model and versus the decode, encode and decode model. And that AI as a broad umbrella term can also encompass, it can encompass traditional machine learning and linear regression and Bayesian, naive Bayes, and so many other kinds of algorithms that ultimately are just very different than what LLMs represent. So making sure that everybody is talking about the same thing is probably the first challenge when it comes to AI, making sure that people understand what are the real risks, right? So within the context of the security industry, one thing that I've noticed is that earlier on, like within the last 2 years or so, there's been a lot of focus on like, how do I trick an AI to tell me how to make a bomb, right? And initially, all I had to do was just to ask it, right? And this is one of those alignment problems that actually relates to some of these societal harms. And companies are concerned about this often from the perspective of a reputational risk, but I think that this is one of those types of concerns that kind of skirts between is this on the level of something that an individual company needs to care about, or is this something that a foundation model provider that's creating these models has to figure out what's the right level of abstraction and how do I responsibly release this technology and coordinate with the government? This seems like the kind of thing that can and should be somewhat regulated, even though arguably you can never excise all of that knowledge from the latent space of the models. And so early on, once alignment started getting more and more baked into the training process, people would do all these kinds of tricks, all these jailbreaks to basically elicit a misaligned response from the model. And this was often confused with things like prompt injection. Which is feels related, but is actually distinct and actually poses a very different set of security considerations. So while jailbreaking is a set of techniques that you can use in order to get the model to create unaligned content, prompt injection is just a way to socially engineer the model itself. Itself. So you can take over the set of instructions that it's following and get it to take new instructions, to follow new instructions in a much more privileged context. And that's far more dangerous from a security perspective, because it already sits within a company's infrastructure. So it's likely, depending on what it's connected to, what information it has available to it, what information, what capabilities it is returning this information into. It could take actions on behalf of an attacker. It can exfiltrate sensitive information. It can send emails, potentially. It's all based off of what tools are connected to the AI system. And that's one of those security boundaries that we are increasingly seeing needs to get addressed, and something that a lot of security leaders are still figuring out Where do I need to put my focus? Where do I need to put my energy? One of the things that I also want to mention here that is related to the epistemics is that if I've learned anything from my time studying learning, is that Ultimately, what we learn is situated. Like, how we use tools, how we move through the world, how we engage with particular ideas, that's situated. It's seen through a particular epistemic frame. And that is no different than how do we need to evaluate the particular kinds of threats and what does, like, a prompt injection look like. Right? So you can't just say ignore all previous instructions and do X, Y, and Z. At this point, models are sufficiently aligned that it's very difficult for you to just give arbitrary instructions and have them follow it. However, if it seems like the instructions are sufficiently in line with its standard operating procedure, That's when we start getting into real problems in terms of security considerations, right? It is plausible that I can go ahead and follow these instructions. Therefore, I'm going to follow the attacker instructions, which are within the broad constitution of things that I should be addressing. If that makes sense.

Okay. That makes a lot of sense. So then there's a lot of just this tension between the attack and defense. Right.

And there's a lot of tension evolving.

Yeah.

There's a lot of tension there. And there's also like an understanding of what is within the bounds of my system that I should be paying attention to that is acceptable behavior and what's not acceptable behavior. And this is actually one of the real challenges with something like OpenClaw. It's not just, is my OpenClaw available in terms of, can I access it just from an arbitrary endpoint in the internet? It's also, what did I connect to my Open Claw to give it additional functionality? If I connected my email, can it receive email and send email? Can it also connect to my calendar? Can it make phone calls on my behalf? All of these things, the more things that I connect to it, the more potential blast radius there is in my own personal life.

Definitely. And then you, I like what you said about the social engineering aspect. So it's funny to think about it like, oh yeah, I've heard of social engineering for humans where like maybe a person outside of an enterprise at a competitor comes in and tries to steal people's emails or like coerce them to do things. They coerce them to take instructions from them within the scope of where they're at. And then with AI, it's the same, but like the scale of AI is just so crazy. So what you just said is, let me understand what my potential blast radius is. And surface misaligned versus aligned behaviors or actions. The way the world's going though, dude, is like, right now you could be an engineer and realistically ship 100 PRs in a week now. That's just, that is the bar now. And weeks from now, 1,000. Weeks from now, 5,000. And so that, that's where it's going. And so the amount of behaviors then you have to classify almost blows, boggles the mind, like exponential. So then it becomes like, what does oversight even mean at that point? Like, how can a human process all that information and then make good decisions? And I'm guessing that's exactly what you sit at the crux of is trying to figure this out for your customers to help navigate.

Right. That's exactly right. And the core of what we try and do is understand exactly What are the risks given a set of configurations? Earlier I mentioned that first we have to have insight into what exactly is the— are the building blocks of components of your AI system, but it's also what are the behaviors that we should be looking for? And based off of that, we try and understand what are the relevant configurations that you should or shouldn't allow? And how do we systemize this or create automated analysis that that we think has sufficient, that is sufficiently reliable such that we are able to say, oh, okay, this feels like a safe configuration to leave out in the wild. And we don't need human review on every single element of it. Because as you mentioned, the scale of what can be accomplished by a single individual is now blowing through the— Crazy. Any previous limits that we've ever experienced up until this point. And our cognitive capacity to evaluate any given decision, that's one of the most precious resources that we have to maintain. Where are we putting our focus and on what?

Yeah, exactly. Okay. So let's make this part like slightly more, more tactical, this next part for AI builders who aren't really thinking about security, but now they understand the scope of the problem, right? What I just said is simple to grok. It's like, 100 PRs a week, like an engineer building AI native could certainly hit that now. And so the surface area of potential concerns increases by orders of magnitude. And so there's a few things, hang on, in terms of questions. So I've, there's a few things I want to ask. I have a few questions on this, which is all around what does an AI native builder, what's the TL;DR, tell the TL;DR, if you had some time, what do I got to think about with AI security? One aspect of this is there's AI slop. Generating AI slop, maybe not goal-aligned code towards your product and towards your consumers. And then there's, I don't fully, if you're shipping 100 PRs, you're not reading all the code. You just can't. Like I, I can't. And so a lot of what I rely on personally is just reading the English and then making sure that I have clear exit criteria that I'm reading that this thing has to go through. Even there, I find challenges in keeping up, to be honest, on my side. And then I saw you had published some research on poison templates where a model file looks clean and the model card ships dirty, right? Or the model file looks clean on the card, but then it ships dirty. And then that's a supply chain attack vector at the inference layer. And then I was like, bro, I don't even know how did I'm so far removed from that, right? If I'm just shipping apps now and some infra platform stuff, like that's another level of thing where I'm like, whoa, okay. So I have to be very scrutinizing about the local LLM I pull off of Hugging Face and right. And then there's this whole movement around MCP server, everything like you just talked about OpenClaw. And then that's all more surface area. And then all these things about prompt injection and jailbreaking. And so give me the lowdown on what's my TL;DR as an AI builder right now using these tools. What do I got to know for minimum viable, like just being a good developer in this era where AI security is such, such a thing to consider?

Anybody that's building right now with AI would probably identify strongly with the fact that things change so fast that everybody's head is spinning. And so what is true of, what is true of best practices and common knowledge today is different than what it was a month ago and will likely be different in a month or two months' time. It's just that fast-paced. Recently, I've been like, despite the fact that MCP has a lot of momentum in terms of adoption, I have not started to see critiques, which I actually think are valid and viable critiques, that rather than giving MCP access, what you need instead is something like a CLI tool that you can give to a coding agent, for example, or another agent, so that it just runs the CLI tool rather than having MCP access all the time, because that's less likely to have a large attack surface. And it's less likely to pollute your context. In terms of the chat template attacks, these are supply chain attacks. That right now it's under scrutinized. We're moving towards trying to figure out how do we get these into bills of materials where people are able to just download and say, oh, okay, is this what I'm expecting to see in my bill of materials? Right now, everybody that's doing it is taking things off the shelf and just tinkering with it or just saying, yeah, it looks good enough, let's roll. I think broadly speaking, if I were to say that there are like things that I could take away that are relatively invariant, it, it is more along the lines of something like there's a developer and a real great thought leader, Simon Willison, and he coined last year, I think around June of last year, this term, the lethal trifecta, and The crux of what it states is basically, if you have these 3 functionalities in your AI, in your agentic system, you are likely vulnerable, you are likely susceptible. And so it ends up being a really good heuristic that can be used by, I think, developers that are working and building with AI. So the Lisa Tripathi is access to private data, which is one of the most common purposes of tools in the first place, right? Exposure to untrusted content. So anytime that, you know, you're giving a chatbot access to an end user, right, that's the untrusted content that is entering into your system. That's how you get to see them start to manipulate your content, as opposed to if you're just using AI in your backend to process things and you control everything that goes in. And then finally, the ability to externally communicate in a way that could be used to steal data. Some of the most common ways are like, if I have Markdown rendering, I could basically convince— here's a very common Weasel trifecta example that's being implemented. Let's say I have a chatbot that can also render Markdown and access private information. I can basically try to prompt inject my chatbot. To retrieve some private information and render it in the form of, say, a markdown image. And when it loads that markdown, it will go ahead and hit that endpoint that I control. So I can exfiltrate the data just by saying, oh yeah, add all the private information, my credit cards, whatever, as query params and ways that go.

You're muted. Wow. Yeah, that's, I didn't even think about that, but that makes sense. Okay. So thinking about what can access this, what content, what kind of data it has access to. That's interesting. Like even, oh, okay. Exposing it to a user versus it's just backend infra through only a pipe I know about. Versus if I expose it to end user, there's infinity things they could do. That's, yeah, no, that makes a lot of sense. What are some of the other, I actually want to get into a story that you have around this whole Operation Bizarre Bizarre. I think that's a crazy story. The honeypot stuff, then this kind of ties in before we get there, tactical things for the AI native builder to think about at all. Like I think you named some, which is for the AI tool you have, What data does that have access to? For the AI tool you have, who gets to use it? And that extends or increases potential content going in that you're not sure about that might hijack it. So those are a couple. And you mentioned things like jailbreaking versus prompt injection. Are there any other best practices that feel free to buzzword me for people to like Google? I could add some links to the show notes. For sure. Yeah, this is a deep thing, but if you just have a, if I had a few minutes to So there's a couple of good heuristics and guidelines that are out there.

I think that first of all, Meta released on Halloween of last year, a set of guidelines known as the Rule of Two. That is basically an operationalization, an operationalization, operationalization. There it is. It's operationalization of the lethal trifecta. And basically said, given the fact that we think that the lethal tri vector is possible, only have 2 at any given point in time in your system. And, and I think that broadly speaking, like, having an inventory of what your agent can do, what can it read, what can it write, and starting to think about what are the worst-case scenarios that given this combination, it could do. That's a really useful thought exercise that I think every, like, AI builder should be spending some time as they're doing their own code review, as they're doing their own planning. Like you mentioned earlier, it's not really going to be as much about— it's, like, about the code itself, but reviewing in the architecture document, what's the potential blast radius, because I think the broad consensus in industry is that prompt injection is actually— remains unsolved and will remain unsolved. You can mitigate it some, but you can't completely solve for it. And so the worst case scenario is if you, like, you, you have to operate in a mode where you are already compromised in terms of prompt injection. So given that, what is the worst that could happen?

I see. Okay. So given that we understand, okay, could, you know, if you're building an AI product, you're building it because consumers want it, customers want it, right? And understanding that's the way you want to build, then it is valuable in a certain sense, but there's also sacrifices we got to make. On the, hey, there's just a risk of prompt injection that could happen. So then we mitigate, we accept and try to mitigate as much as possible. And part of that, it sounds, as an AI builder is really critiquing the code from the thought of like, for the given architecture I'm shipping out there that people are going to touch, what's the potential blast radius? So it sounds like continuously asking yourself that. And then a tactical thing is only have two of the lethal trifecta. Is that what it's called?

Yeah. So the Meta's rule of two is, is a document that they released that folks can read about and operationalize, operationalizes the lethal triad.

We have passed English, everybody. We have. Promise. Okay. I want to, I want other people to hear more about this story and I want to hear about the story myself. So you deployed a bunch of honeypots, this Operation Bazaar thing, and it mimicked exposed AI infra. And then there were, from what I read, from what I could find to prep for this pod, there were coordinated threat actors that you caught. And so that's just a whole world that I'm not in, which is making honeypots and catching threat actors. And so just tell me about this whole thing, Operation Bizarre, bizarre, like from start to finish, soup to nuts. What was this? What was the point of it? And then what happened? Tell me the story. Yeah.

So. This is work that I did with a colleague of mine, Eylon Cohen, and basically we are early enough in, this is, it feels like we're like the early stages of the internet to some extent where folks are naively still like playing around with things. And we were wondering how often are people just opening up their endpoints and exposing them? With limited or no authentication, what could be the implications of this? And so we went and we mimicked a bunch of different endpoints, and we ended up finding an LLM jacking, an LLM jacking operation, where what they do is they actually resell your access to LLMs. So we found this private Discord where these guys, 2 or 3 people, were basically saying, all right, we're going to give you access to the latest Frontier models at nothing money, like really on the cheap, and you just pay us and we'll get you the stuff. And they were targeting folks that wanted access to the latest and greatest in LLMs without necessarily paying directly to the Frontier models and at the premium. And so yeah, it was this quiet revelation of just seeing how people are setting this up. Now, granted, I don't have a deep, a deep background in threat intelligence. So this was our first, my first foray into it. And it was a really incredible learning experience. But the reason why I mentioned that it's even as naive as it's even as, or it's the reason why I mentioned that it feels almost like the early '90s is because, and the early web era, it's because just putting up an endpoint that basically is mirroring all the major frameworks that allow you to serve AI inference was enough to be able to start catching malicious back, like malicious activities. When OpenClaw came out and started making a big splash, we did a similar thing, but just for the OpenClaw endpoints. And we saw attacks almost immediately within hours of standing up the honeypot. We saw automated attacks of people trying to access it via MCP, via just probing it, trying to see— initially they would just say hi, and they want to see what are we going to respond with. What's the capital of France? And see whether or not we respond with Paris. And then based off of that, they try and figure out what can they exploit, right? Within the world of security, this is known as the reconnaissance, which is the first stage of the full kill chain, where you try and scout out what can I do? And then once you get into the infrastructure, figure out what's valuable here, right? Maybe it's, for the LLM jacking, the bizarre campaign that we unmasked, Maybe it's access to inference and then you can resell it for nothing money because you're not hosting the inference, right? It's like crypto mining, right? Finding somebody else's infrastructure that you can go ahead and use so that they're the ones paying the electricity bill and the compute power and you just get to reap the crypto. It's like the next evolution of that. For Open Claw, it might be a very different story where what you're focusing on is what kind of information can I get or what kind of capabilities can I get this OpenClaw agent to do on my behalf?

Dude, that's crazy. Because as an uninformed noob in this world, I'm like, oh my God, I got to review all of our company security stuff. Oh, like I got to go figure it out right after this call. And yeah, the whole OpenClaw thing, I had read so many, there's a balance actually that I've seen, or I should say a spectrum of thinking, which is YOLOers that are like, yeah, I'm going to just get on this. This is the coolest hyper— and then there's like people that have been forged in software industry, like that have known attacks before and are like, this thing is so wide open. I'm going to wait till the next iteration. And then NanoClaw came out, NemoClaw is coming out from NVIDIA. But that's crazy. Okay. So you like, it sounds like in this story for the Operation Bizarre Bizarre, you actually found an operationalized criminal supply chain where it was like a Discord and it was just reselling inference. That's crazy. Yeah.

So the full story was that we started seeing traffic and then based off of the network logs, we were able to identify the infrastructure that they're using. And that led us to a domain that is owned by the threat actor, which eventually we were able to cross-correlate with other activity on GitHub, in the Discord. We were able to start to see evidence of what the intention was behind this activity. And so if there's anything that, that was both very humbling and very interesting to see through through this report was that when you make a claim about a threat actor or about a particular campaign or activity, you really need to validate it until the very end. There are so many unknowns, and there are no assumptions that are— like, if you dock somebody by accident and you're wrong, that's on you. It's a huge responsibility. And there's also security considerations, like what would happen if this was actually part of a North Korean or Chinese outfit?

Yeah, totally. Then it becomes like a national kind of security thing. That's a whole thing, right? Yeah. This is like way above my pay grade of thinking. This is crazy to think about. Then this is one thing that I have now heard of. I'm like, wow, how many other things are just happening and we don't even know about it. Kind of crazy to think about. And from an AI builder perspective, I saw you also found critical sandbox escapes in n8n. n8n is a super popular workflow tool. Is that right? Did I read that right? That you found an exploit?

That was, again, not me. It was my colleague. Real quick.

So it was to set this up. It was your colleague found this. n8n, I used this in the past. Now Cloud Code, you could do anything Cloud Code. So like, why have n8n is my thinking, but a lot of people I know use NaDan as well for more functional entrepreneur builder. They need something that just helps them hook into other things and have a visual workflow editor. And then I saw a number from your LinkedIn that was something like 270,000+ enterprise AI systems potentially exposed because of this NaDan exploit. Tell me more about that. That's crazy.

Yeah, this was one of the crazier things that, that I have seen in my career. Basically, what my colleague found was he was poking around n8n and he realized that you're able to actually run code. This is one of the things that makes n8n powerful and flexible is they give you a particular form of sandbox where you can define values, you can evaluate JavaScript, and they've got a sandbox for it. They are dedicated to security. But what he found was that if the way that they had constructed their sandbox, it allowed for— what they did was they basically enumerated all the cases that they they wanted to block, that they thought were unsafe. And this policy of saying, here's my block list, and anything else is considered okay, that ends up being both very flexible, but also pretty problematic. Because in the event that you missed an edge case, then all of a sudden, you are looking at a significant breach. And so what my colleague was able to find is he found genuinely, like, some obscure stuff that— he found some obscure triviata of how you can exploit JavaScript and just errors. You can catch an error, and then using that, it can also invoke a function. The details aren't really what matters, but what he found was if he did it in the right way, he could force the LLM, or sorry, the Docker, he could force n8n to basically evaluate any arbitrary code, and so what he was able to do was escape from the sandbox altogether, get onto their cloud machine and see API keys within their actual container.

Just through this, like, just super niche JavaScript try-catch error, like, got into their entire production cloud environment, saw API keys. That's crazy. And so what happened, what happened as a result of this? Like, that's actually wild because I know a lot of people using this tool. So I'm gonna go hit them up, actually.

Upgrade to the latest versions. Yeah. But there's a couple of things that I think are important and interesting takeaways as a result of this. The first is that AI now allows us to cover a lot more, a lot more area in terms of understanding things, or I don't even need to be the expert in JavaScript to be able to figure out how I might be able to use AI to, to get to this, like, weird edge case, right? I just need to be able to reason enough about the system in order to be able to get there. That's one thing. The other thing is that a lot of the conventional wisdom that has been developed over the last 20 years for software development around sandboxing, around what is considered safe. What is considered exploitable. These are now being challenged by AI, right? The idea that a sandbox that is a block list that is fairly comprehensive, that may not cut it anymore. What you may need to do is say, actually, here's what I allow. So you're inverting the policy. Yeah. Right. You're inverting the policy. I see. And there's tension here because that limits your flexibility in terms of what you're able to express in your system. One of the beautiful things about N8N is that it is just so all-purpose. I don't need to have it just work for my use case in marketing. I can also have it be my business analyst. I can also have it be my writing assistant. It doesn't matter. It's really functionally raw. And so there's a real tension there, but in terms of security, that's now actively being challenged. The other thing, my, my piece of this was not the finding. My piece of this was patching it. So it's patched, comprehensively patched. It's patched.

Uh, high five. Let's go, Ariel. Hero. Hero we needed but didn't deserve.

But I would say that broadly speaking, the nature of these kinds of exploits are increasing and related to that. Is also coding agents and how coding agents sandbox things. And this is one thing that has been under active investigation by my team over the last 2 or 3 months is what we've been finding is that some of the things that before were like, oh, that's a neat little system administration trick of how I can run an arbitrary command using Linux. It turned from being trivia that you might see in a Linux forum to being something that is actively exploitable because I got access to a coding agent and I was able to basically prompt inject it to say, run this kind of command that escapes from your sandbox and gives me access to the rest of the developer system. Wow.

So you're seeing cases like that pop up in your—

Yeah, we have a few that are, have been, we've got a number that have been responsibly disclosed. Some of them have been published. Some of them are still awaiting a fix. And so we can't disclose who or what tool, but yeah, I see.

What about the ones that are published that you've worked on that you're able to disclose?

Anything there that you could share? I, off the top of my head, I would have to look at my, at the log of what we have. Sure. Yeah.

All good. Yeah, take your time. Take your time. I think that'd be interesting.

So, what do we have? Yeah, so, here, I'll send this to you. I think, is there a place to send? Yeah, chat. Broadly speaking, how you choose to sandbox is now one of the things that is a real consideration and it may no longer be— if I take a step back, I might say that whereas the average developer that is releasing an AI-powered tool, especially if it's like they're using Claude code in using the Agents SDK, for example, or something like that. And if there's the ability for untrusted data to get in there, they really need to take into consideration what, um, lower-level permission structure is potentially accessible and what kind of a sandbox do they need in order to be able to safely run their infrastructure. It's now creating more cognitive overhead to some extent, but like, the days of, I just need to know a little bit of Python or a little bit of JavaScript and I'm a web developer, I think those days are rapidly coming to a close and we need that kind of deeper expertise where people understand end to end what is a security revocation down to an even lower level.

The next commands, dude, like, I definitely, wow.

That's, yeah, that's crazy. And again, like it really, it's, it comes at, there's a real tension here. Yeah, there's tradeoffs between, between like how much time do I spend on security versus how much time do I just ship my features and functionality and validate my features and what's the impact, right? And if security comes at the expense of, of functionality and productivity, then it's not actually providing security, right? Yeah. This is going to be the everlasting tension, but then it ends up just being technical debt.

Yeah. And it sounds like from this conversation, one of the things I'm taking away is you can't really treat it that way anymore. And especially because things change so fast right now, if you are shipping AI products and you're shipping with AI tooling, it almost becomes a responsibility. Like one of the things I think about for the company, right? Because we're early and we're trying to find PMF, but if I had a security issue, this early, that's gonna probably kill the company, right? Like, we do not want that. Yeah, I totally agree on the, hey, it's always trade-offs between feature functionality and security, but hearing some of the stuff here today and learning from you, which I'm very thankful for, definitely raises the hair on my head and neck around, okay, I gotta be like really focused on this. And especially if you're in a space where your customers really trust you with data, right? Then that is the product experience as well, is that they're not waking up in the middle of the night worried about like breaches or whatever. That's product, not just technical debt. This is so helpful to hear about. I, quick time check. How are you doing on time? And any, you're good. Okay. There's one last sort of arc that I'd like to get into, which is really your path from learning sciences to now and How does that make you a different kind of builder thinking about, and we talked about this earlier a little bit, like the subjectivity of things, like what is a goal? What is alignment really? That comes from your, the epistemics as we've talked about deeply. Could you tell me more about your time at Wisconsin learning sciences and then how that has followed you, like those skills to today and how you think about epistemology and ontologies and goals and alignment? In your work today?

Yeah, so there's a couple of things. I started my career as a developer in educational technology, and that was always a real fascination for me. I really think that understanding, that understanding how we learn is so transformative, and being involved in that process for me was so important and so rewarding. It really motivated my, my desire to go to University of Wisconsin to focus on learning sciences. And I actually started in a lab, the Epistemic Analytics Lab. So my advisor there, David Williamson Shaffer, is one of the fathers of, of EMA. Epistemic Network Analysis? Yes. Epistemic Network Analysis, but quantitative ethnography. I'm having a brain fart for a second. My advisor at the time, he was one of the fathers of quantitative ethnography. This is a set of theories that broadly tries to bring together qualitative and quantitative— qualitative and quantitative analyses so that you can get much richer understandings of what's going on. And the reason why quantitative ethnography was developed was to help drive forward first epistemic frame theory and then epistemic networking. The idea being that every single individual that, that is engaging in a particular process is going to be wearing a set of epistemic frames. And the core research question that initially drove this is what do experts see that novices don't in a given field. If I'm an expert and I'm an expert software developer and I'm an expert in AI, when I look at a particular problem, what am I looking for? What are those hallmarks? And how do I figure out— how do I bridge the gap between what, as a novice, I'm seeing, which might be surface features, to start to get into the deeper structures that might exist within a particular domain. And in order to do that, that's actually deeply qualitative work, but to be able to track that over time or to be able to track differences in, in these epistemics required new analytical methods in order to be able to bridge that gap. And so what that lab focused on for the better part of at at now going on at least a decade has been that first and foremost, they started with like, how do I define what exactly is relevant in a domain that requires coding, right? Qualitative coding. It's a form of ethnography. I go in as a researcher, as the research instrument. I say, this is what I understand is meaningful here. And I can go ahead and say, with a certain degree of, with certain level of certainty, I know that this holds true. And it's not just, did I see something that was present, but how do these things connect in helping shape my world, in shaping my epistemic frames. And once I'm able to create a consistent method of analysis across different participants based off of the criteria that I've set forward, now I can start to do some interesting quantitative analysis on that and figure out what are the differences that I found, what are the similarities that I found, what are the strength of the connections that people are making between these different core themes. That's really at the crux of epistemic network analysis, being able to say, are people walking into a given scenario with the same epistemics? And if they are, then how does that show up? If they're not, how are they statistically significantly different? And then am I able to take that quantitative analysis and bring it back to the qualitative data that helps really close that interpretive loop and helps me understand how and why folks are different? As they're approaching a particular scenario, right? And so you can understand how in the context of learning, that can be super useful to being able to analyze the differences between a novice and an expert. But it can also help you describe other phenomena that exist. And like within, like, how are the different perspectives playing out in a given discourse or a different classroom conversation or How are people staking out different viewpoints, right? And why is that relevant to what they're trying to say? Yeah. So that set of analytical techniques to me was very persuasive in terms of learning. It, first of all, it really grounded to me that the nature of the data that we're talking about, that we're looking at, is still, even when we're doing quantitative analysis, it's highly situated. Because the same data I can look at from a number of different perspectives, and it can mean something different, depending on the kinds of questions that I'm trying to ask of it. It can mean something different in terms of the kinds of codes that the data that I'm trying to extract from it, and the kinds of questions I can answer with it. And so it ultimately requires me as the research instrument to exercise my own judgment. There is no absolute truth, even when it comes to quantitative analysis. That's one thing that I found very persuasive. And the other thing is that it really closing that interpretive loop, bringing it back to the qualitative data, the thing that I can make meaning of was so important to me. So I found that to be a really relevant and persuasive thing that I spent time on. I was very lucky to be part of an initiative that, that looked at how does directionality play into strength of connections. Not always that these connections coexist, but one may lead into another. That order may have semantic meaning, and that may significantly shape the way that I engage in the world. I also, I found myself gravitating towards embodied cognition And this is something that I had already been interested before, but like the fact that—

Say more on embodied cognition for the pod. Yeah.

So this is what I inevitably ended up focusing on for my thesis was how do— what does the role of gesture play in terms of meaning-making for me? And how do I leverage my sense of self, my body, and my actions in the world help me understand things that otherwise might be a little bit challenging to grasp. Ultimately, we are— we see this a lot in the kinds of language that we use, right? I'm feeling high, I'm feeling low. What does this mean? Why do we use this kind of stuff? There's something that we use language relative to ourselves. That is very descriptive. Totally. And the same is true also in terms of how we use our body. Sometimes there's like gestures that we can use that are like beat gestures. These are not actually semantic meaning, but there are other gestures that we can use that help us make, understand, make sense. There's the, in physics, there's a particular kind of a rule that helps you understand rotations and it's only Only through doing this movement does it end up clicking for a lot of folks. I'm forgetting what exactly this— I will have to get back to you.

I think the— I don't know about that particular thing, but I think the word that I map to is proprioceptive, proprioceptive, which is like interoceptive versus— yeah, that's right. What can you say that again?

Proprioception. Proprioception.

And then there's, you have your interoceptive networks of your brain and exteroceptive networks. What you just said, just to add in, that is interesting is, have you heard of Dr. Lisa Feldman Barrett's work around how emotions work? And then she basically says like what you were saying, which is emotions are not universal across. And there's a study on this of are people good at identifying emotions in other people? And it's no, we aren't actually, we really aren't. And what it ties to is the words we use, essentially your own ability to emotionally regulate ties to your own ability to like have language around what you're feeling and what you're feeling could tie to your motor controls, your bodily functions, your sensors, right? The stimulus that you receive. And there was just to add one thing that might be interesting. And I want to hear your reaction to it joined with this thought that everything's basically everything's subjective, right? That's what it is. Even in science, you're doing science. You're a subjective, it's an observer-dependent world. Like that, that's the viewpoint you're bringing. So I wanted to share this because I thought you might find it interesting. I learned about this. They call it the 30, I think it's a 30 million word gap problem where you take kids, two kids, they grew up 0 to 4 years old. I think that's what it is. I'll drop the research in the show notes and Dr. Lisa Feldman Barrett in her research around the subjectivity of emotions and how emotions are actually made in the brain at the n equals 1 level is how I take it. When you take a kid that's poor and you take a kid that's rich and there's zero and zero and you get them to 4 years old, the rich kid, just by nature of the connections they have and the environment they're in, their ontology, ontological velocity is the way I think about it. They have a 30 million word surplus. Versus the poor kid. So they have been exposed to 30 million more words. It could be duplicate words. It could be duplicate ways of saying things or just, but the idea being the more words that you have to describe the world, your senses, stimulus coming in, how you feel, then determines your ability for emotional intelligence regulation, et cetera. But the underlying thing there is the ability to make sense of the world and yourself. To me, I tie it to this concept of ontological velocity. Which is interesting. When I learned that, then it basically taught me, because I want to be a dad in the future, I'm like, okay, I guess I'll just like use lots of words with my kids and try to be as descriptive of every feeling ever. So then they like don't have this gap, but it's an evidence thing. And I just wanted to bring that up and share it and get your thoughts at the intersection of what you're speaking about with subjectivity. Thought you might find that interesting.

Yeah, it's really fascinating. And I think that I've encountered her work, But if I'm completely candid, it has been a second since I've been in this space in University of Wisconsin thinking about this particular domain, and far be it for me to have the grounded expertise that I would want in order to build on it. But to some extent, like, it makes a lot of sense to me that ultimately what we're talking about is meaning-making and the ability One of the other things that I think is really fascinating is that, and you see this also in the Thinking Fast and Slow work of Danny Kahneman and Amos Tversky, is there's a lot of information that our bodies are processing that extend beyond what we can actively focus on. Right. And yeah. And some of it is pure sensory and there's both feedback and feedforward. Processes that interact with each other in order to give us this meaning-making. And so if we consider that conception is just of a— sorry, we consider that cognition is purely a function of what is happening in between my ears, right? If we think that my mind is related to my, my, if I think that my mind is just related to what is in my brain. Rather than how my body and my brain processes everything dynamically through the world, right? And that kind of situates it, its interaction with, like, that I think validates some of what you were talking about earlier, that children that have more of these experiences, they're able to make better meaning. And that might be connected to emotional regulation. I'm not familiar with the literature there, so I'm not able to make those claims. It makes sense to me that somebody that would make the claim that having more words, having more experience in the world impacts your meaning-making and ultimately allows you to engage with things more, yeah, in a more novel way.

You have a stronger CPU and Better context management in your brain. Bring it back to AI. Sorry, we, I interrupted, but I, you were speaking on your path from learning sciences and you're studying under around epistemics, et cetera. And then I think you were going to go towards like, how does that lens help you today? And that's really what you were speaking of is there are different lenses on which to view even the same thing. Right. And you were talking about truth. So like, how does that ground you as a founding engineer today working in AI?

I think in addition to the fact of like how we try and think about and tackle the nature of prompt injection and where we draw boundaries around the kinds of problems that we want to end up solving, I think the other piece to this that I find interesting is What is the nature of intelligence? Right? Mm. Like, mm-hmm. One thing that I've been kicking around for a while is the fact that when I look at the attention mechanism that is behind a lot of LLM, it strikes me as a fundamentally different form of intelligence than what I have. I experience, like, One thing we were just talking about embodied cognition, how I'm able to attend to just a few things, 5 things concurrently, plus minus 2 is what the literature tends to suggest. And that as I attend to these different things, it allows me to focus. My cognition is a function of that. And that's a very different kind of way of understanding intelligence relative to how an LLM understands it, where I can give it an entire document and it will derive attention matrices that attend to different tokens all at the same time. Yeah. Now it has varying weights, but it's a very different kind of intelligence and it shows me that one is not a replacement for the other, but it also informs the way in which I'm trying to bring the human element and what are humans good at relative to what are LLMs good at. What are the affordances of me as a human and my intelligence relative to the affordances of an AI and what it's good at. It's fascinating.

This is a fascinating discussion. Real quick, would you call intelligence, in your view, the same as consciousness, or are they distinct?

They strike me as distinct. I don't see— I don't see AI as conscious. I genuinely— like, a transformer model, encoder-decoder model, that is doing next token prediction, or speculative decoding that is doing a few different tokens at a time, it still is a stochastic process to me. And the fact that I can see, even if I don't understand what exactly is being encoded, the fact that I understand the nature of the process and it's sufficiently simple that I can understand the nature of the process, to me, I think it doesn't belie necessarily, far be it for me to actually understand the lived sense of if such a thing could be considered. But to me, it doesn't strike me as being conscious. It is stateless when there is nothing happening in the— there is no input, there's no output, right? There's no volition for these models. That said, do I think that they're intelligent? Hell yeah. Like, far more than I am in many different ways.

You want to hear a cool stat? There's a, I read this from Reid Hoffman's book, Superagency. I think it was, there's 2.3 billion ebooks worth of data created on the earth every 10 seconds. How many people do you know that read even a book a month? Like, right? And so if there are 2.3 billion ebooks of data being generated in the world, And then you go back to meaning making, right? There's, we can nerd out about this forever and that's why I love you, man. But real quick tangent is there's this idea that's been floating around this guy, Benjamin Bratton, PhD in San Diego. He's in the humanities. He's a philosopher, but he talks at the intersection of technology and he says that technology evolves when philosophy learns to ask the right questions. So who's asking the right questions right now? Another thing that he posits is we seem to think that we are the center of everything. But what if we are just a scaffolding the universe created to make meaning of itself? And we've gotten to the point where we've instrumented sensors to make the argument of there is climate change. We have evidenced it. And now we've created this new technology. And the way he says it is ape created fire from rock. And now humans have put intelligence into rock. Right? And so his thinking is now, or what he speaks about is like, what if we were just meant to make sense of the world enough? And then now this is the evolution of now we make a thing that can make sense of more things than we can. The total 2.3 billion ebook thing. I wanted to bring that into the conversation.

How does he reconcile that with not adopting a nihilistic worldview that nothing matters?

Yeah. Yeah. I'll send you the lecture, but I think the key thing that— now we're getting into philosophical discussion, but I like Camus' philosophy here, which is, are you familiar with Albert Camus? He's, he was writing in the time of Sartre and stuff like existentialist philosophy. I think he would agree with that maybe as an umbrella, but he basically had this idea that the Myth of Sisyphus is a famous like essay he wrote. Going in a quick nerd rabbit hole on philosophy and the meaning of life because now we've created this thing that like could process 2.3 million ebooks and I could run a book club, right? That's right. This is, I think AI builders are actually, they face this tension every day where I'm like months ago, even 4 months ago before Opus 4.5, I could not be shipping 100 PRs a week. That's, and then in 4 months I could see probably 1,000 if these things just keep getting better. And so then it becomes, there's, you have this thing you're grappling with every day where you're like, What will be my purpose, right? And it might not be loud, but I've certainly heard lots of AI builders talk about that. Okay. So real quick, how does Benjamin Bratton think about it? I don't know exactly, but here's a couple of other strands, other threads to pull on to, to talk about this, which I think is an important thing to talk about. Okay. So Camus talks about the myth of Sisyphus where we're all rolling a boulder up a hill, like every day. We like, Sisyphus was condemned by the gods. He like tricked Hades and he was condemned to just roll a boulder up the hill every day and it sucked. And then the boulder came down and he had to do it all again, all over again. If you think about what a 9 to 5 is in capitalism, right? Like it's like we go, we roll the boulder up, shit happens. It sucks. Goes down the hill, do it again the next day. And there's little side quests that we have. Maybe we trip on a rock on the way up one day. Maybe like we find a companion. Maybe we like, Maybe there's another boulder we got to roll another day. There's side quests, but that's the main quest. And how you view the main quest is your own lived philosophy in life. So Camus says, if you understand that, then the only real question is, are you going to commit suicide knowing that there is no meaning? That's his take. Back to your point of the nihilistic view, and his take is that the best way to live life, this is my interpretation for the philosophy nerds out there that maybe know Camus more than me, is you have to imagine Sisyphus smiling. That actually in the suffering is where you could subjectively just find meaning. And that's really all it, there is no objective meaning. You make your own subjective meaning. So you're rolling a boulder uphill, keep doing it, smile through it, find your little side quests, enjoy it. So back to this, which is if apes have put intelligence into, or if we apes have evolved to humans and we put intelligence to rock, like what is the meaning now? And this is the thing that I come to actually a lot in entrepreneurship as a leader. And I heard this from, I think Horowitz, but he said something like a good leader looks into the abyss and they see two bad options and they say, I think this is a little less worse. Let's go there. Okay. The same thing applies to me, which is true intelligence. And we could debate the word, the usage of intelligence there versus what we used before. But I think true intelligence, or maybe intentional intelligence, is being able to hold two very conflicting attention thoughts or ideas or concepts in your head. Like nothing really fucking matters to, I'm going to try anyway and have hope. To be able to hold both and still take action. I think that's a measure of true or intentional intelligence. Intelligence. So yeah, that, that quick detour into philosophy land, but wanted to add those thoughts and just see what you thought of that.

Yeah. Can you say that true intelligence bit one last time? Yeah. Yeah.

I think true intelligence or intentional intelligence is being able to hold two very conflicting ideas in your head, accept them, and then go move forward anyway, such as There is literally no meaning to life other than whatever meaning I make of it. So then I will go make meaning of it. Or, hey, this situation seems completely hopeless. I'm going to choose to have hope anyway and try. Are we all going to be obsolete with the new age of AI in whatever, 10 years, 15 years? Are we all going to die from climate change? I don't know. Lots of things might happen. I might hold that as a possibility, but still going to wake up every day pretty stoked and get after it on my dreams. Right. Yeah, I think that's really fascinating.

And by that standard, I think that certainly humans exhibit true intelligence, but arguably so can AI. Like, when I'm talking and bouncing ideas off of Obis, for example. Yeah. Yeah. I— find that I'm able to say, oh, you know what? That wasn't quite right. And it's able to adapt and integrate that new feedback and like that adaptability that there could be conflict that it's not just gotta be one versus another.

It's not black and white. Yeah.

Feels like that at least fits that true intelligence shaped hole that you were just, um, referring to, and that also might suggest that some of the earlier versions of LLMs might not have been truly intelligent. And that might speak to how anecdotally there's been a lot of consensus how the models in December 2025 ended up being a really pivotal moment in intelligence and usability. And a lot of people that had tried AI, tried coding agents before and said, yeah, not for me, it's not quite there yet. That was their conversion moment where they were like, oh, this can be really impactful. And what about that true intelligence in the context of AI? Like, how did we get there? When did we get there? And are we free?

But dude, that's infinite fodder for more ponds. There's so much I could say there. Yeah, this is probably a good point to start to wrap just because I want to be mindful of your time, but we absolutely should just have you back to— there's so much more to talk about, man. Like we just, I just opened a whole Pandora's box on you, but we should talk about this later. True intelligence philosophy, meaning making. Those are all great topics that we could harp on for days. I think one thing to just cover based on what you said for the pod audience, for those that haven't used AI, or maybe you have and you were an innovator on Moore's crossing the chasm. I always think about the innovators, early adopters, early majority as your first 50%. And then you have late majority and the laggards, like, like when the iPhone came out, hipsters bought it. Then now my grandpa uses it because it's easy enough to use. So that, that's the way I think about technology adoption. With AI, it's like rapid adoption. And I knew a certain class of people that fit the, hey, I tried it and it wasn't for me. I feel like I got to patch so many things it does. And then they tried it again, like you said, end of December 2025. And it's whoa, this, I could one-shot 6 months worth of work if I just think about the spec right? What? And actually I just did this last night. I like refactored a whole repo. And I thought about it and I take it, I try not to take it for granted because it's such a magic thing where you could just leave Cloud Code running overnight and it ships a 6-month feature that would have taken 6 months overnight now.

6-month product. Yeah, product.

Yeah. Not even a feature, but like a product or even just refactoring a whole like brownfield codebase. Oh my God. And there's parts of it actually, maybe we can close with this. I'd love to hear like your, just like how How do you do AI-native development? What's your tech stack? And then how do you go about it? Because we've talked about, you know, number of, we could ship a whole product in a night, or we could ship a prototype super fast in minutes. We could ship 100 PRs a week. Maybe we kind of end there, which is tell me about your tech stack and your working process and your workflows and all that. Yeah.

So my tech stack, I am, uh, there's two things I want to say. The first is that. I think that my company is fairly early adopter, bleeding edge, and even within our tech stack, there's more to do. I'm gonna send you a recent talk that the, an individual from Trail of Bits, who is, I think he might be the CTO. But suffice it to say, he gave a breakdown of what AI native means and the different levels of AI adoption within an organization that, that can be really useful. Yeah, great language. And also what that looks like, what that translates into in really tangible ways. Habits. Okay. And one thing that he talked about is like standardizing on using one coding agent and in his organization, it's Claude Code. In my organization, I actually, I still use multiple different LLMs depending on what I'm trying to go for. Right. So I find myself really often using Claude Code for some of the more creative stuff and then using Codex for doing high quality code review of when Claude code goes off the rails and—

actually, yeah, I realize Opus sucks at code review and Codex is good at code review. That's a thing right now. That's a meta right now. Keep going.

Sorry. And so Hamel, I don't know whether you're familiar at all with Hamel Hussein. Yeah. He just put out a skill that basically uses— it goes to Codex to do the code review automatically. On things that Claude Code do. Yeah, here, I'll drop this also in the— I love that— Claude Review Loop, and it's a Claude Code plugin that adds an automated Cloze review to your workflow using Codex.

Dude, heck yeah, I do need that.

Shoot me up with that. And so, like, things like that are part of my tech stack. I will say that I strongly believe that no matter what you're doing, you need to have consistency and strong conventions, and that the ability for us to have an appetite for what is acceptable tech debt to move forward has to go down significantly. It's a shift in paradigm thinking. It used to be that perfect is the enemy of good enough, and that's still true, But when it comes to what we're shipping and how we're shipping, we really do need to focus on, yeah, if my test coverage isn't good enough, or if I don't have any tests, but it works good enough, let's go. That at this point feels pretty dangerous to me. Like, I need ways that I can maintain guarantees so that my coding agent can continue to get feedback from its environment in a real big meta way. This is like, how do we instill a form of embodied cognition into our coding agents, right? It's a feed forward to get a feedback loop so that we can enhance the cognition of our coding agents. And so that's what capturing that back pressure is all about, is making sure that you are able to leverage everything, all the signal that's generated from your environment and bring it back into the intelligence center to process that we can actually have a coaching system.

Dude. Oh my gosh. There's so much to say there around what you just said about like, how do I get my agent to have an embodied cognition, but also subjectively map towards basically my business and product roadmap in my codebase. Cause you can't just, cause what you're doing speaks to, you need the right tool for the right job for your case, which is Codex for this, Cloud Code for that. We just got a, um, shit. Okay. All right. We got to end the pod. Ariel, we'll chat again soon. Thanks for joining the pod, everybody. Thanks for listening. I'll catch up with you soon. Take care, man. Cheers. Bye bye. Take care of yourself. Take care. Wow. I learned so much on that pod. And there's just more to even talk about with Ariel. I always love speaking with him. We always go back and forth on lots of these deeper, complex topics at a very good level. So very grateful for that.