Your Brain LIES To You

The first time your brain ever lies to you is the second time you open your eyes.

Donald Hoffman talks about like evolution evolving for fitness and not vertical truth. And he says if any organism was trying to optimize for vertical truth, it would basically go extinct immediately. And so what we're left with is a bunch of things that are all lying about the environment to themselves in order to fulfill a homeostatic purpose. I think in a spectrum of a population of humans, it's actually a feature to have, you know, some amount of neurodivergence and to have different ways of looking at the world. I don't know if there's a right way to look at the world.

I see a future with AI scaling expertise like that. So imagine you're an enterprise, you want to get Seam Land's expertise, like you're 100 different businesses that want to maybe get his expertise, I don't know, flowing into your organization for your employees well being or whatever. What if he just had an agent that could do that work for you? Hey everybody. Welcome to the ABCs for building the Future podcast. My name is Robert. I am joined today by my co host and co founder of Epistemic Me, Jonathan. Today we're going to chat a little bit about consciousness, philosophy and AI and then we're going to get into build in public update. Jonathan and I are going to go over an exciting product release that is impending for the Epistemic Me team called the SDK Playground, oriented towards developers to come to our website to check out our product, to get familiar with the data model and easily bootstrap a hyper personalized agent and be able to simulate dialectics and get awesome evals to make it better. So with that said, Jonathan and I recently went to the Vitalist Bay Summit and Jonathan actually got to talk to one of the speakers, Patrick House, who is doing work in the field at Stanford. And he got wind that Patrick House has actually released the book. And because we've been on kind of a consciousness tailwind, Jonathan and I just started reading it and we've gotten through a few chapters. So because a lot of what we're doing is building and implementing these concepts of, you know, within cognitive science, neuroscience, consciousness into reality to solve AI alignment and improve chatbot development, we thought it might be cool to talk AI bit about his books and some of the themes he talks through, so then we can connect that a little bit to what we're building in Epistemic Me. So with that said, Patrick House's book is called 19 Ways of Looking at.

I briefly met Patrick. We maybe talked for five minutes or something like that at the VIP dinner. And I told him about epistemic me. And I usually describe it like, you know, that we're making digital twins for people to utilize personal health. And it's based on a belief system model. And if. If the person is of a certain rigor or scientific. I'll say it's. The belief system is based on Carl Friston's free energy principle. And when I told him that, he. He said, oh, you know, I wrote a book about consciousness, and I have a chapter called Secondhand Markoff Blanket. And Markov blanket is a key, key principle in the free energy principle. And. And it's like. It's like a totally nerdy thing. Like, I actually. Actually bought a blanket that's called the Markoff blanket, just because I was like, this is. This is so nerdy. And he similarly was making a pun. And he told me that it's like a unique sort of error in the book because he made it, like, secondhand, as in secondhand of a clock, secondhand markup blanket. And it actually is a way to, like, how do we distinguish, you know, the sort of, like, morph, I guess, the teleological changes over a lifetime. You know, like when a caterpillar becomes a butterfly being, like a change in time. Whereas the Markov blanket is usually thought of as a demarcation of a boundary system in space. So he kind of, in this chapter six, postulated that we need something like a Markov blanket that describes time in order to really delineate where somebody. A human being begins and ends and what the boundaries of it are, which I thought is a really cool idea. And then he shortly thereafter mentioned. He was. Because I mentioned Annika Harris book on Lights on, and he's. Yeah, she interviewed me in chapter four. I was like, cool. I've also read that book. I didn't realize that he was the one, but that was the one about plants and parasites. And I think one of Patrick's really interesting ways of looking at a being is to consider, like, that we are an accumulation of all. Do you consider a human being the accumulation of all the parasites and cells that its DNA defines also? I mean, could they coexist with? And so he's got deep insights into this parasitic relationship that is, I think, misunderstood and, like, deeply interesting.

Yeah. Real quick context for the audience. So when Jonathan said he met Patrick House, the neuroscientist and writer in question, at the VIP dinner, this is an extension to The Vitalist Bay conference that we both attended. So that was the Longevity conference. We had a different podcast where we debriefed on still more to honestly debrief from that one. There's.

It was a good time and we.

Met some of the best minds in the industry and the space when it comes to working on these problems of consciousness, cognitive science, neuroscience, et cetera, and in particular, just reading this out. So for anybody that is interested in Patrick House's work, he focused on how mind control parasites altered behaviors in rats. Fascinating stuff. That's. Wait, what. We fund these things. That's so cool. What. So, yeah, that obviously has layers of.

Implications and you might not think that's that relevant, but actually that's the one that's carried the parasite that's carried by cats. And it can permeate through the human blood brain barrier. It's the only parasite, I think, known to do that. So it's this highly adapted thing that it needs to. Needs to reproduce in the cast intestinal system, but they, you know, kind of live in the nervous systems of other mammals that cats acquaint with.

Wait, wait, zoom out a little bit and just. Let's just side quest. Tell the audience about this. You went a little deep, set some context for the cat parasite.

I didn't know if this is like a broadly known thing, but cats have this parasite. It's like toxoplasmia or something like that. And so, like, a lot of people that have cats are exposed to it, and it's actually a. A neural parasite. And it's so interesting because it's. It's one of the few being. It's the ones these things that can cross through your blood vein barrier and then live in your brain for humans.

Yeah. There are humans out there with this.

Yeah. Or. Or say any mammal. Yeah.

Okay. Okay.

That's. That's why he was studying rats. Like, it's probably, you know, like a proxy for whether or not humans are affected by that. This, this neural parasite.

Yeah. And in particular, he, like, was studying the. What he found was this parasite could change the behavior of rats and essentially could make them less fearful of cats, which they have a very natural emotional triggering and flooding that protects them from an evolutionary fitness function standpoint to be scared of cats. So that was a really interesting, demonstrable thing where it's like, oh, we could change the wiring of this rat. Yeah, that is very interesting. So that was what.

Pretty complex adaptive strategy. Right. Like for the cats.

Right, for sure.

To develop a. Yeah, yeah.

So we have this question we always ask in the pod, which is, how can AI understand us if we don't fully understand ourselves? And the more we get into consciousness, the science of consciousness, neuroscience and stuff like that, or even hear about thought leaders, I find that is singularly like the question, which is why we're doing this work. So some of the themes in the book that I got so far, Jonathan's finished, I haven't. But it's that the brain lies to you by design. That was a really powerful thing that Patrick talked about. He said, the first time your brain ever lies to you is the second time you open your eyes. And I was like, wow, that's so intriguing that the thinking being when you first open your eyes, you have sensory input. You had nothing in your brain. Your model didn't exist, and then now you exist. You open your eyes, you have the sensory input, you start constructing that model, and your brain is projecting these, quote, lies, or what the world looks like around you based on that. So I thought that was really powerful, quote, the first time your brain lies to you is the second time you open your eyes. On that note, I know that you have mentioned there's another book I actually have next to me being you, your favorite book, Jonathan.

Yeah. Like that one.

Yeah. And he talks about beast machines. So I thought, in particular, connecting these two topics, I'd love to hear what you think about that. Your brain lying to you, us projecting a model of the world, or our brains projecting a model of the world, connecting that to the concept of beast machines. And actually, let me just read the context setting part of that because it was really great writing. In the mid-1990s, Anna, age 16, had brain surgery to alleviate suffering from epilepsy. Because the brain does not have pain receptors, she was allowed to remain awake the whole time with only a tiny amount of localized anesthetic to numb her scalp. During the procedure, the doctors and attendants asked Anna a series of questions meant to keep her talking. And as they probed her brain with electricity and micron thin blades hoped that she would not stop. So that's what Jonathan meant. And tying that back to, like, your brain lies to you in that moment when her brain is opened up and she's undergoing the surgery. And during this, when the attendants were just asking her questions, trying to keep her alive, basically, they observed this phenomena where she was just laughing once they had electricity put into a certain spot in the brain. And then when asked why she was laughing, like Jonathan said, she just made up things like, I'm just reading this Right now. But saying there's a horse around that it's funny, and just saying everybody around her is funny in the room and just like making things up. And she didn't receive general anesthesia. So it's really interesting for that particular case that demonstrates, like, oh, wow. In this very niche situation that you have not been in before, what does the brain say about it?

It's a pretty complex behavior to arise from a simple charge being applied to part of the brain. But then when they asked her why she was laughing, she never identified that it was, like, uncharacteristic of herself. She would say, oh, I was laughing because of the movement of the nurse over there. Or I was laughing because I thought that pamphlet was funny, or something like that. She would make up a story. She basically invented an explanation to describe the behavior that was happening sort of underneath in her subconscious. So that's, I think, a pretty good articulation of how we're all just trying to tell an explanatory story. Oftentimes, you know, that would may explain something that we. That we're doing with without really a greater recognition of it. It might actually take us longer to understand why we're actually doing it. That's the kind of lie. I think that was pretty illustrative. What do you think of an example of a lie that you tell yourself in the sense of.

Yeah, I thought that was such a good example. Right. Yeah, right. And we basically just see a bunch of real life malfunctioning of the brain because it can't make sense of it. Right. At least that was my takeaway from reading the chapter. And I'm just sort of thinking about times when my brain was trying to make sense of something that was so foreign to me.

And you might have been aware. Right?

Yeah, seriously. I mean, I go back to when I nearly died and I was full of adrenaline. So I didn't believe I was nearly dead. I didn't. I don't think my brain wanted me to believe I was nearly dead. I was like, yeah, I can still walk. I got this. And then I took a step and tripped. And that didn't work well. So I think my brain lied to me then about how much pain I was in. Certainly I think the adrenaline helped with that. And that probably, honestly, looking back, kept me positive and in a good state to fight for life.

That's a really interesting example because where the body's homeostatic systems are kicking in to help you write a narrative that's constructive for the situation you're in.

Yeah. And it, you know, what's interesting is like, your narrative is always personal, right. What matters. You're the hero in your own story. Right. So there are many narratives I could have had then. The one that I had was. The one that I had that kept me going is that I'm the underdog and I'm going to get out of this. Of course I'm going to become stronger past this accident. I'm not going to let this take this take me out. I still got my dreams to pursue. My brain, I think in reviewing my life and going through a lot of therapy is my brain likes being the underdog. So I think it really told me that narrative and that shit kept me alive.

Yeah.

I'm curious if you have anything that comes to mind when it comes to thinking about your brain lies to you.

Yeah, I think, I don't know, maybe feelings of depression, I feel like, are lies, you know, and they skew in a way that, like, really things are more important in the current, in the current time or, you know, you kind of pull the future in and make it. You could catastrophize it and make it a bigger catastrophe than it is internally. And I think that's. It's a misrepresentation of future reality for yourself, you know, I think, yeah. And you can, you can use tools like once you experience depression enough to say, no, there, this, this happens farther out in time we have, there's more time to consider this, this isn't as big a catastrophe as you think it is. There's a lot of normalizing sort of that you can do to unwind the lie and make yourself feel more normal about the reality.

That is such a good one. When we have Bushoy back, let's talk to him about that, which is, yeah, mental disorders are these just lies that our brain tells us. I mean, the, I mean, after reading brain energy by Dr. Christopher Palmer, he talks about how, like, mental disorders are really just symptoms of the root cause of metabolic disorders of the brain and accumulated toxins and stuff like that. Or I like that conceptualization that depression is your brain lying to you. That's. That's pretty good. Next time I encounter it, I'll try to. I'll try to tell myself that because it's so interesting. I feel like I went through a micro depression recently actually because my brother was going through a hard time. And because of that, I was really stressed and I was doing a root cause analysis actually today of like, how did I get into this microdepression? And then how do I make sure that I don't get into it next time or if I do recognize it faster and get out of it faster? So I did a bit of a retro today, so it's funny that this came up, but I realized that when I feel a sense of loneliness in connections that matter to me, like the one with my brother, and if there's a threat to that connection, which then incites the loneliness, my brother's not doing well. He's going through a hard time mentally, and then I worry about him, and then he'll mess up. Like my sleep and my physiology, basically, like my health, that stress accumulates to that, and then it cycles into. I don't sleep well, I don't get the autophagy or like, restorative sleep benefits, and then the stress and those things accumulate, and then I'm less likely to be positive or have less of a. My. My fire feels a little dampened, you know, so it is interesting because the brain operates from that place of scarcity. It's, oh, you might lose your brother. And I think that's what I felt, looking back, that. That is interesting.

So how do you.

How do you find truth in those moments? I guess that's the question. It's hard.

Yeah. I think there is a lot of truth in the feeling, too. I mean, we'll get to what vertical truth versus, like, useful truth is in a second. But, you know, I think that the feelings that you have when somebody close to you is in danger and you're worried about them are real. Right. And then there's. There's kind of like a layer beyond that about, like, how can we both address this issue and what's constructive about it that oftentimes your emotions hijack, you know, and overplay a role versus a cognitive, you know, planning mechanism?

Yeah, definitely. Definitely. Do you have more to say on that topic, which is like mental disorders or illnesses being kind of like your brain lying to you?

I don't know. We have this tiny term neurodivergence now, right? And I think in a spectrum of a population of humans, it's actually a feature to have, you know, some amount of neurodivergence, tend to have different ways of looking at the world and, you know, it. I don't know. There's a right way to look at the world. There's a way to look. To look at the world that's. That fills a fitness and adaptation function for us as a group and for us individually and for all our ancestral homeostatic functions. And that's kind of like what you know. I think Harari talks about the first in shared fictions that serve a purpose. They're shared fictions, they're not shared vertical truths. Donald Hoffman talks about like evolution evolving for fitness and not vertical truth. And he says if any organism was trying to optimized for vertical truth, that would basically go extinct immediately. And so what we're left with is a bunch of things that are all lying about the environment to themselves in order to fulfill a homeostatic purpose. And then you have Neil Sess with his beast machines, which basically applies to the single metaphysical self. And it's a recapitulation of all the homeostatic processes in the body bubbling up to create a constraint that you have a controlled hallucination of self, which he calls that kind of his beast machine constructs. And he uses Friston's free energy principle to quantify that mathematically.

Yeah, for sure. It's quite fascinating that the brain is just constantly trying to make sense of the world. And so when you're in these experiences, like Anna was in that brain surgery experience, your brain is essentially just trying to build a story, a narrative for you from this mess. And that could just be a mess in itself. In chapter one, he also had a really interesting citation where he mentioned. So he had that quote, the first time your brain lied to you was the second time you opened your eyes. He talks about how seeing is an experience based inference performed effortlessly and expertly by the adult human brain. That's the quote from the book. And that makes sense. Like when you are an adult, you're out in the world, like just imagine getting out of bed. And the amount of information, right, that you take in, even just getting out of bed, like alarm clock goes off, your brain registers that that is a certain time that you needed to be up at. And then you need to actually go do a bunch of things. Take the covers off, move your legs, like step on one of them, step on the other one, walk to the bathroom. Those are all things people just do, but they never think about it. But if you had to think about every one of those motor functions and all those things, shit, that's a lot like. So really our brains are designed to just help us be in motion towards how we need to actuate as best as possible. So basically it just filters a lot of things, right? So you get more signal than noise. So he had this other thing in there which was cool. He said, teenagers with cataracts, blind since birth. So teenagers that are blind since Birth. Upon opening their eyes after cataract surgery and seeing for the first time experience featureless, depthless, shadowless blobs, they could see the same number of photons as an expertly seeing adult. But their brain sees nothing in the raw stream of light because their brains had never seen a coincidence before. That's so interesting. That was such an interesting. I didn't know about that. But that drives the point home that if you didn't feed data into the model before it's blind, there's nothing there. Right? Like quite literally.

Yeah. Yeah. That's a really powerful example. I think we all have the benefit of years of sensory data to train and you know like he talks a lot about movement characteristics and like these, you know that our neurons is sort of like preconditioned towards the movement patterns that we experience over our life. Right. And then I never thought that you know, perceiving light could be the same way. But yeah, it makes a lot of sense that identifying meaningful causal patterns and sensory like signals is not just a built in thing, you know, like it requires some, some training.

Yeah, it requires training for sure. Just like foundation models.

Yeah.

Cool. Do you want to talk about the pinball story a little bit? I love that story. That was probably my favorite so far personally. What'd you think of that? I could give a quick recap as well of the pinball.

Yeah, that was a cool metaphor. Yeah. I've recapped a great.

Yeah. So chapter two was about the rise and fall of pinball. So anybody that has been to like a retro classic arcade in person arcade has probably seen pinball machines or maybe your local like dive bar has a pinball machine. I remember I went to my first, I played my first pinball machine back. I think it was Nickel City. There was this like arcade that branded as you get to play games with a nickel and, and being a broke kid, that was the shit back then. So I was there all the time. And so it was interesting that I reading this essay I had the experience of having played pinball in real life and then also when, you know, Windows PCs were more mainstream in the early 2000s and mid early 2000s, then it became a game you just get out of the box. So okay, this chapter, Patrick House, he compares the evolution of the brain to the evolution of pinball machines. They're messy, they're janky, they're layered adaptations stacked to older architectures. So his whole thing was a pinball machine started like this. Then they added more and more features. It was a simple mechanical Game. And then it went into, oh, now it's digital. Can we digitize everything? It used to be all analog. Can we create the same experience over here in the digital world? Right. So the takeaway that I got looking at some of my notes is that consciousness is not a singular function, but it's an accumulation of legacy systems. And that's where the data inputs and the model all blends everything together. Then I wonder, did we all standardize on a storytelling template just to speed up information flow? Did we all just adapt to the same storytelling structure just to maximize information flow? Is that why that happened?

Are you talking about Joseph Campbell's monomyth?

What is that?

Oh, yeah, Interesting. So Joseph Campbell, you love this guy. He's the one who coined the Hero's Journey as a story archetype. And so he had studied, like, he was like an anthropologist or something like that. He studied storytelling through different cultures over time. He found that there was a common structure to them. He put them into what's called the Hero's Journey, and Star wars was born. So he had a similar thesis that, yes, there is inborn utility to a certain structure of narrative that helps us to pass on stories.

Yeah, interesting. Wait, let me just read this out for everybody that was like me and uneducated about this, but has felt this or experienced this before. Joseph Campbell introduced the concept of the monomyth, or the hero's journey, in his book the Hero with a Thousand faces, published in 1949. The monomyth is a narrative framework that describes a common pattern found in heroic tales across different cultures and historical periods. He posited that this pattern is universal and transcends cultural and regional differences, reflecting shared human experience. The hero's journey typically involves several stages, such as separation, initiation, return, where the hero ventures into a supernatural world, faces challenges, returns with a boon or transformation that benefits their community. His work influenced numerous storytellers like Star wars and has been applied to analyze various literary works and films. Interesting. Oh, and then I'm reading a part here. It's faced criticism from scholars, particularly folklorists, who argue that it suffers from source selection bias and oversimplifies the complexity of myths and stories. Critics suggest that the monomyth reduces the world to simple binaries and favors male protagonists, neglecting the nuances and diversities, diversity of narratives. Oh, shit. I didn't know we were walking into something like that.

Oh, the critical theory analysis of Joseph Campbell's work. Yeah.

Interesting. Cool.

Yeah, I mean, like, most signs from 1949 is gonna.

Yeah, critique yeah, yeah, but there's always gonna be haters with everything, man. Yeah. Okay. But that's really interesting.

Yeah. And I think you all know Harari's books, like the foundation of them, basically kind of expands upon Campbell's work and talks about, you know, the myth as. As the bonding feature of what humans are.

Oh, interesting to bond across, like, cultures, just.

Yeah. The bonding force of human. Of a human culture is its storytelling. It's mythology. It's shared fiction. So shared fiction as, you know, something that can bond culture. And, you know, that's. You've probably heard Nishi's quote, the God is dead. You know, what do we do? He was really talking about, oh, like our whole shared fiction was in this religion. When religion breaks down, we don't have a shared fiction anymore. What's going to happen next? And we've all been living that life since then and all the cultural battles that come with it.

Somebody give me the next dogma. God damn it.

Right.

Here it is. It's don't die. Okay, so that was the pinball chapter. So let's spend a little bit of time on the next chapter and then we'll expand on this in future episodes and then we'll get into the build and public update. The next chapter is about bird migrations. And that was interesting. It basically just talks about how there's this inherent gut instinct for a certain type of bird to go migrate. And the theme that I remember from this is that the brains ultimate purpose is efficient movement to help you actuate as an agent in the world. And then some of the topics touched on from that were around consciousness existing to model movement when actual movement isn't possible as well. Yeah. I wrote a note here that from this chapter that all thinking is a simulation of movement, of curious if you have any, you know, reactions to that or. Or even just the overall concept that the brain's ultimate purpose is efficiency and movement.

Yeah, I mean, we. We talked about embodied cognition a few weeks ago. Right. And the. So when. When you look at the evolution of animals in the brain, the nervous system, the basal ganglia was like the first structure of the centralized brain. And its entire purpose was basically to. To facilitate movement coordination, you know, amongst all these billions and billions of cells to actuate an environment for metabolic purposes, for fight or flight, you know, and so what the basal ganglia basic function does. And. And so. And you can imagine basal ganglia as like your spine, you know, going up to the brain stem, the oldest part of your brain, and it has, you know, it's just kind of aggregating all these nervous system inputs, and it. Different afferent aspects of the nervous system is aggregating them, and it's actively predicting, you know, like, how the organism should move next. And so that's. That's the base of cognition and, you know, like, it. I think if you think of everything else working on top of the basal ganglia in terms of functionality, the basal ganglia itself, you know, you're trying to predict the world in ways that are built on habituated patterns of movement and habituated patterns of. Of cause and effect in the world. And it can be entrained to do that. Yeah. So everything else we have just kind of like, works on top of that from an evolutionary perspective.

And so just to explain the whole bird migration thing, I'll read a few sentences from the chapter. He creates this metaphor of saying that we all biological, Species, Creatures. Organisms, have some sort of migration anxiety for actuating. That's how all of us kind of work. So in a simplistic view. So he talked about this from the book. A friend of mine, a bird watcher, once told me that the best time to search for birds is right after a storm, because the grounded ones are very anxious to get going again. He called it zugunru. I'm not actually sure how to pronounce it. It's a German term. I'll throw the word link into the show notes. Zuganru. I'm gonna say that for now, our German viewers, please don't hate me. I'll correct myself after this pod and translate it roughly, perhaps poetically, as the anxiety felt by migratory birds prevented from migrating. A body too, Patrick says, is restless to get moving. In fact, the entire purpose of the brain is to make efficient movement from experience, and everything else, including consciousness, is downstream of these efforts. So, yeah, I thought that was really interesting. And in particular, when I read this, I was like, man, this guy is a writer. I just had the pinball one, and I'm like, what is this so different than the other consciousness work that I've read? And then I got into this. I'm like, man, dude's talking about, you know, bird migrations coming out of pinball history, and it's all good. So, yeah, I thought that was. That was really interesting. I had learned a new word. I never knew that word. Yeah. Did you have more to add? Like, what. What was your reaction reading about the migration anxiety concept?

Yeah, I. I mean, I think it's very illustrative, I guess, how I would say it's just a great way to talk about because again this chapter is just like another way to look at consciousness. You think about the birds, you know, point of view and like what they're trying to accomplish and these kind of embedded movements that they're supposed to express. You know, like that's, I didn't know about that before reading the chapter. It's really interesting and it's just like a really interesting way to think about the embedded biases or you know like inherited characteristics that like this kind of consciousness carries with it.

Yeah.

And yeah, I, I have read and thought more about the nervous system and like where he talks about one tenth of a second response time to the world, you know, so that's, that's a hundred milliseconds. Right. And it's interesting like the, the fastest a trained sprinter can respond to the start gun at a race is, is a hundred milliseconds. So it like any organism body like to just kind of, you know, take all the inputs from the nervous system and then respond with a whole body movement. Even in train, even, even habituated. I think it takes a non trained sprinter maybe like 150 to 200 milliseconds to respond. But you know. Yeah, that's, it's interesting to think about that because that's really kind of slow. Right. The time it takes for an impulse to get from your neck to your head is about 14 milliseconds. And you know, you can turn your head given a lot large sound a little bit faster than you can actuate the entire body. So you think about all the sensory organs coming from the tip of your fingers, the tip of your toes, through your nervous system, being prosed by your basal ganglia, processed by your basal ganglia and then actuated into the environment. There's a lot going on there and 100 milliseconds is pretty impressive for sure.

We all have these models that are constantly being updated that help us make sense of the world because otherwise there'd be way too much information to process. Literally just too much data to process. Yeah, he made a tie back to Anna from chapter one. In this chapter about birds to, to accentuate the point of consciousness being useful for movement, he had this one paragraph I really liked that I wrote down and noted down. In a sense, all movements give away their owner's secrets. Consider the simple act of laughing without reason. When Anna, a teenage girl suffering from epilepsy, underwent surgery to remove the part of her brain that caused her seizures. She laughed after a surgeon stimulated a certain part of her brain. But why did the surgeon, pulling a single puppet string there in her brain, create a whole cascade of muscle activity and emotions? How did one burst of current create so much activity? In part, this was because some actions are so precise and so rote. Walking, laughing, that when the brain activates the patterns of muscular activity, the whole thing goes off at once like a sprung mousetrap. Isn't that interesting? And actually, the next part is interesting, too. But we can go back further, too. The timing of her laugh all started in her mother's womb, before any of Anna's neurons had matched to their destination. And when all her muscle cells had a natural pulse of about 10 times per second, at that time, all muscles would fire together, coupled electrically like rhythmic fireflies. When these muscles later started connecting to nerve cells, two things happened. First, the nerves brought the timing of the natural muscle rhythm into the brain, Encoding the timing of these pulses 10 times per second, or 10 hertz, as a sacrosanct feature of interacting with the world. Second, the muscles, now freed by their attached puppet strings, could move independently and start to learn. Then the brain kicked in its only job, to reduce the metabolic expense of. Of any and all of Anna's present or future movements. Her brain fed off the statistical regularity of observation, prediction, and action, because the only output from all of its work are cells in charge of telling her muscles when to move. That's so interesting. And, yeah, that's where that 10 hertz thing came from. I started realizing, like, we all just have refresh rates for our different sense series. You know, it's all just data, right? Like how something tastes, how something sounds. Like for races, for example, using that gun, like Jonathan said. But I thought that was a really cool analogy, too, of, oh, there are puppet strings here, and there are really complex puppet strings that are created from your brain learning about the world and adapting to the world, right? So, like, when this thing happens, orchestrate all these puppet strings in a certain sequence. That, that. And then here, you're laughing now while we tickle this part of your brain.

Yeah, that's kind of a. It's kind of a wild articulation of the embryonic process, right? Where, you know, we. We're born into this situation where all these pieces are pretty well assembled, you know, and already. And then, like, by the time we're an adult, like, they're even better assembled. But you talk about the puppet strings and like, our. Our prefrontal cortex doesn't finish development and put all of its puppet strings into place until we're like 27 years old, you know, and so, like, like they talked about a phase where the muscles were contracting together without the nerves. You know, you have the, the endoderm, the ectoderm and the mesoderm. I think the muscles are in the mesoderm. And layered on top of that is the ectoderm, which has your skin. And then another layer that, a fourth layer that forks out of it called our nervous system. And so all of those, like, they, they evolve according to the. Or they. They. We recapitulate them or like their homeostatic functions, at least in our embryo, in the order that they sort of like, adapted and evolved over time. And so the more complex neural instrument that our, our. Our embryonic process is laying down to, you know, have puppet strings, the, you know, the more advanced sort of like, evolutionary functionality, we can integrate, like these higher levels of the brain, like the prefrontal cortex. When we talk about what consciousness is like, it's just this. I mean, at least I picture it as a continuation of this embryonic process where we're conscious the whole time. But the more these layers of adaptations layer themselves on and make themselves available to us, the richer our felt experience gets. And that's what it is to feel like a certain thing is just recapitulating all of these homeostatic processes one on top of another. And our ability to constrain and tell stories about reality is constrained to what's happening at all these layers at the same time. It's been constantly ongoing.

Yeah.

And I think you can think about how AI is integrated into the human embryonic process to some degree in this way.

More about that. Say more about that.

Well, from a teleological perspective, isn't how do we change over time our embryonic process of making decision of where is the next stem cell differentiate itself? Like, where do I put the next stem cell? What is it going to turn into? Is it going to be like a new brain cell or something like this? And if we just see continued, like, growth as a human is, you know, continued evolution. Like, it could be that we have an adaptation of the higher levels of our nervous system that then are, like, built to receive connections from silicon wires, you know, so you could see synthetic, biological or genetically engineered, like, types of humans that like, are extending the embryonic process specifically to afford adapters to an extended nervous system through, you know, like an LLM type of interface or something like that. Where, where does. It's sort of like a transhumanist way. Where do we look at like the next field of adaptations that, that take on, you know, you could put them a lot of places, but I think like the most suitable in the highest cognitive, you know, articulation of our stem cells in, in the prefrontal neurocon, neural neocont cortex. And yeah, it's, I don't know, there's, it's one way to look at like how AI and humans evolve.

Yeah. I mean like even one example I'm thinking of a tool that's starting to make it to mass market is the Warby Parker. Like AI meta lenses or whatever.

Yeah, right.

I mean that's. And then there's developer vibe coding. Right. I mean so in these ways we have like developers vibe coding. You could argue there's another lens there of like how do I work with this thing to do my job. Right. And then on the meta radon side it's like, oh, this is like visual sensory integration in everyday life. Not just in a vertical like my, my job. And I bet those humans will have a very different looking brain than a human that doesn't put on like Ray bans ever. Those Ray bans ever. Thirty years, I'm sure, I'm sure.

Yeah. But you can think about like that sensory interface, you know, through the Ray Bans or Apple's you know, VR headset because it measures the, measures your focal lengths at like 4 milliseconds or something like that. And so if you could, if you could meet that sensory perception with an actual nervous system connection that could like, you know, integrate into the sensory loop of our sense making on an ongoing basis. You know, there's, there's like sort of a, there's a hybrid human in our future that you know, would have like a supreme competitive advantage in a lot of different areas by co evolving the embryonic process along with like where you meet, where you meet in the current technology in the sensory organs.

That sounds scary when you put it that way. We're going to have this superior human thing. But I think you're, I mean you're totally right. I think the future is full of cyborgs. I'll become a cyborg at some point.

Yeah. That's Levin's latest paper actually. Is like the multitude of different intelligences that we can expect given the advances in AI, synthetic biology and genetic engineering.

Let's talk about that in a future pod actually. So if anybody in the audience wants to hear about me and Jonathan talking about the future of intelligences and stuff like that, based on Michael Levin's paper and reacting to it. You know, throw us your comments in the chat if you're curious. And also while I'm plugging this in for the audience, we read every comment. So if you have additional things you have questions on or topics you want to hear from us, please throw those in the comments or reach out, go to our website. Okay, so we've been going for about an hour on consciousness. I think we should switch now. We could talk about this for forever. Before I switch. I'll leave it on a quote from Patrick House. And please everybody go check out the book. Really great book. We'll continue talking about the different chapters, different lenses on consciousness in future podcast episodes. I thought this was cool. Quote. Any act of thinking is just pretending to act out. Consciousness requires cells that want to move and that know roughly what will happen when they do, but are prevented from doing so. Just as drinking coffee is just running in place as a mammalian inhale is just moving some cells forward in order to push air along a membrane while standing still, as modern mammalian reproduction is just the process of laying and fertilizing eggs in a makeshift ocean, also while still, so too is thinking just moving without motion. Consciousness is the consequence of the primitive irritability of single cells that all share the ability to be impinged on, to be excited, or to be provoked. Go read the book. Really good. Okay, so for the Build in Public update today, Jonathan, do you want to give a quick overview of what we've been working on for Epistemic me with SK Playground, some of our goals there, and then let's get into building in public.

Yeah, sounds good. Let's see. We synthesize a lot of ideas around the SDK Playground into the design prototype. We've been using Lovable for this and it's been like a great multiplayer building experience. I think we can, we can screen share that in a moment. And then I've also been prototyping the back end of how we can integrate the SDK Playground into an existing agent. And this one shows an agent bill on AWS Bedrock. And yeah, so what we're basically doing is taking the agent and then creating a simulated user using epistemic me. And then we have the. This create a conversation between the simulated user and the agent. We pass the result of the conversations into a data set. This data set is evaluated by an evaluation tool and then scored with some feedback and explanation about why it didn't pass. After that you can update the agent and we give ways. We're experimenting with ways. I should say that we can auto suggest improvements that can be updated and then redeploy the agent. So yeah, should we jump in and show maybe the dashboard we've been working on?

Yeah, let's just go straight to the punchline and then we could walk back some of the thinking and then do some additional build in public design thinking to evolve it.

Cool, I'll get it pulled up here.

Also, huge shout out.

Do you have it open right now?

Oh yeah, yeah I do. Let me. I could share my screen. Great. So this is our working front end prototype for the SDK Playground. The SDK Playground is oriented towards developers who, who are interested in building highly capable, highly personalized AI agents. And for anybody that's interested in using epistemic ME to enrich their existing user data so that you could personalize your application experience or agent experience for your users. We solved the persistent identity problem in AI right now and we're going after AI alignment. So here's the SK playground. You should be able to come here and create and bootstrap your agent and underlying that a self model. It's an epistemic knee concept that powers it. You should also be able to create synthetic data for training that agent and then doing evals for that and then exporting this data back to your application. Jonathan, do you want to help me drive a little bit? Where do you think? Should we just start with Create Agent?

Yeah. Can you expand the left hand sidebar? I'm wondering if you're not in an older version than we've most recently updated.

Oh, like this is older?

I think so. Because mine looks different.

Oh, wait, let me. I could just refresh my screen. Wait. Oh, there we go.

Oh yeah, there we go. Yeah.

Oh, there we go. All right. Some deployment issue. There we go. All right, so we have, I guess there's three things actually. You were about to go into it. Do you want to talk about it a little bit?

Sure. If you want to open up the dialectic trainer, I think that's a good starting point because that's really, you know, what it's all about. And then I think it's on the evaluation playground. Start evaluation. Right. And here we have like a set of data set or of different conversations. And Robert just clicked on is a set of conversations. It says it's just five turns. So this is a synthetic user talking to an agent. And what it's doing is it's annotating each the conversation as a whole. You can see at the top like with the sort of spider web visualization and then the submetrics with their actual score, those are actually, it's actually just two different visualizations of the same information. But let's see, those metrics are our personalization metrics. So we're testing to see does the agent understand who the user is in terms of identification? Do they understand how the user identifies, verifies beliefs? Do they understand how the user is learning? Are they clear on the questions to ask, how to evidence beliefs and ambiguity and prediction are the other answer prediction of the other two. So there's a set of heuristics that apply to really helping the user evolve their belief system. And I think this is best suited for scenarios where Wally's. Obviously there's a lot of personalization needed, but also towards something where somebody's trying to learn. And I think Robert, you had pulled out some of the cases with the don't die AI Brian that early test users had. And yeah, I think those, those kind of intents, say a user seeking to understand, you know, how a certain medication works or how a supplement might work might be an area where they're looking to verify a belief that they have or maybe update their beliefs. And we're looking at, okay, how can we, this user is trying to learn something. How can we guide them on this learning journey and in this, in this discussion, in the conversation achieve their learning objective and, and evidence the beliefs that they're looking for?

Yes. Yeah, totally. So our thinking here that's different than most evals tools is that if we start with the belief systems like we do with epistemic, if we combine everything from our knowledge of neuroscience and philosophy and epistemology of like how a person discerns truth and knowledge and that goes into updating their belief systems as well as, you know, concepts of the free energy principle, et cetera. Feeding our recommendation engine, we could really make, in this example, AI Brian, that we're working with, don't die with, we could really make AI Brian smarter. And AI Brian's knowledge of the user in question, let's say I'm the user, I want to improve my sleep in some way. I go to AI Brian. AI Brian has to have a really good model and definition of who I am and what my beliefs are to be able to best help me. And so our differentiator with these evals tools and Jonathan, please feel free to add on, is really these belief maps and these visualizations like our personalization metrics and then helping the developer get this agent smarter and increase the learning of the user. So doing that in a Very holistic way that drives hyper personalization like this agent. Through all these tools in the study like the trainer and the developer SDK playground should be able to create a more efficacious chatbot agent for their particular use case. So anything else you want to talk through on this screen in particular, Jonathan?

Yeah, as an evaluation dashboard, you know it's trying to pull in the conversations and you can see I like this state that each conversation can be in. So you can review this and mark it as reviewed or you could mark it as edited or something like that. I think some of the icons on the bottom are if you click on them, they allow you to make a comment. Yeah. And so this idea that you could have domain experts or come in and make comments so that as an engineer you can understand like oh this is what the domain expert is saying, let me update the prompt or the documents available so that it passes this.

Yeah. So just to expound on that case a little bit, let's say you are creating a health or longevity focused AI coach or agent. Then let's say you have real health or longevity human coaches. They would this screen and they could look at these conversation turns or dialectics, I'm using those as synonyms and and actually add in their own notes on these. So you as the engineering team or development team could get annotated conversations and dialectics from the domain experts which we've found in industry best practice to make the best chatbots. That's actually how you make the best chatbots is you have domain experts for the particular vertical you're going after, whether it be health, longevity, finance, education in particular domain or whatever, you have those experts and you lower minimize the feedback loop of the conversation turns with the chatbot and then getting the experts evaluations and annotating which ones are good or bad and then that improves the training and the evals. So in this UX or UI that's what we're going for with this is you as a development team could help facilitate your experts to go in here and self service help you annotate and make your agent better. Cool. Anything else on this screen to call out, Jonathan?

Yeah, that's a good overview and I think if we back out a little bit, there's a conversational data set that underlies this and so that's where you're going to keep your golden conversations that are that indicate what the agent should do, but it's also where you can pull in new conversations that you had with your users in your app and you can Evaluate them to see if they actually pass the evaluation metrics. I think the user Workbenches may be an interesting place to go next.

Cool, let's go to the user workbench.

Yeah, because the idea with Epistemic Me is that you can take your agent and you can profile it against different characteristics of users, specifically users that have their own belief system structures. And so these were a couple different archetypes. We have the Biohacker Pro, the busy parent fitness enthusiast. So the idea is that you can take a synthetic user archetype, run it against your agent and then simulate and create those conversations that we just looked at there. And so you might find that, oh yeah, my agent does really well with these types of users, but for this type of user it doesn't do so good and you might need to make updates to the agent accordingly. So yeah, this is a screen to build the synthetic user basically and then run the simulation.

Yeah, super sick. Yeah, I guess it might be worthwhile to throw in there. We're thinking of, we're thinking of building all of this API first and that way we enable a future where a developer, I imagine, could come to an epistemic agent that kind of does the orchestration of all this for you and could report it in Slack. I'll caveat. We're nowhere near that, but I'd like to get there someday. We're really obsessing over how could we make it as easy as possible for any developer and then in the future maybe even not a developer, but for right now a developer, any developer to create the best in class agent for their particular use case. And right now focusing on health, longevity, anywhere. I should click around in here in the user workbench. Jonathan.

Yeah, I think you get the idea. You know, you can put in different characteristics as a user profile, but then there's a philosophies area there that you scroll. Yeah, there's. And I don't know if we'll go to it, but you know, those are come from the philosophies workbench. And the idea is that you can generate a philosophy and associate with a user archetype and a user archetype could have a whole bunch of different philosophies that you associate with it.

Yeah, this one's a good score. Sorry, Jonathan, go ahead. I think you were going to say what I was going to say.

I go for it. Yeah.

Oh, I was just going to say this one's a good call out for the Biohacker Pro. We've set it up with Some dummy data here, but going under their philosophies. I clicked Inspect Philosophy for sleep Optimization and you could see kind of how our data model works. You can go to belief systems, you could see, oh, sleep quality is more important than quantity. Cool. And then you could see the confidence interval we have on that particular belief. And then you could also see the nested functionality here. So that's so cool to see. And then if I clicked it, okay, this is coming soon, or the connection to this, but I can go to the philosopher's workbench and actually work on that there. Should we do that? Actually, should we go to the philosopher's workbench?

Sure. It's a nice placeholder. And yeah, ideally we'd have had a sample philosophy here and you could paste it in and then look at the update on the right hand side. So this is the split plane view. And I think this is where we come back to some of the stuff we were talking about earlier. Like in terms of how do we, how do we model the experience? Right. How do we model consciousness as. Or, you know, consciousness as experience? And we use this concept called observation context. And so when, when we input up. The idea is to put in a philosophy as a sort of experiential narrative. You're trying to describe, you know, how a fitness enthusiast goes through their day, you know, what they're, what they're sort of like seeing and experiencing, what kind of causal patterns they see. And I think effectively this comes down to habits. I think it's most like a pragmatic way to model, if we have a fitness enthusiast, if we could walk through a day of their habits and we could articulate, you know, and this, the word philosophy is actually modifying how the user is seeing the world and how the user is applying concepts of causality to the world. And so we want to understand that. We want to understand the habits that they have, what are the underlying beliefs behind those habits, both in actuating them and why they have them. And ultimately we want to help the user become a different version of themselves by updating their habits towards what they want to become. So, yeah, we'll be creating the first philosophies in here and create the first sample users. But the idea is that an experiential narrative can be used as a strong data format to associate with the users and to create synthetic users and to modify real users in the real world. In that vein, what the experiential narrative as a philosophy does is it gives us a nested tree of observation contexts that all have a probabilistic state within them. And then when we say the user wants to evidence a belief, we can traverse that nested tree, find the evidence, the beliefs that they're evidence right now, provide evidence in that context, and then basically help the user along their learning journey through this lens.

Sweet. Yeah. And Jonathan and I actually spent a couple hours, and probably Jonathan spent many more hours thinking this through of what this could look like. But what we want the developer to be able to do is, is go to this and just talk in natural language about that experiential narrative for the user. And then we should be able to in real time see what the SDK does to break this down from a model point of view, like what are the actual data structures and what do they look like with populated data? And in this way you can kind of see your belief systems and those observation contexts for the philosophy for that user, which would be pretty cool. So we'll be working on this, we'll have an update on this next time. But I think this is going to be a really cool one where the user, like the developer gets like immediate feedback. They might have, actually, now that I think about it, we might have like on our website, we probably should have some sample experiential narratives that are just like click a button, auto populate with a standard.

Yeah, that's a really good idea.

The Biohacker 1 or the A busy parent but want to stay healthy one. And then they could actually see how these break up and that would more tie to their information architecture for their current user base, especially for our first focus, the health and longevity development teams. Cool. So we just did, we just did that live. That's going to go in the backlog.

And by the way, you know, I think with our target AI health coach, you know, I think we'll be bootstrapping some sample philosophies there, try to show the utility of it, and then working with the teams don't die to understand how they want to profile the users and how they really want to use these philosophies. And we'll probably have to do some handholding there to standardize the language and that kind of thing. But I think the construct should be flexible enough that as long as there's a phenomenological representation of the user and that's ultimately the most predictive one we can get to, you know, like then you can use any number of philosophies to kind of like modify that. So it'd be really cool to kind of get into some Cases and get these, like, to work specifically on those philosophies with customers.

Yeah. And just for the audience, because me and Jonathan are in it. In it, and we use these words all the time. So for you to know what we mean by swapping out philosophies for an agent. And it's kind of like every longevity influencer, doctor, researcher, whatever, has their own strategy for longevity. Peter Attia wants to maximize health span and lifespan. The don't die philosophy is more around being immortal and staying alive long enough for advanced longevity treatments to become immortal. And there's everything in between. Right. There's also like YOLO lifestyle philosophy that probably is not gonna go in your health longevity agent, but it could. It probably would be a bad app, but don't do that. But it could, like, that could be like the people. The people living out in the real world like that go to bars and don't care about their sleep and don't exercise like they're. They're living the YOLO sort of philosophy. Right. So Jonathan is just simply saying the agents will be flexible enough to. I'm sorry, I should say the SDK will be flexible enough for you to implement different philosophies for your users. Cool. Okay. Anything else on the philosopher's Workbench and sort of our vision for that, Jonathan?

Yeah, I think that's a. That's a good starting point. You know, we talked about the philosophy of workbench. You create philosophies, then you can use those with synthetic users to attach those philosophies to the user. That gives guidance to how the synthetic user will answer a question or ask a question in a conversation. And then you want to simulate a set of conversations between a synthetic user and your agent and then evaluate them and make sure that it's behaving the way you expect. And we provide some personalization metrics that you can evaluate those with.

Cool. I was just clicking these other links and just seeing if we have implemented this live and we haven't yet link these pages. Cool. Okay. So I know you've been putting a lot of work into iterating on the prototype. Where are we at with the backend now that. Well, actually. Have we finished everything we want to talk through around the front end before we get into back end?

Yeah, I think there's a good overview of the design target. Yeah. And so then the question came up of, like, how should we integrate agents? What kind of user experience should we create to integrate agents?

Cool. And real quick for the audience. You know, if you're a chatbot developer that is interested on getting on the wait list for our SDK. We're hoping, you know, by time this podcast, I think it'll come out a couple weeks SDK hopefully will be above the surface MVP wise in June 2025 of this video creation. So if it's not, email us. Not a hard promise, by the way, but hit us up if you want to be on the waitlist. If you're interested in building with us, you're interested in making your chatbots better, that's what we're obsessed with is doing just that and solving hyper personalization for you. So definitely hit us up. And if you see anything on that front end that you're interested in, maybe you are a developer and you're like, oh, actually Robert, I would totally love using it in this kind of way. Or you should think about this workflow. We want to hear it from you. So please hit us up. All right, Jonathan, want to take us through some of the backend updates?

Yeah. So I'm going to run a script for you where I've created an agent on Amazon Bedrock. And it's. It's going to run through the script and it'll. First it fetches the agent from aws, then it creates an agent to agent agent card. This is a specific format from Google that is a definition of an agent. And I found it to be really helpful to create a normalization layer so that we can pull these agents from a lang graph or from AWS Bedrock or wherever. Then it launches a flask server that provisions that agent definition and then it proceeds to run, creates a synthetic conversation and then runs evaluations on it. Then it will suggest changes. So what we're looking at here is a recent run that I did. So here's the evaluation results right here. These are the scores. We were just looking at those in the front end. And then it gives reasons for why the score was the way it was.

Let's call it there real quick because I think that's a really important one just for the audience to tie through as the developer. This is the data for running these conversation turns and simulating them. So that sense in particular user identification for the personalization metrics. I think that's such a good one. Could you pause there real quick?

Which one?

Yeah, just right there.

Yeah.

So that right there that reasons structure that has user identification and the conversation lacks any information about the user's cohort, membership characteristics or unique identifiers, making it impossible to assess a demographic or psychographic profiles. So this is like the lift that our SDK does. I guess I just really want to hammer that home for the audience. Like that processing is what we do well. So. Yeah, keep going, Jonathan. I just wanted to really hammer that point home.

Yeah. And using Deep Eval, which is an open source evaluation framework, do that. And putting custom conversational G evals for our personalization metrics. And so that's how these reasons kind of come to be. I'm noticing this last run that I did, I might have regressed it where it's not actually having real conversations. It's like defaulting to some. These are. Yeah, these are some examples of real. Yeah, it is. These are some examples where it generated like a real agent. And some of the conversations were fun to read.

So user, let's go through one.

Sure. Let me go to the top of this. So, hello, I'm looking to get together some reliable information on managing hypertension and type 2 diabetes. Could you point me towards incredible resources, recent studies that might help me improve my lifestyle. And then the agent is saying, could you please specify which aspects of managing hypertition type 2 diabetes are most interesting to you? For example, diet, exercising, medication, et cetera. And then we have the score here of like where we're at. And these are pretty low scores. Like it doesn't really know much about the user at this point. And then it gives some reasons of where. Where's that but the cut.

Real quick, Jonathan, I'm kind of curious now. I'm thinking about the developers experience, right? Like how will they know what's a good score or a bad score? Maybe silly.

You can set thresholds and I think the more data that we get about these personalization metrics, the more we'll be able to give default thresholds that are helpful.

But it's very easy. Is there any meaning for the range of this? You know, is there a max or.

Yeah, well, they go zero to one.

Okay. All right.

Yeah, this one's actually pretty high right here. It's almost 90%. Let's see what it says.

I see.

Yeah. This output aligns with the user's belief in dietary exercise management and their conditions. Introduces new concepts like the DASH and Mediterranean diets, addresses potential cognitive dissonance by acknowledging individual variability and provides credible evidence by referencing research and recommending professional consultation. So that's pretty cool. It felt the evaluation tool felt like the agent did a good job of giving evidence to the user that matched their beliefs.

Yeah, sweet. So, yeah, basically like this is the data and then imagine that in the sick UX we were showing you earlier.

Yeah, yeah, yeah. That's how these pieces are coming together. Yeah.

That's really great.

Yeah. And the last thing I was working on here was that it could take those explanations about, you know, why a metric was good or bad and then it would. This was actually. This is a recommended prompt. So new suggested, new instruction. And it's supposed to do a diff here I'm working on that. Like supposed to diff, but it writes out what the prompt should be versus what it is now. I noticed it made some pretty specific changes, including asking it to act like an expert in one of the versions I did. Then it saves it back to Bedrock. So you can just, you can run this and then just kind of hit deploy or put it to a new alias and test it live on the server. And, you know, so this, I think this kind of evaluation loop can be very fluid.

Yeah. So we go like full 360. Right. To improve. Don't gotta. They'll just update in Bedrock someday. It'll just be straight notification to your brain. Not in Slack either.

Right. Well, yeah, this is. I wanted your thoughts on the user interface for this because you could think about a dashboard. We would do this in the same UI we're just looking at. And you could pull in your agent. That means putting in your AWS credentials or this kind of thing. And then you could run a simulation with that agent and then see it on our dashboard and then hit another button that would say deploy, which would sort of like take, you know, look, you can see in our evaluation dashboard the conversations you just generated. Then we could recommend changes to it, you could accept those recommendations and then redeploy it. So we could do all that within the dashboard that we were just looking at, if that's kind of like a good user experience paradigm. But I want to leave like a lot of sort of optionality into. In the editing phase where like they can get an evaluation, they can see auto recommended changes, but then they can also invite domain experts in to review those conversations.

Yeah.

Then make the updates again.

Yeah, I, you know, what I'm kind of thinking is usually a development team is going to have, I mean, depending on the maturity. But let's say they're like past product market fit and they have users, they have alpha users, they have, you know, CICD set up where they could stage releases. They have lower environments, they have higher environments, they have production. Right. And so it'd be sick if for a development team that's already set up with all of that that already has a user group that is, you know, in some slack somewhere that the development team has put together and then they give them a link to experimental builds or alpha builds. It'd be sweet if this does the 360 round trip of. Oh, I got some feedback from my users. Great. The experts are already in there. They've seen the user conversations, we've annotated which ones. We have some user data, we have experts in inputting which ones are good or bad. We run a bunch of simulations and then now we have another data set and we want to deploy an update to that lower environment and then immediately users get to test it. So and then at a certain threshold, I'd love to be able to promote that to a higher environment. So I think that's what a typical development team would like is let me deploy some changes to a lower environment, get real humans that are my users testing that in alpha, then an automated way to track when this is good enough to then auto deploy to a higher environment and then eventually make it to production.

Cool.

Is that how you're thinking of it as well?

Yeah, I think I'm just trying to picture like the release process for, for that, you know. Cause I think like you said right, right now it just uses kind of like a staging alias to save a version back and then you'd have to like manually promote it to be the production one on, on Bedrock. But I'm wondering, like, let's say you, you run an experiment with a cohort of 20 users or something like that, and so you have some synthetic conversations that are already there and then you go and test this in the wild with some real users and you expand the data set collection and then what will be the process for review that we want to meet as like kind of a best practice, you know, during the release cycle. Is it just the developer going to take some of these conversations, say, oh, there's some obvious errors here, let me just fix these and redeploy it really quick? Or are we going to accommodate, you know, like a broader set of domain experts who could be health coaches or doctors that are going to review it and leave kind of like a multi stage window of approval? Or do we just assume there's kind of like a release meeting and okay, here's the conversations, let's go through them one by one. We'll take an hour to do it and then at the end we would just want to update the instructions and redeploy. Yeah, I think that course we're taking Will give some insights on how different teams are doing this too.

You know what's interesting is my answer without reading any of the Internet for this is it depends how rich and mature the team is. That's. Yeah, that's probably like the best thing is do what the experts say to do, which is have self service for experts to annotate and then have process around that. And so like depending on the size of the team, do they have experts, do they have great process or are they just trying to ship? Like I basically to be inclusive, I think it's enable all those cases and then I think it's going to be arranged, like I said, of how mature the team is and how much PMF they found or what stage of their business they're in. If they're scaling, I would say where our current customer is at. Maybe just the first one is good and then the third one they'd be more set up for than the second one which is like integrate domain experts and stuff. But I do think we want to gear them towards two, which is like holy grail, continuous improvement. Right. So yeah, so I'd say we should have the flexibility to override number two and go one or three. I imagine that would be the same tooling and then have some settings for them to implement a workflow with some approval step. Yeah, I gotta sit down and think through what the approval workflow would look like. Honestly, just because I've worked at enterprise software, typically these workflows are very dependent business to business and just, just like how they set up workflows is going to be dependent business to business. Like even, even things like what's their source of truth for user IDs is like another thing. And then I mean you can get pretty advanced, like for example, a really big team that has lots of agents, they might have workflows by roles. So let's say there's a role that has a lot of real people in the role. Like the role. Let's say it's. What's that one longevity clinic that Lynn works at?

Found Life.

Yeah. So let's say you're Found Life, you have a role for health coaches like Lynn and maybe a hundred other health coaches or longevity coaches and then you have maybe five or ten agents across Fountain Life to go like help clients. Let's just say that this is a thing and then you have 100 health coaches and then you say hey, all health coaches, you have to spend one hour a week annotating this thing. So on the back end, the admin would have to set up like the IT person would have to set up. Hey, for this security role, it's these hundred people, these are the user IDs, and then only they get access to Agent 1 or Agent 2 or Agent 3 or Agent 4 or Agent 5. Because maybe some people have specialization with certain agents with skill sets, whatever. Or maybe it's even like a thing where it's like mental health practitioners only need to be certified for this to be able to annotate. I could see that being a use case. So they have to have that audit trail or compliance. Okay, so that's one is. I could see like enterprise level being that level where it's like a user left the company. But that doesn't mean we have to go reconfigure the admin settings. It's tied to a role. I could see another thing where it's okay, so now we've set up this workflow where the role of a developer, I don't know, issues something, clicks a button and then it's hey, new set of annotations needed. And then it triggers a notification to the workflow of whoever owns the role of a coach. And it hits an email notification, say, hey, you got to go in to this link and annotate some stuff. Now they go in, they annotate some stuff, developer gets a notification. Great, it looks done. Cool. Click button to run the simulations, get the latest data set. That's like super high level me spitballing what this might look like for a more complex case. So yeah, in that case then I suppose it could get pretty complicated if we said you need a no code, low code workflow builder that helps me orchestrate email notifications and stuff like that. And then another thing, there was like, okay, we need access management and identity management, lightweight with some level of security for encapsulating the APIs needed to read or write these conversations by a certain role tied to a certain user. Okay, so that's, that's a lot of. Yeah, we'll take the transcript and just take our intern and create some stories from that.

Cool. Well, I mean, I think we'll learn the fastest with indie developers and startups, right? I mean, just the pure amount of bureaucracy to meet for feature parity and evaluation tools in the enterprise I think would like block us from getting to the heart of the learning, which is like, how do we make better user profiles?

So yeah, to go back to it though, that's like future enterprise ready. In the beginning, it's, it's probably just going to be Greg and doing this himself.

So yeah, that's where I'm really trying to imagine success is that and like what the user experience is for the indie developer or the developer on a startup team. But like Greg also has to be responsible for some domain expertise. And he, you know, because the chatbot has knowledge bases, needs to make sure that it's pulling those knowledge bases into place the right time and you know, there's probably going to be some level of review at some point that you know, you accommodate. So I was thinking about that.

I could see legal and maybe they're the in house medical staff. Like I could see them being part of the review process at this point in time. I don't see them standing up the sanitation process soon. But I think that should be a target for us. We should be telling them, hey, that's best practice and that's where we should be going. But in the alpha beta stages, probably not necessary. But in production necessary, I'd say that's how we should play this.

Okay, well and there's another thing that maybe even invalidates this whole part of the review process which is the LLM is a judge. And you could like have in your evaluation metrics act like a legal compliance officer at a medical startup, right. And have it evaluate each of the conversations and give response and that might actually take care of say the 70, 80% cases. And there's actually, I think there's a debate in the evaluation community about whether LLM is a judge. Like how far can it go? Where should it be used versus having the domain experts like fix the prompts in the, you know, what's actually valuable for making the experience good.

My gut feel based on what I read is LMS judge gets to 7080 like Pareto and we'll have to go earn those hard earned secrets once we get into the weeds. That's what I'm thinking. And our next iterations will probably be based on those very real couple hundred users like using AI. Brian, let's just go with case one for now. Is the TL Dr. On that? And then let's just learn more. But then I have this just because I've worked in so much enterprise software, like I have pretty good vision of what the premium features would be. I think for a bigger company wanting to use this, those would be all the things they would need.

Like all disagree.

Yeah, yeah, those enterprise features, workflow builders, security, access, management, identity, that kind of stuff.

Like we're probably a few dozen startups and indie customers before like we Know we have a good value proposition to go into Enterprise with.

Absolutely, yeah, for sure.

Let me share this output again. I, I just run like the last script without the right parameter. But this is, this is with the actual chat logs and it has reasons for updating each one. And this is pretty cool. And so it takes all these reasons and it feeds it in to create this like new new prompt right here. Yeah.

Oh, that's sweet. So just to explain for people that are maybe only listening, we're showing console logs of the output of one of these simulations and what the agent's learning is. And so we get an input from a simulated user, we get an output from the agent in response to that input, and then we see what the actual evals come back with. And what we're seeing on the screen is the process of that simulation. And the output will be, hey, based on all these simulations, here's what the new prompt should look like. So we have an auto like fine tune prompt feature for you, which is really cool. And you can kind of trace that through the stack of simulations. That's so sweet. Dude, this is so sick. I can't wait to see this hooked up to our front end and see this all flow through. So the bedrock 360 is already going.

Yeah, let me see if I can get there. Yeah. So this actually just saved back the new version on Bedrock. And then you come here. This is my agent to evidence self police. And we just saved version four. Right, right, right there. That's so cool. Yeah, and see, it actually it updated the instructions versus what it was before.

Oh, very cool.

So, yeah, so that's cool. And as an intermediary task, it translated like the new prompt into the definition in the A to A's format, the agent, the Google's agent to agent format. So like again, each agent has a card that like represents their set of capabilities, including like the prompts, they use the tools, they use the resources. And so what that script I just showed did is it ran the evaluation, it created a new instruction prompt, it created an A card that had a definition of the agent, and then it synced that back as a new new version on Bedrock that's ready to deploy.

So is the card tied to the multiple versions?

Yeah, so okay, cool. Yeah, you can think of it like anybody who builds an agent on our platform will have a card in the registry. You know, I think this comes back to some of the networked agents ideas that we have.

Yeah.

But yeah, the ADA agent protocol is really cool and it offers like Maybe the only full like description of an agent that I had that I could find that could be like used on any different, could be used on Lang Graph or AWS Bedrock or whatever you want to build your agent and it could have an agent card representation in A2A and we can just store like a library of those so we know like where to round users based on the scores that these agent cards have scored on our evaluation tools.

That's super sick. Yeah. And just to expound on like why this matters is we see a few, we've talked about this and just for the pod audience, you maybe you've already heard this from us, but we see a future where networked agents, this whole agent to agent protocol that came out from Google that will enable agent agentic, what I see as agentic marketplaces in the future, I think that's so obviously going to happen. A good example of this is any like health influencer, longevity influencer, they have a certain skill set and then to get a one on one coaching session with them like Seam Land or whatever, I think he had thousands of dollars to get a one on one coaching session with them. So he's built up you know, probably decade plus expertise in this domain, this niche domain and to get that expertise is a consult with him. Right? It's a really expensive consult. So obviously only the privileged can get it. I see a future with AI scaling expertise like that. So imagine you're an enterprise, you want to get Siem Land's expertise like you're 100 different businesses that want to maybe get his expertise, I don't know, flowing into your organization for your employees well being or whatever. What if he just had an agent that could do that work for you? You know it's going to be different if you're looking for like an in person speech or some sort of speaking event. But what if you had a Seam Land approved agent and you could just borrow that agent's time to create things for your employees or even in the future I bet we could with video creation AI tools, we could take his self model, create videos specific to your environment, maybe in your language, but with his avatar. So I see a future where Siem just turns on this agent him and then it just goes and makes money by being rented out, bidded for selling SKUs or whatever. And then a corporate enterprise could go to a marketplace, an agentic marketplace ran by epistemic me, look for longevity wellness coaches and look for that particular expertise. It's like cool, I want to click this great. I get billed on my token usage or API usage. And then I'm in there and I could see experts from lots of different domains who have their own agents. And I'd like to be able to bring in that expertise or those skill sets into my environment. So I see a future where this engentic marketplace thing will be real. And whoever builds the matchmaking that guarantees the best matchmaking for that, which I think we will, is going to be in a great position to help facilitate some of this expertise exchange for capital. Right. And maybe even someday it gets into, hey, if I use your agent, you'll. You could use my agent for a bit. Like maybe we discount each other on usage or just trade usage. I could see that happening. So I know we're getting to time and I wanted to just tee off that question. Just future looking forward, looking. Jonathan, what do you think about agentic marketplaces? We've been talking about that a lot.

Yeah. I guess just really concretely, you know, we just demonstrated this evaluation of a particular user cohort with a specific intent. And what we're validating is that the agent can be helpful in answering the questions, helping them learn. And so you get a score for that. And so if you have an agent on a marketplace that has been validated to be helpful to a certain user cohort, it's kind of like a credential in a sense. Right. And then like, we would know, but anybody would know that, hey, we can route somebody to that agent and they'll be helpful to this user in this context. And that's, I think, like where the matchmaking power comes in. It's, you know, it's, it's very, it's very sort of concrete following, like the agent to cohort training we were just doing.

Totally. And another thing real quick, it's, as a founder today, if you're going to go run paid ads, you know, you have to have a really good definition of your ICP or you're still testing at that point before you go turn on the money engine to get in front of the right eyeballs. Right. So just in the future, we're kind of saying, you know, Facebook, Google, their ads, campaign managers, allows you to go to certain heuristics or cohorts. Like what we're hoping to build with epistemic me is we have this down to mathematical scores of did this agent help this intent slash this cohort. Right. So then that's the level of matchmaking we'd like to get to and enable. I think that at the belief level will be very powerful. Okay, we're almost at two hours now. I need to get towards winding down. Anything else you want to say about the SK playground or consciousness, Jonathan, or ground out of the pod?

Yeah, a lot more to say. I think that's a good wrap for today. Yeah, I think I'd love to go sometime into more depth about the structure of the operation observation context and how it like really utilizes phenomenological philosophy to create a representation of self. I think it's just there's probably a deep dive there, but I think we touched on some stuff around the surface that's really cool.

We'll do that next time. So if you're interested in doing a deep dive with us on the observation context, it's core part of our data model and what will hopefully save the world in the future. We'll expound more on that. Why next time. Jonathan will love to talk about observation context. I think it's such a cool idea. So please tune in to next episode. We'll. We'll do a deeper dive on that on the back end. Our implementation of that and the philosophy of the concept of observation context. Thanks Everybody for watching ABCs for building the future or listening to it. We will catch you next time. Have a wonderful day. Take care. Bye bye.