What It Means to Give AI Agency

The more complex you make the system, the more complex your understanding of the system needs to be to be able to use it correctly. Welcome to the Notion State of Mind podcast, where we're going to talk about what grinds Bryan's gears.

That was last week.

Bryan has some beef. Bryan's beef.

It's not beef. I'm just curious.

Okay. What, what are you curious about?

There's a new feature coming out. It's already all over Twitter and all over LinkedIn. It's this new customizable sidebar. And one of the features is to be able to put a shortcut on there where you can add things to databases. And like conceptually, that would— is so nice. And I think a year ago would've been the most valuable feature ever and would've been highly used by me. But I kind of came to the realization as I was playing around with the sidebar that everything that I can organize in that sidebar are things that I just natively access through Notion AI today or through Claude. And that includes the creation of new data inside of Notion. So I haven't clicked the blue new button or the plus new button on a database in recent memory. And I raised that to you today, Josh, and you're like, that's wild. So anyways, I'm curious as to why you still do that.

Well, I think to level set, I think in the general landscape of people who use software, 99.9% of people are still with me. That said, I'm not saying that it won't go in the direction you're going in. And we've often talked about, well, there was kind of a, I don't know, like a running theme online a few months ago of the future of software is chatbots. And interfaces are dead. But also at the same time, you've kind of talked about the idea of Notion being the human legible surface area, right? So it's like software, at least for the next few years. And what I mean is UI that you interact with and isn't just provided or built on demand based on whatever you talk to your chatbot about, but like consistent structured UI has a place. And for me, it's like That, you know, I always kind of go back to this idea of like writing is thinking. And I think using software is thinking, using your tools helps you become better at the tool, you know? And that's part of it is like me going through the act of organizing something helps me to keep it organized in my thoughts. And so when I create a new action, I am very explicitly creating, let's say, a new task in my task database with this cool new feature that Notion is Probably releasing that I'm able to test, but is out on the internet just as a disclaimer. So in case anybody comes after us, it is out there. So I'm not disclosing it. I'm going through that action, whereas if I brain dump to a chatbot, which by the way, I love doing that. I think there's I'm still such a fan of that, and maybe I do transition to this mode of working. It's less clear if an action is created for me, if it was created in the way that I wanted it created. According to the way I think about it, worded the way that I think might help, you know, make me think the right way about it. And so right now, at least, agency is still really important for me. I wanna use that word well enough, you know, because you're still taking agency, but you're kind of delegating pieces of that creation to the chatbot to do well, and it works for you, which is great. This is a long-winded answer, but that's kind of my Initial thought.

It's fair. I guess let's dig into it just a bit, because I still use Notion, the page, as a surface area. Like my hub is everything. I spend all of my time in my hub. My tabs across the top are clients, engagements, projections, people, people private, task work, contracts, projects, skills, and activities, right? Those are my tabs. And it's a very robust surface area for me to interact with. But I guess the, the concept of manually adding something to any of those rows. Is highlighting an inefficiency in my work in some degree. And there is the reason that I'm saying that tasks are the nuance here. And we can get into that in just a moment, because I still use toggles for that. I just have a skill that files those as tasks and notates if they're complete or not. But the others are like my manual action of creating a client or an engagement or a person or a contract. If I'm manually doing that, I'm not taking advantage of data that exists somewhere else, right? Because the company comes from an email or a calendar invite. Or the engagement comes from an email or a Slack message, or the status update, which I didn't mention as one of them that comes from a transcript, a series of Slack messages, a series of emails. And it's like a starting point is one of those zones. So it's like the idea of me clicking new and then starting in that page just feels like I'm missing something in my process, as opposed to whether it's Notion AI or Claude, I start there and then it provides me a link to the pages created. And then I go in and I refine and I interact with it. It's like I still actively build and manage inside of Notion. I just never click that new button because the data should be created in my mind from some other source that I'm interacting with. Yeah. Yeah.

Okay. So I hear you. 2 main lenses here that I'm seeing to see it differently. Um, one is I think the difference between viewing and creating is interesting. You're still using, you're definitely still using Notion as a viewable interface.

Human legible surface area, 1000%.

But you're kind of offloading the creation piece, or at least the manual creation piece to the chatbot. So I think the other important difference right here is I'm just testing this out just from literally yesterday. And I chose tasks because that felt like the easiest one for me to do. And I do think there's a big difference between your ability to just put the task on the paper and you don't really need a whole lot more context versus going to the chatbot and telling the chatbot, I need this task. You know, it's like they're pretty much one-to-one. However, you brought up some really good examples of stuff that is a lot more I think, uh, system-oriented has a lot more gravity. Yeah. Needs a lot more context in its gravity to get right. I think that's totally appropriate. And I would absolutely use, uh, I would default to Nosy to kind of pull those strings to give me at least a baseline to start with. And I do that all the time. Maybe I don't do it through Nosy or whatever. I might start it in a note, but then work with chat on the note because I know I want to start this note. So maybe I'm just— Talking back against myself that I sometimes still do that, but yeah, I get what you're saying.

That's fair. Are tasks the only thing, or the main thing you can think of as like a, it makes sense to be a manual creation? I have one other very intentional one. Tasks are still manual for me. I just do it in toggles. It's like writing on paper. I just do it in toggles and checkboxes on the page.

Wait, so, so you knew that I was using it for tasks, but you still thought me using the new button was crazy, but it sounds like you still create tasks manually.

No, I use the to-do function to create my task list. I call it my free zone. And then I have a skill that at the end of the day will file it into a database that relates it to the relevant client and the relevant activity. So my whole task management is like pen and paper.

Uh, scratchpad.

Yeah.

Yeah. I actually like that workflow. And I think if, I think to this point, what's really important for the user of these systems is Understanding the system. This is gonna sound really dumb, but I have a particular thought in mind. You need to understand the inner workings of the system, and especially as you offload and delegate to AI or to automations, the less you understand about how it works on the inside, the easier it is to break or to not work the way you want. And then for you to backdoor it because you don't like the way it's working. And so the more complex you make the system, the more complex your understanding of the system needs to be to be able to use it correctly. I think that's a pretty blanket statement that I would still stand by. Maybe there's some nuance there, but yeah.

You're 1,000% right. So one, the— I can't remember the razor, but like, is it all like Gauss's law? The simplest, like, what is it again?

A complex system must be built on top of a previously verified and usable, simpler system. Basically, like, complex systems never work when you just build them complex from the beginning. It needs to start as simple as possible to be useful and actually used. Maybe this comes into our, like, I don't know, change management discussion, but people have to get value out of it first before you complexify it.

Complexify. Let's add that one to the vocabulary.

I love that.

What's Occam's razor? Do you know that one off the top of your head?

That's like the simplest answer. You know, I think people use this one out of context quite a bit, so I don't want to misquote it, but let me—

I'm probably using it out of context because that's why I was thinking about this one. I think they both might be applicable though.

The simplest explanation is usually the best one. That's the AI overview on Google, but I think it goes a little deeper than that. I think people use it in the wrong way.

It's like the jack of all trades comment, right?

Yeah. Master of none.

I love that one, right? Like jack of all trades, but king to none. There's 2 things I'm going to play back on the point that you were making though, because I think it's pretty important. Because one, you're 100% right. You have to be intimately understanding of the solution to be able to use it in the way that I'm using it, which means that you either need to have been the one that built it or you need to have documentation that helps you understand it well enough to be able to play around with it. Because the biggest issue that I think a lot of people are already running into and that we would run into if I didn't have that understanding is AI is not right all the time. It's going to make mistakes and you need to review the log that it provides at the end of its action so you can say like, oh no, no, that went to the wrong place. Or no, like this should have been working that way. So you need to understand the system well enough to be able to correct the mistakes of the AI you're working with. It is an incredible system, but there are mistakes that always happens with it and you need to be able to make sure that you, you fix them.

Yeah. Well, let me give credit to the AI approach. Like I'm not allergic to the AI approach. I think what's important is that you give AI a very clear surface or documentation for simpler, you know, the simplest term of what the system is and its ability to reference it and also update it and self-heal off of Like it's the metasystem, right? It needs an understanding of itself to be able to get better at itself. So that's, I think, the gap that most people miss and why AI feels like it's really powerful at first, but then when you start using it a lot, it just kind of misses the mark. And, you know, it needs an understanding of what you wanted it to do in the first place, which means you have to have an understanding. But yeah. So I think if you can get that right, that's a world where the switch is when you can trust The AI's ability for you to not have to hold the system in your head anymore, to have that complex understanding of the system like I just talked about, knowing that the AI has an understanding of it and you can just tell it, hey, uh, you know, I don't even know how the system works, but something in the system is broken. Look at the system, fix it to solve my problem that I'm having here with your output.

I have never gotten to that level of sophistication with any of my builds yet, but I'm very optimistic and excited about when I get there.

I honestly think this has been my not issue with AI, but what I've— I think right now the way people are using AI that I don't love, just personally, this is like what I wish it would get better, is that it still requires you to call a skill slash use this skill. That means you have to remember all the skills that you have, a registry of them, and when they might be useful. I think AI will absolutely blow people away when, and this is where I like, I tried to build my own kind of harness using Claude Code to understand me. And I was just interacting with Claude Code for everything. And I was trying to give it this like understanding of itself. And when I used a skill, I didn't want to have to tell it, hey, use this skill in these places. I just wanted it to understand, hey, when you have skills, whatever you need to do, like develop your own registry of when you should use it. I don't want to have to tell it when to do You know what it should be good at. So I know it's actually not that hard to bridge that gap and just, you know, like use this skill or do this or whatever, or call this thing. But I think that's the bridge that once AI gets really good at and like self-heals itself, maybe we're talking about AGI, I don't know. That's when it will act like people want it to act, like a Jarvis.

Do you have a timeline for when you expect that to be a thing?

You know, I feel like it's. Maybe conceptually possible now, not AGI, but maybe it's like if you have the tokens to burn, you can make something that does this. But it's such a fascinating exercise and you have to think more like the way a human thinks.

Think more human psychology as opposed to software developer engineering.

Yeah. You have to think in terms of how does a person think and connect ideas and concepts and What's so hard about that is how do humans think?

It's, it's a very, very challenging one to get. There's, there's, I mean, this is a gross oversimplification of it, but like 2 sides of this. One is the concept of there's always a trigger of something. There's always an input. There's always a format and then there's always an output, right? That's one piece of it. It's like, I had a sales conversation. I just want, I just wanted to know that that took place. I don't want it to be polling all the time and spending a lot of tokens polling, but I wanted to just know that that happened and then take that transcript, turn it into all these things, and then add tasks to all these people to sort of review. That's a skill today, very easy to build and very easy to execute if you have a good system. But what you're wanting is the trigger just organically occurs for all of the outputs to just naturally follow. That's one piece. And the second, I think that is the really interesting one that you're getting to is like You just want it to understand why you're asking or talking or saying what you're saying. And it should just organically be able to follow the natural next steps that are optimal for it, as opposed to having to guide it or curate what it does. And that one is a really wild one to get to.

Yeah. You know, what's funny is, uh, so you've been a big proponent of the agency triangle, right?

Agency pyramid.

Agency pyramid. Yeah, that's right. And at the very bottom is somebody that Executes on steps, and that is exactly what you have to teach an agent how to do. They're the best order taker on the planet, but you have to give them the orders. And I think what we're searching for is somebody higher up on that on that pyramid. Give some context on that.

Yeah, the agency pyramid is something that I don't know Alex Lieberman posted about it a few years ago, and I was like, "Whoa, I really appreciate that." It was something that was taught and drilled into me as a Deloitte consultant, and it's something that now I've started to incorporate with a lot of. The growth discussions we have at Notion, it's this concept of how do I become higher agency and more helpful in an organization? Layer 1 is there is a problem and that's where you leave it. Agency level 2 is there's a problem and I found some causes. Somewhat helpful, but still not super helpful. Agency level 3 is here's a problem, here are some possible causes, and here are some possible solutions. Level 4 is here's a problem, here's what I think caused it. Here are some possible solutions and here's the one that I recommend. And then level 5 is I've identified the problem. I figured out what caused it. I've researched how to fix it. It's already fixed. I just wanted to keep you in the loop. And you're looking for your agents to be higher agency, higher up on this pyramid. Maybe not that top one all the time, but somewhere between 3 and 4 feels really good. Yeah.

I want to keep myself honest here because the way it's described there, I do think a lot of times I want to get the right angle on this because When you talk to it, like Claude, like it is amazing. So I feel like its responses are usually higher up on those levels.

So I guess what I'm thinking of is like the Jarvis example is when you're talking to it, it's easy for it to give you a higher agency response. You're asking it to have a more organic, uh, high agency interaction or involvement in your life. It's, yeah, it sees this calendar invite next Tuesday, but it also knows that you had an email with your wife that She's really excited for dinner next Tuesday. And it automatically starts to say like, yo, Josh, here we go. You have a business and you have a family conflict. We know family's higher agency, higher importance for you. We've rescheduled that business conflict to Wednesday. Like, that's kind of what you're looking for more in, uh, uh, your excitement with, with AI.

Yeah. So we could build something like we could tell our Jarvis, you know, our Jarvis Jr. to do that, but that requires us to be like, Hey, um, you need to always check my calendar for events. Always check my messages, look for any sort of overlaps. Give me a review, do it through WhatsApp and send it at this time. Like that's the kind of order taking that I think it needs where it exactly like what you're saying would be helpful.

Yeah. To build the system that you're wanting today, you need an infinite number of scheduled tasks. Maybe not infinite, but you need a high volume of scheduled tasks. that are constantly polling, interacting with one another, probably a crazy system on the backend that you have to create and curate for it, for it to be able to function. And it still wouldn't know.

Yeah.

If we did it now.

Yeah, yeah, yeah, yeah, for sure. Uh, cause it's like, it, it, it needs to be like a human. It needs to be constantly thinking, taking in information, looking for problems, recognizing them without even knowing why, and then doing something about it.

I'm going to say it's not necessarily like a human. It's more like a bunch of if-then statements. plugged together that are told to look at something when, uh, like the human will naturally find that, like today a human will naturally find like, oh, that conflict gets like immediately this. You need to have programmed it to look there, then make this decision framework or evaluate based on a decision framework on what to do. But the human will just, I get it.

It's the, so it's the combo of deterministic work and the fuzzy LLM logic. Which is where I think workers in Notion are pretty fascinating, like workers versus agents. And I think we're actually zoning into maybe the technical gap to fill. When you give it a command, when you give it a deterministic command and you elicit a response from the agent, that fuzzy response is usually really good. But it's the, it's the commands that need to be given that need to just exponentially level up, it needs to just be like constantly triggering itself to think about something, right? And that needs to be based on every single thing that's changing in your life, based on every single thread of input that's happening in your life that a human naturally does, that would cost a billion tokens in our current state for an agent to do.

100%. Also, I just started laughing because I started playing out the example we just talked about in my mind of a worker is the one looking at the schedule, but then it's like, I don't know what Josh would choose. So then an agent spins up and it's like 17 times in the last month he chose work over his wife. Another agent spins up and it's like, does that mean that he wants to do work more than be with his wife? Or does that mean he should lean into family first? New agent spins up. It's just a battle of sub-agents trying to figure out what does Josh want? That's funny.

Yeah. I mean, this, I don't hate to bring it up again, but I think ethics and principles and philosophy play such a huge part in this as we offload our responsibility to agents. We're always still going to be responsible and culpable for that. But I think your principles and, and man, over the last century, talk about an erosion of just principles that people carry in their life, or at least knowing what they are. But principles will be so much more important as bedrocks of what you believe and why you do what you do. And even when you don't want to do the thing, you stick to your principles. Like, you know, you have a compass, a North Star, or, you know, a north on your compass. That's really important because human behavior doesn't always follow what you actually want to do or the kind of person that you want to be. So if AI is only judging you based on your behaviors, it's probably going to paint a much darker picture of yourself than you would care to admit. And I won't bring up the are humans good or humans bad conversation, but I do think it's a relevant discussion.

Such a good one. The 100 years, I mean, hundreds of years versus today is also a really interesting one because it was easier to have tight principles back then because there weren't as many distractions or a variety of options. The fact that we have everything at the tip of our fingertips today makes it very easy. And so many things pulling at our attention makes it very easy to have gray area between them versus when your options were way more limited, it was like, No, these are the 5. And it's only getting harder with the way things are going with the flexibility and capability and opportunity that AI and technology is building. All right. So we started with the new button and why you still click it. And we ended here on human psychology and how we can get Jarvis up and running in our real life and why first principles are going to be the most important thing in this AI world. Hell of a journey, man.

We really need, so, okay, let's do some table talk about the podcast. We want some feedback. And so here's what we're putting on the table, so to speak. Leave a comment. Most thought-provoking or most loved, funniest. I don't know. We're going to pick one that has some effort to it and we're going to send you a t-shirt and it's going to say, I heart Notions. Yeah, we want to get some discussion going. So, you know, who are the listeners out there? We want to connect you together, see who's out there, see who's listening. So leave a comment, whatever platform you're on. If you're on Spotify, YouTube, what else is out there? I don't know, Apple something. Anyways, wherever you're listening, to this, leave a comment and we'll pick the best one for this episode, send a t-shirt. So let us know what you think. More philosophy, less philosophy. All right, that's my pitch. Thanks for listening and we'll catch you in the next one.