Watch now | Recording of AURA - Tim Frazer & Shane Gibson on 5th August 2026

Hey Tim, welcome to another Aura session. This is Aura number 5 for me, Aura number 2 for you. How you going? What's top of your mind for you today or this month?

It's so funny. Yeah, this month. I mean, it's so funny. It's like, I know it was a month ago that we had this conversation. Honestly, it feels like yesterday. We haven't had a catch-up and proper catch-up in a little bit. So I don't know, lots of things going on right now. I do a weekly newsletter that tries to keep me keep my attention span. As you know, I always like reading lots of stuff. Um, I don't know, I mean, we talked, we riffed a little bit before this, and, and, you know, there's a lot of stuff on our mind right now. Where do you want to talk about?

Uh, well, you know, we're always open to questions from the audience, so if anybody wants to jump in and ask a question or give us a topic, we'll, uh, we'll talk about it. Otherwise, we'll just talk shit amongst ourselves. Uh, I don't know, I'm still stuck in Harnessland. Uh, been doing a lot of building, a lot of iteration on the harness to help me build and design and build quicker and faster. Just finding that sometimes it does make me faster and sometimes I feel like I'm spending more time building the pick and the shovel than actually digging a hole, right? Kind of interesting going through that process at the moment. Yeah, it's interesting.

So like I spent a bit of time trying to build my own harnesses and, you know, you and I had that back and forth and I just went full on on this new startup called Human Layer. And it's Dex's startup. And it's been my coding companion for all my, like, I'll call them hobby projects to keep me, just keep me moving forward. It's been working out pretty well. It's just really interesting, even with some of the best tools and context pieces and harnesses. Like, I did this migration from Cloudflare. I realized I want to do Railway. I was doing the very engineering thing, which is make something way too complex to start with. And so I was doing that. I was like, what am I doing? I've just made myself basically an infrastructure maintenance nightmare. And I was butting up against a bunch of nuances of the system. Very good system. But I was like, this is too complicated. So I had to do a migration. And it was like, worked really well. It was like going really well. And then I basically realized I had left all of my Cloudflare-specific code in the codebase, even though I had done a migration. I was like, well, now I have like 30% of my codebase has got to be ripped out just like from there. Even though I said do a migration, I guess I wasn't strict enough where I said do a migration and make sure not to bring along all the crap with it.

Yeah, I'm finding that, yeah, it's that testing, right? It's that validation. And then 'Cause I'm also spawning multiple sessions, and it's something that I've kind of realized is bad, is I sit there, I kick one off, right, and I don't wait for it to finish. I go and start another task or another session, and I've constantly got them running, and I built a little front end called /cogs to keep track of all the ones that are running, primarily because it flashes pink when it's running Fable. 'Cause I kind of found that what I was doing is I'd go and run Fable for one particular thing, 'cause I wanted the, the power of it. I forget it was the last thing I ran in Claude, and then I'd go and spawn 3 different just admin tasks, right? And like, they would all use Fable, and they just kill my token limit. So I kind of did that, and then I realized that actually there were too many flashing, too many going, right? And that, that, that context switching across those sessions was causing me more problems, because I kind of forget where I was up to. And I'd say, where we're up to, and it kind of would remember, but it wouldn't really give me the context of what we were doing, uh, as much as I wanted. So I think that's one of the problems, right? And I've been reading a lot about that is about, we have this, these tools now, right? We have these shovels and these picks that can help us dig holes faster, but lots of people are using lots of shovels to try and dig lots of holes at the same time. And I'm not sure that's a good thing. Yeah.

I mean, we were talking about this and I was just writing about this and it was like, you know, in the news, at least what's getting pushed right now was that, you know, the rewrite of Bund, which is like, you know, JavaScript module management and a lot more. And it was like moving from Zig to Rust. And we were just talking about it. Like, I mean, on the front, it's like, wow, that's incredible. That looks really interesting. And then I was like reading all the, you know, good stuff that Pragmatic Engineer does and everything in between. And I was like, oh, what's this? And then I was like, yeah, but the biggest news is they already had their whole test suite written in TypeScript, which was completely separate. So, they had already spent years of engineering effort creating this really good shape of what software looks like. And I'll say, working on— having had a lot of unfortunate bruises working on old Java codebases, most of them that I got to work on, even at big enterprises that I've worked at, did not have good test suites. Right? So I'm like, well, I'm not going to like— it almost read like the story was like, you too can just convert, you know, convert your codebase with all these tokens. But I was like, yeah, but their test suite was actually really good though. The really hard piece that most of us avoid, you know, data tests, quality tests, test-driven development, like all those, all those pieces that we say we're going to do but never do.

Yeah. I don't know. Again, you know, I read the Anthropic paper for the data thing. There's some stuff in there that I thought made sense. There's some stuff in there that people keep throwing up, and I'm like, that was one line. Why are you tagging to that one, particularly the star schema thing? I remember, I can't remember if we've ever talked about it, but I've got a mate, Sean McGurr, and he kind of comes from the data science background. And he was saying he was at a meetup once, or a presentation, and there was somebody from one of the FAANG companies standing up on stage, and Sean said to them, you know, so when you're doing all this machine learning, like, how do you get the data, right? Like, what's the process? And the guy looked at him funny and kind of said, well, what do you mean? He goes, well, you know, 80% of your time's getting the data and wrangling it right before you actually get into the machine learning. And the guy goes, no, no, 100% of my time's writing machine learning models and testing them. And Sean said, well, you know, how does that work? And the guy looked at him funny again. He said, well, it's easy. Every time I want data, I just log a request and then one of the data team give it to me in the next 10 minutes. And because, you know, they had hundreds, if not thousands, of people sitting there serving the machine learning engineers with data. And, you know, maybe the top, the FAANG companies do that, but the rest of us don't. And so I think often when we read these papers, we forget that it's a different organization with different contexts. You know, the stuff that they have in place, the people they have, we probably don't. And then how much of it's buzzwashing? Yeah, how much of it's really true? You know, it's true, but how much of it is warts and all, right? I think there's a lot of happy stories out there, happy paths, and not the horrendous paths.

Yeah, the dissenting voices, because there's not as many, and there's not, funny enough, dissenting voices don't get a lot of money. It's like, you know, I'm going to create a startup and its hard code ethos is, you know, handwritten Rust. There are very few. Although, in contrast, one of my friends wrote a great piece about that and where open source projects— and he was showing that even the OpenJDK, you know, owned by Oracle, they have a strict no LLM-generated code, like a strict, right? And just chunks of the open source community that are saying, like, You cannot commit LLM code, generate code. A lot of these projects are considering it theft on the codebase and that people aren't owning their outcomes. I'm not saying it's right. I'm not saying it's wrong. I'm just like, it's interesting. And I'm like, why are these, you know, what are they, the graybeards objecting to this movement in technology? I've been reading a lot of pieces about it. I don't know. I think there's definitely going to be space for both. I think there's going to be space for generated code. People still use WYSIWYGs for website generations and people still write code. I think it's just going to be more interesting mixture. I don't know, James, what do you think? I bet you in your work right now, people come up, business will come up to you and be like, here, I drafted all these amazing things. Can you just go off and build it?

Well, not really for us because it's not how we work, right? So we're kind of lucky that it doesn't do that. I mean, we got one at the moment, it's kind of interesting. So we're building an information product for a customer and the way it works is we build it to solve their problem and then we work with them to take it to market as a product that gets resold effectively to solve everybody else's problem. And What's interesting about this is the product's got to have two things. They always do, right? They have to have the right data and they have to have the right way. And so the right data is, you know, all the chunky data work that always is still difficult, right? No matter what everybody says. But the second part of it is how does the information product interpret that data to help people take action and make decisions, take actions and then get an outcome, right? So we don't just want to give them a dashboard of data. We want to give them access to data and then some form of recommendation, right? Some form of insight or knowledge or whatever you want to call it, understanding, however you want to go through the pyramid. And so what's interesting is, as I'm talking to the stakeholder, they're like, oh, you just, can you add these 4 bits of data to it? And I'm like, yep, no problem at all. But what are you going to do when you look at it? Oh, well, I'm going to do this. And it takes me a while to get through that loop of, okay, if I show you this data, What's your unconscious behavior you're doing to use that to triangulate the business process, find out where the problem is, and figure out what action you want to take? Because that's what we want to bake in. And then when I distill that, what's interesting is using the GenAI at the beginning, rather than saying I want those 4 fields, I say to it, here's the outcome we're trying to achieve with the stakeholder. Here's the data they think they use. What should we build? And what's interesting is it's really good at it, but it's incredibly opaque. Like, I then, once it's built it and the stakeholder's like, yeah, actually that's what I do. The gap between what I told it and what it built is really hard for me to work out what it's done, right? Because it is spaghetti, like it works. And so then the question is, should I be testing that it's got the right outcome and not caring what's in the middle? Or do I actually have to understand what it's doing and how it's doing it? And that, for me, is an unanswered question at the moment. What do you reckon? Like, how much do you have to actually understand the codebase if it's doing the job you want and if your tests are passing?

I mean, there's the answer I want to give, which is like, yeah, you should know your shit. But I honestly think that when I was doing test-driven development before, You're always starting with outcomes, like with expectations, right? And how it actually accomplishes it. I would obviously reach, I used to teach boot camps and all that stuff. And I didn't say what they did was efficient, but it would get the job done, right? And sometimes I would say, I'm not so much a heretic here, but I would say that my point of view is that you're always looking for outcomes first, right? Almost like when you talk to the business, like outcomes. What are you trying to achieve? And how it gets accomplished, I think it matters more to the craftsman and depending on what you're looking for and reliability and all kinds of stuff. I think that there's a reason why I chose using frameworks like FastAPI because they're abstractions, right? Abstractions that made me efficient at coding. Made me efficient at doing lots of things, especially for what I wanted to do, especially with tooling for data, data engineering. And then, you know, I didn't know, I never went to look at CPedantic's Rust codebase to see like, you know, how did it enforce the D class? Not that I couldn't do it. That's true. But like, I'll be honest, I very rarely would look into it. Do I have the skills to know what I'm looking at when I read that? I would say yes. And that's where I struggle, right? Like if I'm thinking today is like if I was a junior engineer at Computer Science and just finished, like, I mean, a vast majority are probably using LLMs today to do it, but they don't even, you know, have the concept of what good is. Like I got the concept of good because I failed so many times. Right? And if you have an LLM that gives you like so much confidence of what good is, And you haven't had, like, you know, that staff engineer, like, laying into you about, like, you know, no, you shouldn't, like, write code like this. Like, there's a more elegant way of writing a function. Like, it has, like, there's a very, like, clear, if you adhere to these standards, I guess, at a certain point, is like, are our programming languages actually going to evolve to a point where they actually create the outcomes that we want from these large language models that we're going to be using. And they're not going to go anywhere. They're definitely not.

No, no. I kind of think about it in terms of the architectural bits, right, versus the code itself. So give you an example. So I'm working on Information Product Builder, which helps me build these products faster, right? So think of it as a design system with a bunch of components, kind of like a BI tool that's using a headless platform, right? So I'm focused on the front end and everything else gets pushed to the back end as much as possible. And so I don't, it's all using Svelte, 'cause that's what we use for our product, and I don't understand Svelte, right? I don't even understand that code base at all. But I architecturally understand how it works. So, for example, when we have a list of values on a screen, I know that if the core table behind it has, you know, a list of 500 customers, and that list of values is dropping down and showing customers, uh, that I can let the code go do a SELECT DISTINCT on that table. Yeah, and it's going to perform okay, right? I've tested it, we're all good. But I also know that if that list of values is doing a SELECT DISTINCT on, uh, a table in the backend that's got, you know, 200 million records, that that's gonna suck. And that I have to extract that list of customers out into a view or a table, materialize it, right? So that that list of values is now hitting something that is performant. Because otherwise the user's gonna sit there and it's just gonna go absolutely slow. Now, I also know that maybe I can bring a caching layer in, right? And there's some other architectural techniques, but I've gotta make those decisions. As I'm building because they impact the product, right? The usability, feasibility, the viability. And I know that, but the code itself, yeah, I'm not sure I care. Probably because I can't care because I don't know how to code, right? So that's the question then. So if you don't know how to architect, but you know how to code, like, when do you, what is the minimum viable set of skills you need to build sustainable things with a GNI tool. That's, that's kind of what I'm struggling with at the moment.

I would say, I mean, it depends how deep the rabbit hole you want to go, right? It depends what you're building on first and foremost, right? Um, like if Shane, you're trying to create a new programming language and you don't know how programming languages work, probably not. Um, if you're trying to create, um, fairly— I mean, I think architecturally and being a systems thinker is more important than being a craftsman. But I think that's always been true. You can tell when you talk, I think for a point, for a long time, especially for coding camps, and I've hired from them and they've been really good, but it's like you needed somebody that could do Ruby on Rails in Ruby, right? And they'd be very good at that and they understand the concepts of how that works and they know that. But as soon as they got into, I don't know, the terminal or as soon as they had to God forbid, switch over to Python, they were like, wow, like, I, concepts is like, I don't know, like how this, like, you know, indentation thing works and like, like it changes everything. And, but then like the power of like working with people that have like strong architecture, like there was like this staff architect I work with and I would say he's probably medium on AI stuff now, but he is like, I learned so much from him, just how to think about systems and system design, that no matter how many good O'Reilly books I read, just sitting with him for, like, 35 minutes, I was like, "Wow." And, you know, I think, yeah, for me, it's architecture and system design, it's gonna, is, like, definitely the biggest piece. And the second biggest piece goes on, is actually building the thing, right, rather than just being theory. I think you and I have collaborated enough times to be like, okay, cool, we've done a data model, but it doesn't really matter until it hits reality. Yeah.

Yeah, yeah, yeah. I, yeah, I find that as I'm building the harness out, uh, using it in Anchors when I get the best bang for buck, because that's when it fails, right? I'm like, why'd you do that? Like, I'm sure we had a conversation about this. We're not— yeah, not really conversation because it's not a human, right? But I'm sure, you know, there's been some text typed into the prompt and some feedback, uh, saying that, you know, That's not the way our data— So, like, for example, one of the original— So, with Practi, which is the thing trying to get it to do the data work, Nigel did the first cut of the guides. And because he comes from a Star Schema background, right? And he used LLM to assist him to write it. So, all the way through, it was SED2. And we don't run in SED2s, right? Like, we run change records, but we don't run any CD2. And so what I found was every time Practi was reading a guide, it saw EC2 and it naturally applied DIMMs and facts, and we don't run those. And so it was applying these patterns which could work, but the language was kind of different, right? So, so yeah, it was trying to translate, and then I'd say, no, we don't run a fact table, and it was like, Oh, okay, no, that's right, you don't. But as soon as I went to rewrite the guides to take every term SAP2 out of it and just talk about change records, then it got it, right? Like, the way it was working became so much better. And that's the iteration. But I wouldn't have found that until I was trying to build something with change records and just noticed that it keeps dimensionalizing the bloody thing, and then that was causing it problems, right? So that I suppose that's their understanding if you're building the pick and the shovel versus using it, right? But yeah, but the other thing is there's lots of content out there at the moment in the space, but not in the data and LLM space apart from the BI Semantic Player, right? Is that what you're seeing, right? Like all the really good writing, all the really good use cases, they're not in the data domain, right?

Most of it's just like building, how fast can you build a tool for, right? Like what used to be a spreadsheet can now be a React app and doesn't look amazing. And I would say that vast, vast, vast majority. And then the second piece is that AI is going to be this self-service layer. Like it's just going to be like a come-to-Jesus kind of moment. Sorry to offend anybody, but like it's just like this, this moment of like, finally we have self-serve, like that the data team has always been the holding us back from, from our true, true version of ourselves. And then of course, like, you know, you know, Anthropic drops its self-service data analytics with Claude and it's just reams of things that I mean, I wouldn't say like I told you so, but you read it and it's just like, it's almost like the thesaurus on like, I told you so, of like all the best, it turns out like the best practices in data and data work is just that. It's just lots of hard work and getting people to do stupid things like write it down. And do you agree what a customer is? Do you agree what a subscription is? Do you, can we agree? And then like, you know, if you don't agree, what do you think the LLM or the analyst is is going to be able to figure it out. Like, the answer is no. I mean, it will— it will tell you straightforward it actually does know the answer, um, the vast majority of the time, even with good testing harness.

Yeah.

Um, so that— I mean, the biggest thing in the data world, like, right now, um, is like, what is a BI tool? What does that future look like? I think you and I hold similar points of view on that.

Maybe— come on, let's do this one because I know, uh, Ryan Dolly's kind of, probably pronounced his last name wrong, I always do that. But yeah, Ryan, everybody knows who Ryan is, or they should, is out there kind of on the frontier of that one, right? He's talking to all the BI vendors. He's doing some good stuff around where he thinks it's going. A little bit of sponsorship in there. So I kind of sometimes go, yeah, okay, I can think, but I think the takes he's taking from those vendors are fair. But what's your view? Where does BI tooling sit a year from now?

A year from now, I think more— we're going to have a lot of noise about what changes, but not a lot of movement yet. I think people still have multi-year contracts, and guess what? Those stick.

Or go up.

Or go up. I think that we're still going to want the comfort blanket. Of BI, I think there's going to be a massive consolidation, right? Massive. Like every— and then the ones like everyone wants to do, like the Anthropic play sometimes of like these like small components or like how Evidence is kind of done their Markdown SQL combos and a few other ones. And I would say that massive consolidation, the what is the point of BI comes back to another question. Uh, you know, do I have any like a solid answer for do I know what the next 6 months is going to contain? Honestly, I could see like 8 out of 10 people that are starting companies today don't bother with like solid BI. They don't need it, um, because the vast majority of people are using an ERP, Salesforce, and maybe something else, and they're just like, you know, 3 simple systems and if they quickly realize that what matters more is what they put in the systems to make it clear so that you can draw an analysis. And that's never— has never changed, right? But now those people that couldn't hire a data team are going to be like, come across the same problems that any good data team comes across— shared vocabulary definitions. And then it doesn't matter how fancy your visualization layer is on top of it. It's not organized.

Yeah, so you reckon that, you know, the dream's always been that we push everything back into the source systems, that they are modeled based on business language and business processes, not extracted out. So you reckon that is actually gonna, is starting to happen, that?

Starting to happen. Well, it's interesting, like, did you see Rippling came out with like this whole like HRS, like their own like whole data play too? Like they're like, hey, vast majority of people reporting out of our system is actually We can give you guys a whole lake house right out of our huge system and we're connected to vast majority of your systems, so why go anywhere else? Don't know about those kinds of plays. Is the nirvana going to happen? No, I don't think so. I think we're going to try it and maybe some systems will, maybe. But just look between HubSpot to an accounting system like QuickBooks. They have very clear different definitions of what a customer is and from different perspectives. Then you're going to have what is the unifying glue here?

Yeah.

I don't think, I mean, you and I both talked about this before. We both think Anthropic AI, OpenAI, and those guys, I definitely see them buying a vendor in my point of view. I don't see why not. Why wouldn't Anthropic go out and buy ClickHouse essentially?

Yeah, I don't know if they need to. Like, well, whenever somebody says, why doesn't somebody buy somebody? I look at it and say, is their business model built for acquisition, right? Because the problem is buying somebody and then bringing them in is really, really disruptive, right, for a whole bunch of reasons. So are they good at acquiring other companies and then integrating the bits they want and then being brutal with the bits and the people they don't want? Um, you know, and you get people like Oracle. I mean, I remember when Oracle bought PeopleSoft, uh, because, oh man, yeah, yeah. Yeah, and it was, you know, never meant to happen because it was a poison pill and yada yada yada. And then a bunch of PeopleSoft people went on Christmas break and then they got an email saying, company's being bought and you no longer have a job. It was brutal. It was, here's the day in January you can come into the office for your half-hour slot with your cardboard box to pick up your desk. That was it. And it's Christmas as well. It's like, wow. So yeah, I think that's the question, is, is Anthropic, uh, a company that buys and acquires companies? It's definitely a company that buys and acquires people.

Yeah, that's a good point. Like, they might just buy a really slim, a really smart, small, slim startup that's got a little bit of an edge, it's got a very specific, um, tangible difference, and they buy that, right? But they're essentially buying skills. Like, rather than spending, you know, 3, 6 months trying to recruit a team, they might just be like, we're just going to buy the team.

And if you think, uh, who's the guy from Anthropic that basically does all the great posts? Um, you look at his work, you know, he, he's about getting the engine to run 24/7, building up its pipeline and automating it because he's got unlimited tokens. So if you think about it that way, if you wanted a database, You know, you hire an expert in databases and then give them unlimited tokens to build a new database. Like that, you know, that's, that's why you wouldn't buy it, right? You just buy a person who knows how to build it and get them.

Yeah. I mean, that's what Cloudflare did, right? They bought a small team and they're rolling out their own, you know, iceberg-based data store. And it's pretty interesting. Haven't used it much. I think that's an interesting play. But I mean, for me, it's just like, yeah, I mean, I do wonder whether these, you know, the Anthropics and like all the tooling they're doing, I'm not sure if they want to go into the ownership because then you're like obviously like crossing that like line, right? Like a vast majority of the enterprise licensing, they're like, we don't train on your data, we don't have your data, we just have the shape of your data. And now like if you do, obviously if you're running a data company, you're now like, going almost essentially into ownership, uh, and more responsibility and more vectors of attack.

Um, yeah, yeah, nice. That's good. 28 minutes this time before it—

28 minutes before auto shutdown. Yeah, nice.

Well, why Tim's, uh, he's back, right? You know what happened?

I don't know why I have a 30-minute like battery cons— conserve, conserve energy, but I'm like, I'm here, I'm like talking, but Like, Substack, come on. You're supposed to be like maintaining that I'm active here. See, I'm good. I'm not multitasking, right?

And that's surprising because normally in the past—

Normally I multitask all the time. Shane will be like, focus, Tim. And I was like, I am.

You're like, focus for 30 seconds. You're looking left to see how your agent's running at the moment on something.

Oh my God. I am so bad at that. Oh God. I've definitely been in too many important meetings. I've had to hide my terminals for myself.

Oh yeah, I just minimize it. The one I get is because basically when it's building something, it knows now to pop it up so I can test it. And so I'm running something in the background and I'm like, I don't want to kill it. So I just minimize Claude. And then I'm in the middle of doing something like a meeting and then it pops the screen with the latest thing and it always goes right over top of the meeting. And so I'm like, ah, I gotta minimize that.

I was such a good boy. And then you poked me like my old notification.

Tipped me. You tipped me with the latest build to test.

Isn't it short story? Sorry. Like, so quick segue, folks. I think this is really important. I find working with LLMs and everything else, like, is almost the— especially if you're like business aware in building, like building things and building businesses, it is like a video game is what it feels like. It feels like the gamification of You're like, oh my God, why wouldn't I create a process map? And like, why wouldn't I create like all this, like all these, like all these artifacts and so on and so forth. And I think a vast majority of people I've seen, like they get to a certain point and they're like, eh, they stop, they get to like level 2 maybe if they're lucky. And then they're like, I don't know, this is too much work. Yeah, I mean, you know, I don't meet very many people, Shane, that like are your level of like, you know what, I am gonna persist on my like testing harnesses and keep going.

I'm just cutting, right? Bucks, uh, I have no choice, uh, but I definitely find that just adding one more feature is so easy and so free. And I like your idea, right? Hopefully I'm gonna get to level 3 where my sack's full, I gotta dump out some of those rusty old swords, right, so I can actually fit in the double-handed broadsword that I just found. And, you know, uh, yeah, it's really, uh, so one of the other things is, uh, because we run multiple repos, and we run Dependabot on all of them. And I've just moved all my components for the BI tools to npm packages so I can get a better process for distributing it across the repos, and get more reuse. And I hate GitHub with a passion. And so now I've got all these bloody PRs, right? And Dependabot keeps popping me emails saying, bump this and bump that. So I got clawed. To write me a little app, uh, and, and it's called, uh, Fleet. And basically every PR is a ship, uh, and over time it, it goes red and then it starts getting rust and then it starts sinking. And that's how I know I need to go and actually do that PR. And so I can, I can see my fleet, how far out from dock it is, how degraded it is, and then I just basically click on it, opens the PR for me, I look at it and go, yeah, yeah. Yeah, merge that puppy or whatever it's called. But again, I find that really, really useful. But to your point, I now find it when I'm building stuff too easy to add another feature. And so I think, you know, from a product management point of view, we're going to have to bake into the harnesses this idea of which features aren't being used and deprecate them. Yeah, because otherwise it's just so easy to, so easy and free and cheap to add one more thing.

Yeah, but I mean, there's so many practices in architecture and design that we need to take in that more is not better. Simplicity is everything. And I think that that's actually going to be really, really hard. Right now, I was nerding out with Simon Sasti. He has, is it ssp.sh? He's writing a data engineering book on data patterns. And I was nerding out with him because he has an open brain, which is massive, and he's prolific writer.

Obsidian. Yep, he got me onto Obsidian. Yep.

Oh yeah, he's, he's incredible. I said, wow, you're just so, so organized. But I mean, he— but like, so then I was like taking what I did, I started going down because obviously I've done— I mean, you probably know this about me, like I did Tiago Forte's Second Brain, so the PARA method. I've done, uh, Zettelkasten. I've done— you name the thing that the next whatever fly I should follow or try trend. And I'm like, oh yeah, yeah, yeah, I'll get better at it. And then my natural brain before AI was like, yeah, yeah, like 90 days later it's completely like just like gone back to its normal habits. So yeah, I've been working on a simplified workflow and I was gonna ask you about this because like I basically have simplified it cuz to a point where I have a system where I use Readwise for highlighting articles and things, because I figure if my rule now is I don't just read it if it's on my, on my browser anymore, I throw it into this thing because then I distract myself later. Otherwise I'll just do it right then and there and I'll be off again. And then I'm like trying to get better at how to manage to-dos because I got like a fairly large family and the business. And it's just like keeping track of everything needs to get done. And I was like, you know what's really good at doing this and populating my to-do list and nagging me? Anthropic.

Yeah, I, uh, I find that, uh, for my personal bot, uh, it does some— it has added value, right? There's some things it does that used to take me too long or very drossy, and I hated them, so I never did them. The one thing I haven't nailed though is my to-do list, my work list. Yeah, I just, I, and it's me, not, not the system, right? I haven't found a pattern that helps me focus on to-dos. Yeah.

I mean, I've read the book, you know, Get Things Done. I read The To-Do List Manifesto, you know, like all of the things to try to save me. Myself. And it is the hardest thing. Typically what I do, and I think Shane, you know, that's why I always like, I'll hire people to save me, right? It's like, if you're not good at accounting, hire a good accountant. So I find, we'll find people that are very good at it. I find that AI is helping me, but I agree with you. I don't know what Drossy actually means, I'll be honest. Too much noise.

Drossy is like boring. Boring as batshit. Yeah, Dross, boring hard work that you don't want to do. Yeah.

Or what happens is it frequently— and this is like my morning brief. I don't know what's going on right now with the large language models, but the language, all my directives that are essentially being ignored on how to write, and it's reverted to the mean of intentionally bad, annoying languages despite all of my skill layered on top of it. Sometimes I'm like, what is going on here? Like, I have a very clear directive of how not to have this crap freaking— like, it seems to just go off.

And, but have you, um, have you run doctor, um, /doctor against your skills?

Oh, that's a good question. I'm trying to think. I think I did a little while ago, but not recently.

Yeah, I'd keep doing it now because what I— a lot of the stuff I'm reading and the stuff I'm kind of seeing is, uh, they're moving so much stuff back into the foundational model or into Claude itself that the really specific skill or prompts that we hardcoded now actually degrade the experience because it's fighting with what's native in there.

Yeah.

But I don't know, like, how do you know, right? Like, how do you know that, you know, unless you ask it? And The more and more I think about the to-dos is I'm going to go and experiment with more of an observation-based approach. So I know that when I direct it to use any kind of to-do or task-based system, right, it follows those systems to the letter of the law, but I don't, right? It doesn't matter whether I get it to bucket urgent or not urgent, prompt and nag me, Right? Like there's, I just, I don't follow them. So my new theory is get it to observe what I'm doing and then tell it my goal, right? Is to get better at doing things and then see what it sees. Cause I don't think giving me more tasks is the answer. Maybe it's making me quicker at certain tasks. Maybe it's stopping me doing tasks. I don't know, but you know, it'll have the data to look at what I'm doing., and then potentially see a pattern that I can't see.

So yeah, that's interesting. Actually, that's a good point because like, you know, uh, what is it, like Eat Your Frogs? Eat Your Frogs First is already like a good book. That's the one I resonate with. Honestly, my one for getting, um, to getting even my to-do list done is, is almost like the calendar drives my to-do list, right? So it's like, it's like it's merciless because there's only 24 hours a day and I only have X time. And it— I need to like say like, oh, I promise all this stuff. And if I look at my calendar and it's like, uh, I have 7 hours available if I'm like working normal hours, you're like, well, it has to fit in that, right? And it just naturally drives efficiency, or, you know, work in progress. Um, and you're just like, well, you don't need to be simpler than that. Like, how much—

you didn't—

he's like, how much— don't let Claude try to flesh out project for time delivery. We all know this at this point, right? Like, it, it's all based on like human ability to deliver and code and stuff like that. So it's really funny when it says like, based on my estimates, this is 4 weeks of delivery, and then like 5 lines later, done. Um, so I imagine when like to-do lists, like you don't want it prioritizing, or even it can kind of score, help you score based on what you've done it, but like, I don't know, we still keep a lot of stuff in our head, so Yeah, I need to work on two bits.

So I want it, so I started off where it started giving me times and then like you said to come back, you know, this is 4 hours and 2 minutes later I'd done it. So then I got it to t-shirt size because that's my natural reaction, but actually they didn't tell me shit either. What I needed to do is tell me complexity.

I like that.

Yeah, impacts, like you make this change, how many other things are we changing?, right? What's my risk, uh, or my blast radius? That's what I want to know. And then the second thing is, because as I build out more and more testing, uh, the testing is now taking way longer. I've got a funny feeling some of the, uh, stuff behind it's been degraded without me. Um, just at the moment seems to run slower and dumber. Um, but I want to know how long it's going to take because I have to— then I'm still human in the loop, right? So Is it going to be 40 minutes? Is it going to be an hour? Is it going to be 10 minutes? And so I want to know that so I know when to context switch myself to go back and look at it, right? So those are the two bits of time estimation that would be useful for me right now. Blast radius to say, how much other shit am I going to impact? And when do you think it'll be ready for me to just confirm, not Not the build, but the T-Step before I deploy.

I like complexity. I mean, I love the measurement of complexity or Fermi estimates, right? Like, those are some of my favorite things because then it gives you, I don't know, a taste for the cost, right? I think a lot of humans are horrible, especially in software engineering. Like, I don't— very few projects are delivered on time, on budget, or frankly, what even what the person actually wanted in the end. And so, I mean, when we've worked together, like, that was one of the things that we talked about. It's like, well, not—

and it—

and then, and realize, you know, I think it was a good lesson over the years we've talked back and forth is like how bad we're at understanding complexity. And like how people, like, including myself, constantly will underplay because you're like almost assuming that everybody on your team has the same skills or way of thinking. And then when you're actually applying it, Um, and that's like— what did Joe write recently? And, and I think you've written about it and I've written about it, but like the Dunning-Kruger effect, where it's like AI thinks this, like you feel like you're an absolute rocket scientist, right? And you're like, well, obviously. And then 3 prompts later, you're like, huh, I don't seem to be solving the thing. But you feel like an expert, right? So, yeah, I do wonder if, like, its abilities and using abilities to understand complexity.

And also, it's so much easier to add one more thing. Oh, yeah. I had this problem all the time. You know, oh, let me just add one more thing to the harness to stop me having that problem. And you never go back and go, okay, tear it down, rebuild it, right? Rebuild it simpler. And in theory, that cost of rebuild, that cost of change should be low, right? We should have massive agility where we can go in and say, hey, go and take everything that we've written together for the last 3 months, tear it down, and rebuild it again, but simpler. And like you said, if we already got the tests in place, test, make sure all the tests pass, and then I don't care, right? Because I know it's working. Now, if I don't have tests, I care, right? And the only thing I can do is look at the code or manually test it, right? So, I think it goes back to that bit you said at the beginning, right? That if you've got really good testing in place, if you have evals, if you have surety of when a thing is meeting the outcome it's meant to meet, then our ability to rebuild the thing in the middle is much easier. And we have tooling now that allows us to do it. Whereas before, humans were expensive, right?

Yeah, I mean, but that's the whole, I mean, that's, I mean, we're always like with even like really good, Yet again, it goes to all the patterns, like, that we've been using for decades, right, with good software engineering, is, like, when coding was really expensive, and I think even, like, if you think about other programming language, like C, and, like, you know, how much effort it took, like, a lot of the time, you really wanna start with a very clear goal and then work in all the code, because all the functions and memory movements, It takes a little bit of time to get there. If you're very unclear, now I'm not saying iteration, agile software development, but the whole concept was deliver thin slicing or deliverable slices. Still relevant today. I still think even with AI development, it's good. I think it's way too tempting to do migrations from Cloudflare to Railway in one single shot. It's so funny, I'm talking about this and I was like, I literally did that. Over that, like, over the weekend night. And I was like, what? Like, did I not learn anything from what I just said?

Yeah, yeah, it's like, well, how hard can this be? I'll go make this big change. But I've got my, I've got my LLM friend, so it's gonna make it so much easier. So yeah, normally it'd be 2 days worth of effort, but hey, I'm sure it's just like half an hour, and I've got half an hour, so let's do it now. Um, the other thing I find is I reckon stack, uh, so I'll build something small and I'll like test them. Yeah, that's good. And then now that the testing phase takes longer, I'm like, in the past I'd like, yep, you go, you go and deploy it for me, right? And then once it's deployed, I'll come back and I'll do the next bit, right? Small chunks deploy every time. Um, yeah, now that the testing phase takes longer elapsed time. I go, ah yeah, let's just build this other thing that I know we need at the same time, and then we'll deploy both of them because then I only have to wait once. And I was reading an article about somebody that was running— they were grokking, uh, 200 Claude 20x max accounts.

Oh my God.

And they built a router. Um, so basically, uh, they kept all the context out of Claude, uh, effectively built a backlog and a knowledge base. And then they had a router that would round-robin all the tasks across all their Macs accounts overnight. So they were trying to get to a stage where they could run 8 hours of programming while they slept. Um, and one of the things they said is they were breaking GitHub in the CI/CD process because every time they were committing It was a 30-minute round robin. And they were pumping so much code and so many changes into it that actually they blew the window. And so now they're trying to figure out, well, how do they fix that? And they just talked about a whole lot of patterns they used to break things up. And they naturally tried to overcomplicate it. They naturally started to do queuing and putting things together and saying, well, if that ran at the same time, it made sense, and that kind of thing. And you can see how it happens, right? You can just, yeah. Yeah.

I mean, then it's like, you know, I'm just as guilty. Like I, you know, I built this app to scratch my itch like everyone else here. And then I naturally, as an engineer, because I also know all these patterns, I start to like scale things more than I should. And it's like, you know, recently I'm just like, slap my hand. I'm like, no, don't add, stop, stop adding that feature. Oh, I just want to do this. Like one thing, stop it. Stop it. Like don't add features because you can. Get really good at a couple of things and then people will find value from that thing. And I find that even, you know, inside— it's such an important lesson I have to constantly remind myself.

Yeah. And what would happen if the LLM, if you gave it a skill where it would talk back to you? Like, would you listen to it? Would you not?

I don't know. Yeah, man. I mean, I have so many apps on my phone. Like I worked at a certain behavior change company and it's like push notifications and like MS Teams, like annoy you bot. Like you should change your habit, you should go for a walk, you should drink water. Like I think the biggest stat I got from that was like a vast majority of people ignore, ignore, ignore, or like, you know, get rid of it.

Or they'd say, yeah, Do you reckon, do you reckon the early adopters are all multitaskers?

Oh yeah, I could see that. I could see like the people that like the dopamine. It's so funny, I mean, there's like some studies I was reading about like attention span and this like a rapid decline in people's ability to pay attention. Like, like Christopher Nolan's like 3-hour movie, it's so successful because I think people have to get locked into the theater for 3 hours And but if you like put it like home video, it's just like, I wonder how many like chores and websites would be built during that movie.

Well, how many times you stop for a toilet break and then just check where Claude's up to, kick it off again. What you doing?

I feel, I'm feeling very, I think this is interesting. I think like LLMs to me are even way more addictive than social media because it gives you the theater of the theater of feeling effective, right?

And also the subscription model, right? So if I was paying for the APIs, then I'd be far more conscious about when something ran. Because I have a subscription, I just keep an eye on how close to 100% I can get it every week. And so, you know, I am running stuff that I probably don't need to run, you know, because it comes as either free or I've paid for it, so I need to use it. So that's kind of an interesting behavior that I've found.

I think that's— I think that's totally spot on. It's, um, I think when you're in a subscription, subscription, and you're just like, ah, just one more, just one more. Although I think you and I both noticed, like, every time, um, Anthropic did a new update, all of a sudden, like, my models' tokens burnt faster mysteriously, and all of a sudden now I'm on like a Max Pro XXXL size. Oh wow, I went from the $20 a month one to the $200 a month. And I was like, oh yeah, but I'm being effective, right?

Yeah, yeah, yeah. I mean, what would it cost for an actual engineer to do that amount of work for me? I got one for you to kind of get your idea on. So we've got this idea of context and knowledge, right? But let's call it context. I think I had, Yuha was on last time, and he talks about knowledge plane, and I talk about context plane. And I've got to go back and check the transcript, but I think he was saying the word context more than knowledge by the end of it. So, win for the context team. Team context. So, I'm seeing a lot of people now talking about getting documentation from an organization and effectively putting it into some form of brain, and then using that as a way of distilling knowledge of the organization for the LLMs. Have you, have you seen that? Like, what are you seeing in that space? Have you read anything about it?

Yeah, I mean, I think that's a good segue. So I've been working on that from, for myself. Um, I talked to, you know, Trevor Fox. I think we, we talked about it briefly, and I just did an interview with him, and he has an open source project on that where he deploys that. I think a lot of people try to sell it. I think it's like Glean and a bunch of other startups that basically try to sell you on, on this company brain. But I think even if you're like even medium technical, low technical, like you people are standing them up themselves because Karpathy did like a thing on Twitter. I'm not calling it, and he, you know, just dumped an idea about like, you know, LLM writing basically a markdown file about a point in time. Formatting it, and then like you move on, right? And it creates this huge, as you say, context, right? And it is in— because it's used like a lot of people start using Obsidian, which is like basic linking, um, the LLM is really good at like rocking this. It's a very straightforward schema, very easy with— and it's been fairly, um, effective. I see the one thing that's not very good at, and but thankfully for— in my opinion, it's like, thankfully the definition of a customer subscription, all those things don't change that often. It's like, consolidated it, generally you don't go back to it. But like capturing that and making sure everybody's speaking the same language, yet again, old practices, new again, are really good. Like, one of the companies I work for has this company brain, and I think that it's been really helpful. Like, I think at first I was just like, huh, like, this is fine. And then I've been making it with— I had this another thing I started called like the Data Sense Maker, where I smashed some of the work that we've done together and I smash against like what is the company's brain. And I'm like, oh wow, I missed this thing. I didn't get that. Or it's like too vague and I'm like, okay, well now we need a clear definition. This is clearly not like a thing. So yeah, I mean, I'm starting to see it in more places. I actually think it's super powerful. It's like Do you remember when, like, I got all the, all the internal wikis that are, like, including the internet, like the, the Confluences, the Notions, and they're frankly never updated, and then, you know, the context that was written down, and it was like, um, I think if there's, like, a company brain and you're comfortable with recording, which everybody's kind of used to it, they're AI overlords at this point, um, that is It's pretty powerful.

But it's got to be automated and it's got to be no human in the loop, right?

So yeah, yeah, yeah.

So it has to have access to all the emails, all the conversations, and then, yeah, and then it just keeps needing to poll and self-organizing itself. Yeah.

So that's like this interesting, like privacy, because you remember the pushback from like Meta said, we're going to record all our employees, their keyboard strokes, their eye movements, their bowel movements, you know, the whole of it. And there was huge— it turns out, turns out you cannot be an NSA even on your own place. There are actual laws. And so they had to roll back some of the more extreme, extreme pieces of that. But I think that to a certain extent, yeah, like it has to be live. So I'm like right now personally, I'm playing between a Karpathy method, which is like Markdown files so I can understand and read it when I want to. And then what I would consider like just-in-time references. So that's a mixture of like, you know, embeddings, right? Very quickly, like if I ask this question or similar questions before, it points to an article. Okay, great. I've already done that in all my writing. And I also, I do have it suck up my email, not all my email because it's a total waste of time to suck in all my email, all the spam and everything that you get. Like it would just get so fat and useless. It would just— you would ask about a thing and then it would probably reference 50 data tools that somebody had just spammed LinkedIn or my Gmail account. But it says data and it says context and it says this, right? But it wasn't like, I would say, relevant.

Uh, actually, I was thinking about that because I still get the, um, you know, the spams that are obviously auto-generated. And I hadn't thought about actually, yeah, if you do enough of those and somebody is grokking an LLM across their inbox, eventually you're going to influence it, right? It's like, yeah, it's like that text LLM, text or whatever, you know, it's like in theory.

Right. And I don't have an engineering team behind this, like to develop this stuff. So like, how do you create simple rules that are easy to follow? Kind of like what you're talking about, like less is more. Like, what do you, you do? So like my process right now, like we were saying earlier, is like I get an article or podcast or thing I want, I put into Readwise. That's my gate, my quality gate. If it doesn't go in there, um, I read a lot of my books either on paper, I mean, if I like it, I'll take a photo of it and then I'll put into Readwise. Or, um, or yeah, and that's— and then I'll take it there and then it'll sync., and then I'll be like, okay, well now I have an, you know, then I'll write down my own opinion basically. Like, and, or dictate recently. I just, I find that dictating for me is, is, is faster and then write, rewriting it in my own language is, is my preference. Um, but yeah, go back to your question about, um, company brands. I think it's super powerful. I think it's gonna be like, imagine, could you imagine, like, I know McKinsey and all those guys are like standing up some of the things, but like if you did a privacy privacy-first company brain that people are like, you know, don't get destroyed on the value. I think it's going to be super important.

Gonna be back in a second. That was the 28-minute timeout. I was like, yeah, 28 minutes, spot on. Um, yeah, I can, I can— the privacy thing, I think it's going to be a problem. Um, but I can see, I can see the value of it, right? I can see, uh, if it's taking the summary context, right, the stuff that's useful, and storing that and getting rid of everything else, uh, I can see how it's going to be really useful. So I think we're going to see a lot of noise in that space, right? Um, because that's something where theory, somebody building tooling for that would be useful because that's—

yeah, I completely agree. I think privacy— I think people underestimate privacy like to like a crazy extent, right? I, I still think like I use a tool called Super Whisper, um, you know, because I do local, local transcriptions most of the time because I am still conscious about, about that. Um, I still think that local LLMs and open weight LLMs will have a chance. They're never going to be as good, I think, frontier, but I think the evidence is starting to show that you can get like, I don't know, 80% of the value for 10%, 20% of the cost is kind of, you know, Pareto. And I really am starting to, we're starting to see it. I do like how everybody was like, you can't use a frontier model to train your cheap model. That should be illegal. And I was like, isn't that like you guys using all the books and videos and content, scraping it and complaining, whatever. Yeah.

Yeah, I think the other thing that's going to be interesting though is that, you know, we've talked about data teams dealing with unstructured data before, you know, voice, video, all that kind of stuff. I think now it's going to become a reality. It'll be interesting to see whether that corporate brain, whether that sits in the data team's remit, in the data domain, or we get a new equivalent of the CRM team who end up doing that. So where does it live?

Yeah, I don't know. I mean, right now I'm like, I'm seeing it very clearly in engineering teams potentially. And then a lot of people are spinning them up themselves and then they're like, they're trying, everybody's like, you know, I've seen this a lot of places where like an engineering person is like, understands that knowledge is power. Like, I don't know, most data people and it's like, well, nobody's interested in doing this, so I'm just gonna spin up my own thing. Right. And then they just, they do that and they start to do a layered architecture and you're like, oh, you mean you're doing data engineering and data analysis and then you're creating— Welcome to my world.

Yeah, yeah, yeah. Capture the history, design the data so it makes sense, and then make it consumable. Wow. Wow. Wow.

And then they're going to ask the question, it's like, what's the real Slim Shady? I don't remember. And they'll ask questions around like, Oh, I said this thing and you said the same thing like 5 times over a year and you're like, but which one's true? I don't know.

And we just then just have to talk about loops, graphs, right? And what else? You know, there's all the buzzwords we've got to put on top of it.

I told you, Boris.

In theory, maybe that's where the brain should live, right? And library sciences, maybe that's the part of the organization that should own the corporate knowledge base effectively.

'Oh my God, are you saying something we've been doing for thousands of years is what we should probably be using?' And we thought about it like for a really long time. I don't know what you're talking about, Shane.

But we've never used it in corporates, right? Well, very rarely.

Yeah.

Is there a knowledge person, a librarian person in a corporate, right? It's always outside in domains that are bound to that.

So, I mean, I think there's still a lot and we know a lot of people that have like library science backgrounds and I mean, I still think there's a lot of it. I still have a hard time applying it to the business mindset that I've been stuck on so long. Yeah. And I'm hoping I can, at one point, I'll convert over to it. I mean, Anthropic said basically metadata is everything. And I was like, okay, but when you say metadata—

Yeah. Yeah. What do you mean? Excellent. All right. Hey, look, that was two good 28-minute sessions.

You know, just now I changed the screensaver inactive. It's now 90 minutes. It. So our conversation never goes over it. I just say, this is not the first time I've done this to Shane.

Excellent. All right, next time I need to remember that I should have a joke ready so that I don't just sit there live going, do I say something? Do I wait for him? How long is it going to take? Yeah, excellent. All right, hey, uh, we'll catch you in a month's time and see, uh, what we've both been working on.

And absolutely, mind, yeah, I have some ideas and I'll pass them your, your, your way. I think they're they're fun and you can tell me if I'm like losing my brain.

Yeah, well, you could, yeah, just send it to my bot and it will put it in my to-do list and then I'll give it a good stiff ignoring. So, or maybe I'll solve that one in the next month. We'll see how it goes. Excellent. All right, good to catch up as always. Catch you later.