Listen now | Recording of AURA - Tim Frazer & Shane Gibson on 1st July 2026

The inaugural. I don't even know what I'm gonna call it. What's it like? Aura, aura, aura.

Like an aura ring.

Aura ring. Yes. I mean, it's kind of a take on Aurora, right? I was like, yeah. And obviously the image was vibe coded, so I was like, yeah. I can't remember if I. I think I used Nano Banana for it, or could have been Claude Design.

I can't remember.

It's one of the one of those things. But anyway, thanks for coming on and being. The first, first of the, uh, Aura foursome.

Um, the irony is we have these conversations anyway and we're like, why don't we just like record it?

We do, although there'll probably be a little bit less swearing, or maybe not.

Yeah, I'll try to be nice. I'll try to be— I'll try to be nicer around my swearing. Sorry, I have a background in operations logistics working on trucks and shipping, so It, uh, it comes through strong at times.

Yeah, and I just come from New Zealand, so, uh, we're all potty mouths over there.

Because I'm the Canadian in the group, by the way, happy Canada Day to anybody listening. Um, that, that I wouldn't, wouldn't swear on. I'd be like super polite, but I think you know me long enough now that polite's not— I'm like, I'm, I'm kind, but maybe polite's not my strength.

Ah, but the question is, when you're vibe coding, are you polite to the machine just in case the, the AI Yeah, actually, I'm nicer than I than I probably should be.

I think my language is really like courteous. I don't know why. I'm like, I know like it's not a sentient thing, but like you know, you're just like fooled into it because the way that it and you notice is like remember when you were first using like ChatGPT like two or whatever and like how just horrible it was? Like sorry, like. has changed so much that I actually am super annoyed with the Lex Anthropic one because it's so wordy, which is like, I don't wanna date myself. I already speak way too much. And it's like, I'm having this conversation and I'm like, okay, fewer words please. Like under 50 words. And if you don't like, and also don't blow smoke up my armpits, I guess. Like really just be— Be direct as you can and, you know, don't stop. But then of course it ignores me like 3 prompts later anyway.

Yeah, I mean, my harness, I've got a whole lot of prompts in there about what I like and what I don't like, and it kind of refers to them. Um, I don't know, I keep looking at things like Caveman and going, oh, should I? I don't care about— because I'm on, you know, a max plan, so I don't care that much about tokens yet. Uh, well, at some stage. Um, but I'm thinking, well, yeah, if you're less verbose Uh, it might be useful because I don't read half of it anyway. Um, but yeah, the politeness. Yeah, I'm, I'm still polite. I don't know why. Even, even when I'm yelling at it, right, I kind of like, yeah, you just, you know, you lied to me. It was, um, funny today I was getting one Claude session to write a prompt for another Claude session. So basically a person, my personal iOS talking to work harness. And, um, and, and the personal iOS one, it wrote some pretty, pretty gnarly prompt. It was like, I want you to do this and I don't want you to pretend, I want you to actually go and do the work. And so it was like basically saying it knows Claude lies even though it's Claude. And it gave me the prompt and I went past it into the other session and then moved back again. So I was kind of— that was kind of funny.

Yeah, that's— yeah, that's really interesting. Uh, I think it's like diverging this conversation a little bit, like talk about, like, so I was at the Snowflake Summit. We could talk, I think we should talk about it a little bit like that. Yeah. Because obviously, you know, you do a lot of work in like GCP. I, you know, I've worked in there. I haven't done a ton of work in Databricks and like Azure, done some in Azure and everything like that. But the one thing I noticed is like, and I don't think this is like a hot take necessarily, but just said, I can't even use that word anymore. Every data vendor is like consolidating at like a crazy level. Speeds that I, you know, a year ago I probably personally wouldn't have predicted quite how fast it is. But like, you look at like warehouse pipelines, BI, semantic layer, they're all like folding in at like very intense speeds. If you walk into any conference floor, the only thing you're going to hear is context. I mean, this year's like the whatever, ontologies, context, context, semantic layers. And the one thing I had a really, really, really hard time when I was at Snowflake is competitive advantage showing amongst any of the vendors. Like it was every investment that I see right now in any medium-sized vendors is like a huge risk because like if you do a 1-year investment, Snowflake could very much release something that's maybe 80% good within the next 3 months. Same with Databricks. Like, you could just see them just chomping through anybody that says chat with your data. I, I have a very hard time seeing how they're better than what the big guys are doing.

Yeah, I think, I think it's the fact that basically they're, um, veneering over the top, right? So they're taking a foundational model, they're taking a use case, and then they're kind of making it easy for you to solve that use case using the foundational model. But as you said, the big players are watching what you're doing and they're effectively building that into the platform, right? If you look at what Claude Code's done to Cursor, just listening to a podcast today with SpaceX, xAI, whatever it's called this week, taking it out with a share swap. They basically said Cursor was leading, but what they had done is done an IDE over the top of a foundational model. And Anthropic just came along and went, good use case. Um, you look at the way Anthropic have published their article on their data agents. Yeah, sure as shit they're going to come after the data market, right? If you think about complex problems that need to be solved, they're going to come after this one hard because it's a massive— it's a massive TAM for them, right? Target market. Um, and, and complex problems to be solved, and they got a shit ton of data to help solve it.

I mean, did you hear about Rippling starting up their whole data play?

Oh my—

yeah, the full— they're going full lakehouse play because their perspective is that, and they're not wrong, is that in the HRIS systems, payments, everything, like a lot of data, a lot of things that you're reporting on are actually coming from those systems. So they've launched their whole, like a whole new data platform. I don't have specifics. I just read that and I was like, this is crazy. This is like, what is going on? And I was like, well, Maybe we're going to start seeing more of these where people are getting these open-source blocks and then, yeah, they can just stand up on Iceberg and just computation layer is now commoditized.

Yeah, but we saw it before where the transactional systems, Salesforce, SAP, tried to own the data space and they suck at it. I don't know why.

I didn't say they were going to be good at it. Definitely didn't say that. And yes, I do remember like, you know, like NetSuite, like these guys like kind of going into these spaces to try to do stuff. And then, you know, NetSuite got bought by Oracle, so it's just a giant Oracle database anyway.

And then the question is, even if they do make it work for their own product, their own product only holds half, you know, really holds more than half the context and half the data of the organization. So you're still gonna need something else. So are they thinking that you're gonna go move the rest of your data make it available, and then you're going to go pump all your context into their platform.

Another platform for platforms. But I think one of the things that we were talking about earlier is, and it's still not that uncommon, but I think Amazon— call them out for this— is probably the most infamous for this play, which is where Snowflake and Databricks are the landlord and they become the tenant. Think about the Sigmas, the Omnis, the Lookers, the Tableaus and stuff like that, where they're like, they're their partners for now. And then, you know, they do classic platform trap. Now it's just like, oh, you could get 80% of these BI tools, but like, you know, what, 20% of the cost? If, because you're already on our layers, we'll give it to you like super, super cheap. Same thing, like you were just saying, SAP and all those other guys playing this. But like, now you see like Snowflake is doubling down on the Iceberg play. Right? So they're doubling down on their ability to move data, and you see them launch something called Coco. I think it's called Coco or maybe Cowork now. I can't remember what it's now. The name's changed so many times, similar to Salesforce. But the gist is, is like, it's a chat interface like Claude, like OpenAI on your data, and Snowflake has the computation and the money to train these models themselves. Maybe their own models only, and they do have one 90% as good as the frontier frontier models, but then they just start swapping them in and you don't really most of the time notice the difference. And they start getting you more and more into this space. And same with Databricks. Like, I wouldn't say there's like a forced adoption between Delta and Iceberg, but you see Delta and Iceberg converging for this open table format. And now everybody's going to start controlling these standards. Yeah.

Standards.

Yeah.

Oh, it's like, yeah, Open Semantic Interface standard, right?

It's like, um, I remember, remember there's like this time, this era when you could sell on Facebook Marketplace?

Where you going?

Literally no idea. I just had a Mac takeover. All right, I'm back now. Um, that was weird. I've never heard that name before. The, uh, where Amazon's like, come to our Facebook— or Facebook Marketplace— Amazon Marketplace and sell your batteries, sell your whatever, your goods. And my couple of the companies I worked for did that. And then one year we were selling and there was like this pedometer that looked exactly the same, and it was like It was like one of Amazon's sub-brands or whatever, but it was not called Amazon because they had so many other brands. But then they sell in the same marketplace and compete against us. And we were like, well, you actually know our margin. You know how much we're selling to you for doing all these things. You have all this insider information and now you're cannibalizing us. But it's Amazon, so you couldn't really argue with them. File a Zendesk ticket or order Amazon ones for those things. So like, you know, is Databricks and Snowflake doing the same thing?

I don't know. I don't know. I mean, for Agile Data, we're on Google, right? We've been on Google Cloud for 7, 8 years now. Uh, one of the things we like is, in theory, uh, all our customer data and all our behavior is, is isolated to our projects, our tenancies, and Google don't look at them. That's what the terms and conditions say. Um, I've never read the Snowflake terms and conditions or the Databricks ones. There's an interesting thing. I listened to a Lean podcast. Yeah, I listened to a Lean podcast ages ago. Brian Belfer was talking about the kind of cycle of these big companies, and he talked about marketplaces, and he was kind of talking about who's going to dominate the AI agent marketplace. But he had this playbook, and it was like perfect. It was like you start out early, yeah, and then you get early adopter partners on, and you're all friendly to them, and you give them a shitload of money and free credits and, you know, a lot of loving.

Yeah.

And then effectively, you know, once you got some success, you then start having partner tiers, you know, bronze, silver, gold. Um, and then after that, you open up the marketplace and everybody starts, you know, putting their stuff up to sell. And then you start carving out the products that are really, really successful, right? And you start replicating them. I don't know if you ever saw the story about what Amazon did to Trifactor, but they effectively, uh, Amazon Glue or whatever it was called, um, was a complete scrape of Trifactor's interface and the way it worked. Um, what was the other one, that open source one that they then picked up and actually embedded on AWS? Uh, Redis?

Oh, was it one of the ones for licensing? And then obviously then they didn't like that because then Amazon could, couldn't just like suck 90% of the, of the engineering for free.

Yeah, but the thing that interested me is, is how few data professionals or data practitioners actually look at the market and where the vendors are and bring that into their strategy, right? Like, if you can, if you can go fully on Snowflake as a full end-to-end platform now, and they're cost competitive, right, and they're constantly adding features, uh, and they're not going to go and sell out to Salesforce or get, get acquired, then it's probably a good strategy. If you go to somebody that's venering, you know, uh, one use case on top of a foundational model, you've got to, you've got to risk that that puppy's going to get bought, right? And we've seen the waves of, you know, start a company, get to market share, and then get bought out. And we've seen the consolidation from the modern data stack. So it always surprises me how many people, when they kind of invest in these things, don't actually look at, uh, the life cycle of companies and what it means to their company.

Well, I mean, I guess your hope is that your, your company outlasts platform. Yeah, I mean, it's really interesting to me. It's just like, and also I think a lot of people under-leverage the platforms, but then they're adding so many new features. And then we've had conversations before around the quality of those features just because more is not always better. You go to use it and then you're just like, did you read the fine print? Did you do— I've been trying to use Snowflake evals and I would say, beta-ware at best. And then I'm like, okay, but how long is that going to stay beta at the speed of acceleration that we're seeing today? But I don't know, what do you think the thinner slice ones? So you have these bigger platforms, you could say there's— ClickHouse is almost like a dark horse right now, but I actually don't think it's as dark horse as people think it is. I think it's, it's, it's growing, growing quite fast. Um, I think that, you know, then you have Google, always sneaky, sneaky. It's just so funny because you use Google, I've used it before in BigQuery, but like honestly 90% of the interviews and companies I've talked to in the past, like they use it for Google Analytics and that's like it.

Yeah, I mean, they've got their Trojan, right? You know, their GA4. integration into BigQuery, their free Data Studio. Sorry, Looker Studio. Sorry, Data Studio because it's just been renamed. But Data Studio has no cost, no license cost. Free as in puppies, not free as in beer. But I hope— because we used to use it as our front end for our customers because there was no license, it was fully integrated to BigQuery, we got really good performance out of it. Now we're moving to more building our own information products. But I hopped into Data Studio the other day, And now there's conversational analytics embedded there at no cost. No cost as in free as in puppies, right?

Yeah.

And so I'm like, excellent, click on the button. And then it goes, oh no, now you've got to go into GCP console and set up your agents. And, you know, I love Google's technology. I love their pricing models because you can run it on a smell of an oily rag. But it's a technical tool and you've got to cobble it all together. And so you know, they've come out with their context graph or their knowledge, whatever it is, which is part of Dataplex, which was part of something else, which was part of something else. Um, it's incredibly ugly, it's still hard to use, but if you're an engineer, you can put it all together. Um, the thing is, great technology, shit marketing, shit partnering. Um, which is why, you know, all you ever hear is Databricks and, uh, Snowflake.

Right.

Maybe a bit of Fabric if somebody's keen on PowerPoint. I hardly hear— I mean, I hear a bit of Amazon Bedrock. You never hear Redshift anymore. You don't really hear Amazon coming to this. It's definitely a race of Snowflake versus Databricks.

I mean, every conversation, I completely agree with that. I would say they dominate the— I would say when I was doing some interviews last year, before my current role. And I would say I think I did about 10 solid interviews for different positions trying to figure out where I wanted to go. Done the consulting, having kids, so I was just like, I need more stable lifestyle. I don't know what that actually means in the startup world these days or any business work. But so many people, so many people were talking about moving from Snowflake to Databricks because it was going to be cheaper. And I just, I had such a hard time with that and I couldn't get like a straight answer. So many people at the time that I was talking to, that was the move. I was like, so you're going to move from like, you know, Snowflake's compute engine to a smart compute engine and it's going to be radically different. Maybe, maybe. I just had a lot of like, but there's like so much other cost around operating these. systems. And I was like, there's probably other ways you can do in your process that is far simpler than moving a whole platform in play. And it was not like, you know, I mean, I can barely— I barely hear anybody talking about Redshift these days. And if I do, it's usually, how are we moving off of Redshift? Uh, so yeah, Amazon from this conversation, at least from my perspective, has mostly disappeared. And then Microsoft in this conversation I mean, I hear about Fabric, but I personally haven't seen a ton of it deployed in my conversations. There's a lot of people talking about it, but even the people that I know that are on Azure that I've worked with, they use Databricks. Which honestly, I thought 2, 3 years ago that Microsoft was going to buy Databricks.

Yeah, I thought so too. I mean, it's going to be interesting to see what happens to Power BI given what's happening in the BI reporting space. and the dashboarding space. It's going to be really interesting to see whether things like Streamlit take out— if you've got a Power BI constant consume last mile sitting on top of Snowflake, how many people are going to go to building out their own Streamlit apps, which is obviously where Snowflake's going. I mean, the other thing that confuses me is when I talk to organizations that have Snowflake and Databricks, and I'm like, well, is this a layered architecture? You've got 3 layers, effectively what we call history. Some raw staging, persistent stage, and then the place you do all the heavy lifting, your design, whatever you want to call it, and then your consumption layer. Snowflake, your consumption, and Databricks, your hidden mirror. But yeah, I have to go see their architecture diagrams if they exist. We've got a comment here from Daylan Webb. Excuse everybody if I ever pronounce your name wrong, it's going to happen every time. My org is exploring a move from Fabric Power BI to Databricks Genie Spaces. Power BI is sticky through in large orgs.

Oh yeah, it's like MS Teams problem. Remember when everybody's on Slack before the Salesforce acquisition and then Microsoft did the Internet Explorer play again? I was like, did you get like caught out for like a monopoly play on this? And it's just like you've tied everything to— you tie it all together and you're like, well, you get free Power BI, you get free this, you get free that, you get free Microsoft Excel, you get free like all these things from is one low-time fee, your soul.

Um, but before Slack, I used to use Yammer. I used to love it. It was great because I could follow the bloody threads. And then the bastards Microsoft bought it and they killed it, and I had to go to Slack like everybody else, and I still can't find any threads. I, you know, the Databricks Genie thing's really interesting, and we've got Chris coming on in a week's time, and he's been to the Databricks conference, right? So actually, at some stage, we should probably have both of you on and have a punch-up about the conference. But, um, yeah, they were announcing this whole Genie ontology and Graphy thing, and was it 4.5 million ontology snippets so it can work out how your organization works? But I suppose my question is, if you're going to run Databricks Genie and you're going to run Power BI, like, how do they integrate, right? Like, if I'm going to go do chat Ask AI, do I go to Databricks? If I'm going to go traditional dashboarding, am I going to go to Power BI? Are you going to embed the Databricks Genie in Power BI? This integration piece, and I still don't believe that OSI is going to give us one standard to rule them all for the BI semantic layer, let alone the context plane. Do you couple those 2 together? Do you build a new interface? for the consumer that then in the background talks to each of those? Like, how do you architect this stuff? And more importantly, how do you architect it today knowing that in 6 months' time it's all going to completely change? Like, are you just going to be getting good at rebuilding everything?

Get really good at foundations. Put your semantics in. Yeah, I mean, uh, people I know, like, uh, I think Valplan had a conversation about this recently and I was like, yeah, they're not too wrong about this, but like It's in your code. Keep it as close to your code. Keep it as raw text as possible and then model on top of it. We've talked about that. If you're in text and you're practically speaking in Git with your Markdown files, with SQL, with code, everything's kind of together, then it really doesn't matter where you're moving around because you can just transform it with all these large language models. But the further away you go into any vertical on these vendors, they have their special sauce. And then yeah, you're trapped, right? Like, either you're trapped in DataWorks— not that that's wrong. I think a lot of people avoid being trapped in a vendor at times because they're like, oh my God, it's the worst thing ever. I think there's just like sometimes bigger things to fry out there than like getting trapped.

Yeah, I mean, it's like this open source bullshit where people tell me they do it because it's transportable, right? And I'm like, ah, it may cost you nothing to move to it, but the cost of the person, right? The cost of the people to replumb your entire stack onto the open source stuff or the other way. That's where it really hurts, right? Rebuilding it and then retesting it.

Yeah.

So testing always kills us. And maybe with some of the AI, GenAI tooling now, we're getting better automated testing. Kind of impressed with that. But my view is, is figure out what's cattle and what's a pet. Yeah, take the, take the DevOps approach. So for me, the context, right, the text about your organization is your pet. So protect that and put it somewhere where you own it, and then hydrate it to those other platforms as cattle. So if you're going to use DataOps Genie, master that context somewhere else and then just push it into Genie. And then if you have to later on, you can push it into Coco, or you can push it into anything else. Google Knowledge, whatever. Hasn't that great branding, right? So I— we, we use Google Cloud every day I know I can say Databricks Genie and Snowflake Coco more often than I could try and remember what the context knowledge graph thing that Google has natively is.

Yeah. Wow. I know they have it, but it's one of those things that just slips from my mind. So wild. I had to go into Google BigQuery the other day for some Google Analytics stuff that I was working on, but it was just like the interface had changed again. I've only been out of here for 90 days and I was like, What is going on here? I was like, how do I query a table? And I don't want to use tokens. I just want to write like a select. I know, forgive me, select star from. I just wanted to see it. I know that like the limit in BigQuery is not— it's like fake. It's like cosplay. But like I just wanted to see that.

Limit limits the data that comes back. It does not limit the cost. Yeah, it limits the data that comes back, not the size of the query.

And I was just like, where do you go? Where did you put my beer? I can't find it. I think that the— but there's all these other theses, the counter thesis, I would say, for some of these other players. If you think about the people that aren't tenants, but they're trying to dissolve a platform. So for example, look at, you got Motherduck, right? So some of the— you have the Duck Lake, but essentially they're pushing people, I think, to improve Iceberg and Motherduck in their play. is like a swappable compute engine, not a fortress, like completely the opposite of MPP, right? It's just like a very different play that they're going for. And marketing is brilliant. Like everything that Microsoft seems to do is just like, clearly they've done it well. Big fan. They really know how to tell a story shockingly good using ducks. And but then it's, but it's so interesting because of Databricks. Snowflake almost force adopting Iceberg and everybody's converging on these open table formats so then everybody's having to move upstream. Even Google was starting to allow me to play on Iceberg and they really didn't want me to do that stuff. It's getting better, but we're in each of these bets.

But I'm seeing MotherDuckDB change. I've just seen them change their messaging. Yes, they were around DuckDB, around edge, around small data, fast data, that stuff. But I think they've flipped to a platform play now. They bought in the diet.

We got diets.

Yep. Um, and actually they've just come out with cookbooks, right? Which is, you know, everybody's tried this every, every wave, every decade. We all try to come out with, uh, cookbooks that are reusable and can be, you know, 80% of it and then 20% can be, uh, tweaked, right? And I've never seen it work, right, commercially. Uh, be interesting to see what happens now, right? Because if we can provide that context as a cookbook to Claude Code or whatever, and it can do 80% of the work for us, uh, that's really, really useful, right? And the interface is different to how we used to have to do it. But it's like that poll I put on LinkedIn the other day, you know, what's harder to define, customer or subscription?

Well, I'll be honest, I was I was club subscription because I've had to design so many of them over time. And to me it's like, it gets like this, especially when you get into corporate finance. It's like, what is the subscription? We get paid upfront with this customer and this customer pays later. How do you amortize? It just gets so nuanced. I would say customer too could get like pretty gnarly cuz then you're trying to get the customer to define even the stages of a customer.

But yeah, I was doing some Yeah, and so how does that work with a cookbook, right? So I can get a cookbook for Stripe, I can get a cookbook for Chargify, but that's not going to give me the definition of subscription for any organization that I've ever done it for because there's always the if-then-but-else, right? Oh well, we do trials and actually we didn't really want to do them properly, so what we do is yada yada yada yada, right, in Stripe. And that's the context that's almost impossible, uh, to put into a cookbook. Or maybe it's not. Maybe, you know, every platform that's seeing that behavior will then strip out the context of the 5 different ways everybody does it. Because, you know, in theory, I'm assuming there's 5 to 10 to 20 patterns across every subscription company on how they deal with all those— not edge cases, but all those cases.

You know, you know, a subscription Or a customer. The irony is every business is like a snowflake. The devil's in the details, like all those things. It's so wild. It's so wild. Like every person I've ever consulted or done work for, every time I think I'm about to apply a pattern or a certain set of rules, it's just like it has to be unique to that customer or that company. And I was like, okay, why, why did you have to redefine this? this thing. And then maybe it's important for some parts of the business, and maybe that's like how they define it.

But, um, but I think there's some value in the cookbook idea if it tells me the patterns I've got to watch out for, right? So, you know, first time I rocked up to a company that wanted me to go and help code, you know, physically codify subscriptions and count them, I'm like, yeah, subscriptions, baby! Like, we'll do customer last because that's hard. How hard can a subscription be? And then I learned And so I wonder whether actually, uh, those cookbooks, right, where it says 80% of, you know, of the subscription process looks like this, everybody pretty much follows it, then here's all the edge cases or the things that everybody does various patterns for. So ask them about trials, uh, ask them about prepayment, or, you know, uh, ask them what happens about cancellations. Ask them what happens when somebody subscribes, cancels, and then resubscribes, right?

Yeah.

that funnel, because there's a bunch of use cases that really kill you. And then maybe that's where the value is. So when you're going to that domain for the first time, you're kind of getting the hints, right, of things to look for and how to look for them. Maybe that's where the first round of value of those pattern cookbooks will be.

Yeah, pattern cookbooks is also like honestly why I like doing data detective work, right? And even like, you know, think about, you know, think about the cookbooks, but even how you investigate like data structures and companies today is very different. I mean, obviously it depends on the client or it depends on the company working on the access to AI stuff, but like building a data sense maker, right? And like, how do you make sense of a company? And then you're like looking at the data and then you're looking at the report, you're looking at the reports and you're like, these are 2 different things. Like, I don't understand, like where's the context? And you realize it's like in Bob's head and you're like, okay Bob, I can look at your SQL. And now like thankfully you know, I can sit down with Claude or whatever LLM of choice this week and like kind of reverse engineer it a bit to get a pretty good understanding. But then of course the hardest part is like, now you got to change Bob's logic and you're like, okay.

Like, I'll tell you, one of the things I do when I coach a team is at some stage I'll ask them how the company makes money because I'm kind of interested if I don't know already, but I'm also interested if they know. Like, how does a company make money, right? Because that's kind of important if you're a data team. I had this the other day, somebody was saying, oh, their team's structured here and they kind of, I think it was in IT under the CTO or whatever, and they were looking to move, you know, should they move the team into another part of the business and if so, how do they determine which part? And my answer to them was, which part of the business, how does the business make money? They told me.

Yeah.

I said, who's the most senior person in the organization apart from the chief executive that runs that part of the business that makes money? And I think they said chief marketing officer or something. And I was like, that's who you want to work for, right? Because they've got the problems that actually will make a difference to the organization using data. So because I asked teams there, I wonder if actually if I asked an LLM that's been running in an organization for a while, how does the company make money? That would be interesting, right? Well, like, could it actually detect where that value is.

Yeah, some of them, I mean, it just really depends on how good the player is, right? The person that gets in command of these harnesses and stuff that we've talked about before. I think that's really, really important of basically how you're creating almost like, you know, the wiki, everybody had those wikis Like everybody had to have a wiki that nobody maintained. We all said we're going to do it, but no one did. It's crazy on how important that has become. It's like the downside is that then if you go and of course use that old wiki to go into your new data models, it's so wrong because none of them got updated. So it's like, you know, everybody says like, what was it? Like everybody should document, but nobody wants to. And thus it continues. And now you have AI helping you document it. So I mean, maybe that will, that will get better now that we can all be lazy about our documentation. I mean, okay, hang on, I'm gonna go blast to the past maybe. Okay. Remember Dragon NaturallySpeaking?

Yep.

Yeah. Like I remember using so many versions and I'm like, you know, this is the like early 2000s and I was like, saved so much money to go buy this $200 piece of software, get the DVD, probably CD at the time, install it and use it. And you're like, you see all these commercials or magazine things about how good it's gonna be. Dragon NaturallySpeaking, come back here, Mark. And it was so bad. It was so bad. But the ad in PC Magazine was so good. But oh boy. But now, now the language models, like dictating. I use, I use Super Whisper for privacy. I like it. Whisper for privacy. and it was everybody's vibe, but I like Super Whisperer. And its translation and dictation is wild. I recently was yelling at Siri because it just could not understand me, and my Canadian accent is not that thick. I'm sorry. Or at least I don't think it is.

No, try and be a Kiwi where, you know, everything's fush and chops. Hey, just a shout out to Mark Freeman for joining us and saying hello. Anybody who's on actually wants to ask a question, that is, uh, ask us anything. But as I said in the, in the promos, otherwise we'll just talk shit for a while. Um, That was the first one.

I managed to dodge a few of them. Okay, gloves off now.

Not where I come from. Um, like, one of the people— one of the teams I was working with a while ago, we, we experimented with, uh, the developers, the engineers, actually writing, uh, the context of what they were doing as they were coding, uh, separately. Right? So this idea of just write your thought stream, uh, no structure, no template, right? Try and make it really, really easy, uh, and then grab all that context over time, whack it into an LLM, and see how much insight it gave. And that was really interesting. And, and I kind of thought, well, actually, why don't we just dictate it? I mean, I don't— I'm not a dictator, right? Um, yeah, uh, so I don't, I don't think that way. I always Yeah, I'm going to go try and change it. But maybe that's what we should be doing, right? As we're coding, as we're figuring— well, as we're telling toward the code, we should be kind of dictating why we're doing it. And remember, there was that guy Nico and somebody else that did— what was their product called? They did like a version that was— their first version of the company was orchestrating the modern data stack. Uh, kind of thing. But the second one was, uh, Analyst Journal. And so basically, as an analyst, you were working in this IDE where you were kind of querying the data, writing code, and looping through it all. And on the side, you were keeping notes, uh, about why you did that. And, and their, their idea was when you come back 6 months later and you're like, why the hell did I put that WHERE clause in there, you had your journal notes on the right-hand side, right? Uh, so it was highly valuable Maybe you'd share it with the rest of your team depending on how you feel about job security. And so my theory was like, wow, that was great product too early. Because right now, if you were doing that, that context of why you're writing that code is really valuable. Maybe try just using Whisper, but go back to—

that's your pet. Nico, I miss your open— Nico, I forgot his co-founder's name, sorry. but like reconfigured. And it was— yes, I was, I was a paying customer.

Oh, you are so early adopter. I reckon every time we do this every month, we're gonna have to have Tim's top pick for bleeding edge shit to play with this month.

Oh man, I'm like, okay, shout out to Mark Freeman because I've been using Petri. Sorry man, I'm still confused. I'll figure it out, I swear.

Okay, so that's the one though. Is that our top— Tim's top tip for next— this month? So, because you're at the beginning of the month, right? Yeah, leading-edge tool to play with, Petri.

Uh, God, whatever we got today. I'm using—

no sponsorship.

Oh, I got one. No sponsorship, by the way. Uh, but I'm using one called Human Layer. Oh man, if you love deep coding and like, like harnesses and all the concept stuff, but like we were talking about it and working— me working through it, and you're like, oh my God, it's so much like front-loading, and it's like the whole structured process And it's like, it's like BMAT on steroids.

Yeah, let's talk about that, right? Because, um, I tried to use superpowers and I kept asking me too many questions and I'm like, just shut up and go away. And then I deleted the skill. So I have this theory that actually, because, you know, vibe coding allows people like me that are completely unstructured and can't code for shit to do really cool or dangerous things. And so I think there's kind of 3 people, right? I think there's people like you that come from the background of doing it as a practitioner and are very structured, right? So you don't mind writing a quite detailed spec, uh, with acceptance criteria up front and then let the engines run off and actually build it, right? And then, then test it. That's one extreme, one side. And then there's people like me where I'm just going to go natural language, this shit, and then test it and just keep iterating as fast as I can until it actually does something that I like. And then I think these people in the middle, right, that are kind of not willing to ride it all up front, but not willing to just vibe it, you know, to YOLO it or bareback it. And so they're kind of stuck in the middle. And I'm not sure I'd want to be the person in the middle, right? I think you want to be one of the two. And there's a whole lot of harnesses now, right? So, you know, we've talked a while ago, we talked about NOW and KTX, and we talk about them because they're doing a brilliant paid influencer thing at the moment, right? They're kind of popping up in the feed all over the place. There's another one which is Open UI or Open Web UI or Web UI, which isn't for data people, but is another harnessy framework for putting in your prompts and all your agentic context and all the other contexts. So before we do that, do you want to have a go at explaining what a harness is?

I mean, I could try. I feel like every time I try to do it, I'm like Same here. I'm like, I don't get it. I mean, a harness to me is like an orchestration of markdown files. It's like the least— it's just like almost like a giant if-then-then-else component of it. Like a harness to me, and I'm probably wrong here, folks, and despite reading like 100 articles on this stuff, I mean, it's almost like, you know, a mini Airflow or a mini DAG. What's the difference between a DAG and loop engineering?

Uh, so my answer to that would be don't loop engineer until you have a harness. So the way I think about harness is it's a bunch of guardrails.

And by the way, I mean, you think of an IFS team as a guardrail, but sure.

Yeah, yeah. So the mechanics is a bunch of MD files or a bunch of if statements I tend to, I tend to take my coaching harness and apply it to the LLMs. Kind of interesting. So I have a—

Is it all in caps though? Do you like keeping—

What's the one where you do like, you put 2 words together and the first letter of each one is caps?

Camel case?

Yeah, yeah, I camel case everything where I don't underscore it and I don't know if I'm a tabs or, uh, Yes. So I have a persona file which tells the agent how I want it to behave and who it is. I have a policy file, which is the rules it can't break, right? Very few of them, but it's like, you cannot do these things. I have a way of working file, which basically says, this is how we work and how we do tasks together and all those kind of things. And then I have a retro file where I'm just working through some kind of self-looping at the moment where I have a red-blue team and a judge reading all my logs, all our chat logs, and then putting stuff into the retro that it thinks it should update. And I'm just finishing off the bit where there's another task that goes through and reads the retros and then updates those other 3 files. And so, and then of course, if you use Claude Code, these skills, right, which makes it even slightly harder because that's a combination, and there's Claude Then there's your memory, which is a persistent store of all the things you've done when you start a new session.

Yeah.

All those things are there to allow you to start a new session and it remember where you were and not lose the context of all the work you've done with the LLM before. They're there to stop it doing things you don't like or things you do like. That's it. Loop engineering, because I've only started reading on this, is this idea that actually I can now give an LLM a goal, and then using my guardrails, it will then just keep looping itself around and around until it achieves that goal. Which, if we go back to me, that means I have to write acceptance criteria and eval criteria, what success looks like. So I'm never going to loop successfully. For you, yeah, you could probably go, this is what, this is what good looks like, right? When you get to here, finished. Um, but that's how I see the difference. What do you think of that?

Well, yeah, I mean, that's, that's very good. I was still like, he's like, you know, uh, Mark's still online if you're still here, because he's got a scientist background. You can see Petri, like, it's like built on that kind of like looping from what I can understand. And but you look at like, um, Karpathy, like he did, like he's done so many interesting things. And one of the ones was like his wiki was really interesting. Um, but the other thing was his like auto research. which is basically, it was just a big loop constantly going over, giving it the tools. It's 100 lines of Python. And it was like, same thing. What's, you know, ultimately what's the goal that you're trying to achieve here? I think one of the fun things as the models have gotten improved and you can test, obviously you can attest against this, is like the weird wiggles that all the models have when you're trying to ascertain a goal and how they all get to it and also how crazy different they can Like, yeah, um, I think, yeah, the thing is when we talk about harnesses, so we'll codify some guardrails or some behavior in our harness, and then when the next model comes out, it's actually better at dealing with that problem than the codified way we did it.

And so if you don't remove it out of your harness, then actually the model is not as good as it could be. And so I don't know how you do that, right? How do you go back and say to it, Well, look at the latest model because I noticed that, um, Fable 5's just been released, been coming back on slowly. Don't have it in my one, but, um, Sonnet 5's out now, right? So how do I get it to look at Sonnet 5, figure out what it's better at than I am, than my harness is, and then remove that from my harness because the latest foundational model deals with it? Like, there's too many moving parts now, right? Too many moving parts.

The famous, the famous, what, 12-month rule or 14-month rule of prompt engineering is already gone at Swansong. Um, the prompt engineering rules, damn it. Um, it's really interesting. So I'm like, I'm working, I'm like, you know, working in this space obviously. Um, and I'm still very much like hands-on coding. Well, let's be honest, 90% AI coding, 10% practical coding. I did, I did teach myself Rust by hand. without using too much AI. And that was brutal. It's so hard, so hard than I thought it was gonna be. I was like, God, only 2 years ago, this is like, it would've been, I'm not gonna say it would've been easier to learn, but I would've probably been like less addicted to just asking a quick question. And God, we gotta talk about that. Like the ability to understand stuff with all this AI offloading. But yeah, like getting consistent output. So like how I started working on this, stuff is like, I work on the testing, so I use a coding language called— not a vendor spiel, guys, but BAML. They're out of Seattle and it's like a coding language for LLMs. I've really liked using it because it's like I'm putting in my tests around it and expected outputs and I can let the LLM go off and do its crazy whatever it's going to do. But I'm like, no, I need this JSON blob or this output of the thing. to have this consistency and the model in the background can do all its crazy stuff and I can have my prompts going through it and then I can know relatively how this non-deterministic thing is doing all these outputs. Let's be honest, most of us want deterministic outputs on a lot of the things that we do today.

Yeah.

I want to know if I'm doing image recognition on something like a cup is a cup, And if it's a donkey, you probably want it to be like— because there's a donkey on the mug. It's like, which one's more important? These large language models, I was using some older ones recently just to see what it was like. It was wild, the quality changes I've seen. It's just like this crazy curve in possibly both.

I think that's a good point though, is that my view is right now, if you're entering into the space, you should be using the LLMs to help you write deterministic code, not to write it to do it without you. So, you know, one of the things we've got for an information product is, is we're building at the moment is we'll do a brief. So customer will pick a thing, let's just say product, and then it will give them a bit of text, right, with numbers embedded in it, which is the brief around that product. You know, this product is your bestseller, you know, had a 12 over the last 12 months it increased by yada yada yada, your margins so much, right? And, uh, your main channel is blah blah blah. And so the first question the customer said is, is the LLM generating that? And it's like, no, the LLM generated the code, right, that we use. But whenever that code's running, it's now deterministic, right? It's running a select statement to say, you know, top-selling product, and here's the logic. And then it's running one saying what's its increase over the last 12 months. And so every time it runs, it's running deterministic, but we used LLMs to help us write that code. Well, I did anyway. I want to play with the next one, which is putting another section in that information product page that is, did you know, in bright yellow, and then actually let the LLM just kind of guess at some stuff that might be interesting, right? But putting big warnings around it about it made this shit up after time. Um, so I think if you're going to start out, that's what you should be doing, right, is you should be looking at using the tools to make you a little bit faster. And Nick's got a, um, point in there. He's saying, you know, I was at a CTO/CPO event last night, and one of the lines that stayed with me from a CTO was, 90% of my skills are now worth zero, but the remaining 10% are worth 1,000 times more. And so this comes back to this argument, uh, is LLMs and GenAI going to kill juniors? or seniors, uh, I think what we're going to end up with is 80 to 90% of everything that's done is going to be commodified GenAI slop to a degree, and 5% of it is going to be massively high quality, differentiated. And if you're working in the space, you need to understand which one do you want to play in.

Yeah.

Are you going to be the top 5% of your domain, uh, or are you going to be really good at using those tools and you you know, doing throughput or token maxing or whatever the metric for your company is that, uh, apart from, you know, better business, making more money, serving better customers, uh, shareholder growth, what are the other metrics that you've got to gamify? I mean, what's your thought?

Yeah, I mean, it's gonna be too, too different, um, in perspective. Yeah, I'm gonna say there's, you know, there's gonna be a lot of people that build the jets, but there's only very few people are going to fly them, right? So you got to figure out kind of where you're going to be. And most people shouldn't and can't fly jets, right? Or are going to be able to be in that top 5%. It's just, it's going to require such a crazy— like, I mean, you can see that, like, the people that are exceptional at using a lot of these tools are, like, good at multiple domains, like, across. Like, there's this little TED Talk about being a multipotentialite, and I think that's, you know, ADHD. And I was just like, it's now almost like a superpower, like your ability to be a master of so many different things. Before, it used to be everybody's kryptonite. Now you're like, well, actually, I understand the business, I understand the technical, I understand the architecture, I understand, you know, all these different, like, components that make up a business. And now the people that have those skills, I think they're going to be very good. But then like you've got to have some depth there. Like you can't just be surface level on everything.

And you're gonna be able to multitask because I don't know about you, but you know, I'm time slicing all the time now. I'm running multiple things and trying to do some other work because I'm not sitting there waiting for the thing to finish, right? I'm like, yeah, go do this, it's the next step, then I'll go do something else, then I'll forget. Like I actually got one of them actually sends me a Signal message when it's waiting for me Because it's on my Mac Mini and I never see the screen. It's like it sits here for too long. Like, I'm like, all right, so now it sends a message saying I'm ready.

That just feeds the beast even more. I mean, I'm so bad with that stuff. Like, I actually think focus is so important. I have like an actual physical hourglass on my desk that's like 15 minutes, because if I can't— like, and that's why, probably why I spent so much time on a PRD, or, you know, or specs or something like that. And I think really long and I dictate on it. And I like to focus, force myself to focus, which is not my strength, and trained over years, um, to focus on the thing. And so I have clarity of exactly what the mission is going to be. And then I go off. And yeah, of course I have like 3 or 4 of my, like, what do you call them, cattle going off and doing my things and getting slaughtered in the field. Um, because like, oh please. What's going on?

Out of there?

Uh, I don't know. It's just like I touched my desk. Maybe my power's going off. Mind you, it's super, super hot here, so my electrical in my house built in the 1800s is not doing great. Um, the, um, but anyway, yeah, going off my agents, go off and do the work. It's just been really interesting as we work through these processes over the past couple of years, just how dictatorial we get. Also, the one thing I do do is I actually read the code. Not all the time, it depends what level I get. Sometimes I'll scan it and sometimes I'll get to a place and then I'll have it explain certain things so I understand what it's actually performing and what is actually going on. Can't say inspect YAML very much. Like actual logic, I'll actually want to understand it because I consider it my responsibility as an engineer. What I deploy into the field, I absolutely have to own it. And I guess like the agents are going to get obviously getting better, but I would say I have a healthy suspicion about all the different QA agent coding skills.

Yeah, yeah. I mean, I've read something the other day that you should never ask the agent, the LLM, your work session you're working in to peer review the work it's doing. You should always ask another session, and then you should always ask that session to peer review it with a different model. So if you're—

Yeah, yeah.

So if you're kind of going along and doing, uh, you know, Sonnet, then you should— whatever it's called— yeah, you should change the model, um, if not the vendor, because that's the only way you're going to get a true review.

Yeah, I agree. I use a tool called RoboRev. It's the guy that created Pandas.

That's Wes, right?

Yeah, Wes. I'm pretty diehard on that one. Usually I do use different language models to inspect. I find OpenAI's one is very pedantic. Sometimes Gemini, if it doesn't break, it also allows me to have form up different, like, different pieces. I also like, like, breaking up my work, like, really, really small chunks. Um, I figured, like, seeing, like, you, it's like, as soon as your context or, like, your work stream— like, any PR for any person gets too big, funny enough, it gets really squirrely. But if you, like, chunk it up, like, really small and tight, um, I have found better, um, capabilities. And by the way, I'm nowhere close to being in the top 5% of people yet. I'm still—

Nearby.

I just sometimes I find the tools change so much so often that every time I think I've like nailed it, another one comes out. And you know me, I'm like always like, you know, playing with the toys.

And everything changes. Um, you know, that PR review one, so when we— so when I get Claude to submit the PR, then Gemini automatically reviews it on the, on the GitHubby thing. Um, and they were having a fight today because I had a list of values and, uh, There's like 451 unique variables in that list of values as a dropdown on the information product. And so I'm like, yeah, that's tested, that performance was okay, cool, you know, like, that's good, just generate the whole list every time they go in there. And then pushed it, and then Gemini is like, oh no, that's going to have a performance problem, it's going to crash their browser, you need to limit to 100. And so Claude goes, oh yeah, I'll limit to 100. And I'm like, no, you won't, because I want all 451. It's like, okay, they pushed it back, and then Gemini's like, no, it has to be 100, I'm not letting it go. And they were just bloody arguing until I overrode them. So just like a team, right? It's like 2 people arguing. I think the other thing that you mentioned about is like the number of times I'll go and you'll mention something and I'm like, oh, it's another harness. And then I'll go look at it, oh, I like that bit. And you're just constantly trying to find something that just does what want to do perfectly, and you never do, right? You find patterns that are useful, and you should adopt them if they have value. Um, but I'd say 80% of the harness that I've built for myself, I don't actually use or get any value out. Like, I've built a whole personal OS. Nick Zed started me off on this one. Thanks, Nick. Um, and it tries to task manage me, right? I've spent ages trying to help get it to help me manage which tasks are the most important to work on next. Of course, I, I ignore it because I don't work that way. So I know, right?

You're trying to like get things done, get like that whole list, and you're just like, it's just so counter to like you as a, like a personality. You're like, I just like, it doesn't matter. You're like, was like task list meant to-do list manifesto? Like, or like, I mean, mine is like calendar. I put stuff on my calendar to do. Like, that's my to-do list. It's like, if it's not on the calendar, then it's going to be put into like the whatever list.

Yep. But, but what I do do is in my harness, I force the LLMs to keep, uh, a to-do, a doing, and a done. Um, because when I run out of tokens, right, when they run out of tokens in the middle of a task, uh, and 3 hours later I can kick them off again, I want them to know where they caught up, where they, they're sitting, right? And so they just go back and read the file. Um, before I had that, it was constantly going back and reading all the context memory and trying to figure out where it was, and then forgetting it had done half the job and rewriting it. And yeah, so again, the interesting thing for me is I'm bringing those human behaviors in, and I'm not sure that's right. You know, I'm treating them like employees and agents. I'm going, this is how I manage it, to help a team manage themselves. I keep going back to them with Clear Sessions, to the LLMs, going If you were going to run yourself, how would you do it? I never get a good answer.

So, but like, but like, you know, like, but that's how the marketing machine works, and that's why it's been so effective, right? Like, I would argue the reason these systems are so effective and have gone so widespread is because they're like the Turing machine. Like, the very thing that we always like, yeah, it won't happen, won't happen, what happened? You know, it's happened, and you're just like Wow. I mean, I think it's incepted my brain faster than any social media ever did. And it's just like, oh, whoops, I didn't realize that the shortcut is language. And obviously lots of researchers knew that, but like, the way that they like incept is so intense. And I think in this space, and this is where I get like worried a little bit, is like, you're like using Claude, you downgrade. And it goes, not that Claude— Anthropic claims they'll never do ads. We'll see, right? And also you're doing that and it's like, have you thought about Databricks? And you're just like, wait a minute. You know, or maybe, you know, even worse though, in these systems, it becomes more subtle, right?

Yeah.

Unfortunately, we can't solve this problem because you've chosen an inferior system, like in fancier language, you should think about Context Plane, brought to you by whatever. I don't know. And you're like, damn it.

Oh, and it's got the magic button. Would you like me to migrate you all magically for 1,000 seconds?

Don't worry about it. We already have your credit card on file. Yeah, we'll move you over. Yeah, yeah, yeah, we're almost at time. But you know, that's the funniest thing is Remember when we did that era and it still exists, it's like those massive migrations, like moving from say like an ERP system or accounting systems. Oh my God, that whole space with these tools now, it's going to go so much faster.

It's easier to map the 2 systems. It's easier to get them to help us write the code to do the migration, and it's much easier to write LLM tooling that actually tests it for us, right, and picks up that 80% we always miss. All right, we're up on the hour. Uh, what's the one thing— if you're— if you— somebody, a data practitioner, said to you, what should I do this month to kind of get my toe in the water for this stuff, what would you say? What would you recommend they, they do?

Um, build your own— build your own brain, uh, like your own memory and your LLMs right now, like your own proprietary thing for your household, for whatever it is, really just get good at that because then it doesn't really— and don't attach it to a system. I know a lot of people who just said, who cares if you're on a platform? But be honest, keep your information yours to a certain extent. I think that's a very important piece, kind of like that CTO, CPO piece. If you're dictating, doing all these things, create up your own. It's really easy to do Embeddings, vector embeddings. I mean, this is a data podcast, so you probably know how to vector embeddings, but take all that stuff, that context, all that information that you're taking down every day and just start building a memory bank for yourself. All those articles, all those things, because ultimately becomes your self-referential library and it can also be in your own voice and then allow you to, I don't know, go against it. I think that would be my thing. And I know a lot of people are trying to work into that space, but I have found that personally the most rewarding.

Yeah, and something like Obsidian makes it a little bit easier to store it and see it in a UI as well as, as code. Uh, yeah, so my one is go, go read an article that talks about some of the shit, and if the article sounds interesting and gels with you, go and try and build something. that is related to it. Like, even if it's just one small part of what that article is talking about, right? Just go do it, because as you play with these things, you'll learn so much more about how they work, how they don't work. Um, so just, yeah, it's time to play. Like, everybody's learning. There's no actual answers yet. So, uh, yeah, go play with it and learn. Like, yeah, enjoy the experience of having a complete change of technology and way of working. Uh, come on us.

I know, right, man? What a time to be alive to have this going on. I'm super— it scares crap out of me. At the same time, it's like, as an engineer, it's like the ultimate toy box.

Yeah. Oh, you got it again.

Yeah. All right, talk to you later.

Excellent. All right, we'll talk to you next month. Catch ya.