Separating AI Hype From Real Developer Gains

I've seen some discourse around, you know, AI generated slop and how all these SaaS platforms are just degrading and whatnot. I think yes, that is true, but more than one thing can be true at once, right? I think some people, us included, are writing some of the best, most secure, most performant code they've ever written because it is easier to generate that code. And when you do know what you're doing and the bottleneck isn't typing each keystroke, you really can orchestrate it at a lot higher level. Hello and welcome to Even Keeled, the podcast about the craft of creating software and building effective development teams. I'm John Kinney, founding partner and CTO at Headway, a product design and development agency. For more than a decade, we've been helping startups and enterprise teams build better products faster. From launching new products, adding new features, building powerful design systems, and even transforming the way their teams operate. Every discussion we share is a window into what we've learned, how our teams are leveraging new tools, thoughtful perspectives on the latest trends in AI development, and more. In today's episode, we talk about how we're helping clients leverage AI, from early ChatGPT integrations to today's frontier models. We dig into the work that Tim and Dan have been doing with design systems and why giving AI agents better context makes a huge difference. We also share what's working and what's not with tools like Claude Code Cursor and Windsurf. And I walk through an internal tool I've been building to streamline our estimation and sales process. Enjoy.

How do we help an organization leverage AI? What does that look like right now? Like, how do those conversations begin? Is it usually like, hey, we have this idea or we don't know even know what to do? What are you guys doing right now with other clients? However you want, I guess.

It's progressed so rapidly over the last even year. I think one of our first uses in the wild was just sort of exposing ChatGPT, so to speak, to the end users with a prompt that would guide them towards sports specific questions. So this was a baseball app that would help advanced teams learn their playbook more effectively. And you could just ask it if you were maybe on the edge of being an advanced team, but maybe your coaching experience wasn't quite there. You could say, hey, I've got a first and third situation, you know, what do I do for defense here that's most effective. And it would know based on the system prompt that we gave it, you know, sort of embedded into the deployed application that it should help you understand the options and if you gave it more context like you know, these are, It's a traveling 12 year old team or whatever, it could help understand probably with some level of discernment based on the training data that was inherently available to it. This wasn't super advanced. We weren't doing ML and Human in the loop or anything, but just kind of off the shelf, you know, Sonnet Claude Sonnet type type integration. And it would, it would be generally good at what you should do in those particular situations as a up and cominging baseball coach or if you were a college coach or you know, minor leaguer, which we've had some folks use the app in that, in that regard it would, you know, the questions you ask, the responses you get would be more tailored and more advanced based on the more advanced prompts. So I think that was one of our first kind of in the wild deployments maybe 18 months ago.

I was going to say. Yeah, it was almost two years ago probably.

Yeah. And since then I think, you know, we've started using it a lot more just behind the scenes for all of the administration of projects. The actual coding has changed so much even in just the last six months and especially with the latest frontier models that have been put out with Opus 4.5. And I heard, I don't know if it dropped yet, but it's February 3rd today, rumored that Sonnet 5 is coming out today. So I guess we'll see if that changes the game. But really the biggest thing I think that's still unanswered is with the new capabilities of these frontier models, how can you safely use them in the context of giving it access to enough data where it can make meaningful, you know, decisions or help you make meaningful decisions as an organization around a key metric or idea or a go to market plan or, you know, anything that you would typically need to take multiple people looking at for days or weeks. A lot of analysis, maybe report building that, you know, used to take a long time because you had to collate all the data. It can just do it so much faster. But how do we do it in a secure way? So sort of just answering business questions, I think is where a lot of people will enter that stack. Um, but we so far have really tried to just lean on our expertise as builders of systems and helping people create more structured, more reliable, faster systems. And so I know Tim and Dan have been doing a lot of work in design systems and being able to help structure a code Base in a way where when you tell agents, hey, I want to be able to create this new component, or I want to, you know, really create this screen and it knows the list of components that are available and they're well defined, it can really go a lot faster and help understand what it is that you need to compose that resulting ui. So I don't know, Tim or Dan, if you want to speak to a little bit more of the nuts and bolts of some of the things we've been doing there for a couple of our clients.

I think what Dan and I have been seeing is that like the, the MCP offerings, the agentic aids that companies are offering. So if you just take like Figma's MCP offering, for example, what Figma is providing is not well informed of the design system that we've helped build out. And so, so their MCP offering is helping the agent that we're using, often Claude code. It's helping Claude code produce a better, a better artifact, a more accurate artifact. But it's still, it's still missing key ingredients, it's still missing design tokens or components that we've created that are a crucial part of what we're building. So, so being able to add that context to the workflow, the agentic workflow that we use or that our teammates use, you know, with the client, has made a pretty substantial difference in, in how accurate the product is that we get from an agent.

How do you and Dan, like, kind of like divide and conquer on that front? Like, I know Dan, you're more, more of a mobile expert. So I'm curious, like, as far as like design systems and the connectivity there, how are you guys kind of like, oh, Tim's focused on this aspect of it and Dan's focused on that aspect. Or is it just kind of like, hey, yep, you're kind of like leapfrogging each other, like, Yep, I have 10 this week. I'll make some more progress. And then you kind of hand off to each other, like carry the torch, so to speak.

It's more that leapfrog approach because what tends to happen is that that split between, you know, web and mobile becomes a lot less important, especially with the way that we, we build design systems and that we can build them in a really cross platform and platform agnostic way. When we come at it from that approach, then the way that the AI tooling comes into play really matters less about the platforms and more about the split between, like that deep back end work that Tim has, has done a lot of on this project and then how we map that to the front end work. So what we're finding is that there's a lot of tooling that still needs to be built and it's getting introduced in more of like a platform way with things like the Figma mcp. But it's still not great, it's still not turnkey enough I think for us or for our clients to really pick it up and just use it. So we're really having to fill in the gaps in a lot of ways either through tooling to, you know, get some of that information out of figma in a way that the AI systems can use it in a really like thorough and thoughtful way, or building more of a bridge from our AI tools into the front end. Sometimes with things like Figma Code Connect, that's been really useful, but it's not useful in a broad spectrum yet. And so taking kind of some of the best of all of that and building our own recipes of how we use them has seems to be how things are shaking out that it makes any sense.

Yeah, no, it does, yeah. Again, I was just curious because you know, in the past it's like hey, this person's going to focus on this aspect. But like you said, it just seems more agnostic now. It's like, hey, these tools can kind of understand no matter what we're dealing with. But I think a lot of it's like you were saying, like filling in those, those connectivity gaps, like hey, it knows this but it doesn't understand that yet. How do we give it that information for now until like that gap is closed by some other tool or whatever that might be?

Well, and the, the trick is that, you know, these large language models are they're not item potent and what I mean by that is you, the same input is not going to get the same output every time.

Right.

Which is one of the things we strive for in software development, especially around things like testing and deployments. You want the exact same thing to happen every time and that's just not guaranteed. When you ask a question of AI and you know, to further get into the weeds, they're a stateless model right now, which means every API call to an LLM has a certain defined amount of context and it doesn't necessarily, unless you build systems and tools around it, have long term memory. So it knows what you asked at that time and you append another thing to it and it now knows a little bit more because there's a new message in the thread and now it knows what you asked at that time. But that's where these context windows, you know, come into play and the agents get less predictable and less smart and even start to cut corners, you know, when the context windows fill up and those context windows have been getting larger. But that also isn't necessarily the only thing that's going to solve that problem. And so I think ultimately being able to have more predictable guardrails around what aspects of automation? Because really that's where all of this is going, right? It's automation. What aspects of automation truly need generative AI and large language models and what aspects of this automation have really already been solved and now we're just making it harder by trying to put them into these, you know, non deterministic systems.

Yeah, yeah, yeah. There was an interesting study done by SparkToro around just, just search and like you know the Google, the over AI overviews from Google or searching in ChatGPT or Claude or Perplexity or whatever and they just had a bunch of people use the exact same question and use it multiple times in, in like a week or a month. And like so everyone was asking the same exact thing and the results would be different like you said, even from the same person, 24 hours later the results could be completely different. And I know like normal search can be like that as things like start to rank or there's news or whatever, but usually like the top 10 on a page, you know, the first 10 links or whatever are almost the same. Like, but like it just, it just does seem like you know, when you're thinking about like hey, even if an organization has all this data into a data set and they're using an LLM to ask it questions, today they could be asking for a go to market strategy and get one answer and next week it'd be a different answer. And so that's like, really, that's like a big deal. Like so it's like how do you know? And I like you said it's the state, you said you call it stateless.

Stateless, yeah, yeah.

And it's like so how do we, is there a path to like fit fixing that and what, what would it be? Are there people or are there people trying to fix it and what have been the results been?

Well I think what we're, we're finding success with right now, I don't know if this will persist is, is treating it in the same way that in the past we would treat it as like a software spike. Like we're just going to write the code that gets the job done. Maybe it doesn't follow all of the architectural patterns we'd want. It's. It's a little bit more spaghetti, it's not as clean. And then we clean those things up once we understand kind of the problem space. I think what we're finding success in is doing that with AI, throwing everything at the AI kitchen sink and then seeing where we want things to go, what the outputs are, and then pulling it back, trying to really systematize it, trying to get AI out of the mix as much as possible, except for the spots that it needs. Seems to be a really reliable way of getting back to something that's not. It's like quasi deterministic. You know, it's not. We're not getting exactly the same input and output, but inside of these specific bounds where we're okay with content being more malleable, everything around that, we can make sure that we get back what we would expect. And that seems to be a good way to move forward. I was actually just working on a thing earlier today that is very much that.

Where we're.

We're breaking out some designs, having AI do some analysis, and then returning that analysis to us, along with some structured formatting of that design, which AI could do all of it if we wanted. But what we know is that what it's going to give us back from run one is going to be different from run two, and we don't trust it enough to be correct in both runs. And so we're gonna, we're gonna hedge and make sure that we know we can get reliable information. But we also do wanna pepper in the detailed, like, analysis that AI can do about a large structure of code and some other various inputs to get back something that is really useful and would take somebody, you know, some, some amount of time to actually do manually where we can. We can do this entire flow in, you know, 30 to 45 seconds.

I was just going to say, Dan, what, what I think you're talking about is really trying to control the slop factor, right? Which has become kind of somewhat of a buzzword along with AI vibe coding. Right. And so people are, you know, context, engineering all these things. I think slop factor I initially was, was like, well, yeah, AI slop, sure. I wonder where that came from. But it occurred to me that it is kind of already a technical term because in iOS development, there's a thing called hit slop, right? Which is how much off from hitting a button you can be. And it still works. That button still gets Triggered. So I don't know if there was any of that that came into play around slop or if people just think, hey, the code is sloppy and I don't care. But in any case, kind of a funny little connection there. But the, the thing that's interesting is I've seen some discourse around, you know, AI generated slop and how all these SaaS platforms are just degrading and whatnot. I think, yes, that is true. But more than one thing can be true at once, right? I think some people, us included, are writing some of the best, most secure, most performant code they've ever written because it is easier to generate that code. And when you do know what you're doing and the bottleneck isn't typing each keystroke, you really can orchestrate it at a lot higher level. And it's way easier to get rid of the human slop that used to be in the code base. Like we've all had code bases where, oh yeah, I guess, you know, that function is just 500 lines and it is what it is. We don't really have a way to clean it up because there's never enough time allocated in the sprint to really go back and fix that. Well, you can point that function at AI now, tell it what you actually wanted it to look like and it'll generate it in under a minute. And so the ability to remove our own cruft and fix our own bugs in the code that we know are there, but didn't really have enough time or energy or weren't given enough budget to fix or change, it's so much easier now to tell it how to do the right things. And I think that kind of just. I want to piggyback that on some other ways that we at Headway are really able to stand on the shoulders of what seem like extremely insightful decisions now. But we're kind of just the product of making code better for humans is also making it better for AI agents. And that is convention over configuration in the tools that we use. And so that's the case for Ruby on Rails, which is one of the main things that we use to develop products. That's the case for Elixir and Phoenix and both of those languages have very succinct English like expressive code. And so it's easy for the AIs to generate that code and it's token efficient. And so the technology that we've been building on for the last 10 years at Headway, and much longer than that, before we all started this is really well positioned to take us into this next agentic development phase. I think maybe a little less so on the JavaScript front, just because there are so many ways to do things and that's where, you know, you ask it the same question and you get a different result, you type the same search and you get a different result. That is very much the case when you're trying to solve a JavaScript problem, because so many folks have done things so many different ways and it isn't that you can't solve the problem, it's just probably going to be somewhat of a different answer depending on how you couch the question and depending on what context your particular project is in it. That's not to say it can't do it, of course. You can say, well, something like a Rails has really strong convention over configuration, a long history of open source, so there's a lot for it to train on. But the amount of training data pales in comparison to JavaScript. So it's a bit of a trade off. Right? Like there's maybe a little bit, there's still tons of Ruby and Rail stuff, don't get me wrong, but there's less of it than JavaScript. But what is there is maybe a little bit more idiomatic and so a

smaller pool of higher quality training data, right?

Yeah. And that's not to say that some JavaScript code can't be of high quality. I mean, web apps suck without any JavaScript. It just is what it is. But you want to make sure that you're doing it intentionally because, yeah, I mean, hey, my, my Vibe coded app works great. Except the problem is it has to download 15 megs of JavaScript before it can run. Right?

Like that's, oh man, that's not good.

That's not universally the case. But somebody who doesn't know what they're looking out for, that can very easily be the case. So yeah, we're seeing more and more clients kind of bring a Vibe coded app to us for us to take to the next level. And it's interesting because on the one hand you can almost throw it away because it's so easy to recreate. Oh sure. But on the other hand, you know, trying to understand all of the nuance that they prompted into it is trickier because they don't have good specs or requirements. It's just, hey, I built this thing

to add on to that. Like the app that you recently helped build a native version of, which was initially, was it initially web or do they just have a React native and you help migrate to Expo.

Yeah, it was React native from, from. From the cli, sort of the old style React native build and we converted it to Expo through, through the help of Claude code. And yeah, I mean it was, I, I know like one shot gets way overused, but it wasn't far off like it was.

But I think it's already interrupt. It was pretty close. But I think part of that is because give Dan credit, his knowledge of both web and native platforms helps make the AI one shot promise kind of work. Right.

Knowing.

Knowing how to. Knowing how to divide up your component definition and your module definition so that that same module can be shared across platforms. Whereas. Well, I know that this is different on native than it is on web and therefore I need a different definition, I need a different approach to how I build out this feature or function. Just having that knowledge leads me to a more efficient prompt and I can help AI help me, but I. With that knowledge that we all share and that Dan has been so instrumental in helping establish, it's kind of a crucial ingredient to how you've been able, like how you saw that migration from React native CLI to Expo happen so smoothly.

Yeah, well, and to be fair, I had worked on that conversion a few months prior. We've engaged with this client a couple of different times, times with various contracts and we worked on the Expo conversion and kind of set it aside because the team wasn't ready to bring it into production yet. And we had a couple other features and things that we needed to finish and build and they didn't want to disrupt the whole app deployment and workflow just yet. And some of the changes that I had to make for Expo meant fundamentally rewriting a few of the ways that they worked with file uploads and storage and things. And they wanted explicitly to be able to allow files to be accessible in your My files area depending on iOS or Android, not just within the app, but ideally it could be both. It was only external before Expo and then I had it only internal in my first go around and then we kind of scrapped that because it honestly ended up getting about three or four months out of date from other. From, you know, the main branch. And so then when we started the conversion again, we just started the Expo conversion from the latest branch of the code and it, it went smoother because Expo had more tooling to be able to help with agentic, you know, workflows. I think they even have an MCP now too. And additionally we kind of knew some more of the edges of the system and we were able to make the thing like that file access happen both inside and outside of the app, which is the ideal state. So it ended up. Yeah, I mean there was some experimentation, there was some learning, some failures, some restarts and then once you know all that, yeah, the prompt can be so much more focused. And again, the tooling and ecosystem helped quite a bit there too. But it did go quite well in the second go around for the, for the upgrade.

Yeah, four months is, is a lifetime in these tools and so in the end it kind of becomes a blessing being able to set it down for that period of time when you come back to it, because it's like moving from a computer to a supercomputer. And I've seen that in a number of things, things that I tried even just, just like toy projects in the early days of AI, which we're talking 18 months ago, right. Early days that were very, they required a lot of like manual intervention and thinking through how and planning that now the tools really can one shot them. And it's fascinating how quickly that's changed. I think touching back on a few of the things that you were talking about earlier, John, it's crazy how just the regular valuable things in our jobs that we would do as software developers, writing clean code, writing good tests, all of that, it all just feeds back into the system. And so the convention over configuration stuff, the tools benefit greatly from all of that, just like developers benefit greatly from all of that. And so what we've found is like the more we follow these good practices, the more the AI can work efficiently when those things don't exist, like when we get these projects that John's talking about where you know, we're being asked to vet them and figure out like how do we, how do we take them to the next level. And none of that extra, extra work, extra infrastructure exists. The AI has gotten really good at building that too. So if you know that what we have, you know, in my hand is the thing that should work. It's pretty, pretty great at extracting out all of that testing. It's really good at looking at that 500 line of code function and figuring out how to break that out and in ways that would just be not feasible for a human in the past because it would, like John said, it would just tie up all of your sprint time for, you know, stuff that we as developers know is very valuable but to the end product is less valuable than other tasks. And so now with that all off the table, it seems like the quality of code and products we can ship is just fundamentally going to continue to increase.

Today's episode is brought to you by Headway, a digital product agency based in Wisconsin. Do you need to design and build a world class user experience for your software, but feeling like you just can't get there on your own? That's where Headway comes in. We're the folks who help ambitious startups and enterprise teams bring their product ideas to life through design, development and product strategy. We don't just give you user interface designs and leave you to figure out the rest. We work with you and your target customers to create beautiful user experience experiences, build scalable design systems and provide Silicon Valley level talent to get it done

right the first time.

Whether you're launching a new software or just looking to add features, we've got your back. We've worked with startups in industries like logistics, healthcare, fintech and edtech, helping them solve their biggest software challenges. And here's the thing, no matter how complex your product is, we can help you gain clarity and get the job done together. We're so confident that we'll be the right fit that we offer a 50% money back guarantee. If you don't think we're working out within the first 45 days, ready to see how we can help you and your team? Just head over to Headway I.O. and book a free consultation today.

What else has or have you guys been using within like your tooling or like your day to day tasks to, to be more effective in your work? Like with, with AI obviously like there's things like cloud code and things like that, but are there, is there anything else about how you're, how you're working every day, whether it's communication stuff with clients or actual, actual code. I'm just curious what you guys been using.

I just wanted to share this. Quick, Jacob, a tweet here from Gary Tan who is one of the president and the CEO of Y Combinator. So you know, if you're not familiar with them, they fund all kinds of startups every year. There's batches, right, that people apply to and I forget even how many they're up to now. 20, something like that. And he said people are sleeping a bit on how much Ruby on Rails and Claude code is a crazy unlock, right? So Rails was designed for people, people love syntactic sugar and LLMs are sugar fiends. And so just a bit of calling out of some of the ways that that convention over configuration can really help empower those large language models to take the next step without needing as much prompting, which is really nice. So you can, you can get a feature built out pretty end to end without having to do a lot of intermediary coaching and then just review the totality of it, which still sometimes needs changes and tweaks. And especially the front end scenario with Rails is still a little bit fraught with some of that JavaScript ecosystem bifurcation where, you know, it's somebody does it all in css, somebody does it in Tailwind. You know, these people use Hotwire and Stimulus and Turbo and these other people use React with a Rails app and other people use different. What's the JavaScript library, Tim, that. That you've found some good luck with lately?

Well, I've been using Inertia to tie the Inertia.

Yep.

Phoenix to. To React. So.

So there's just different. Yeah, there's. That's kind of, I think, still a big piece of the puzzle that we're trying to solve here at Headway with some of our sort of Figma to code initiatives and design system capabilities. And it's a little trickier to solve that kind of thing in the context of some of our enterprise work because we don't always control the full stack that the whole organization uses end to end. But I think for any of the products that we're building or new, zero to one MVPs were definitely two feet in on trying to make that be really effective.

Like other tooling I. I've tried, I just sort of this past weekend for the first time gave pencil.dev a go. Pencil.dev is they're taking inspiration for how Figma's attempted to integrate AI into designing and prototyping. And so pencil.dev connects to cloud code when I've got my typical workflow. If I'm working in a Phoenix app or a Rails app, most recently though, it'd be Phoenix. I've got tidewave connecting to cloud code and now when I'm asking for some designs I'm just, I'm exploring like I'm trying to get ideas that I've got down on paper, so to speak, that I can then work on actually implementing that design. Pencil.dev got me pretty far with very little effort to download it, make sure it's connected to cloud code, which it just did. Like it identified and connected immediately prompt and I had some very reasonable designs that I could work with and then, and then it was a simple workflow. Select this layer, go back to tidewave, prompt, ultimately Prompting cloud code through Tidewave implement the design that's in pencil and it. And it did. Right. It gave me a pretty high fidelity first go at that design. Not perfect. Right. Like this is. This is more there. There's sort of a sliding scale. Right. Like this. What I'm describing right now is I just. I want to prototype. I want to explore this idea. I'm not concerned with the need for a design system and in building out a production app yet. I'm not there yet. Right. So that. That this world is. Is ideal for just like Dan said, kind of quickly throwing things at the wall.

So I can.

That the concept that flow doesn't necessarily work in enterprise the day to day that we. We help clients with. But. But it does. It does help there too. It's just. It's a different workflow and calls for. Really calls for a design system to help maintain brand standards and consistency throughout product development.

Yeah.

What's interesting with Tide Wave, Tim, is I've done some digging on this because I've been trying to figure out. So I've got a Claude Max plan. Right. So it's 200 bucks a month and it has a certain amount of tokens, 20x more than Pro. Right. You're. How could I ever run out of that? Well, you can if you're running a bunch of stuff in loops all the time. Right. So people have probably heard of RALPH loops and whatnot. I don't maybe I'm using them so to speak sometimes but I'm not like setting out to write a RALPH loop specifically. But what's interesting is there's a. The concept of a harness like a model harness. And so CLAUDE Code is a harness that you can use to talk to anthropic models. And Opus 4.5 is the. The current as of February 3rd current kind of state of the art model and. But you can also configure it to talk to sonnet. Right. Sonnet 4.5 I think is the. Is the latest and 5 is coming out soon as we alluded to. But if you're using a max plan they really want you to use Anthropic. That is that plan inside of CLAUDE Code. But there's another company called Open Code that is a different harness that can still use any of these frontier models. You could choose something from OpenAI right. Codex if you wanted to. And they have their own app that just launched today too. Desktop Mac application, which is cool. But there's just this weird world that we're in right now where tools like Tidewave actually somehow are able to authenticate with your Mac's plan, but they aren't directly in the tool call loop with the, that's not the right way to put it. They aren't controlling the agent loop, if that makes sense. They're making API calls. So it's different than open code, where open code, you can give it some browser capabilities and you know, playwright, mcp, different things, and you can kind of approximate what Tidewave does. But it's, they're just different approaches and it's, it's going to be interesting to see where the sweet spot is and from a pricing perspective how these things shake out. Because Tidewave works really well with that theoretical kneecap of not having full access to the loop. Right. So it's making API calls, stateless API calls, but it tracks some of that stuff and, and it knows what framework you're using. It knows how to read that framework's logging, it knows how to look up the docs for that framework. And it's all very specific. So it's token efficient. And like you alluded to Tim, you can choose kind of like in Chrome dev tools, you can choose to click on a particular DOM node and it will then know even more context about what it's trying to fix. But it's still just interesting that like Tidewave specifically, and I validated this with Jose, one of the creators, is, is, is using the SDK, the API calls, not sort of man in the middle masquerading as cloud code like, like open code was doing for a while.

Yeah, that's, that's a really interesting distinction. I mean, I'm curious what pencil.de for that matter, if they're playing in a similar way with Claude as Tide Wave is.

If you don't, they come after you and try to shut you down is what we've learned. So I would imagine that's what, what most people who want to be able to grow and scale do is try to play by the rules. Right now I'm just curious how long those rules are going to stay the way that they are. Yeah,

yeah, something I, I, it came up, I didn't mean to share it right away, it is auto shared, but I shared this with the design team last week and there's this guy that does really great interviews with folks, a lot of AI designers within the past year, folks at like, at Perplexity and Anthropic and stuff like that. But he, he was just talking about like, he has this matrix of like, here's the tools I'M using right now and when I use them because like you guys are saying like right now it's like, oh, this makes sense for this, but not for that. And then like, but then at the same time like if this guy has subscriptions to all these tools all the time, all of a sudden it's like, you know, your, your budget and your margins start to go away. If you're a freelancer or an agency, it's like there's just so much happening. Or if you're an internal team, a product team and you're just like all of a sudden your costs are inflating because you're using all these things that you weren't using before. It just, it is interesting to think through and you start to have to start being more selective on like, oh, we don't really need that as much as we think we do. And obviously these, this matrix will change every three to six months. Hey, this new thing here, the new thing there. So yeah, it's just really interesting. I'm curious if there was anything else that you guys have been using on actual project work or. Yeah, seem, seem promising.

Yeah, I'm kind of in a different world than John as far as like AI usage goes. And I think it's interesting to like the spectrum that we have just internally a headway of how we use these tools. And same with Tim, like Tim mentioned he uses Tidewave quite a bit. I don't. And part of that is a technology stack thing. Right. If I'm not working inside of Rails or Elixir, it's less beneficial to use Tidewave and the type of work that I tend to be doing doesn't sit in either of those realms right now. So what I'm finding though is really good success with like a more hand rolled curated set of tools that is very small. So I think we're all using this tool called beads, which is a like task management tool. But it's very AI focused. And so the whole point of it is you can instruct your, your AI to use it instead of its built in to do system. And then the BEADS tasks can, can depend on each other, they can block each other. And where it really shines is that there's this, there's this concept like when we talk about the context window at a certain point with at least with clog code you, when it fills up, it has to kind of wipe the slate clean. And so it can do that in two ways. It can compact your conversation where it takes all of the tokens that it has and it compresses it down into just what it needs to know to move on to like the next thing or it can clear it or just wipes it totally away. And beats lets you kind of leapfrog around that with this concept of priming. So what it can do is you track all of that work inside of beads. Beads can come back and after the context clear basically re inject what it needs to know to continue to work, but in a, in a more detailed way you can provide enough context into each individual task to work on it in isolation. And so pulling that into my suite of tools and bumping up from Claude Pro, which was like the $20 a month offering to Claude Max, but not, not quite as high as John's using, I'm, I'm on the hundred dollar plan that has been to me like the sweet spot I've not run into. So back on the Pro plan I would run into the token usage limits fairly often where they kind of block you until I forget three hours later. I take, there's like a reset period once I bumped up to that $100 level. I've never run into that again. And adding beads into the mix has, has been enough that I can continue to work and sometimes have two or three things going on across different projects. Like I was working on this design system stuff, but I also had a couple of our other clients, like native apps that I needed to upgrade to meet an Apple deadline. And I had the three, all three of those running, you know, at the same time and switching back between them. As you know, the AI prompted me for input and it's just really efficient. I'm able to like do three times the work I guess essentially. And it's still really high quality work without it losing the thread, without it being frustrating for me. Like I have a pretty high, what I call like a jank factor where I can, you know, I would happily work off of a Chromebook just for the fun of it with all the limitations that are inside of that. But I don't hit that. I don't hit any of this. This jank things just continue to work and I don't have to babysit the model and try to like remind it of a decision it made five minutes ago. And so I don't know, I keep thinking, I've been thinking about this a little bit and I, I think when. So I'm a big fan of like science fiction books and you know, Neal Stephenson and William Gibson and stuff. And, and I've been thinking about how like, this concept of, of cyber decks in those books is kind of where we're at with these AI models. Everybody's got their own, their own deck. They're building up of, of tools, and it can look vastly different from the other person's. And you can have it. You know, the spend on it could be crazy, but you can also be really productive with a, a fairly cheap set of tools too, which I, I think is pretty cool. You still can't really do that with the free tier. I think, like, it's enough to wet your whistle and then realize, like, okay, I do actually have the pony up. But even at a, at a, you know, a $20 spend, I think you're, you're getting a ton of bang for your buck.

Yeah. How, how have we been kind of changing, like, timelines of things, knowing that, hey, we can get this thing off the ground pretty quickly, and then after, you know, the last few weeks is refinement and, and review and all that kind of stuff. How have we been thinking about that from a development side?

It's Hot Topic, Jacob. Hot Topic. It's, it's interesting, right, because code is cheap is kind of the saying that you're seeing around nowadays. And, you know, when anything can be built, don't build everything. Because now taste matters way more than execution, which isn't 100% true. You still, I mean, otherwise we're just back into the slop world, right? An insecure world. I mean, if you were on vacation last week, you don't know that the AI agents now have a church that they can attend without humans. Like, stuff just has gotten kind of crazy with Claude Bot, which got renamed to Molt Bot, which got renamed to Open Claude openclaw. And yeah. So in any case, you can do a lot quickly, but knowing what's the right way to do it and what to and what not to do is super important. And so specifically to your question, what, what I've sort of faced in my technical, you know, leadership role at Headway, I have to do a lot of technical sales with our CEO, Andrew. And so we'll hop on with prospects and we'll talk about what pain points they have, what needs they have. Again, a lot of times nowadays they're coming with some AI generated something, whether it's just documentation or ideas or, you know, different tools that they are prescribing without even really knowing what they do but saying, you know, we should use this because XYZ said we should. Sometimes it's right and sometimes it's something I hadn't heard of. And that's really cool. And when a, when a customer can help teach you something new and then you can learn about it and then in turn take your, you know, I'm an expert in five minutes, in 10 years, right? Like in 20, 20 years, whatever it is, right?

Yeah.

TLDR all this stuff, it's, it's the, the collective experience with a new set of data that really helps you be the expert. And that's what we get to do. But back to the estimation piece. I had to take things that clients were telling us, research them, distill them, understand with their budget constraints and timelines and our team availability and expertise, what was going to be the right thing for any one of our customers. And it took me not forever, but long enough that it was a pain point, right, where it's like, all right, I got to do this estimate shoot. I'm going to need like a four hour focus session to dive into this, to take notes on that, to aggregate this information and then, you know, research xyz sometimes I'll do a little bit of prototyping and just understanding what's possible. And I wanted to distill that down into, at least for me, a workflow that would help me be more effective and efficient without losing much, if any of that human in the loop expertise around what we're trying to help solve. And so I started building a set of tools in a Rails app that would allow me to ingest that data and sort of have a conversation around that data and build up the context in a way where I could then start to use large language models to extract features based on the context of the conversation and the nuance that I was able to help impart to the, the conversation that wasn't necessarily there in the documents.

Right.

So I take what they give us, I lend my expertise and experience, talk about similar things we've built in the past and timelines and technologies, and then I'm able to actually create a pretty nice set of estimates around how many people for how long with what features, and then turn that into a timeline and an estimate ultimately. And so it isn't like a fire and forget, and it isn't yet something that we're providing access to clients for them to do this. I think the first step would probably be continuing to use it internally just with me and Andrew. And then perhaps some folks that are on our sales team could start to get ideas around some of the conversations they're having before I get involved. You know, timeline, scope, and those kinds of things. And then we'll always have that human aspect of presenting what we're going to help you do and bring to market or change inside of your organization if we're coming into a bigger team. But it really does standardize the way in which we can break down the needs of a project and understand how we approach it with our tech, with our tooling, in a way that's going to be hopefully theoretically much more concise and accurate over time. And we get to reinforce that learning that we're doing on every project rather than sort of just retrospectively looking back and going, yeah, we did go a little bit over on that one. We should probably add an extra week or we should do something different by another, you know, development lead. Should, should come over top of this project for a day, a week, you know, in the second half or something. It'll be much more quantifiable going forward to be able to, to have that in place. So, and then like any good product born out of need, we are potentially looking to make that a SaaS product at some point. So we're excited to, to continue to, to build on that and iterate on it a little bit internally and then open it up to, to other folks at some point.

Again, looking forward to doing more, more conversations like this around things that the development team's working on, how we're working on things, what we're seeing in the industry, what we're seeing on projects. And honestly, like how we feel about a lot of this stuff that's going on, because I think that's half the battle sometimes is like, is this really where we should go? You know, as a business, is this where our clients should be looking to, to advance? Like, with technology, things like that, there's always a lot, a lot more variables involved when it comes to all this agentic stuff and data and security and, and predictability and all that kind of stuff. So.

And remember, human interaction is better than a bot.

That's right. Human interaction is better than a bot.

My bot responds faster than you though, Tim.

Oh man, is that what life's about though? Response rates, patience to use, the God,

response speed, and also telling me how correct I am.

Well, that wraps it up for today. If this content is helpful for you, it mean a lot to me if you could rate and review the podcast on itunes or wherever you get your podcasts from so that we can help share this info with even more people. Until next time, I'm your host, John Kinney, and this is the even keeled podcast.