Understanding what's possible, doable & scalable

Foreign. If you work in artificial intelligence, aspire to, or are curious how AI related tech is changing the world, this is the show for you. Thank you to our partners at fly IO. Fly transforms containers into micro VMs that run on their hardware in 30 plus regions on six continents. So you can launch your app near your users. Learn more at fly IO. Okay, friends, I'm here with Andy Sexton over at fly Annie. You know we use fly here at changeload. We love fly. It is such an awesome platform and we love building on it. But for those who don't know much about Fly, what's special about building on Fly?

Fly gives you a lot of flexibility, like a lot of flexibility on multiple fronts. And on top of that you get. So I've talked a lot about the networking and that's obviously one thing, but there's various data stores that we partner with that are really easy to use. Actually, one of my favorite partners is Tigris. I can't say enough good things about them when it comes to object storage. I never in my life thought I would have so many opinions about object storage, but I do now. Tigris is a partner of fly and it's S3 compatible object storage. That basically seems like it's a CDN, but it's not. It's basically object stor that's globally distributed without needing to actually set up a CDN at all. It's like automatically distributed around the world. And it's also incredibly easy to use and set up. Like creating a bucket is literally one command. So it's partners like that that I think are this sort of extra icing on top of Fly that really makes it sort of the platform that has everything that you need.

So we use Tigris here at Changelog. Are they built on top of Fly? Is this one of those examples of being able to build on Fly?

Yeah. So tigris is built on top of FL infrastructure and that's what allows it to be globally distributed. I do have a video on this, but basically the way it works is whenever, like, let's say a user uploads an asset to a particular bucket, well, that gets uploaded directly to the region closest to the user. Whereas with a cdn, there's sort of like a centralized place where assets need to get copied to and then eventually they get sort of trickled out to all of the different global locations. Whereas with tigris, the moment you upload something, it's available in that region instantly and then it's eventually cached in all the other regions as well as it's recorded requested. In fact, with tigris, you don't even have to select which regions things are stored in. You just get these regions for free. And then on top of that, it is so much easier to work with. I feel like the way they manage permissions, the way they handle bucket creation, making things public or private is just so much simpler than other solutions. And the good news is that you don't actually need to change your code if you're already using S3, it's S3 compatible. So like whatever SDK you're using is probably just fine. And all you got to do is update the credentials. So it' super easy.

Very cool. Thanks, Annie. So Fly has everything you need. Over 3 million applications, including ours here at Changelog. Multiple applications have launched on Fly, boosted by global anycast, load balancing, zero configuration, private networking, hardware isolation, instant wireguard, VPN connections, push button deployments that scale to thousands of instances. It's all there for you right now. Deploy your app in five minutes. Go to Fly I.O. again. Fly I.O.

Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I am founder and CEO at Prediction Guard, and I'm really privileged today to have in the Prediction Guard offices one of my good friends, Mike Lewis, who I've known for a little while now in the AI space, and I think have been really intrigued to see how Mike, as an architect, has been able to solution certain AI tools with his customers, with his partners. He's currently the chief AI architect at Synthe down in Cincinnati. And yeah, it's great to have you here, Mike.

Thank you, Daniel. And I've been looking forward to this for a long time. Obviously a huge fan of the show and have learned so much by listening to you and Chris, but I just am so excited to dig in here.

Well, one of the things that I think is really intriguing is that I've heard a lot recently that there's some disappointment a lot of times and people trying to find value with AI, especially in enterprise corporate environments. I've heard things like the tools aren't there yet. We have yet to kind of fully realize the value of AI, and there's some disillusionment maybe, but I've known you for a while now, a couple of years at least. And even before we had GPT4, GPT3, you were creating and architecting solutions for people that actually led to value and eventually led to your business getting acquired by a company that's engaged with Fortune 50 companies. And you continue to Prove that you can find solutions with this latest wave of AI technology. And so there's a disconnect for me here from your perspective as that sort of architect. People are talking about this disillusionment, but you've also been finding these solutions with your partners. And so, yeah, what's your perspective on that? Are we not there yet? What is the state of these tools, that sort of thing?

Frankly, I've done a lot of head scratching on this topic. There are astronomers and there are astronauts and you know, the astronomers get out their telescopes and they look at the stars and they tell us about all these things and the astronauts get up there and work on the satellites. Right. And I think you and I both have been up, you know, been up in orbit for a while now. And you know, I tend to not pay too much att to all the people who tell me the things these tools can't do. Most of the time I'll go to a conference or, you know, I wind up in an environment where a lot of, you know, there's a lot of talk about the tools and I'm just astonished at the use cases where I will hear that there's no value currently. And we're waiting on that. And then at the same time it feels like the same person will tell me all the things it can do. And I'm scratching my head on that. Well, how'd you get it to do that? You know, and so the reality is it's just, I mean, there's a whole universe of problems and solutions and we were talking about it earlier that, you know, to be involved with even just generative AI, you know, not even counting all the stuff going back a decade or decade and a half or since the 50s, I suppose, but even just the recent developments, golly, I mean, I don't think any human could really understand it. All right. And so we all tend to carve out a niche. I know you have a phenomenal niche carved out and mine tends to be, you know, get to know the client, find the low hanging fruit, prove out value. Right. And then start making decisions about, you know, is now the time to, you know, put something in production. That said, I understand, you know, I mean, if you hadn't been tinkering and you haven't built 150 tools already, then it could feel like they don't do a lot. Because frankly, I worked really hard to figure out how to solve all those problems. It's not like I'm running around telling everyone how I solve them all. I hope to do that today? A little bit of that, yeah. It can't do it until it can. And sometimes it's not that the tool can't, it's just that the user hasn't been creative enough yet.

Yeah. And maybe this might be good context for certain people because even before the show, I think it was interesting to talk to you about perceptions in this space. There's sort of a perception maybe by larger organizations or teams that don't have kind of data science, AI expertise, that we sort of don't have the know how or the expertise or the skills to be able to adopt this technology in a meaningful way. And I think it's interesting that your background, although you have some technical background, it's not like decades and decades as an AI researcher at Meta or something like that. Can you describe a little bit of your background and kind of how that's influenced maybe how you've come into this space and how you've come into this technology? I think that might be relevant for people.

Yeah. As quick as I can, I'll say. Initially I aspired to be an industrial designer. And so even in high school I worked. When I was 17 years old, I worked as an industrial designer. And that might not seem possible, but the reality is I got an internship at a medical device company and I went from like printer paper, coffee boy to designer within like six months because I knew AutoCAD and the company was hurting for like warm bodies to like draw up, you know, sort of. And so I had this amazing mentor who taught me problem solving from just. He just had a great way of explaining how to tackle problems. And so that was the framework that I kind of initially formed professional aspirations. I went off to college to become an industrial designer and I got an internship at NASA Aerospace was where I wanted to go. And so that was all highly technical. And then my career took this massive turn. We won't go too deep into it, but essentially I was presented with the opportunity to be a full time professional portrait artist and just got really lucky in that my art was taking off. I spent the next 20 years essentially just traveling all around the world painting portraits and, you know, all commissioned works. I did about 2,000 of those, stepped away from that in 2016. We just completed a successorship and I really didn't know what I was going to do next. And I decided to just work with a couple of nonprofits just to keep myself busy. One was homeless shelter, the biggest one in Cincinnati. And then another one was a publishing company that operated a homeless shelter. So I was just kind of. I just kind of wanted to do that. And the publishing company ended up being bigger than I knew. They sort of had a global presence, aside from several book lines. They had a magazine that had similar subscriber base to say, like Newsweek or Sports Illustrated. So that's how many tables it was on. Sort of a niche publication. I wound up being asked to be the director of innovation for them, which suddenly I'm the director of innovation for a publishing company and you can kind of start to see some overlap. Took that job in 2000, probably 16 or 17. Yeah, and. And it really hit around 19, I think, with the AI stuff when, you know, we started reading about, you know, oh, these tools are, you know, coming for us. It might have been 20 around Covid, I think is when some of this stuff started to. The language models were surfacing. Not only was that going on, so I had that incentive. That alone would have been enough to really familiarize myself with the tool. At the same time, I'm hearing that there's a tool out there that you just type in words and it can spit out images. That's interesting to me because I sell images. In fact, we charge design images. We charge a lot. In fact, I'm uncomfortable with how much we have to charge. I feel like it's a deal breaker in a lot of scenarios because I know takes us a month to land on. So I just wondered, is this something that we could leverage as an efficiency in our fine arts business? So OpenAI at the time was you could apply to become a beta tester just to get early access to Dall E. And don't hold me to any of this. The way I remember it, there were three criteria. You needed to own a commercial arts business, you needed to have a large online following, which we did. And you need to have plans to commercialize. And like, I could demonstrate all that. And I think I was approved within a week or so to get early access to Dall E, which wasn't super impressive, actually. I kind of remember being a little disappointed. Like, you know, that got me in sort of their, like, ecosystem really early. And so whenever, you know, the API hit, I was in there and I had already. I already kind of knew how to code a little bit. So the Python, you know, Python pretty much looks. Looks like plain English. So here we are.

Awesome. Yeah. So I guess as you were kind of first starting to develop those solutions, first kind of your interest from the fine arts side. But then as you started to get into architecting these solutions with Smaller businesses and then medium businesses and larger businesses. What have you found? Because I often. Well, we sell infrastructure for this sort of thing at Prediction Guard and so oftentimes we hear people that they have found maybe the right infrastructure, but they might not know where really the value at the application layer is unlocked in these models, in the functionality that they have. So I know you spent a lot of time kind of categorizing certain things that you've seen as trends and commonalities among the solutions that you've developed over time. And I really like how you frame these in. You know, earlier today we were at lunch and talking about some of these. Less from the functional level, maybe a knowledge retrieval or information extraction, and more from the kind of task or goal level. So like generating new ideas with AI is one of the ones that you mentioned to me. Preserving important know how I think these sorts of ideas that can connect with a less technical audience. So how did you come on these trends and anyone that maybe we'll go through a couple, but any one of these that you would want to highlight?

Yeah, so I think you're talking about our solution archetypes. And I initially I was asked by my team to identify all of the different archetypes of solutions that I've tackled and sort of solved for. So not none of the ones that we couldn't figure out, only the ones that we were able to, you know, get the client sit forwarded in their seat and say, holy smokes, it can do this. That Right. And I identified 35 and I brought it back to the team. They said 35 is too many of anything. Get, get that down. Right. And so we got it down to 15. And so inside of each of the 15, what you'll see is like three or four or five examples of like either full blown applications or just a simple script that runs in the background. Here's what I'll say for the people who look at these tools who've spent any time around them and will tell you, nah, they're not there yet, they're still baking, you know, whatever. They're probably people who have really only interacted with them in a chat bot kind of environment, like Chat GPT. I realize chatgpt is more than a bot, right? But I can definitely appreciate how you could come to that conclusion. If, you know, you're a business leader and you derp around on chat GPT for an afternoon and you kind of go like, how would I use this? Right. But it isn't until you start to bake Personas or agents or if anyone's listening, don't fully understand what that means. Just to say an AI that has an awareness of a discrete task and purpose. So it's kind of like I do this specific thing when you start to learn how to program very good Personas and park them in the middle of code where you can kind of automate them and have them, oh, this file hits this folder, trip off this chain reaction, and then this is the moment that used to be unsolvable or would have cost $10 million to figure out because there's so many if there either. And you know, like, forget all that, like, let's just aim this Persona at it. Poof. Now we know which direction to go. And that's really like the simplest version of, you know, what to say. I will say, you know, for those of you who have perked up and thought, yeah, but how can we trust it? It's like, hold on, like, hit the brakes. How do you trust your co workers? Right? Like, we work with people all day and the same concerns that we have about these AI agents, I think most of them you could map onto your co workers. And the reality is like, how can we trust it? The answer is because we worked really hard to make it very good. Or maybe you can't trust it 100% of the time, but maybe there are jobs that 90% of the time is good enough, 99% of the time is good enough, 30% of the time is good enough. There's all sorts of work where it's just kind of like something is better than nothing or no one was gonna do this anyway, or this just doesn't even make sense for a human to do because you got to do it 10,000 times a minute and we're not gonna pay that many people. You know.

Foreign, What's up, friends? I'm here with a friend of mine, a good friend of mine, Michael Greenwich, CEO and founder of Work os. Work OS is the all in one enterprise SSO and a whole lot more solution for everyone from a brand new startup to a enterprise and all the AI apps in between. So Michael, when is too early or too late to begin to think about being enterprise ready?

It's not just a single point in time where people make this transition. It occurs at many steps of the business enterprise single sign on like Saml auth. You usually don't need that until you have users. You're not going to need that when you're getting started and we call it an enterprise feature. But I think what you'll find is there's companies when you sell to like a 50 person company, they might want this. They actually, especially if they care about security, they might want that capability in it. So it's more of like SMB features even if they're tech forward at work os, we provide a ton of other stuff that we give away for free for people earlier in their lifecycle. We just don't charge you for it. So that Auth Kit stuff I mentioned, that identity service, we give that away for free up to a million users, 1 million users. And this competes with Auth0 and other platforms that have much, much lower free plans. I'm talking like 10,000, 50,000, like we give you a million free because we really want to give developers the best tools and capabilities to build their products faster, you know, and to go to market much, much faster. And where we charge people money for the service is on these enterprise things. If you end up being successful and grow and scale up market, that's where we monetize. And that's also when you're making money as a business. So we really like to align, you know, our incentives across that. So we have people using Auth Kit that are brand new apps just getting started, companies in Y combinator side projects, hackathon things, you know, things that are not necessarily commercial focus but could be someday. They're kind of future proofing their tech stack by using work os. On the other side we have companies much, much later that are really big who typically don't like us talking about them, their logos, you know, because they're big, big customers. But they say, hey, we tried to build this stuff or we have some existing technology but we're sort of unhappy with it. The developer that built it maybe has left. I was talking last week with a company that does over a billion in revenue each year and their skim connection, the user provisioning was written last summer by an intern who's no longer obviously at the company. And the thing doesn't really work and so they're looking for a solution for that. So there's a really wide spectrum. We'll, we'll serve companies that are in a, you know, their office is in a coffee shop or their living room all the way through. They have a, you know, their own building in downtown San Francisco or New York or something. And it's the same platform, same technology, same tools on both sides. The volume is obviously different and sometimes the way we support them from a kind of customer support perspective is a little bit different. Their needs are different. But same technology, same platform, just like AWS, right? You can use AWS and pay them $10 a month. You can also pay them $10 million a month, same product or more, for sure.

Or more.

Well, no matter where you're at on your Enterprise Ready journey, Work OS has a solution for you. They're trusted by Perplexity Copy AI Loom Vercel. Indeed. And so many more. You can learn more and check them out at Work O s dot com. That's W o r K again, Work os dot com.

Yeah, I know one of the things, and it's super interesting to me, but I think also it's unlocked a little bit in terms of what you've talked about in our meetings is that there's a certain element, element of operational efficiencies that can be gained with these tools. If you look at a human process and think, you know, how could we use a Genai tool? How could we use a language model to improve the efficiency of that process? So you have like five things in a sequence. How can it help me do two or three of those things faster or something like that? That's certainly true. But I love how you've talked a little bit about finding something upstream or going upstream of that to really about. Maybe this is part of the thing that people are missing when they're trying to find the real value that that AI unlocks in their organization is that they have a set process in their mind and they could see, well, yeah, we could do this, you know, 10% faster, but is that worth the money that they're investing to an LLM system at a certain scale? So, yeah, could you help maybe talk us through that upstream value and how you think about honing in on that?

The reality is that when you are solutioning, it is often a barrier for the client to visualize a solution because we tend to map sort of a biomorphic representation of the process onto the problem and the solution. And we're trying to imagine a robot doing a human's job, and just instantly you give it arms and legs and a head, right? And so it's like, for instance, you know, if I were to say, Daniel, how long do you think it'll be before robots replace cashiers at the grocery store? So we have Kroger in Cincinnati. So at Kroger, how long? And instantly where's your mind go? It's kind of like, well, I'm just trying to imagine, you know, okay, the Tesla Optimus bot or one of these, you know, you know, trying to fumble around with your delicate groceries and not crushing it and scanning it and just seems so hard to think through and pull off and, and you would just map that so far out into the future. And what you don't know is that my little brother who works with the robots for Kroger look nothing like what is popped into your head. They're like these little square garbage can looking things that, that run around on tracks and it's, it's a, it's like they really operate on the Z axis. So it's like they're stacked on top of each other in these grid systems and they're dropping products from one to another and, and in the end in the bottom, you know, a bag of someone's order comes out and it loads right onto human, doesn't touch it. And this is an example of how it's just really hard when you're staring at a problem, trying to think of a solution. It's really hard to think way upstream. But the reality is the solutions tend to live up there. We start so late, we often start at letter Z when we should be starting with what's an Alphabet?

And yeah, why are we using an Alphabet?

Yeah, why are we using an Alphabet? And I know that sounds like, oh, that sounds expensive. What are we going to be talking for days? No, all I'm saying is the example we used at lunch was like, oh, I think that for a developer these copilot tools are handy because it can kind of double check their code. Well, what I'm talking about is what if an organization had been mindful about tagging and consolidating all of the relevant context for the problem that the developer's been asked to solve. And it's, it's programmatically, it's just automatically, you know, injected into the context of that copilot so that it's aware of the voice of the client, it's aware of the complaints online. It's, I mean, I know this is far down the road, but this is probably going to be built for us pretty soon. I think it's a new paradigm of work, a new way of working. The way we work now is legacy. The new paradigm will be that way, upstream of our work, way upstream of our work. Special care has been put into collecting and curating and injecting the appropriate information into something like a language model or some more sophisticated, maybe multimodal version of that so that it can augment our workflow. And I only say that because it's how I work now. So I don't Let anything go to waste. I mean, use it all. Every scrap of communication I like to transcribe if I can and make sure it exists in some sort of secure context so that it's just double checking my work and helping me ideate. And I find that like it's muscle memory for me now, but the time requirement to set that up pays off exponentially down the road when the client and I need to reopen a case. And I've got this context that is fully locked in, completely aligned to the who, the what, the when, the where, the why of what we're being asked to do. And I just, I'm really excited for a future where that's just handled for people, you know, who aren't weird like me.

And yeah, I guess that gets a little bit. I know one of your passions is thinking about how your work generally, but also the technology and other things impacts people at a, at a human level. Right. And in particular even vulnerable people or marginalized groups. And it's interesting for me to hear you talk about that where a lot of times what people hear is, hey, how can we make this role more efficient? So that maybe we have, you know, this focus workforce wise on efficiency and you know, how do we make this role more efficient? We're going to need less of these people, which is certainly a relevant thing that we need to talk about. But here what you're talking about is more of a new paradigm where those people that are working are enabled to do actually net new things. And I also imagine maybe you could speak to this if you have an idea about it. But opportunities and things are unlocked for maybe people that are in vulnerable or marginalized situations that maybe could, you know, would be opportunities or things that they would have access to that they didn't before. And so this is maybe a different paradigm of looking at the workforce impact of AI, I guess.

I love that you've asked me about this. I've had a few different big light bulb moments when it comes to just thinking through imagining future products that might exist as a result of this Brand new technology is brand new thing to people, right? Like, and I imagined once that there might be a vulnerable single mother sitting at home who maybe doesn't realize she's being exploited. I mean, I'm imagining a fictitious character, I'm not, I don't know this person. But you know, maybe there's a vulnerable single mother sitting who's being exploited and doesn't even know. And can you imagine the cost of running. These are trending towards zero like we see that. Right? So it's not hard to imagine a wearable that like ingests contexts and sort of helps someone navigate life, maybe maybe might make them aware of their rights, maybe might be willing to able to act agentically on their behalf by maybe filing a complaint somewhere or putting a situation where some marginalized person is underrepresented. In a way, it excites me when I realize that a primary difference between the forgotten, the neglected, the marginalized primary difference between them and others is that the others generally just have someone willing and able to act on their behalf because it makes financial sense when the cost of cognitive work and some of these automated or these agentic type actions are free or could even be ad supported because they're like one tenth of a penny for that type of help. That's exciting to me and I do look forward to a future where people have access to advocacy, even though there's no real advocate.

Yeah, that's so encouraging. Certainly, like I say, there's an element of workforce impact of a. That needs to be considered on the negative side, I guess, if you want to consider it that way. But also there's a positive side and opportunities that are unlocked, for sure.

I have to say, that tool will never exist if no one builds it. Daniel. And so this is just a challenge. I'm doing this personally and we can unpack how. But if you've been given gifts that overlap with an expertise here, it's incumbent on each of us to think through, like, how can I make the world just a smidge better? Right. Because I didn't just ignore all the need that overlaps with my expertise. And it's not just about writing a check at Christmas time, you know. Yeah. Hunt that out.

It's about, yeah, creating that outpost of goodness and beauty in what we create. And not just like in your case, in your background with art and actual sort of fine art, but also in the technologies and the products and the tools that we're building. I love that. I'd love to maybe talk, maybe just highlight a couple of these solution archetypes, because I think this is also kind of helpful maybe for those in our audience, maybe they're on the business side and they would connect with them, but also maybe they're on the technical side and they're just struggling to connect certain ways in which they can communicate the potential of this technology to stakeholders. So I love the one that you have here of generating new ideas with AI. How do you think about that as a solution archetype?

Well, the obvious way to do this that anyone would just think of on their own is you could go to something like ChatGPT and say go a step beyond what a normal sort of ho hum user would do, which is to use it like Google, but to ask it for help with inspiration. And so anyone could just lean forward in their Chair and open ChatGPT and do that right now. But the use case that we highlight in our solution archetypes documentation, there are two or three under that specific archetype. But the one that I'm most proud of and like we'll talk about today is the first app I ever built. And this app was a Persona named Andy. And so Andy is a roundtable discussion like focus group style facilitator. So there's essentially, it's really just a system command plus some tuned hyper parameters that help Andy do his job well. But Andy's supported in Python with the ability to synthesize or spawn new Personas. And so what Andy will do is listen to the transcript of a meeting. What I would love if there was like a low enough latency audio, you know, like the files that OpenAI is kind of working on right now, I'd love to be able to get Andy like and the whole group participating in real time. But for right now, the way it works is you can kind of give them a chunk of transcript. And what Andy will do is kind of look at the problem you're trying to solve and spin up a focus group. So Andy creates, I think it was six or eight, I think maybe eight additional Personas. So now all that means is in the Python code there's like eight new things that Andy just created, system messages and some hyper parameter tuning. So that now in the code the tool has the functionality to summon any of these additional Personas to weigh in on the conversation. And so Andy as a facilitator can pick which one has a turn to talk. And so then Andy opens the group and you can use the world knowledge for this bit. And I did. And he just understood how to run a focus group, which actually is cool because I didn't know how to do that when I had the idea to do it. And so Andy will say, you know, like, okay, today we're going to talk about new product ideas for X industry. We happen to do this for the publishing company where I worked, so I was still working for them when I made this tool. And Andy could, I think it was like a fireman, a nurse, a web developer, a doctor, you know, around the table and the problem this company had was their new product development team was struggling to come up with new products. And pretty much at the end of each meeting it would just be the same product. And this was a problem. And so Andy understood my job is to think of new ideas but then kick it to the group for discussion. And it was so fascinating to watch them talk about it. In this instance, the very first time we ever used it, someone in the group came up with an idea for something that was very close to the products that they had been making, but a little different. And then the nurse piped up and said, that's great, but that form factor would not work at all for my job. And if I wanted to bring that to work and benefit from it, it would need to be smaller so they could fit in my pocket or purse. And so then someone else built on it and said, you're describing a deck of cards and this could work because. And then the web developer kicked in behind and said, that's a great idea. If there was a QR code on the cards, you could then link it to an account that they have here. Let me write that right now. Of course the code didn't like fully work, but it was cool. You could see it try to write the code and this, we were just letting it run. Right. They loved the idea. It was the freshest, best idea they'd had in the seven year existence of the new product development team. It was a brand new product idea and it was the first app I'd written and it was the first time we ran it. And that for me was a major moment. And that was on GPT. Yeah.

And there's an element to this solutioning that you've been doing that makes me think about how quickly you're able to get through to a proof of concept or demonstrating the value that you had in mind with a certain AI use case. And I'm wondering as you're doing this solution, I'm guessing that there's a balance or maybe potential objections here. You know, in this scenario I could see how a company might object to putting in so much product information or IP information into one of these systems for privacy reasons. Or maybe they're thinking, oh, what you're describing is going to take so many, so many LLM calls, it's going to be slow or maybe costly, which maybe are relevant concerns. But also you kind of need to know what is possible before you get to some of that optimization and also try to get to an end to end solution. So how do you think about as you're solutioning, balancing this desire to move quickly and get to a solution and end to end thing versus privacy security types of concerns and balancing those things that maybe are optimizations or scale issues or environment issues with the ability to get to an end to end solution.

Yeah. Sitting across the table from maybe the best person in the world at this, the best I've ever met when it comes to thoughtfully considering security and the risks associated with it. I will say this is not something to ignore ever. And we've learned to start with security when it comes to forming relationships and assurances with a new client. And we need to be good at that. But you gotta start somewhere. And sometimes what we do is we start with synthetic data. So it's kind of like hey pharma company, we're going to invent a synthetic drug and we're going to make all the synthetic assets that a company would have and we're going to, you know, and what's neat about that is these tools do that for you. So I mean if you have a Persona, remember I said like the new paradigm is having a Persona fully aligned. Well that that Persona that I've taken the time to get up to speed on the project can just spit out all this stuff. I need to prove to them that what they want to do is possible. Right. There are so many trust me's floating around in this space. Right. Like it can do it. We'll get you there in 36 months. You only need to pay us, you know, X million dollars to build out your. And it's BS maybe that works for big co who has a metabolism that is the same as a, you know, a 20 ton whale. But like, you know, for Midco, mid small that has a normal human metabolism and would like to see something before Christmas. Right. Like you know, there are ways to get to POC proof of concept in sometimes hours. I have had persuasive proof of concept scripts written before the end of meetings where the client is talking about a solution they want. And, and how do you do that? Because I probably went into the meeting with an aligned Persona on the goals. I hear a few little things in the meeting. And I have a bot that I've created called Persona Craft that I can just throw in a few things and you know, bang, bang, bang. Oh, okay. Throw that in python. Is this what you're talking about? Like holy smokes. So yeah, What's up friends?

I've got something exciting to share with you today. A sleep technology that's pushing the boundaries of what's possible in our bedrooms. Let me introduce you to Eight Sleep and their cutting edge Pod 4 Ultra. Now, I haven't gotten mine yet, but it's on its way. I'm literally counting the days. So what exactly is the pod for? Ultra? Imagine a high tech mattress cover that you can easily add to any bed. But this isn't just any cover. It is packed with sensors, heating and cooling elements, and it's all controlled by sophisticated AI algorithms. It's like having a sleep lab, a smart thermostat and a personal sleep coach all rolled into a single device. It uses a network of sensors to track a wide array of biometrics while you sleep. Sleep stages heart rate variability, respiratory rate, temperature, and more. It uses precision temperature control to regulate your body's sleep cycles. It can cool you down to a chilly 55 degrees Fahrenheit or warm you up to a good nice solar temperature of 110Fahrenheit. And it does this separately for each side of the bed. This means you and your partner can have your own ideal sleep temperatures. But the really cool part is that the pod uses AI and it uses machine learning to learn your sleep patterns over time. And it uses this data to automatically adjust the temperature in your bed throughout the night according to your body's preferences. Instead of just giving you some stats, it understands them and it does something about it. Your bed literally gets smarter as you sleep over time. And all this functionality is accessible through a comprehensive mobile app. You get sleep analytics trends over time and you even get a daily sleep fitness score. Now, I don't have mine yet. It is on its way thanks to our friends over at eight Sleep and I'm literally counting the days. I get it because I love this stuff. But if you're ready to take your sleep and your recovery to the next level, head over to 8sleep.com PracticalAI and use our code PracticalAI to get 350 bucks off your very own Pod 4 Ultra. Now, you could try it free for 30 days. I don't think you want to send it back, but you can if you want to. They're currently shipping to the US, Canada, United Kingdom, Europe and Australia. Again8sleep.com PracticalAI.

Yeah, certainly there's an element of this where you are acting as an architect and a solution developer getting to that proof of concept showing value. But I know one of the other things that you're particularly passionate about and have been exploring is actually finding kind of individuals in organizations that are champions or could be champions of AI Maybe they're not from a kind of technical programming background, maybe they are, but kind of teaching them how to utilize these tools effectively in a sort of crash course kind of way. So how do you think about that and what are some of the things that you're taking people through to help them understand how to use these tools in an effective way?

I just want to preface this bit by saying I am trying to become a master at what I'm about to describe and I'm not there yet. This has been the hardest thing for me to navigate so far. Here's the reality. Companies need more than code. Like what they really, whether or not they realize it when they tell me we want to become AI enabled, I think some part of it is they want their people to know and understand this stuff. And I think if you just kind of trust it to luck that people who have a really demanding job are going to develop an accurate sense for what's going on under the hood, you're going to be disappointed. And so we have developed. The company that bought my small business happened to be a change management like an instructional design company that they have helped big Co for decades navigate massive change. And so it's just kind of lucky. I mean, that's actually part of why I liked the offer. But I have worked with their experts to develop a pretty well defined approach with mixed results. Where I want to get better is I want to get better at identifying the people who are really going to become the super user rock stars. So we've developed this tool called a fit checker that like essentially as a quiz. It's like some sliders, they slide and it helps us kind of score. I don't even really know if that's going to work great or not. But what we are doing is letting them get a peek under the hood at what's going on with these tools because we want to demystify it. Right. But they think it's magic or BS or both. Can I just run you through the beginning of what it would be like in one of our early sessions with them so you can see. Okay. And feel free to try this in your organization if you're kind of trying to align with someone on like what's actually happening with these tools? And so what we'll do in the very beginning is I'll stand in front of the room or one of our other coaches will stand in front of the room and say, hey everybody, I'm gonna say something and I'm gonna say a few words and what I want you to do. And you can maybe even do this, like, in your car or like, if you're listening to this at home, you just blurt out what pops in your mind, right? And so I'm gonna say some words, and then you're gonna blurt out what pops in your mind. Are you ready?

Here we go.

Peanut butter and jelly. It's jelly every time. It's always jelly. And the whole room says jelly. No one says anything except jelly, right? And then what we do is we explain to them that they essentially provided a completion to our prompt. And jelly happens to be a token. And peanut butter. Peanut, if it's a Capital P, is 3 tokens. Butter is a token and is a Token. That's the GPT4 tokenizer. And so then we can open up a tokenizer and we can show them those individual tokens, and we can click on the token IDs, and we can show them that those are just numbers and that there's this language model that is trained to spot weird patterns to what tends to come next as a number. When you give it a string of numbers, those were converted to bits or entire words, and then it just spit back jelly. And so when you understand that, and then we show them, we tell them, like. And look, if it spits a sentence back, it's just kind of a total token, token, getting longer, getting longer, getting longer, you know, and it's starting from the beginning, and they get a sense for the, like, oh, it's just mimicking language. It's mimicking what a next word that a human might say when they heard a thing. When you understand that it's working that way, it helps you use it better. Okay, so then while I'm talking about all that, I might start talking about blues and rock and roll. I might mention Memphis Cadillacs. Maybe I'll say something about Graceland, and I'm talking about all this. And then I'll say, let's try it again. Peanut butter. And it's like, jelly, Jelly. Someone yells, banana. And behind me on the board, I take a thing off and we've written banana on the board, right? And then we just have this weird moment because some people in the room are young and they'd have never. They don't even hardly know who Elvis is. Right. And I don't even know if you would have gotten that, Daniel.

Yeah, yeah, I definitely would have gotten the Elvis, Graceland, et cetera. But I would have been somewhere there.

Yeah. And so I think it's that Elvis, like peanut butter and banana. Sandwiches or something. And so now we can just unpack the concept of context, because by inserting some new ideas, we sort of shifted the probability of that index of potential tokens that, you know, in the order that they would be. And we've shifted it enough so that banana could potentially show up as worth picking. And someone did. And that just sort of demonstrates. Well, most of you still said jelly, but someone said banana. I'm just telling you, it has never not worked like it always works. And so then we start talking about, okay, so these are tokens. They're parts of words, they're numbers. They go together. It can predict the next one to mimic human communication or thought. And then there's only a certain amount of these that can fit in its brain or its memory, its context window. And we'll show them on a board, we just put little hashes for like, this is token. This is token. This is. Well, what happens when we get to the end? Because, Daniel, when I started working with these tools, we only had 2,000 to work with, with. And now it feels infinite, but not really. But because you learn early, right? Because you learn early, you learn to work within the limitations of the tool, and you learn elegant workflow then, right? Because we. We appreciate even though it feels big, it's still scarce because they got to pay for what we submit. And so we show them. These are how the tokens stack up. And. And then we show them, like, when you start getting toward the end, if we fill up this 2000, what's it going to do? It's chopping off the ones in the beginning. And then we asked him, like, have you ever had an experience with ChatGPT where it feels like. I've already told you that. Oh, yeah, you see some. Oh, oh. Because it didn't. Forget it. It selectively. It deleted it, you know, and. And so we just run them through, you know, that's like the first 10 minutes. And we just build on that and build on that and build on that so that they build good AI usage habits, and they can interact with the tool in a way that. That leverages the strength of the tool and with an awareness of what can go wrong.

Yeah, I love this approach. And in my own workshops, when I start going through some of this, of how a generation actually happens, how text is generated, I often see that a number of really important and interesting questions kind of naturally pop out, out of that. Like, hey, if this is just producing these tokens in this way, how could it produce anything meaningful? And how does it seem so coherent, like there's an intent behind it. Why would it produce anything actually valuable and connected to the real world? So yeah, I love this.

So because we get all those same questions, we have identified what we call layer one, layer two and layer three knowledge. We only teach layer one knowledge in our boot camp. We will address layer two and layer three questions individually, but not as a group because the layer 1 knowledge is, is the strict information set that is required to use the tool well and so we won't. We would never talk about prompt injection, attack and the group setting because what we don't want to do is scare off the non technical person and in fact we target the non technical people. This is their moment. You finally program computers with plain language.

Yeah, really good points. And on the data side, I think one of the interesting things is like you say, not going into some of those details in that context, but definitely getting to a point where people can realize how data starts to become integrated with these AI systems in a meaningful way. An example I often give is I go into a chat interface, I say summarize this email for me. And of course you haven't pasted in an email. The model doesn't know how to summarize an email that it's not given. Right. But everyone knows what to do. They could paste in the email that they want summarized. And you know, ignoring the kind of policy, potentially policy related things around that, it immediately makes sense that people can bring the right data to the table, inject it into these existing models without having to do complicated fine tuning and actually produce value.

Can I tell you our five step process to effective reasoning? So we have a slide on this and we teach it, I think, I hope I can remember all five because I just told you there's five. What we teach when it comes to like reasoning over data or just processing information, it could really be almost any project we teach to do it in this order. So we say what we like to do is teach him to start with end goal alignment. And so because we know that, I think it's known that these models tend to pay a little more attention to what's going on early and late. In your context, we want to get the real end goal articulated very early. So a neat trick I have for end goal. And so this actually can be fun. So this is a way you can work from your hot topic. Maybe you should open the show with that clip. So this is the way you can work from your hot tub. I like to put in my AirPods and hit record on My phone and just talk about what I need to get done on this project. Just talk, talk, talk, talk, talk. Get it out this way. I don't have to sit in front of my computer at my desk. I might go on a walk or on a treadmill and just talk through. This is the who, the what, the when, there, where, the why. Right? And get it all out there, there. Okay. Now what we do is we transcribe that. I use Mac Whisperer, so I'm doing that locally. So it doesn't matter if this is something for a top secret project. I can transcribe that locally and then I can inject that into my, essentially my first prompt. And it might be big. I've had those go on for three hours or as quick as 10 seconds. And what we tend to do is try to rush to work. But there's a bunch of steps before we start working. So step one is establishing the end goal. Step two would be to validate the end goal. So now we can go back to that same model and say, tell me what you think we're gonna do. Like just lay it all out, give me bullet points, you know, and so it will, it will go through and say, I'm expecting you to do this and you to do this. And then I'm gonna do this and we're gonna do this, this, and what the client wants is this. And so we're probably going to do that. And you know, and you can look at it and go, actually, this or that is wrong. Now what I like to do instead of playing AI whack a mole at this point, right? So anyway, AI Nerd knows exactly what I'm talking about is I use these next few bits of conversation to identify where it misunderstands the end goal, if it does at all. And then what I do is I go back to my source material and I modify my source material to clarify. And so what we do is we get rid of all those subsequent prompts for validation. Validation is for me. It's not anything that needs to linger in the context, so I delete all that. So step two was validation of end goal alignment. Step three is we curate and ingest our assets. Our. Like you said the email, remember I said curate and that is big. You don't always have to do it, but it's helpful. Remember I said I learned on 2000 tokens. This is why I make a habit of curating my content before I put it in there. What's the least. Now there's a balancing act here. There's a trade off cost benefit. You don't want to overdo this step, but it is helpful to trim the fat out of transcripts. You know, if you've got a transcript saved as vtts and you look at and you open that up in Notepad, 80% of the tokens are time code. So, you know, maybe we get a docx that just has speaker names and, you know, so we're just mindful of like, what are we throwing in next that isn't going to distract the model that's relevant. Okay, let's pretend like we've done all that work. What do we do now? Do we start working? No, we validate again. So we go back and we say, what do you understand that I just gave you? And how do you understand that it relates to our end goal? And it will talk about it. Usually this part it gets right if we've done a good job of angle alignment and curation. But sometimes you'll spot, oh, it can't. See the infographics. I assumed by attaching these PDFs that, okay, what color is the biggest slice on page 14 of the pie chart? That's what a validation thing might look like over your data, right? Okay, you can see that pie chart. Okay, you see the orange? Okay, you know what that is? How's that relation to our project? Okay, you understand? Okay, you validate all that. Now get rid of that. Because that doesn't need to persist. But I need to be comfortable. This model understands the goal. So it's aligned now. It's finally time to start work. We've done all this work and boy, can you burn through a project when you've done those steps. And it's just, this is the fun part really, where we just start. Okay, you can almost just say, okay, go at this point. And it starts to do the work. And then I think the fifth step, I haven't been numbering them on purpose. The fifth step is just letting it synthesize assets. And this can not only be like, if you're using ChatGPT, you can have it create spreadsheets or whatever, but you can also, if it understands, hey, I want to use UDIO to make a song about that, or I want to use ideogram 2 to like create slides for that, or I want to use. And you can just have it start to create prompts for other things and go off and do it. And it's really fun. Now the. So we've done all that and we've got that context persistent. The client sends us an email and complains about whatever, what do we do? Copy, paste, boom, done. Right. Like, it's just so little work.

Yeah, that's super encouraging. I love this example because, you know, oftentimes you might be, you know, I might be in a meeting and you have to respond very quickly. But. But a lot of times it's those slow times where you're able to think about something. For me, like on a walk around the block or on a hike or something like that, where I'm really present and I'm maybe more calm and thinking through things in a slower fashion. But I also want to capture that and I want to summarize it. I want to bring it into conversations in the midst of my workday. So I love that example. And I think it also illustrates how you've integrated this technology. You're practicing what you preach to some degree, kind of integrating this technology into your daily rhythms. And that brings me to maybe a last question here as we're starting to wrap up is as you look to the future and you're thinking about the new solutions that you're creating for people, the things that are on your mind, the things that you're architecting. What's most exciting for you now? What are you excited to explore? What are you excited to kind of prototype with the partners and the customers that you're working with? What excites you looking forward?

Well, I will stand by what I said initially, that if we never got another innovation, these tools are good enough as they are to justify going down on the timeline of humanity as a massive technological revolution, evolution. And the tools that I wish for that are already sort of like they're almost done baking, right? The tools that I wish for are really low latency in and out audio, video, image, agents. And so, you know, I've built so many tools that interact with people. We call it. One class is called like mind Mappers, where we're essentially trying to convert tacit knowledge to documented process. Right? And so like, you know, you've got the single point of failure person in your organization. How do you map out, like their subject matter expertise? So what would be awesome is if there was a low latency conversant model. So, like what OpenAI has, but it's walled off right now like an API where I could give a person an option between an email chain, text messages, a phone call, and right now I can't do the phone call. And that bothers me. And then I think too, just as people start to see the value of the tools that Maybe they're a little less weirded out and fearful of interacting with them. And so of course it makes sense that people feel nervous that they're going to lose their job. The reality is they are going to lose their job as it exists today. If there's knowledge work, they should, because that's like continuing to want to farm land with an ox and whatever the thing behind a plow, I guess, like, you know, like that's wishing for keeping that, you know, when the tractor dealership just showed up and like, no, we, we're not going to want to go back to like, you know, the days of like trudging through tedious, you know, knowledge work that could be automated. But it is going to be a process that requires empathy on the part of us, the AI engineers, the architects, the developers, the, you know, the leaders. And to appreciate like this is a scary thing. And so another thing I wish for is just to get past this adolescent phase to where the tools feel less scary. And then I, I, I gotta go back to like. I am very excited to see what it means for learners who maybe their teachers don't have time to go on every curiosity journey with them or explain every little lesson. And so that's really big. I could give you such a long list, so I'll just stop there. I am very excited for how this technology develops, but I will say the good guys have to have hand on the wheel here, Daniel. And if you leave it up to Big co, we won't have the stuff that really makes life better for a wide range of people.

Yeah, well, I think that's a great note to end on. Thank you Mike so much for joining and I would definitely encourage you to heed Mike's call. Get your hands dirty, think about those solutions that are gonna create goodness and create new paradigms of the way that we work. So yeah, thank you so much, Mike Mike for joining.

Yeah, thank you.

All right, that is Practical AI for this week. Subscribe now if you haven't already. Head to PracticalAI FM for all the ways and join our Free Slack team where you can hang out with Daniel, Chris and the entire Channel Changelog community. Sign up today at PracticalAI FM Community. Thanks again to our partners at Fly IO, to our beat freaking Residents Breakmaster Cylinder, and to you for listening. We appreciate you spending time with us.

That's all for now.

We'll talk to you again next time.