Adam Brekane, co-founder of GatewayX, hired an AI engineer who specializes in no-code platforms like Make and Zapier instead of traditional software engineers. They fine-tune language models using prompt-response pairs in CSV files to embed company style and knowledge, building models for LinkedIn posts, copy editing, and recruiting. Their AI agents have a 25% success rate, and they're training non-technical team members to fine-tune models themselves.
Chapters
We hired an AI engineer for the first time. We have all these cool ideas. I wish I could fine-tune. I never want to talk to a non-fine-tuned model again. Well, that means, man, someone's got to go and gather all this data. I'm very excited about fine-tuning. I don't see a lot of content about it. It just isn't out there. I think there's something that we're learning here about who are the right people to get excited about AI solutions. How do you get them excited? And it's a little more complex than I thought.
I'm Andrew Warner from Bootstrap Giants. I've got— who with me? Actually, Adam, I would introduce you myself, but I'm afraid of mispronouncing your last name.
It's Adam Brekane. You're gonna break the cane. Brekane.
Okay. So I am Andrew Warner, and I've got Adam Brekane with me from GatewayX. And Adam's venture studio is working on— actually, what's the goal? How would you describe what your vision is for or GatewayX in AI?
I would describe it as rapidly evolving, but the broad vision is we want to be in the top 1% of 1% of aggressively learning how to apply these AI tools in order to make businesses work better. We don't want that for fun. We want it because we think, hey, if we could buy businesses and make them better, start businesses and make them better, And if we're the first ones at doing this, we're gonna have huge competitive advantage for years and years. We'll just be ahead of everybody else.
All right. And we're gonna have these semi-regular conversations where we get an update on Adam's progress. Last time we talked, he told us how he built 4 AI agents in 2 weeks using no coding, development, programming, none of that. It was just really easier than, than it And then he expected— I wouldn't necessarily say it was easy that you built these agents, and I would say that it was more powerfully surprising because they were able to help you find people that could end up being customers.
Yeah.
And we demoed that last time.
Yeah. Yeah. I think one, this time, one, it's not easy. Part of it is actually learning how to accept the failures too. You know, we have, let's call it a 25% hit rate of building an agent and having it work, and the other 75% are in the trash or on the shelf to come back to later. So some of it is like, oh, how do we readjust our predictions of success? When will it work? What will work?
That is one of the frustrating things. When AI doesn't work, there's a temptation that I have to say, ah, this is clearly overhyped. All it is is a more intelligent Google search that lets me find what I need. I've just brought into this hype and walk away. It's a lot harder to say, I'm going to tweak and tweak and tweak. All right, so one of the topics that we're going to talk about today is fine-tuning and how excited you are about that. And you're saying, look, I don't see a lot of content about it. I'm kind of figuring this out on my own. I asked you what fine-tuning was, and I think I didn't fully even understand where you were fine-tuning. You're not— when you're fine-tuning ChatGPT, you're not going into the ChatGPT desktop app like I am and fine-tuning it. Where are you going? And What does that mean?
So you would go to chat— you are going to chat.openai.com. I'm going to platform.openai.com. So ChatGPT is, is like this application that uses these big large language models that, that you're, you know, reading about. There are a lot of other ways to access those same models, to either customize them or talk to them in different ways. And so think of ChatGPT as one application built on the OpenAI platform. You can build your own applications on top of the OpenAI platform. You can customize bits of the OpenAI platform. So when we're doing it, it's like the platform level. It's a little more foundational than like using ChatGPT, which is a great tool, but it's also only good at the things it's good at, and you can't change that.
A consumer product.
Yeah, that's good.
And so when you are fine-tuning, what are you trying to change? And then we'll get into how you're doing it and what you're learning.
Yeah, you're trying to change— in my case, you're trying to change the intuition of the model and the style of the model. So when you talk to ChatGPT, you know, at the upper left, you get to pick a model. If you're not picking intentionally, you should be picking intentionally. If you click the little dropdown, it'll tell you this one's good at code, this one's good at advanced reasoning, this one's good at, you know, whatever. They're all good at different things and it's changing as they release new ones. But you can take a model like 4o, which is good at— it has a very long context so it can look at a lot of input all at one time and understand it well and generate an output pretty quickly. The problem is when OpenAI fine-tuned when they finished that model. They had to make it work well for everyone that used it. So every person on the planet that's talking to 4.0, they're going to get the same model, they're going to get the same style. And so it's average. It has an average point of view. It doesn't want to step on your toes. It's very affirming. It's very positive. It's very helpful. And it's like a little nice and bland, honestly. And so if you ask, you know, go to it and say, oh, like, write me a great LinkedIn post or a blog post, it's going to be a little bland and it's going to be average. And, oh, that's not what I wanted. And you could fight back and forth and give it feedback. Or you could say, no, I don't want to use this tool that by definition is kind of average. I want to go into the platform layer. I want to build a version of GPT-4o that has my intuition of what good content looks like, that has some like specific knowledge about, you know, what's my proprietary knowledge, what's my approach, uh, how do I like to talk. I'm more direct, I'm more concise, like I love metaphors. So fine-tuning is going to go in and actually build the version of 4.0 that is a little bit honed into just me. It's a little more focused and custom to what I want. Now, that also means it's worse at everything else. It is for me.
Why are you excited about that? Actually, why are you excited and then how are you even using it?
I'm excited about it because my frustration with a lot of the models is that the output is bland and that the output— let's say you want to ask it to make a decision. Look at all of this data. you know, look at these 5 LinkedIn profiles and here's a job description. Who should work? You know, who should get the job? You could ask it that, right? And it would give you like a pretty good like, well, this person is good for these reasons. This person is bad for these reasons. I would recommend hiring this person. But it's the same as you going to any recruiter on the street and saying, here's profiles, here's a job description, who would you hire? But without giving that recruiter context on Here's what I like. Here are the intangibles that I like. Here's, here's like why I like this type of previous experience. And I want to see this journey in someone's work experience. Well, my fine-tuned model. So first of all, you could maybe get ChatGPT and ChatGPT-4o. You could maybe get it with a bunch of chats and like really long prompt engineering. You might be able to get it to kind of understand your approach. But every single time you open a new chat, you're going to need to do that exact same thing, right? You're going to need to give it your approach again and either paste in some big block of text or have a long conversation. And honestly, that's exhausting. It's exhausting to copy and paste. It's exhausting to have lots of chats. What if there was a version of GPT-4o that already shared your intuition about what makes a candidate great? So now when you go to chat with it, you say, I've got these profiles. I've got this job description. Who should I hire? And that's the only bit of the prompt that you need to give it because it knows generally what's my process, what do I like, what do I not like, why. And so the first level of output is better because it's tuned to me, because I fine-tuned the model to work in the way that I want it to work.
Okay. And how are you using it? Are you literally using it now to hire candidates?
So that is, that is one. It's, it's on a, there's a ticket for that this week. Uh, I think, you know, you opened this with, you know, we built 4 agents in 2 weeks and these ones are working and that's awesome. You know, the last 2 weeks, like last Monday or 2 Mondays ago, I built my first fine-tuned model and I stepped out of that thinking, oh, this dang thing works really well and it was incredibly easy to do. Now there's probably about 8 fine-tuned models in our account.
Yeah.
And by a month from now, there'll probably be 50 fine-tuned models in our account. Because why would anyone use the standard model if you're building a workflow? Why would you use the standard model if you could have a fine-tuned one that had better intuition? So yes, this recruiting one, that, that is exactly one that we're going to build. The ones we've built so far are— think of them as like, as like writer, writing assistants, copyright, like editors, like stuff like that, that that have a really good sense for our tone, our style, and they can give really direct feedback like someone on the team would give feedback. So if you write a LinkedIn post, you know, it could— because it understands our LinkedIn posts, we've trained a model on our like best performing LinkedIn posts. It, it has intuition about what makes a great LinkedIn post for us specifically. It's not a broad one. It's not here's what's good for LinkedIn. It's like, here's what we like. So it's copy editors, copywriters, content generators, and then to some extent research as well. Like, hey, if you have this really long call transcript, we have a model for this now as well. You have this really long call transcript, I want it to summarize and pull bits of the content out. Instead of just writing a prompt for that, I'm going to write a prompt and have a fine-tuned model that is good at synthesizing the content in the way that works for us.
You know, the problem I have with that is that as a user of this version of ChatGPT, I can't tweak it. So if it's not giving me the answer I want, I've gotta come back to you and say, Adam, this is weird. Can you fine-tune it? Then you've gotta take my feedback and try to fine-tune it and then come back to me and it becomes right back to the old developer and user experience.
So that's a good point. One, like, absolutely by design, like, these things are all going to take away my job, right? So I need to make myself important. But more realistically, it is—
it—
that's temporary. I don't think— like, again, fine-tuning these models is actually pretty easy. I trained a guy on our team how to do it. He is not a technical person at all, and he's now fine-tuned more models than I have. And it's because I'm—
what I mean is, can, can we share the fine-tuning experience?
Yeah, I can. I have like a Notion doc, uh, guide that, that walks you through it. It's, it's very easy to follow and anyone, anyone even like non-technical could do it. You just have to be like slightly nerdy and you could do it.
But you can create a version of it and say, Andrew, don't touch my main one. Yeah. Because then we're gonna end up like stepping on each other's work. Tweak it all you want.
Yeah.
And see if it gets you the answer you're looking for.
Yeah, I think it's probably a little less— let me demystify it for you because it was to me a little bit mysterious, like what the heck was fine-tuning? This is my problem. I wish there was more content about it because I could have done this months ago and wish I had. Fine-tuning is really basic. The workflow is annoying because there aren't great tool sets for it, but it's non-technical. But at its core, all you're doing is saying, hey, ChatGPT, I'm gonna hand you sets of messages. When you talk to ChatGPT, you write a prompt and ChatGPT gives you a response, right? So there's, there's this like pair between prompt-response, prompt-response. And if you prompt it 50 different things, you'll get 50 different outputs. The fine-tuning is just telling it in advance, hey, Here's— you give it an Excel file of here's a prompt and here's a great response for that. Another line, here's a prompt, here's a response. You could say, write an article about this topic, and then you actually have to feed it a full article about that topic. When I say we trained it on LinkedIn messages, what we did was we grabbed our 50 best LinkedIn messages by the number of likes and fed those in as the response because that's what the model in theory should be writing for us. Then we wrote a prompt for each one of those that could have generated that output. We hand all of that into the platform tool. OpenAI gives you this tool that consumes it. What it's doing is you're just saying, hey, when you get another prompt like this, this is the output I expect. When you get another prompt like this, this is the output I expect.
Okay.
It's specifically overweighting the process overweights now the output on the things that you gave it to fine-tune. So now, man, when you ask it— if you ask ChatGPT to write me a LinkedIn post, it'll write you a blah post, right? If you ask this fine-tuned model, write me a LinkedIn post, it's like, oh, I was told 50 different ways to write a LinkedIn post, and here's what a great output looks like. And it's the stuff we wrote, so it's like by definition our style, it's what we like. And so now it thinks that's what a LinkedIn post should look like. So when you ask it to write another one, it's going to write one that like seems like the stuff that, that's already there. So, so it's not like it's not that hard. You do have to do a little thinking of, well, what do I want to train it on? And like this guy on our team, he's like, well, you know, you probably don't want to do it on, on likes. You probably want to do it on engagements. And, and actually you might want to sprinkle in. He's like, I wish our content was better. And so I also grabbed a couple of posts from other creators that we aspire to be like. And like I fed in like a ratio, 70/30, like 70% our content, 30% our aspirational. So there's the like fine-tuning, like, yeah, it takes a little logic. You want to think like, hmm, what do I want it to actually act like? What traits would I like it to have? And then you gather all that content, which is also kind of annoying. It's like, well, how do you export all your posts from LinkedIn? I do. Like you have to gather it. But then you just basically upload it into a tool and OpenAI does the rest. Now you can chat with your new model. It's not— it is not difficult.
All right, let's talk about the AI engineer that you hired. What's an AI engineer?
Yeah, so I don't really know, but my experience with software engineers and this like new GenAI, like using GenAI, is they typically are very slow. because they want to do everything in code. It's like how they were trained. It's their whole experience. That's the momentum. And so I'm starting to find a lot more success on AI solution building with, with workflow people. You know, go find someone that has a Make.com certification or a Zapier certification, someone that's like, oh yeah, like I plug stuff in together. Those people, when you layer on top— and I'm really good at prompt engineering. And I know like N8N or one of these tools that's a little bit more like workflow AI, like AI native, those people are really powerful at churning out like actually good workflows that actually work, that are actually useful out of the gate. So we hired a guy, like we went looking because essentially, you know, we have all these cool ideas. I wish I could fine-tune I want 50 models, not one. I never want to talk to a non-fine-tuned model again. Well, that means, man, someone's got to go and gather all this data. Well, cool. Now instead of me having to spend all my time doing that and running out of time in the week, I've got this guy that's helping.
Give me more specific stuff.
What would you do specifically? Yeah. Take this LinkedIn editor. The ChatGPT interface, it's great. It's very familiar. When you go and fine-tune a model for whatever reason, OpenAI doesn't let you chat with it in their standard interface. If they did, that'd be great. They don't right now. Maybe they will. This guy on our team is— this week he's building out a very basic interface so that we can start to chat with all of our different fine-tuned models and even make it really easy in the chat to switch between all the different fine-tuned models. Because even in one, you might find, oh, this one is Let's go back to the LinkedIn one for a second. You might have a LinkedIn fine-tuned model that's fine-tuned on likes. You might have a different one that's fine-tuned on comments and engagements. You might have a different one that's fine-tuned for recruiting-type posts. When you go in, you might say, oh, what's the goal? What is the goal of my content? I'm trying to get this. Okay, here's my draft. You would select the fine-tuned model that we've custom-built that's specific to your to your goal for this post. You might even in this workflow say, hey, real quick, I don't want one that's already there. I want to give you a bunch of new content because I'm Andrew Warner and I want to create one of these fine-tuned models, but I don't want to know how the platform tool works. I actually just want to give you this big block of content, go and create another fine-tuned model for me real quick, and then let me chat with it in this window. We're building this little internal tool that lets you Like move through all of our custom models seamlessly.
And this person can do AI engineering using tools like Zapier and Make, which are just automation tools with a what-you-see-is-what-you-get type of interface. And that's it.
Yeah.
They're not, are they developers?
Uh, he has like some engineering experience. Honestly, I wasn't like, that wasn't a requirement in, in my search. And if anything, again, was like a negative signal for me. But no, he's like more of an automator. His past jobs were like automation engineer, which meant I'm building these workflows. You know, I think low-code is the best way to think about it for now. We lack integrations in a lot of places. So having a little bit of code is great, but 80% of the code this dude writes is written by ChatGPT. So it's not really a like standard engineering.
We're essentially talking about Just flowcharts.
Yeah.
Right.
Yeah. You can think through the problem and diagram it on a whiteboard. You can build it. That's essentially what these, what these low-code or no-code, uh, workflows.
Got it. And so what you need is someone who first knows the tools and doesn't have to keep hunting around for how do I drag this box on my flowchart to that box on the flowchart and someone who can reason well and say, when this happens, I want this other thing to happen. Got it. And you're gonna start to create more tools. I keep asking, give me examples of tools that you wanna create. So we've talked a little bit about hiring, we talked a little bit about content writing. What else are you thinking of?
Yeah, there's, there's, I mean, there's a lot. Like a really simple one is one of our businesses, they every day they have their bank account send them a summary of, you you know, the transactions from the day before over email, because that's what the— that's what their tool allows. That's what the bank allows. They send it there, but it goes to this person, and it needs to be shared with multiple people, and they're trying to make decisions. So then they set up a little integration somewhere between Gmail and Slack. So when it hits that guy's inbox, it also gets forwarded into Slack so that everyone can see essentially the PDF of this email and then make decisions. But it's really annoying, right? It's like there's this pretty clunky and no one, no one cares about the details. They're just trying to say, you know, do we need to draw more on our credit line? Is there enough in the bank to cover, you know, our expected expenses for the next week? And well, hold on. Why are they showing this to humans at all? Like, the AI can consume that email that the bank can— is already sending. So instead of sending it to, you know, Max email, We'll just have it forward into one of these AI workflows that says, okay, well, look at the expenses, look at the revenue we had. Does it meet these qualifications? And if it does, if everything is green, then don't send a message. Like, don't warn the business. Don't like use up our time every single day thinking about this thing. And if it needs attention in one of these 3 different ways, send a very specific message to a very specific person Hey, go figure this out, go make this change in the next day. And then they can drop in and do it without, without like annoying everyone else, without like missing it. Because it's like, it's not this, it's not this, you know, if you get a message every single day, eventually you're gonna ignore it, right? Like, oh yeah, that's the notification I get at 9:30 every day.
Right.
Like, I'll go back and check later. And then you'll, you'll forget. Well, what if it only alerted you when there was actually a problem? Every time it happened, you'd look at it, right? So not only is the workflow a little better and more directed, it'll probably be more successful than, than what we have today. So that's like one, like, teeny tiny example. And you could do that.
What about this?
You don't need to do that.
Before we started, I asked you, what are you excited about? And you told me about fine-tuning. That's exciting. And you just don't see enough people talking about it and thinking about it. You talked about hiring an engineer. All these things sound really far ahead. And then you said, I'm really excited about what people are doing internally with Gamma Decks. And I go, that seems like a very consumery thing.
Yeah.
Why are you adding that to the list of things that we could talk about today?
I think it's kind of interesting. I'm going to pull open— I, I just sent this like summary to, to, you know, Jesse, uh, the CEO yesterday. It's like, hey, like I wanted to, I wanted to present some ideas to Jesse. And, and Jesse is, you know, this is the CEO of this studio. He helps run all of these businesses as well as studio, he's thinking about 1,000 different things. If I want to get the most out of him, I need to communicate very clearly. I need to set up like, here's, here's the context you need, here's the decision I want help making, here's what I want you to know. And that has to be really, really clear. That's like not necessarily one of my strong suits. It's not my default. So what I thought was, you know what, like, let me, let me test out AI for this. There's this tool called Gamma which is in the middle of this workflow. And Gamma is basically a deck builder. So you could give it, you give it a prompt, you give it a PDF if it's context, you give it a call transcript, give it whatever you want basically. It's just an LLM prompt at the beginning to create a deck from nothing. And you could give it as much feedback on, I want it to be 30 slides, I want it to be 2 slides, or you don't even tell it, then it decides the number of slides and the style and all of that. You give it that and it'll generate a pretty convincing looking deck out of the gate. The cool thing is if you use it as part of a broader AI workflow, you could create full awesome decks that are really compelling, that really distill down the concepts with almost no work. So my workflow yesterday was, it's like, man, I want to communicate to Jesse. I've like demoed a bunch of AI stuff and I've learned how to demo better and who to demo to and why. And, you know, how could I predict when someone's going to be ready to adopt this AI workflow we're trying to build? And so I was thinking about that. I want to communicate it to him because I want his brain on this too. But it's a lot of like meta concepts, and I'm not even sure that I understand it that well yet. So I thought for 3 minutes, and then I didn't start writing. I didn't open up Google Slides. I pressed Option+Space on my computer, which for me is an AI transcriber. And so the transcriber is just dropping into ChatGPT at the moment, but it's just transcribing. And I talked out loud for 5 to 7 minutes, like, all right, it's this, and I've noticed this, and I introduced some features to Andrew, and here was Andrew's response. I introduced some features over here. I'm just talking. I'm just like vomiting out my story. And then I end. with a short prompt, again out loud, saying, all right, I want you— now I'm talking specifically to ChatGPT— I want you to take everything I've just said, distill it out into what I've learned, uh, tell me, break it into probably 3 profiles that I've identified. I give it this prompt and okay, then I let go of Option and Space. All of that drops into ChatGPT and I hit Enter. ChatGPT drops out this like great summary. I read through it real quick. I'm like, oh yeah, this is it. I paste all of that into Gamma as the source of the deck, and Gamma makes me like these 10 slides that go through and say, okay, here's the concept. This is pretty good, but like I want a better intro. Well, Gamma has a chat, and so I open the chat in Gamma And say, you know, I want a better intro that actually breaks down this whole thing into, you know, I've learned about like what traits does a person need to have, how do I need to demo, and then like what sort of response could we expect? Okay, it does it. It reorganizes the deck for me. I still have at no point typed words that end up in this deck, right? Zero characters. I'm just telling the AI like what I want the deck to be. I edit a couple times. I go in and I say, you know what, like, I actually want you to come up with some, like, catchy names. I built these profiles around people, and Andrew, you're, you're one of the profiles in, in, in this. It's like, instead of the people's names, like, I want catchy names. So I tell that. It comes up with Pained Pioneer, Reluctant Renegade, which is you, and then the Curious Captain, which is Jesse. Like, all these different people then, and, and, like, different ways that they relate to AI. Oh yeah, this is pretty good. I think I spent maybe 1 minute making direct edits and I hit the publish button and sent it to Jesse. And like, wow, that workflow of— I spent 100% of my time— like, you could think about like, oh, it made a pretty deck, whatever. The genius of a workflow like that is I did not waste any time at all with the tools or typing or design or putting this image where it belongs. 100% of it was me thinking and doing that reflection and like spending my valuable time on the most valuable thing, which is what the heck did we learn and how do I want to communicate that to Jesse? And everything else was done by like my copilot tools, ChatGPT and Gamma.
I've got to use that more often. I keep thinking about, especially in relation to you, all these high-level tools, but sometimes just the more consumer-y things end up Being super helpful during the day. What's the tool that you use for your audio transcription?
It's called Super Whisper. I think there's a, there's a thousand different ones. It, I, I was looking for one that had no features. I don't, I want there to be one button and all that button does is type what I'm saying into the thing. I had no clutter, no nothing. Just really, really easy. Super Whisper did that for me. I, but there's a handful of them for Mac. Um, and I'm sure a bunch of—
they're so good. The Super Whisper I think is available for Mac. I use Whisper Flow and the ability to just hit a button, talk, and then let go.
Yeah.
And it, it types it in.
Yeah.
It's super helpful. I do wish that while I was talking it would type it in so I could see what I've said.
Hmm.
But as long as it's fast, I'll take it.
You know what, man? Like, I think that's one of those things that within the next 6 months, I'm sure that's how they'll all work. Like, that's part of the fun part of AI is it's just changing so dramatically. And you're not the— I've thought the same thing. And all of these AI models generally support what's called streaming output. You know, when you talk to ChatGPT, it doesn't, it doesn't generate the whole response and then drop it in your window. You're watching it like type back to you. Well, Whisper, I don't know if Whisper, the model under the, under the hood there, if that supports streaming yet, but for sure it will soon if it, if it doesn't. And if it does, like all these consumer apps will, will start to use it as well. Yeah.
And they all make the built-in Mac dictation feature look so stupid.
Yeah, I was, I was using that before. It's like, oh, just like doesn't work. It doesn't— so many, like, you know, it just doesn't understand you. Yeah, it's phenomenal how good they are. So bad.
All right. For anyone who wants to connect with you, Adam, what's a good way for them to reach out? And I do like that people are reaching out to you.
Yeah. Yeah. A couple of people have emailed or sent me a message on LinkedIn. You can find me on LinkedIn, Adam Bray-Kane. Uh, or reach out direct, adam@gateway.xyz.
All right, thanks.
Bye, everyone.
