Supercharge Your Team With AI (Loops, Skills & Agents)
Eric Siu walks through building and scaling AI workflows called skills. He covers concrete examples like an AI finance skill that found $500K in savings, plus governance, quality testing, agentic loops, and how to get teams to actually use them.
Chapters
Today we are going to start to talk about skills evals loops as it relates to growing your business, as it relates to cloning your best employees, because that's what we all want. It's all about the process, right? 95% of the time it's not a people problem, it's a process problem. I'm going to show you how we're going about doing this right now. And I'm going to also take you through our, our Skills Dojo, a couple of skills that are really cool that we have. And also there's an AI marketing skills repo that we have that has about 2,600 stars on it or so. Let's get into it. I'm going to show you a couple of things. Maybe I'll start with some results here. So this is my X account. And really, this is last week of December. That's when Opus 4 or 5 started to really take off, or especially on X, right? Maybe November for some people, but December, at least for me, that's when it really started to take off. And then January comes and then OpenClaw, boom, OpenClaw comes out, right? And then I just start writing more and more X articles. Okay. But then with OpenClaw and with skills, I'm like, well, I don't want to have to keep repeating the same process every single time. And so we basically made an X long-form skill. And then what happened was, as I continued to get busier because I could do more stuff, basically we had great members of the team, multiple people now are actually using skills and they're compounding off the different skills that we have, right? So they're forking the skills. Basically, when you fork a skill, that means that you can make it into your own. You can upgrade it, right? And then if I decide that I like it down the road, I can just merge it into the main skill and it can become the main version. And you can see we're only in June 18th right now, at June 18th, right? And we've already broken the record. So I think we'll probably break 3 million views. But going from, call it December, like 183K, 275 over here, but really I wasn't getting, let's just call it maybe an average of $100K before on impressions. This month will be $3 million and then maybe $3.5. And I think we're starting to get ourselves in the flow, right? And so, but you can't really do this without compounding off of skills. You know, recently there's been some people that have joined the team and it's been easy for me to work with these people directly. And I can't do this with everyone obviously, but it's been easy for me to work with these people and mentor these people because of these skills. These skills ultimately add a lot of leverage, right? Right now on X, you see maybe like a month or two ago, people like Gary Tan from Y Combinator, He's like, you want to have fat skills, thin harness, right? And what that means is you want to have really skills that go deep and then the harness that you use doesn't matter as much. The skills need to be portable. So this is one of the examples, okay? But not only that, you're like, oh, Eric, that's cool. Like you're getting more views. Yeah, we actually get conversions from this too. We actually get leads from this. And because of the different harnesses that, or sorry, the different skills that I have, it was able to launch our Single Brain website that actually drives leads, right? And I'll show you what that looks like right now. So this is the Single Brain website. Now, is it the perfect— is it the most perfect website? No. But because we have a skill now that allows us to deploy these pages quickly with the right analytics, with the right tracking, that makes it easy for us to spin up new landing pages and things like that, right? So this is it for Single Brain. So all that to say is I'm going to show you some of the skills over here, and then I'm going to go into our Skills Dojo as well. And I think everybody needs to be making a Skills Dojo, and I'm going to talk about Kind of the methodology and the thinking behind it as well. So this is the AI marketing skills repo we have. And if you go to singlegrain.com/skills, it will redirect you to this page. Okay. So what you'll see, what you see here is that we have a build. Here's a skill clone website. So if you like the single brain website, the one that I just showed you, you can use a clone website skill. Okay. Or there's a content eval here that you have. Okay. Or there's one for like a lead dossier. So like, let's say a new lead comes into your website. Well, what if you want it to You wanted to do multi-source account research, you wanted to cascade enrichment and lead score it. Now when a lead comes in, we'll see how fast is this company growing right now? Who are the main stakeholders? How do our products and services fit in? What news seems relevant for us to talk about for the discovery call? And we should be running this every time. You can see the architecture for this and all this over here. So instead of you having to think about this, I would just recommend for me, like I see a lot of people sharing interesting stuff. Oftentimes I'll just go and try a skill. Probably 80% of the time I'm not going to like it, but maybe 20% of the time I'm going to like it and maybe I'll just steal it. And then maybe a small percentage of time I might upgrade that skill for my own purposes, right? Because we all have different goals. I have this little finance ops skill here that saved me $500 grand. Like literally I ran the skill and you can see, okay, this is your AI CFO that finds hidden costs in 30 minutes, right? Am I saying cut your CFO? No, I'm not saying that. I'm just saying that instead of waiting days or weeks to get your data pulls, you can do this a lot faster. So you can pull your QuickBooks exports and then you can pull whatever else that you're using. Maybe you're using Ramp or something like that. And it'll give you an executive financial summary, profitability analysis, people cost breakdown, tool subscription audit. And so basically for me, it said, hey, you're actually spending a lot on these tools over here and you're spending a lot over here on maybe contractors that you might not need based on the goals that I know that you have. And they'll call it anomalies, they'll call it maybe duplicate spend as well, because duplicate spend happens. And so in a span of like me running this in the next 3 days, I cut that $500 grand in costs, right? And so that's pretty cool. And again, you can just go out there and steal this. So there's conversion ops skills, there's SEO skills, there's podcast ops, there's revenue intelligence, There's this X long-form post, like the skill that the— you saw my X just growing like that. You could just take this. So we have— we combine skills in here. There's an X long-form post writer and a humanizer. This writes long-form X articles. It takes my voice, right? So you got to think about what your voice is, and then it tries to humanize it. It takes out this whole negative parallelism stuff, right? It's not about AI, it's about human, right? Like, you, you see that a lot in AI writing. It's— you see a lot of em dashes as well and vague attributions. We don't want that too. So we want— we also want like diagrams in it too. So we want ASCII diagrams. And so a lot of people ask, oh my God, Eric, you're spending so much time on these articles. How do you do these diagrams? It looks like you work really hard at— nope, didn't do that. It literally is ingesting a lot of my content from other sources. And that's how I used to use this X long-form skill, right? So I'm going to jump over to our Skills Dojo just to give you a little preview. And then I'm going to talk about how to think about skills, evals, and loops and how these all combine. So this is our Skills Dojo. Okay. And what this does is it It has a lot of our— we just added this, I think maybe a couple days ago this week. So this is now internal to my team. So again, it's cool if I have 360 skills, but I want my team to have access to as many skills as possible. And some of them you can't give access to your team on, but I'm just telling you, if you have a Skills Dojo like this, this is great. So we have the X Long Form Post Writer. So you can go in here, think of it like GitHub for skills. You can call it Skills Hub, right? But the skill that I just showed you earlier, so this will write long form X posts. You can see how many bookmarks it has, how many upvotes it has. You can copy the prompt for it as well. You can see the adoptions by different teams that we have too, the voice, the structure, the ASCII diagrams. And honestly, this looks better than the GitHub repo that I showed. And so, and here's like a, here's a pattern checklist as well. Okay, here's a humanizer scoring thing. So the whole idea here is if someone on my team, they decide that they like this, they can download this or they can upvote it. So it's almost like, it's like a forum, it's like Reddit almost. And you can see what kind of pops to the top and what pops to the bottom. So in essence, it's almost like a little app store as well. Right? So keep in mind, this is going to continue to grow over time. And by the way, if I publish a bunch of poo-poo skills, then they're just not going to rise to the top. Just think about how much garbage there is in the Apple App Store or the Android App Store. There's a lot of garbage, right? And so the whole idea here is we're just going to continue to make this better and we want to continue to add our fat skills to this thing so our team can use this. And also at the same time, you're going to see who's going crazy with this stuff and who's uploading the most used skills. You're also going to see who needs some help too. And then your team can go out or you can go out and help these people. You're also going to see who needs help, who accepts the help, and who needs help but doesn't want the help. Okay? So you want people who are willing on your team, and as long as they're willing, there's usually a way, right? But if they're unwilling and they're capable, then that's a business decision that you have to make, right? That's— I'll just leave it at that because I do think a lot of these things are very teachable. And as long as, again, as people— Yeah. As long as people are coachable, you're going to be fine. Again, we have— these are just kind of the intro skills here, but later today I'm going to be adding skills to this. I'm going to be adding the 360 skills that I recovered. So we have a published skill section over here, so we can publish a new skill. You can publish a new version. I'm going to be combining them. I'm going to have my Hermes basically zip the ones that are safe and that are high leverage. So maybe it gets cut down to 100 skills or something like that, and then I'm going to upload it here, and then it's I can decide on the safety and the privilege level over here. Like, can people— is it draft only? Is it read only? Internal, right? Can they send with it? Whatever it is exactly, right? And so again, this is very much a work in progress for us, but the whole idea here is when we send this as well, this is going to sanitize it, right? So your skills might have certain API keys in there, for example, or certain passwords in there, which you don't want other people to have. And so you're going to— this is going to— when you're uploading it to a Skills Dojo, right, or our Skills Dojo, then you're going to— it's going to be sanitized, right? And before it gets published, and then the machine will also double sanitize it before it goes out there. So again, I think everyone should have a version of a Skills Dojo. I think from a metric standpoint, again, you can see kind of who's adopting it, who isn't, and who's engaged and who isn't, and what's the most downloaded skill. So all that to say is, if it's true that most of the time the problems that you have in a company are a process problem, well then it's on you to make sure that most people, like for repeatable things that you're doing for customers all the time, Most of them should probably be skillified in some way, shape, or form. And then the best versions are going to rise to the top. And sometimes the best forks are going to rise to the top as well. That like, and when, let's say, you know, someone on my team decides to fork a version and it ends up getting way more votes than mine, it gets way more usage, then yes, it's democratic, right? Also, we're like, let's say someone's joining my team and they're on the sales team, for example, or maybe they're on the paid media team or they're on the SEO team. So we're going to have these skills packs for people. Where we can just say, hey, you're onboarding to the company here, just take some of these packs over here and go run with it, right? And then over time, they're going to improve the skills too. And again, the most used, the best used stuff should rise to the top. And yeah, it should be democratic in that way. So by the way, we have— we're going to talk about skills we kind of broke down already, but let's talk about evals and loops too, right? And so there's also verification checklists too, to make sure that whenever a skill is like— when we're reviewing these skills, right, making sure that There's a clear owner for each of these. There's a clear team that it's dedicated to, that there's an outcome statement, that's, you know, when to use it and all that, right? So it's important for you to have these checklists in there. Otherwise, if everyone just kind of YOLOs with this, it's not going to be a good look, right? So that's what I would say. This is admin look, so I can see who's trying to upload stuff and then who wants to, you know, approve or disapprove it. Probably have a machine help with this too, just to do like a first pass. But that's, that's the Skills Dojo. Okay. And a fun fact, I tried to buy skills.co today. I offered $20 for it. The counteroffer was $175,000. So anyway, these domains aren't cheap either. By the way, quick note, I'm going to be speaking at the MozCon conference in New York City. This is happening on July 14th, which is coming up very soon. And I'm going to be joined with a bunch of other amazing SEOs and AEO experts from around the world. I'll be sharing my concepts around how AI is evolving, not just from a search standpoint, But also a business standpoint. I'm going to be talking about real money-making workflows. So you can check it out, just search up MozCon. If you go to moz.com/mozcon, that's M-O-Z-C-O-N, again, M-O-Z-C-O-N, you can find out more information there. And we'll see you on the other side. So let's go over to this because this is what I've been training the team on, right? Because I've been doing these apprenticeships, almost like Jedi trainings with the team, because it does work for a few people on the team. Like, okay, well, I want to see how we can roll this out to maybe a broader team and then like maybe a couple people on the team and then they can roll it out to the broader team, right? Because I want these people to then become Jedi Masters on their own and then they can train other people, right? Because then I can scale myself ultimately. And so we have about 3, 4 weeks of training left and this is my teacher's kit that I made using, I think I used Claude code to build this by the end of this training. And so Yes, we have a Skills Dojo, but it's like, okay, that's not that helpful if you don't train your team on how to use it. So part of the reason I'm also doing this webinar too is so we can reuse this training. One, Max, this is content, but two, we should make this more repeatable because as I continue to update my views on this stuff, this will help in terms of our group training. So by the end of our 6-week program here, every person can build and use Cloud Code/Codex skills, and they have 1 to 2 real skills shaped to the Skills Dojo. With evals. There's that word again, right? And has the confidence to train others on this. One person doing the work of many, 4 to 5, even 10 jobs by leveraging AI. And so I was at an AI mastermind with a bunch of founders a couple weeks ago. And one of the guys said, so he used to be a pro, let's just say pro hockey player, right? And he works in, let's say he's in the services business. I'll just give you an example. And he's like, you know, in my industry, you know, I'm always looking for people that can do 5 or 6 different jobs. And then the rest of us who are kind of in software, we're like, yeah, 5 or 6 different jobs, right? Because the reality now is, you know, Coinbase talks about this concept of this pod of one, right? Which is where you might have not just an engineer, you might have an engineer that functions as a product person, as a designer, and then kind of you can swap these around, right? If you're a designer, you can do some product stuff, you can do some engineering work now. So I see this no different with my services company, my marketing services company. Why can't— like, I know I can do a lot more now and I've seen people on the team, they can do a lot more now and they're having fun while they're doing it. The only time where they feel held back is when other people aren't doing it, right? And so anyway, all that to say is, okay, well, how can I create the conditions for success with these skills, with these evals, with these loops so everyone can come along or everyone has the option to come along? Not everyone's going to come along, but if your job as a leader is to create the conditions for success, And if they want to come along, they come along. If they don't, that's on them. Okay. This program over here, you can see like each week we have this, every week there's a run of show, right? Where we kind of unpack things, we build, we show to each other. What I want to call out here is we kind of talked about what a skill is, right? Just at a high level, a skill is not a clever prompt. It's a packaged capability that the team can reach for again and again. 5 traits separate a skill from a one-off. It's one clear job. It's reusable. It's documented. It encodes your standard and it ships with Eval, which we're going to talk about right now. So what is a good eval, right? When I saw people write about evals maybe a year ago on Twitter or X, I was like, man, that's just for developers. That's not for me, right? It sounds scary. What is a cron job, right? I didn't know what a cron job was before. I'm like, oh, that's just for developers, not for me. An eval, I mean, it's judgment, right? It's judgment that you have, and you're trying to pass on this judgment. People are like, oh, humans are for judgment. The machines are for doing. Well, no, you need the machines to judge as well, right? You need the machines to think. But if you, the more context you give it, the more judgment you pass to it, the easier your life is. going to be. And so eval is just— it just writes that judgment down so a computer or a teammate can run without you the same way every single time. So our rule in our Skills Dojo is if you're going to write a skill, you must have— if you want to submit it to the dojo at least, if you don't have an eval, you don't define it, we're going to reject you. And that's just how it is. You have to have some type of quality in there, right? So you have good skills, you have good evals. Now the next step is a loop, right? And the easiest way to think about an agentic loop right now, everyone has their own definition, If you know Boris, the creator of Claude Code, is like, "I don't prompt anymore. I just type loops." Really, it's just a version. I'm just looking at my Codex on the on the right side. I'm using Claude Code as well, so I use both. But I just keep spamming spamming slash goal. As long as I've defined what I'm looking for, I just keep spamming slash goal, right? And you can do you can do slash loop as well inside of inside of Claude Code, right? And so to me, it's just you're able to run these these tasks longer, and if you plan it out more. You can use the Grill Me skill from Matt Van Horn, or you can just ask it, use any of the plan solutions from each of these. The more deeper your plan is, the longer your /goal or your /loop is going to run, right? To me, it's just a version of that. And so I think this is all on the way to recursive self-improvement. I don't think we're there yet, where the system gets better on its own, right? We're not quite there yet. I think we're going to get there within the next 12 months or so. But right now, I think it's like A couple months ago, we were talking about Ralph loops, right? The Ralph Wiggum loops. That was like a version to get to /goals. Now we have /goals, there's /loop as well. And then eventually, it's going to get better and better over time. The models are getting better and better over time. And then eventually, it's going to get into recursive self-improvement. But if you have good skills that you daisy chain with your plans, it's just going to be able to run longer. If you are very broad with how you've defined something, you don't have good skills, you don't have the fat skills, okay? you don't have a good plan in there as well, then it's not going to go very far. And part of a loop to me is also if you're defining like a really like, you want a long-running thing to go, like a long-running command, like, yes, what I just mentioned, that's important. But you can also say a version of a loop is a bunch of these skills daisy-chained together, right? With evals in there. So an example of this might be around like we do AEO, SEO, we do Reddit services, right? On the agency side, it's like, okay, well, what do we have to do on the SEO side? We have to think about keyword research. We have to think about competitor research. Okay. We have to think about internal linking. We have to think about, oh, if we're going to do data storytelling, how's that going to look? How does Reddit fit into this? How does YouTube fit into this? These are all skills that maybe can be invoked based on whatever the customer is looking for initially. And then there's an onboarding skill for that too. Based on that onboarding skill, what other skills activate here? And then that is basically your onboarding loop. Okay. And then from there, maybe you want it to be a longer running loop. Then you can decide if you want to run it through a slash goal. And so, I will say this, nobody has a perfect definition of what a loop is right now, but that's my interpretation. And I find more people are saying that right now than not. And when I did the talk with Matt Van Horn, it was a great talk and he talked about a lot of different skills that he has like Last 30 Days and Grow.me and things like that, like his printing press CLI. Those are all cool things that I recommend you watching that one. It's on my YouTube channel. But I would just say that I don't think he had the best definition either. And he kind of just said like, everyone's just kind of making it up as we go, right? And so, I think we'll eventually settle on one, but in my opinion, it's You're looping. Looping is basically you're combining a bunch of skills together that makes sense to combine with evals in there that makes it clean. And then if you wanted to go deeper, then you can decide if you want to execute on like a slash goal there. This is what it looks like. And then we even have like skills and skill.md and eval templates in here. So I would encourage you all, I think it's one thing for you to watch me talking about this right now, but it's another thing to actually train your team on it. And I have a— it's a joy for me to train the people that want to be trained. It is hell for me and also probably hell for the other person when they don't want to learn, right? And it's so easy to see. Anyway, catch you all later. Thank you all for joining and have a great rest of your day. Thank you.
