
Eric Siu presents Jev, a classifier layer that plugs into AI agents to cut hallucinations and automate decisions. It evaluates content overlap and claim strength for SEO, scores recruiting candidates against hiring bars, and checks transcript-title alignment in video production before publishing or outreach.
Chapters
Jev is like having God mode for AI. So all the harnesses that you're using, it doesn't matter what you're using, you can plug it into all of them and think of it as a classifier layer that you're adding in on top. It helps you make better decisions, ultimately helps you classify a bunch of files in volume a lot faster, and it hallucinates a lot less. They're claiming 0%. And when you look at what they've had, I mean, one, it's very cheap. Two, it doesn't really hallucinate according to what they're saying. And then two, it processes data a lot faster. So what I'm going to do in this video is I'm actually going to show you how it's being used within my different harnesses. So it could be GrokBot, could be Instinct, could be MetaMuse as well. And there's a bunch of different harnesses that I use. I use Codex, I use Hermes. I'm going to show you how I'm going about using it and how you can continue to refine your use cases, right? And so once you watch this, you should be able to understand, okay, I see how Eric is using it right now. And then this is— then I think about my use case is what's the highest leverage thing that I can do? And this is how I should use it, right? So let's start with GrokBot first because I have an AEO SEO background and what I like here is that it is continually helping me with this bot. What this bot does is it will ingest a bunch of AEO SEO ideas from the content that I put out or maybe conversations that my team is having with customers, customer objections, things like that. Those are all content spikes for ideas that we should be generating for AEO SEO purposes. Now, you might be wondering, how does Jev actually help here? Well, here's the cool thing. You look at these columns up here. You can see, okay, number one, this evaluator over here. Okay. So does this overlap with other content that you have? Okay. Clear, clear, clear, clear, clear, clear. Okay. That's good. Right. And then job delta. Okay. So is this, again, is this overlapping with any other jobs of the post itself? Right. And then the claim, like, is the claim thin? Right. So if the claim is thin, then we shouldn't publish this, right? If we don't have much evidence behind it, we shouldn't publish it. And then the Eric tape thickness. Okay, so this one, I mean, this is more so, is there anything that it's pulling from where I might have related content? Okay, so this one means it's stronger in terms of the evidence that I have, right? Okay, so this is giving you some ideas on how to approach this. AI tells. So this one's important too. I don't want a bunch of content that I'm publishing that's very easy, easily, you can easily tell that it's generated by AI, right? So what I've been doing here, by the way, it took some time to get here, but I'm constantly iterating with this over and over in terms of the right eval gates that I should have. And so I'm constantly working with— I'm like, okay, how do we fix 3 and 4 then? And then I'm like, hey, how do we get the AITell piece to low across the board for the content that we're publishing? But we need to try to get it low across the board. And then when we're weak on Eric tape thickness, maybe we should go get more. Let's lock that for all in. This is how I'm working with it and you can see, okay, the AI tell now is low. Let's see what it added. Then it's like, hey, the call here is this is clear, this is good to publish, this one not so much. This example right here should be a microcosm of how you will work with this. In fact, why don't I just show you how I've been working with my little chief of staff bot. What this one did, It looked through all the people that have been applying to work at my company, Single Grain, right? So this is an example. Juicebox is one of our tools over here, right? It helps with agentic recruiting. So it shortlisted 100 people and said, Jeff scored them against the frozen hire bar. Okay, so that's the hiring bar that we have, right? None were marked ready for InMail. Okay, so when we're unsure or thin, we hold, we don't pretend that they're hire ready, right? So this is stop emailing people who only look good on the title. So before we didn't have this before, we weren't able to do that. And so it got really messy, right? There's just more to sort through when you don't have a classifier. So tier 1 to tier 3 leads, these are higher level leads, right? Which ones do we suppress? Which ones do we stop emailing? Which customers do we not email when it comes to doing sales outreach or recent contacts, like people we talked to maybe in the last 90 days or so, and maybe we didn't win the deal or whatever it is exactly. And so we've added it here. Oh, this one's really cool. We had a HaggleBot inside of GrokBot. So what it did was it— what it does is it basically will look for opportunities to get discounts on software, right? Or look to cut down, on our seats. For example, like we're paying for Asana. I realized that most of our team doesn't even use Asana and we're just paying this stupid bill. And so what it does now is it basically added in the ability to see usage. And so Jev will basically score on usage and it has all these connectors in here. So basically say, hey, like, are you actually using it or not? If not, then let's figure out what we can cut it down to. And it'll just show you, hey, here's what your actual usage is and here's what you discount and buy, which is, it's numbers-based, it's evidence-based, right? Even when we look at our— we have a speed to lead bot. We just— what we've been doing with Jev is we've basically been upgrading the main workflows that we've been using. And my team uses GrokBot pretty heavily. I would say we use Claude pretty heavily as well. I personally use Codex a bit. And so whatever you're using a lot of, I would just consider this to be an upgrade to what you're doing. And that's why I call it God mode, right? Okay, so this is me using my Instinct agent. So it said, hey, I was like, hey, what have we done with Jev so far and how could it make things better? And so it said, okay, here's what we did so far. So mapped wins and losses. It stress tested your whole candidate pipeline. 40 isolated checks across 6 live candidates agreed with deterministic, every deterministic blocker, zero false advances. It caught a real bug. Batch grading said, this person over here, been there, done that, when the truth was they haven't actually been there, done that. It also graded 2 podcast drafts before you approved them, found the shared flaw, no defensible proof point, too much category speak with concrete fixes. Okay. So where it makes things better from here, every eval gets a second opinion in 250 milliseconds, calibration data towards the 25 case activation gate. Every outward pitch gets credibility score before it ships, and candidate screening gets a cheap second signal at scale. What it won't do, it won't write anything or make final calls. You're deterministic eval stated decider until the data says it is earned more. But it takes— it basically helps you make decisions at scale. That's what I would say, right? Because once you've set these evals in place, the definition of success, and then you can continue to tweak these evals over time as well, right? That part. Yeah. It's very important. By the way, if you're enjoying this right now, and if you need help with marketing implementation around AEO, SEO, paid media, ad creative scaling, what we do is we build these marketing agents and we manage them for you, and they live inside your Slack and your Microsoft Teams. Just go to singlebrain.com with the B, singlebrain.com, and we'll see you on the other side. So when I look at this, I'm like, hey, okay, well, just we're making this video right now, right? It's like, okay, well, did you already, uh, did you do these And how high leverage are these 1 through 100? And so I always like asking that question. So, you know, prescreen rebuild done permanent 75 out of 100. It kills a whole class of glading errors whether or not Jeff stays. Okay, that that's powerful because we don't want to outreach to the wrong people, right? Okay, now content QA before shipping done on the two podcast drafts now runs on every outward pitch. This is 85 out of 100 highest leverage. Your name goes on every cold pitch. This catches. Weak credibility, pre-send at volume for free. Okay, great. Second opinion on every eval done and running. Okay, this is 65 out of 100. So far agreed with everything. Calibration activation in progress now, need 25 real cases for workflow. Okay, great. So you can see how I'm asking the question, but even after you get the answer on how high leverage it is, the things that you've built already, you now ask the next question. How do we make this better? Now that you built this, how do we rebuild this, right? Or how do we enhance this even more? Okay, so these are 4 moves. Right? And so feed it reality, the biggest one. Right now, Jeff's labels sit next to deterministic labels, but nobody's grading against outcomes, which candidates you actually advance, which pitches get replies. So it needs that feedback loop, right? Wire that feedback in. Okay, great. That sounds fine. Make content QA a pre-send gate. So every outward pitch like YPO invites, recruiting outreach, sponsorship DMs, speaking abstracts get Jeff scored automatically. Okay, I like that idea. The shadow discipline is what's making the data trustworthy. Okay, shadow discipline. I like that. Okay, so now we're on Meta Muse, right? So Meta Muse is their autonomous agent, right? So I just, I asked the same question here. I was like, hey, how high leverage is this? Honest version. So it says, it has supercharged my judgment calls. We call it shadow discipline, right? Shadow judgment, whatever you want to call it. So Monday packaging. Okay, so every content concept now gets scored 0 to 4 on buyer fit, hook strength, and format match before it becomes a shoot card. Backtested 9 out of 9 on my old concepts. It would have killed the flops and kept the winners. Cost per decision is 0.00002, under a second. So under a second to do that, minimal cost, helps you make a better decision. I'm with it, right? But I prefer to use it for business, for work, right? So the eval gate, this is the actual superpower and probably your video's contrarian take. Nothing gets to make judgment calls until it proves it can replicate your past decisions. No ungated decisions ever, okay? So smartest use cases for us, ranked idea triage before production. The packaging gate is a template. Inbound scoring, DMs, applications, deal flow, you only touch 3+. Call intelligence, score Gong/Granola calls. So that's like sales intelligence or note-taking for a buyer fit instead of relisting. The meta use, anywhere a judgment call currently costs you 10 minutes or a full LLM call, Jev does it in milliseconds for fractions of a cent after it passes your eval. I really like that. So I make AI prove it can think like me before I let it decide anything. I mean, this is great, right? But again, this is why I say Jev is God mode for AI because you have that decision layer that most other people don't know what to— they don't know about this stuff right now. The fact that you're watching this right now means you're ahead. You just got to go out there and do it. And as of— I'll just say this, anybody can access Jev now, right? You don't have to worry about not being able to access it anymore. So we talked about Muse, we talked about Instinct, We talked about GrokBot. Now let's move over to Codex because now I can show you how that work looks inside of Codex. Jev for Codex is crazy because take a look at this. This was— it helped me prepare 2 Jev videos. I got one out today, I got one out yesterday, and these videos are both performing really well on my YouTube channel right now, which is why I'm even cranking out another one right now. So what it did was it evaluated text criteria across 4 shorts, 36 judgments. This cost $0.000302. And then you can see what it evaluated here. So it looked at the work, it looked at marketing, SEO, business leads. Okay. So it's just saying, hey, what Jeff evaluated. So this is for— these are for videos that we're putting out here. So each one of these columns over here is a video, right? So these are the evals down here, right? So did the transcript support the title? Okay. Supported. Is the thought complete with the resolved ending? Incomplete. Uncertain. Right? So it's basically just classifying it. Each of these, row by row. Is the essential context missing? No, no, no, no. If it does, it doesn't pass. It doesn't get to pass go. It doesn't get to go to the next level. And so what has happened now, so it's like, okay, what repair is needed? Does the title include JEV? Use case payoff? Yes, yes, yes. Okay, great. I can continue to improve this over time. Every time we run this now, this eval, I can just keep asking, hey, how do we make this better? Now that you build this, how do we rebuild it for next time? Now that you've taken the context of this thread, How do we rebuild this eval for next time? So your job is to continue to tweak, give it feedback, continue to improve the eval over time. You can watch this video right now. I'm not going to say this is the best edit, but it's way better than before. I'm actually curious to see how these perform because the videos that are going onto my YouTube channel right now, they're AI edited. And I'm not going to say they're the best. I would say they're like a 7.5 out of 10 or so, but I just want to get it out there quickly. I would say the thumbnails are like an 8.5, 9 out of 10. Because I have the thumbnail skill and I just kind of tweak it. I spend like 5 to 10 minutes on it. So that 5 to 10 minutes goes a long way. That's the judgment and taste piece. So let me just show you what the short form looks like for this because it was really bad before earlier this morning. Um, but using Jev has made it a lot better. So let's just play it real quick. And so when you think you— when you have like a lead come in, right? Um, or maybe you're reaching out to someone from a recruiting standpoint, maybe you want them to have 2 promotions at 2 different companies and you want To see an average tenure of 3 years. So it can actually score and it can also give a confidence score. You see this? This is pretty cool. So you can see this is scanning. It's real. There's motion. I asked it to include the Jev icon here and I said just put Jev for Work so people know what they're looking at. And so this is cool. Am I going to say it's perfect? No. Again, I think there's a 7, 7.5 out of 10. One more thing, we have the Single Brain Gateway. This allows you to connect all of your different tools. And instead of having all your different API keys and your agents need to, you need to connect them one by one. You can just connect it to one single gateway. And the cool thing is you connect it to the gateway and you define what your marketing goals are, what your revenue goals are. And the cool thing is we can even service other workflows that you can be using to grow a lot faster, right? And so we're constantly building and building more workflows, and you can compound your business a lot faster by also, one, having a secure gateway, two, having everything in one place that your team can also use as well. But the third thing is If we are able to understand your goals, then we can recommend the right connectors for you so you can actually grow your business a lot faster because the way we work is changing. And I believe that the way we work is going to be— we're all going to be managers of these agents and workflows in the future. So if you want to learn more, just go to singlebrain.com/gateway and we'll see you on the other side. And by the way, I said, hey, put like the B-roll stuff in there so people can see what's happening. So you keep the viewer engaged, right? And so these are all being scheduled now. And I like Codex because it can— I just work within it, right? And so it scheduled all these videos. It used the Metricool API, which schedules for social. It figured out the scheduling and it put the video— it edited the entire video. It used Jev to eval it. And then from there, it made a bunch of other articles as well, like X articles. It eval'd those articles and then it eval'd the short forms, the mid forms, the long forms. And this is the missing piece. Again, that's why I say it's God mode. And so check this out. This is me using Jeff again yesterday, right? So I'm with marketing school, Neil and I, we crank out so much content. And so I'm just trying to make like these— it first made like this like slop cannon, like characters, like this one over here, like this wasn't the best, right? I didn't like these. It kind of looked like the Will Smith like slop when he was eating spaghetti back in the day. But here's how I started off. I was like, hey, if we use Higgs field, can we do this over here? Okay. So then naturally I started to continue on and I'm like, hey, how do we layer Jev in here? How high impact is it? So what I just did right here as we're recording this video, I said, hey, so show me a table of what Jev eval over here. So it looks at the hook, it looks at the story, it looks at the teaching, it looks at the visual relevance. So keep in mind, you're not limited to the different level of criteria that you can put in here. Payoff, adequate, call to action, preserve source meaning, complete thought, missing context, needs visual proof, yes. If it doesn't pass go on these, then we just don't continue. Then it's like, hey, this fails so far because of this. And then you can set an agent that says, okay, if you have this result, it should path over to this agent over here, then it can take it and do the work in terms of like, oh, okay, maybe there's a payoff agent, maybe there's a lip sync agent, whatever it is exactly. So when you're making these videos, it's $11 using SeedDance for this one and then Kling Turbo. I'm just going to continue to make the evals better, but let's just— I'm just going to, for entertainment, I'm just going to show you this video over here. Every new client needs a different report. That doesn't mean you need a different process. Save a template once, put the new client's data through it, let AI draft the summary, then check the numbers and add your judgment. Next week, use that same workflow, new data goes in. Okay. So it's like 7 out of 10, but I can see this going somewhere because the teaching is up there. We're kind of sitting next to each other. I want the lips to be moving. The pauses are going for too long, right? So these are all things that we can fix. So for me, as a business creator, creating is not my job, right? And so for me, I'm like, I need something that just will crank out the right formats for me. I need to understand the best formats. And then I can just work with Codex and he can interview me on the things that are top of mind for me, the things that are annoying me, the things that I might disagree with someone else on because controversial stuff will typically drive views, but I want to make sure that I'm driving leads as well. When I look at this one, I'm like, okay, show me a table what Jeff eval'd here. It looked at which— these are the ideas down here that it evaluated. Which marketing agent should I build first? Format fit over here. Which marketing metrics matter most with AI? How much autonomy would I give AI tasks? What actually matters when putting AI to work? Which AI tool categories to add first? These are all scored out of 3, I believe. And you can see, it basically— Codex helped me make these studios over here on the different tier, the different ideas that I should be working with because I'm just looking for ideas that are easy for me to come up with, to yap. And then also it's easy for the AI to edit. And it's also, I don't have to do all these weird videos where I might need to have different scenes and all that, right? Because if you're in business, you tend to have a little credibility. And so you can just kind of narrate what you're going through there, right? Running through this 9-call production fit run cost, okay, cost $0.00074 calculated from token usage, no new calls ran. So it also says this, Jev detected the obvious test problems, but its scores for the real briefs are clustered near the top. That is not enough evidence, okay? So what I'm going to just say here is this. Okay, well, now that we built this on the Jev part, what do we need to upgrade it to make it more reliable or trustworthy, or do we think this is good enough? Let me know. And so again, you've seen how I work with this. You probably work differently within your harness, but again, you're adding that judgment layer or the— what is it? Let's just call it shadow judgment if you want to call that, or shadow discipline. And ultimately, if you're able to take a huge load off your shoulders when it comes to decision-making, you're going to be able to move a lot faster. So anyway, let me know what you think about this video and we'll catch you in the next one. Thanks, Eric.










