Eric Siu uses Jev, a Codex-powered tool, to evaluate YouTube Shorts against 36 criteria across four videos. The tool checks whether transcripts match titles, thoughts have resolved endings, and titles include a payoff. Siu can then ask Jev how to improve the evaluation itself, creating a loop that refines both the assessment and the videos.
Jev for Codex is crazy because take a look at this. This was, it helped me prepare 2 Jev videos. I got one out today, I got one out yesterday, and these videos are both performing really well on my YouTube channel. So what it evaluated, text criteria across 4 Shorts, 36 judgments. Okay. And then you can see what it evaluated. It looked at the work, it looked at marketing, SEO, business leads. Okay. These are for videos that we're putting out here. Each one of these columns over here is a video, right? So these are the evals down here, right? Did Did the transcript support the title? Supported. Is the thought complete with a resolved ending? Incomplete. Uncertain. Right? It's basically just classifying it each of these row by row. Is the essential context missing? No, no. If it does, like it doesn't pass, it doesn't get to go to the next level, right? What has happened now? What repair is needed? Okay. Does the title include Jev use case payoff? Yes, yes. I can continue to improve this over time. Every time we run this eval, I can just keep asking, hey, how do we make this better? Now that you build this, how do we rebuild it for next time? Now that you've taken the context of this thread, how do we rebuild this eval for next time? Right? Job is to continue to to tweak, give it feedback, continue to improve the eval over time.
