In this presentation, Robert Ta introduces Typesafe's Jev, a System 1 decision model that classifies and routes inputs without generating text. Jev is positioned as a dramatically cheaper and faster alternative to traditional LLM judges, offering up to 440× lower cost and 109× lower latency while achieving comparable accuracy on benchmark decision‑making tasks. The talk walks through early benchmark results (Jev 68% accuracy at $0.0004 and 0.4 s per case versus GPT‑5.6 Terra 68% at $0.03 and 10 s, Opus 5 73% at $0.18 and 38 s), outlines a spectrum of decision‑making approaches (logistic regression on embeddings, fine‑tuned encoders, cross‑encoders, small‑team judges, frontier LLM judges, and Jev), and provides a cost‑latency‑explanation matrix for choosing among them. Jev’s zero‑shot classification can reach 98.3% accuracy without task‑specific training, but it does not produce textual explanations, which can be critical for debugging and iterative improvement. The speaker recommends a hybrid architecture: run Jev on every trace for cheap, high‑coverage scoring, and selectively invoke an LLM judge on uncertain or failure cases to obtain explanations. This strategy balances cost, speed, and the need for interpretability, and has significant implications for AI evaluation pipelines, autonomous agent loops, and overall application architecture.
Chapters
So Arise has a great blog. I would recommend people sign up for their blog. Typesafe's Jev. So Jev is the latest hotness, 440 times cheaper, 109 times faster than the competition with their System 1 models, which introduces a new class of models. One of the use cases is LM judges. So can Typesafe's Jev decision models replace LM judges? Typesafe's Jev classifies scores and routes without generating text. Up to hundreds of times cheaper than an LM judge. What that changes for evals, conference routing, and application architecture. So, JEV versus LLMs, early benchmarks on accuracy, cost, and latency. If you're an AI builder and you are trying to figure out your evals loop for closed autonomous, closed-loop autonomous agents, you want to listen up here. Everybody wants to get to this amazing LM judge for some domain-specific workflow, for some domain-specific harness that will eat up everything. And the more data you put into it, the more of a moat and IP it builds and accrues. But you know, that there's the promise and the idealism, and then there's like reality. So I love this, which is comparing the use case with Jev with their structured outputs and free output tokens and much cheaper and faster application to this particular workflow. So they published an eval for decision workflows, one of which is reviewing a finished agent trace to decide whether a human needs to look at it. Averaged across the 4 workflows, JEV lands at 68% accuracy at $0.0004 and 0.4 seconds per case. GPT-5.6 Terra is at 68% for 3 cents. So that is a couple or 3, yeah, a couple orders of magnitude difference and then 10 seconds. So couple orders of magnitude difference on both costs and speed. Opus 5 is at 73% for 18 cents and 38 seconds. So that's 5 points behind Opus 5, which is the leader. But on the other hand, it's 440x cheaper. That's a very interesting cost-benefit trade-off, such a radical one that may change how we architect applications. So this article goes into a little bit of why did LLM as a judge become a default for AI evaluation? Why are you using LLMs to make decisions in the first place? The reason is simple. They are able to do it without huge expensive training sets. Which is what most ML solutions prior to LLMs required. So here are some options. Approach: logistic regression on embeddings, fine-tuned encoder, cross-encoder re-ranked, small team judge, frontier LLM as judge, decision model JEV. So here's a little cost table on the different approaches to make structured decisions in your ML pipeline and eval pipeline. And you can really boil down to a few heuristics. Do you need labels? Cost per decision. Latency, which affects user experience, probability you can act on it, and then explanation required or not. So for your AI workflow, LLM-based product workflows, you should ask about these things. Um, Jav is down here, the newest entrant to maybe a forming market here, but you could see they outperform everything in terms of latency and cost here. So it's definitely a strong tool in the toolbox to consider. Zero-shot classification with JEV decisions without task-specific training. It takes the same plain English criteria you put in a judge prompt, needs no labels, and returns a probability that actually has reached 98.3% accuracy without task-specific fine-tuning or labeled examples in its request, which is really promising. What might you lose when you replace an LLM judge with JEV? The biggest loss is the explanation of the reasoning. TypeSafe's docs say plainly that System 1 models don't generate explanations of their reasoning. In NearHere's test noted the same thing. A category and probabilities came back, nothing else. Depending on your use case, that could matter a lot. LM judge explanations can be incredibly valuable. That tells you not just what was wrong, but why. That provides real signal en masse back to a coding agent to automatically improve the software. JEP just gives you a probability. It's much less directional. But what Arise recommends is actually a meta for AI builders, which is why not both? Porque no los dos. Why not that? Right? All traces, this is the architecture they're kind of recommending, use both for cheap coverage and expensive explanation. So all traces, every run, every tool call or agent step you afford, you can afford to score, throw that into JEV. It's 440x cheaper. Label and probability assignment, broad measurement, no written rationale. That's what JEV does. Categorizes into confidence cases, so it keeps your score, monitors drifts, no extra spend. And then rerun a sample through an LM judge for this red area or failures or gray zone scores. And then that should go into overall a coding agent or a human plus coding agent to fix and iterate on. So the TLDR meta for AI builders with, hey, I have an evals pipeline and I have LLM judges. What do I do now that Jev has come out? The recommendation from Arise here is do both. Jev is really cheap and it's a great classifier. So for the classification use case, have it on every trace for 100% coverage. And then for things that you're unsure about, like sample failures, uncertain scores, gray zone kind of areas where you want to bring in an expert to annotate, or you want to bring in LLM judge where you use more tokens from their frontier intelligence and you need a written explanation to improve the system, bring that in. This is like a recommended use case for them here. Another view of the same. So today's default is pass/fail, route A/B, pick a label, frontier LLM judge. Split stack would be typed questions. Decision model goes to classify score route version swap this layer on its own, and then text that's got to be written. Lm that goes through the lm for tokens for summaries replies explanations reasoning, and then you pay for generation from there because again, Jev has no cost on output tokens. Lms do, so that that really is a huge benefit. New tool in the toolbox for AI builders to improve their pipelines from a cost efficiency and speed standpoint.
