Jev vs LLMs: Where a Decision Model Beats Prompting, and Where It Doesn't

It is not a "better or worse" question

Jev and an LLM are answering different questions. Jev answers which one, how much, or how likely; an LLM answers what should be written.

	Jev	A generative LLM
Output	Typed answers with a probability per option and a confidence value	Text, optionally constrained to a JSON schema
Cost basis	Input tokens only; output is free	Input plus output tokens
Latency shape	One forward pass; questions fan out in parallel over one state	Sequentially generated tokens; reasoning models add a long hidden pass
Confidence	A calibrated number you can threshold	Usually none, or a self-reported sentence
Schema	Matches by construction	Matches until the model decides it does not
Known input	Text and structured state only, no images	Text, and images for multimodal models
Beats it at	Narrow, high-volume judgments	Generation, reasoning, explanation, arithmetic

The moment you stop needing text out, the arithmetic changes. A classifier-shaped task that produces twenty tokens of prose per item is paying for those tokens on every single item, forever.

The accuracy question, with real numbers

Marketing comparisons usually compare each vendor's favourite benchmark against the other's worst. The useful published test we found is Emil Lindfors's early-access write-up, An early-access test of TypeSafe's Jev, run on 24 responses to Norway's 2022 hearing on the salmon farming resource rent tax.

He labelled everything with Claude Fable 5.1 in two independent passes first, then compared Jev 1.13 against DeepSeek V4.1 Flash through OpenRouter — same questions in one prompt, JSON out, once with reasoning on and once with it off.

Task	Jev 1.13	DeepSeek, reasoning off	DeepSeek, reasoning on
Stance, 4 options	20 of 24	20 of 24	22 of 24
Respondent type, 6 options	21 of 23	22 of 23	23 of 23
Arguments, 192 yes/no	0.86	0.89	0.88
Substance, exact level	19 of 24	14 of 24	14 of 24

Two things are worth reading off that table. First, the author is explicit that these are agreement with a frontier model's labels, not correctness — where the reference is wrong and Jev is right, Jev scores as wrong. With 24 documents, the 95% interval on a stance number is about ±15 points, so the three columns are the same on stance and arguments. Second, the one clear win is on the ordered Score scale, 19 against 14: that is the primitive doing exactly what it was built for, and it is the kind of result you would not get from a text answer at all.

Anyone claiming on this evidence that a decision model is categorically more accurate than an LLM is overreading a 24-document sample. The interesting claim is the narrower one.

The cost question

The same test priced the work per 1,000 documents:

	Jev 1.13	DeepSeek, reasoning off	DeepSeek, reasoning on
Cost per 1,000 documents	$0.22	$1.31	$3.08
Median latency	0.32 s	2.7 s	26 s
Slowest request	1.3 s	17.9 s	250 s

Roughly a sixth and a fourteenth of the two LLM configurations, at roughly an eighth and an eightieth of the median latency. The author's own summary of the cause: dropping text generation is why the price falls that far, and the reasoning tokens bought two extra stance labels out of 24 for ten times the latency.

Those are one workload, one language, one day. Your numbers will differ — tokenizer efficiency alone moves them. The same write-up measured Jev's tokenizer at about 2.06 characters per token on Norwegian, against the ~4 characters per token you would assume from English, which cuts the usable state budget from roughly 120,000 characters to about 64,000.

Calibration is the actual difference

This is the part that decides whether you can automate. A model that is right 86% of the time and knows which 14% it is wrong on is a different tool from a model that is right 88% of the time and sounds equally certain about everything.

From the same run, on 192 argument judgments:

Jev's stated probability	Judgments	Reference agreed
0.0 – 0.1	14	0%
0.1 – 0.3	56	4%
0.3 – 0.7	41	34%
0.7 – 0.9	38	97%
0.9 – 1.0	43	98%

The direction is right at every step, and the model runs slightly underconfident at both ends — TypeSafe's own target is that outcomes assigned 0.8 should occur about 80% of the time, and the middle bin came in at 34% rather than ~50%.

The practical version of that table is a threshold. Splitting the choice questions on the top probability:

	Top probability ≥ 0.9	Below 0.9
Stance	14 of 15 agree	6 of 9 agree
Respondent type	20 of 20 agree	1 of 3 agree

That is the operating pattern: accept the confident majority, escalate the rest. TypeSafe's own SEC-filings cookbook reports 27 of 30 correct at 0.9 or above in English; the Norwegian run got 14 of 15.

The comparison runs one way only. With reasoning enabled, DeepSeek put 84 of its 192 argument answers above 0.9 — and in the 0.7–0.9 window its labels matched the reference 48% of the time. A model whose confidence is not calibrated cannot be used as a gate, no matter how good its answers are.

Where an LLM is still the right call

• Generation. Jev will not write the summary, the reply, or the commit message. TypeSafe's own documentation sends generation to a generative model.
• Reasoning and multi-hop questions. TypeSafe lists indirection as a known weakness: questions with double negatives or multiple hops cost accuracy.
• Arithmetic, dates, and counting. Documented as unreliable, and the error grows with the size of the thing being counted. Keep it in code.
• Images and audio. Jev is text-only. In TypeSafe's Jev Can't See. I Made It Guess What I Drew Anyway, Bartosz Mikulski converted 400 sketches into SVG coordinate strings and asked Jev to name them. It beat a ten-way chance baseline clearly, lost to Claude Sonnet 5, and answered "airplane" for more than half of the drawings. The same content encoded as base64 landed on chance — a useful reminder that a text model reading numbers is reading them as text.
• Anything needing an explanation. "Why did you label this one that way" is not a question a probability answers.

TypeSafe's founder makes a related argument on the LLM side that is worth knowing about, since it cuts against a common workaround: on the launch thread he argued that constrained decoding of the kind OpenAI-style structured outputs use makes models dumber, because masking invalid tokens is not the same as the model not being confused about them. Treat that as a vendor's position rather than a settled result — but it does mean "just force JSON out of a cheap model" is not automatically equivalent to a typed answer.

The pattern that works

The most useful conclusion in the Norwegian write-up is that the choice is not either/or. The author's own project used three models for three jobs:

Job	Model	Why
48 careful reference labels	Fable 5.1	Few documents, judgment calls, cost does not matter
Every document, every question	Jev	Low-cost, fast, and says when it is unsure
A second opinion on the uncertain third	DeepSeek or a person	Slower and dearer, so only where it is needed

You can run that shape today with a single OpenAI-compatible key, using a low-cost tier as the first pass and a stronger one only for the cases the first pass cannot settle:

1. First pass on the cheapest tier that clears your bar. gpt-5.6-luna at $0.022 / $0.134 per 1M, or glm-5.3-flash at $0.105 / $0.35, or deepseek-flash at $0.15 / $0.60.
2. Force the decision into a closed set in the prompt, and ask for a confidence score alongside it. Treat the score as a hint, not as a calibrated probability — that is the gap a decision model fills and prompt engineering does not.
3. Escalate only what falls below your threshold. gpt-5.6-sol from $0.297 / $1.781 across 19 routes, or gemini-3.8-flash from $0.20 / $1.00 across 7.
4. Keep the arithmetic, dates and counting in your own code, for both tiers. It is cheaper and it is correct.
5. Measure your own agreement rate before scaling. Fifty hand-labelled items will tell you more than any benchmark table, including this one.

The economics of that pattern are why routing exists as a category at all: the first pass is where nearly all the volume goes, and the escalation tier is where nearly all the quality comes from. You only need the second one for the minority you cannot classify.

Create an APIMaster account, add pay-as-you-go credit from $1, create a key in the console, and point an OpenAI-compatible client at https://apimaster.ai/v1 with any of the model IDs above. One key covers the GPT, GLM, DeepSeek, Gemini and Claude families, so the escalation path stays on the same integration. Validate a route with the model tester before you move production traffic.

FAQ

Is Jev a language model? No. TypeSafe describes it as a structured data model, and the founder's own wording on the launch thread is that it is "technically not a language model (it doesn't generate language)". It reads text and returns decisions.

Is Jev more accurate than an LLM? Not measurably, on the published evidence. In a 24-document Norwegian test, Jev and DeepSeek V4.1 Flash agreed with the reference labels at the same rate on stance and argument questions. Jev was clearly ahead on one ordered-scale task.

Is Jev cheaper than an LLM? Yes, per task — not per input token. Jev's $0.042 per 1M input tokens is undercut by gpt-5.6-luna at $0.022 on input, but Jev emits no output tokens at all, and generative models are billed for output. In the published workload, that came to $0.22 per 1,000 documents against $1.31 and $3.08.

What is "LLM as a judge" and how is it different? LLM-as-a-judge means asking a generative model to rate or label something and reading the answer out of its text. It works, but you pay for the text, you wait for the tokens, and the confidence you get back is not a calibrated probability. A decision model returns the same judgment as a typed answer you can threshold.

Can I just force JSON output from a cheap model instead? It gets you the schema, not the calibration. Constrained decoding makes the output parse; it does not tell you which answers to distrust, and TypeSafe argues it can degrade quality by masking tokens the model was genuinely uncertain about.

Which one should I use for classification? If you make a few judgments where accuracy is everything and you can review them, a good LLM is fine. If you make many judgments and need to automate, use whatever returns a usable confidence signal and escalate the uncertain ones.

Does Jev handle non-English text? It does, with caveats. TypeSafe says English is where accuracy is best and other languages including CJK are handled but not equally well. The Norwegian test above found it reading scanned council minutes correctly, and also measured tokenizer efficiency at about 2.06 characters per token — worth checking on your own language before planning around the context limit.

Can Jev see images? No. It is text-only. Converting an image to text and asking Jev about it can beat chance, but it loses badly to a model that can actually see.

Is Jev available on APIMaster? Not on sale yet — Jev is in onboarding on APIMaster, and this page will be updated once the integration is live. The models on the other side of this comparison are available now.

Which low-cost model should I start with for a first pass? gpt-5.6-luna is the cheapest tier on the marketplace at $0.022 / $0.134 per 1M tokens across 3 routes — the lowest input and output rate in this article's table. glm-5.3-flash at $0.105 / $0.35 and deepseek-flash at $0.15 / $0.60 are the next steps up. Measure on your own data before committing volume.

Sources and further reading

• Emil Lindfors — An early-access test of TypeSafe's Jev — 24 Norwegian hearing documents, Jev against DeepSeek V4.1 Flash with and without reasoning, with per-task agreement, calibration bins, cost and latency
• Bartosz Mikulski — TypeSafe's Jev Can't See. I Made It Guess What I Drew Anyway — 400 sketches as coordinate strings, chance baseline and the Sonnet 5 comparison
• TypeSafe — Models — model ID, $42/Btok pricing, rate limits, context budget and language support
• TypeSafe — Jev 1.13 jaggedness — the published scope notes on literal reading, arithmetic, dates, indirection and generation
• TypeSafe — Introducing System One Models and Jev — the 70–500ms end-to-end figure and the 40×–200× comparison
• Hacker News launch thread — the founder's comments on what Jev is not, on constrained decoding and on why output is free

Third-party test results are reproduced as their authors published them; neither author was paid for their write-up or reviewed this page. APIMaster route prices were read on September 20, 2026. Jev's APIMaster onboarding is in progress and this page will be updated as it advances.