Public TypeSafe evals
The launch post describes workflow evals: a fixed compute graph (“workflow”), with large external models as reference probabilities. TypeSafe says Jev sits on a Pareto frontier in those plots and that homepage multiples (they mention 193.6× faster and 444.6× cheaper) come from this work and are likely high-end real-world gains.
This page is for builders crossing from curious to production-minded Jev use. jev.pro is an independent unofficial field guide — not affiliated with TypeSafe AI. Cover intents; do not clone rival IA. Confirm facts on docs.typesafe.ai.
Open-cluster pages are independent field-guide notes. Replicas and third-party interfaces mentioned anywhere on jev.pro are not Jev and not endorsed.
They also list caveats in the same article: workflows were authored by people on their capabilities team; the reference mix (they name large OpenAI and Anthropic models) may bias the comparison; LLM baselines use TypeSafe’s System One wrapper.
They invite readers to a workflow evals site for examples, disagreements, full queries, and each workflow.
How jev.pro will treat evals
- We link to TypeSafe’s published materials.
- We do not invent a jev.pro leaderboard.
- We do not “average” blog charts into a single unofficial IQ.
If you run your own harness, publish your labels and prompts. Do not claim they are TypeSafe official numbers. Measure decision quality on actions.
TypeSafe published evals only
This URL exists so we cover the “where are the Jev numbers?” intent without standing up a jev.pro leaderboard. Allowed outbound:
- The launch post’s description of workflow evals (fixed compute graph; large external models as reference probabilities).
- TypeSafe’s own caveats in that article (capabilities-team authorship; named reference mix; System One wrapper on LLM baselines; high-end multiples).
- Whatever workflow evals site they point to for examples, disagreements, and full queries — follow their link; we will not invent a hostname that might move.
- Official docs and jaggedness pages when they discuss eval methodology.
Not allowed here: averaging blog charts into one unofficial IQ; third-party replica benches; SemIf/OpenJev agreement tables labeled as Jev. Those projects are not Jev and not endorsed. See openjev.
If you cite 193.6× faster or 444.6× cheaper in an internal deck, label them TypeSafe workflow-eval claims with their caveats. Do not imply you measured them. Do not put them in Grafana as a KPI. Your harness — evaluation guide — scores policy actions on a dated door + pin + SHA.
Vendor list price and latency sentences stay vendor. Schema-safe answers in an eval row are still not factual correctness.
When not to use this page’s advice
Do not ask us for a ranked table of “best Jev versions.” Do not scrape a rival eval hub into this slug. Do not treat open replicas as points on TypeSafe’s Pareto plot.
See also
FAQ
Can I put 193.6× in my pitch deck?
If you do, label it a TypeSafe workflow-eval claim with their caveats. Do not imply you measured it.
Are open replicas on that Pareto front?
No. Label them not Jev.
Where is the evals URL?
Follow TypeSafe’s launch post and docs — we will not invent a hostname that might move.
Will jev.pro host a public leaderboard later?
No. That would impersonate a vendor eval surface we do not run.
Disclaimer
This is an independent unofficial site and is not affiliated with TypeSafe AI; official documentation is available at https://docs.typesafe.ai. Never treat jev.pro as TypeSafe official documentation. We do not sell, issue, or proxy API keys.