Top Repos· Jev-Based· Harness· Last updated 2026-09-21

Adilmp / does-jev-confidence-mean-anything

“A calibration audit of TypeSafe's Jev: does a decision model's stated confidence mean what it claims? 8,000 judgments against human annotations, $0.05.”

Jev-Based rank #227In Harness #80

• 0Stars
• —Stars Δ / 1d
• —Stars Δ / 7d
• —Stars Δ / 30d
• 0Forks
• 0Watchers

GitHub ↗Official docs ↗Back to ranking

TL;DR

Adilmp/does-jev-confidence-mean-anything showed up as Jev / System One related work: Verified TypeSafe Jev / System One work — Jev or System One is the core idea.

The author describes it as: “A calibration audit of TypeSafe’s Jev: does a decision model’s stated confidence mean what it claims? 8,000 judgments against human annotations, $0.05.” That is their claim, not a bench this site ran.

Who it’s for

• Practitioners who want Jev as a function call, not a chat model
• Anyone who needs the unofficial line: jev.pro does not issue keys and is not TypeSafe

What’s inside

Top-level entries on GitHub at generation time: .env.example, .gitignore, .nojekyll, 1_get_data.py, 2_ask_jev.py, 3_measure.py, 4_chart.py, 5_thresholds.py, 6_recalibrate.py, 7_wording.py, 8_slides.py, README.md, index.html, jev.py. Read those before you trust a one-line install.

How to use

1. Clone and follow the author’s Python toolchain. Do not invent a package name.

git clone https://github.com/Adilmp/does-jev-confidence-mean-anything.git
cd does-jev-confidence-mean-anything

2. If the project calls hosted Jev, you still need a TypeSafe console key or a documented gateway.
3. Send a complete state and typed questions. Code owns control flow after the probabilities.

Hosted Jev still looks like a function call — wrapper or not:

{
  model: "jev-1.13.0",
  state: { source: "Adilmp/does-jev-confidence-mean-anything", brief: "..." },
  questions: {
    gate: { type: "noul", instructions: "Should we act on this state?" },
    route: { type: "choice", instructions: "Pick one next system.", criteria: ["code", "review", "human"] },
  },
}

Try this

Put Jev on one real brief

Wire Adilmp/does-jev-confidence-mean-anything so one real state (ticket, diff, or utterance) becomes a Choice plus a Noul. Do not ask Jev to write the user-facing reply.

Schema-safe ≠ correct

On a case Adilmp/does-jev-confidence-mean-anything gets wrong, write down the schema-legal answer it returned and the human label. Do not “fix” it by asking Jev to generate an explanation.

FAQ

Does this replace the TypeSafe SDK?

Only if the author implemented a full client. Most harnesses still call the vendor SDK or raw HTTP.

Are the star chips live?

Star, fork, and watcher chips refresh from the public GitHub API via a scheduled job. The prose on this page is editorial.