Moderation decision workflow with Jev
UGC needs a category, a severity, and an allow/review/remove decision. Jev scores the text you provide against your policy excerpt. Code enforces.
This unofficial page is the decision workflow slice of the content moderation pack. Intent: apply the Jev (TypeSafe System One) decision model to content moderation decision workflow. Primary search language: Moderation Jev decision workflow. Confirm patterns on docs.typesafe.ai. This site does not sell, issue, or proxy TypeSafe keys. Use a credential you already have from the console or a documented gateway.
Independent angle (cover ≠ clone): Policy-as-criteria + confidence abort + human pack — not a clone of a moderation-API landing page or rival recipe IA.
Moderation use-case context
Content moderation is a workflow, not a chat. Assemble a narrow state, ask the primitives below, and let the moderation worker branch. TypeSafe’s docs say a good question is a snap decision a knowledgeable person could make in a few seconds — not an open-ended analysis of the user-generated post or message.
Hub: Use cases. Compare, when the other tool is the real job: moderation APIs.
Decision Workflow inputs
Keep only fields the questions name:
{
"post": { "id": "p-209", "text": "…", "locale": "en" },
"policy": { "hate": "…", "spam": "…", "illegal": "…" },
"author": { "strikes": 1, "age_gate": "18+" }
}Point instructions at post.text, policy.hate, policy.spam. Drop image/video bits (not accepted — run a vision stack elsewhere and pass text labels); the author’s entire post history as one blob.
Decision signals and actions
| Id | Type | Job |
|---|---|---|
category |
Choice | ok / spam / hate / harassment / illegal / other |
severity |
Score | nuisance → severe harm |
allow |
Noul | Would a trained moderator leave this up given policy.*? |
All of these share state and run in parallel. Code owns the graph:
def moderate(ans):
cat = ans["category"]
if cat.confidence < FLOOR or cat.choice == "other":
return "review"
if cat.choice == "illegal" or ans["severity"].score >= 2:
return "remove_and_escalate"
if ans["allow"].noul < T_ALLOW:
return "review"
return "keep"Do not treat a Noul of 0.5 as a “medium” content moderation score — it means yes and no are equally likely. Conjunctions stay in your code.
Guardrails and escalation
TypeSafe’s confidence-gated examples use a lower bar for recoverable reads than for irreversible actions. Those numbers are illustrations. For content moderation, treat remove_or_ban as the high bar (removing content or issuing a ban). Tune on labels — see offline evaluation.
Low confidence, category == other, or a policy miss → human or safe default; do not remove content or issue a ban.
Evaluation and rollout notes
Shadow: Show Jev next to the current vendor API; take the old action.
Canary: Auto-remove only high-precision spam; keep hate/illegal on humans.
Pin jev-1.13.0 (the versioned id) after you fit thresholds. jev-latest and the marketing line jev-1.13 can move. Log the response model. TypeSafe’s published list price for jev-1.13 is $0.042 per million input tokens (vendor claim — confirm on the models page); output tokens are free on that same page. Unused distractors still bill as input.
Official Python and JavaScript SDKs read TYPESAFE_API_KEY and retry documented 429/529. This site does not sell, issue, or proxy TypeSafe keys. Use a credential you already have from the console or a documented gateway.
Pack map
| Slice | Page |
|---|---|
| Graph and primitives | you are here |
What may enter state |
input contracts |
| What to gather first | evidence collection |
| Atomic rules | policy checks |
| Act / review / abstain | confidence thresholds |
| Reviewer payload | human handoff |
| What to persist | audit trail |
| How it breaks | failure modes |
| Labeled replay | evaluation |
| Shadow → canary | production rollout |
FAQ
Does Jev execute the moderation worker action? No. It returns typed answers. Your moderation worker code calls queues, models, or humans.
Why several questions in one request? TypeSafe’s fan-out pattern: extra questions are cheap versus another HTTP call. Category + severity + allow on one post, one call. Extra Nouls for each policy atom beat one vague “is this bad?” Score.
Where is the rest of the Moderation pack? Start with Moderation input contracts and Moderation confidence thresholds. Cluster hub: Use cases.
Should we replace our moderation vendor with Jev? Only after a labeled bake-off you run. This page does not publish one. See Jev vs moderation APIs.
Can Jev moderate images? Not directly. State is text. Run a vision system, put labels/transcripts in state, then ask typed questions.
What this page does not claim
- Not a trust-and-safety certification.
- No published precision/recall.
- Not official TypeSafe.
- Official TypeSafe status, or that jev.pro issues API keys.
- That a schema-constrained answer is automatically factually correct.
Disclaimer
This is an independent unofficial site and is not affiliated with TypeSafe AI; official documentation is available at https://docs.typesafe.ai.
Primary documentation: https://docs.typesafe.ai. Hub: Use cases.
Sources
Public TypeSafe or adjacent documentation only. No private claims.