Customer support evaluation with Jev
Triage already picked a queue. This pack decides what happens next: is the customer asking for a refund, does a drafted reply violate policy, should a macro fire? Jev judges; macros, agents, or humans act. Jev does not send the reply.
This unofficial page is the evaluation slice of the customer support actions pack. Intent: apply the Jev (TypeSafe System One) decision model to customer support actions evaluation. Primary search language: Customer support Jev evaluation. Confirm patterns on docs.typesafe.ai. This site does not sell, issue, or proxy TypeSafe keys. Use a credential you already have from the console or a documented gateway.
Independent angle (cover ≠ clone): Full support-action pack (contracts → thresholds → handoff → audit) beats a single support-triage-choice recipe. This pack is refunds, commitments, and policy replies — not a clone of rival recipe IA, and not a duplicate of the triage routing pack.
Customer support use-case context
Evaluation for customer support actions is a frozen harness, not a vibe check and not an opinion-blog “Jev review.” Labels: send_macro / hold / escalate gold from senior agents, plus invented-promise gold. We publish no unofficial accuracy.
Hub: Support triage hub. Compare, when the other tool is the real job: ticket queues.
Evaluation inputs
Replay the same contract you ship:
{
"ticket": { "id": "T-2201", "latest": "You promised a refund last week. Charge is still there." },
"draft": { "body": "I issued a $400 goodwill credit." },
"account": { "plan": "pro", "refunds_ytd_cents": 0 },
"policy": { "refunds": "Auto-refund language only if amount extracted in code is under cap.", "promises": "Do not invent credits." }
}Freeze questions, criteria, and jev-1.13.0. Record the response model.
Decision signals and actions
Score these, not a blog-grade star rating:
- False-sends on planted invented credits
- False-holds (handle time)
- Overlap errors vs the triage pack — department misroutes belong on triage eval, not here
Pair auto-act errors with handoff rate. If the support action gate never acts, you have not evaluated customer support actions — you have evaluated a human queue.
Do not treat a Noul of 0.5 as a “medium” customer support actions score — it means yes and no are equally likely. Conjunctions stay in your code.
Guardrails and escalation
Promote a threshold only when the harness says auto-act error ≤ SLA and reviewers still catch the residual. TypeSafe’s confidence-gated examples use a lower bar for recoverable reads than for irreversible actions. Those numbers are illustrations. For customer support actions, treat send_customer_reply as the high bar (sending a customer reply or issuing a credit flag). Tune on labels — see offline evaluation.
Evaluation and rollout notes
After any criteria edit, rerun before production. Cookbook lifts you see on TypeSafe pages are vendor claims — re-measure on your ticket thread + drafted replys. Pin jev-1.13.0 (the versioned id) after you fit thresholds. jev-latest and the marketing line jev-1.13 can move. Log the response model. TypeSafe’s published list price for jev-1.13 is $0.042 per million input tokens (vendor claim — confirm on the models page); output tokens are free on that same page. Unused distractors still bill as input.
Official Python and JavaScript SDKs read TYPESAFE_API_KEY and retry documented 429/529. This site does not sell, issue, or proxy TypeSafe keys. Use a credential you already have from the console or a documented gateway.
Pack map
| Slice | Page |
|---|---|
| Graph and primitives | decision workflow |
What may enter state |
input contracts |
| What to gather first | evidence collection |
| Atomic rules | policy checks |
| Act / review / abstain | confidence thresholds |
| Reviewer payload | human handoff |
| What to persist | audit trail |
| How it breaks | failure modes |
| Labeled replay | you are here |
| Shadow → canary | production rollout |
FAQ
Will jev.pro publish a leaderboard for this use case? No. Measure on your labels. Vendor cookbook figures stay labeled as vendor claims.
What must stay frozen? Questions, criteria, and the pinned model id. Aliases can move.
Where is the rest of the Customer support pack? Start with Customer support failure modes and Customer support production rollout. Cluster hub: Use cases.
Is this the same as the support-triage recipe rivals publish? No. Triage routes the ticket. This pack gates actions after routing. Cover the intent; do not clone their slug or outline.
Can Jev replace the agent? No. It classifies and scores. See the support triage hub.
What this page does not claim
- Not a helpdesk or payments product.
- No invented CSAT/AHT numbers.
- Not official TypeSafe.
- Official TypeSafe status, or that jev.pro issues API keys.
- That a schema-constrained answer is automatically factually correct.
Disclaimer
This is an independent unofficial site and is not affiliated with TypeSafe AI; official documentation is available at https://docs.typesafe.ai.
Primary documentation: https://docs.typesafe.ai. Hub: Use cases.
Sources
Public TypeSafe or adjacent documentation only. No private claims.