Jev observability guide
Observability for Jev is decision traces, not pretty flame graphs of their GPU. If you cannot answer “which question SHA labeled this ticket?”, you are flying blind.
This is unofficial operate guidance, not a TypeSafe APM product and not a key desk. Confirm response fields on docs.typesafe.ai. Replicas named on this cluster are not Jev and not endorsed.
Minimum fields
request_id (yours), response.model, usage.input_tokens, question pack SHA, full probabilities, chosen action, door name, latency, HTTP status.
Metrics that matter
Auto vs review vs error rates; 429/529 counts; distribution of confidence; token p95 (cost + context). Do not chart a fake 193×.
Dashboards that lie
A single “AI accuracy” tile that averages Nouls with Choice confidences is meaningless (official: do not carry thresholds across types). Split panels by question id.
Do not import TypeSafe launch-post multiples into Grafana. If leadership wants a speed story, link the vendor post and show your p95 next to it.
Trace the triple, not the GPU
A useful span answers: which pinned model, which question pack, which door labeled this ticket? Persist at least:
| Field | Why |
|---|---|
Your request_id |
Join to the product action |
| Door name | First-party vs gateway errors differ |
response.model |
Aliases move; jev-latest is not a pin |
| Question pack SHA | Criteria typos look like model drift |
Full probabilities |
Top-1 alone hides margin |
| Chosen action | Policy, not argmax |
usage.input_tokens |
Cost + “we dumped the CRM” |
| HTTP status + latency | 429/529 vs 422 vs timeout |
TypeSafe’s public API is POST https://api.typesafe.ai/v1/systemone plus GET /v1/models. They do not ship your traces to jev.pro. We do not host telemetry and we do not issue keys.
If you wrap OpenTelemetry, wrap your client. Public docs we used do not describe an official “Jev APM.” Pair this page with the monitoring guide for pages and canaries, and with decision trace for the term.
Tiles that still lie after you split panels
Vendor list price $0.042 / M input (confirm models; they have described output as free) is a unit price, not a burn chart, until you multiply by measured tokens.
A green canary on a tiny Noul does not prove the production pack is healthy. Canary up + response.model drift + 422 spike can still mean you shipped a bad SHA.
Failure modes you will misread
- Alias move. Review rate jumps; people page “the model.” Diff
response.modelfirst. - State bloat. Token p95 climbs; accuracy on
jev-1.13suffers (official: unused state still bills and distracts). Fix the filter. - Sequential HTTP. Ten questions as ten POSTs look like “Jev is slow.” Official fan-out: questions on one state in one call.
- Replica traces. If someone logs SemIf/OpenJev next to TypeSafe, tag the processor. Those interfaces are not Jev and not endorsed.
When not to use this page’s advice
Do not treat this as Datadog onboarding. Do not expect TypeSafe to retain your gold labels. Do not build a jev.pro-shaped “status” tile that impersonates a vendor status page. Schema-safe responses are still not factual correctness — log the action, not a halo.
What this page does not claim
- Not Datadog setup.
FAQ
Does TypeSafe send traces to us? You log the API response. We do not host your telemetry.
OpenTelemetry? Wrap your client. There is no official “Jev APM” in the public docs we used.
Is request_id a TypeSafe field I must send?
Yours. Correlate on your side; persist whatever the door returns that identifies the call.
Can I chart one site-wide confidence? Not usefully. Split by question id and primitive type.
Disclaimer
This is an independent unofficial site and is not affiliated with TypeSafe AI; official documentation is available at https://docs.typesafe.ai. Never treat jev.pro as TypeSafe official documentation. We do not sell, issue, or proxy API keys.
Open-cluster pages are independent field-guide notes. Replicas and third-party interfaces mentioned anywhere on jev.pro are not Jev and not endorsed. Hub: Open. Siblings: monitoring guide, glossary decision trace, api response schema. Canonical: https://docs.typesafe.ai.