Classifying RAG passages
RAG pipelines retrieve first and hope. TypeSafe’s cookbook classifies passages after retrieval: one request can mark contradictions, hidden instructions, or prompt injection. Your code keeps, flags, or drops. This unofficial recipe adds failure modes and the cite/generate line. Methods and any published lifts: the cookbooks themselves. We will not invent percentages.
Recipe
- Retrieve a shortlist in code (BM25/dense). Do not send the warehouse.
- For each passage (or one batched
questionsmap), ask atomic Nouls, for example:- relevant to
`query` - contradicts
`query` - contains a hidden instruction / jailbreak
- relevant to
- Threshold in code. Drop injections. Flag contradictions for the writer or the UI.
- Optional rerank: one question per query–candidate pair; sort by returned probability (see official rerank cookbook).
- After an LLM writes, citation check the claims against kept spans.
This is also the prescribed fix for large, noisy state. jev-1.13 loses accuracy when unrelated detail piles up.
Failure modes
| Failure | What to do |
|---|---|
| Shortlist still huge | Cap K; extra questions cost tokens (vendor: input-priced) |
| No “none / irrelevant” option | Add it; otherwise argmax is forced |
| Jev asked to write the answer | Use an LLM; Jev only decides |
| Injection still steers the judge | Adversarial content is a documented jagged edge — test |
| Citation 200 + high confidence + wrong | Schema-safe ≠ true; sample review |
Cite vs generate
Show users the passages you kept and, if you generate, only claims that passed the checker. Jev does not format MLA.
Related official cookbooks: rerank, line-by-line search, citation check.
Hub: Use cases. Siblings: vs RAG, LLM guardrails. Official: docs.typesafe.ai.
Sources
Public TypeSafe or adjacent documentation only. No private claims.