# Catch the confident lie before your customer does.

Built-in judges score sampled production runs for factuality and faithfulness against the context the agent actually retrieved. A grounding failure shows up as a red score on the trace and a Slack alert, not as a support ticket.

## The problem: Hallucinations do not throw exceptions

The run succeeds, the response reads beautifully, and the cited policy does not exist. Nothing in your error tracking fires because nothing errored. Hallucination is a quality failure wearing a success status code.

## How RunAgain does it

1. **Judges built for grounding.** Ready-made LLM judges score factuality against a reference (subset, superset, equal or disagree) and faithfulness against the documents the run actually retrieved, with no prompt to write.
2. **Online, on real traffic.** Attach the judge as an online eval, scope it to the agents where truth matters most, and sample a percentage of runs so coverage and cost stay balanced.
3. **The failure, in context.** A failed grounding score sits inline on the trace next to the retrieval spans, so you can see what the model was given versus what it claimed, in one view.
4. **Alerts and drift, not dashboards.** Failed evals post to Slack as coalesced digests with deep links, and a drift monitor on the faithfulness score catches slow degradation after a model or prompt change.

## FAQ

**Q: How do you detect hallucinations without ground truth?**

A: Faithfulness judging compares the response against the context the agent retrieved in that same run, so the reference is the run's own evidence. Where curated ground truth exists, the factuality judge scores against it as well.

**Q: Do I have to run a judge on every single request?**

A: No. Online evals have a coverage slider and scoping, so you might judge 10% of all traffic but 100% of the agents where a wrong answer is expensive.

**Q: How trustworthy is an LLM judging another LLM?**

A: Measurably trustworthy or not: annotation queues collect human labels on the same score names, and the judge-alignment table shows how often the judge agrees with your reviewers before you rely on it.

## Built on these solutions

- [Simulation & evals](https://runagain.ai/solutions/agent-simulation-evals.md)
- [Monitoring & alerting](https://runagain.ai/solutions/agent-monitoring-alerting.md)
- [Agent observability](https://runagain.ai/solutions/agent-observability.md)

Book a demo: https://cal.com/tamas-szuromi/30min · Get in touch: tamas@runagain.ai

---

Markdown mirror of https://runagain.ai/use-cases/hallucination-detection for agents and LLMs. Append .md to any runagain.ai page URL for its markdown twin. Overview: https://runagain.ai/llms.txt · For agents: https://runagain.ai/for-agents.md