# Score every run, with the method that fits.

LLM-judge, assertion, rubric, dataset and trajectory scoring, applied online to live traffic or offline against versioned datasets. Scope each eval to the agents and runs it should grade, and calibrate judges against human labels.

## The problem: LLMs are easy to eval. Agents aren't.

A benchmark scores one answer. An agent plans, calls tools and loops; the answer can look fine while the trajectory was a nine-tool detour that got lucky. Scoring agents means scoring the path, not just the destination.

## How RunAgain solves it

1. **A full scoring catalog.** A dozen heuristic scorers (exact match, BLEU, ROUGE-L, JSON diff, regex…), built-in LLM judges for factuality, correctness, relevance, conciseness and safety, weighted multi-criterion rubrics, and dataset comparison. Or bring your own judge prompt.
2. **Trajectory evals, not just outputs.** Model-free tool-trajectory scoring grades how well the agent used its tools, with recovery invalidation so an agent that errored and recovered isn't punished like one that failed.
3. **Online, scoped, sampled.** Attach evals to live traffic and scope them to all runs, one agent, or any metadata attribute, with a coverage slider so an expensive judge samples 10% while a cheap check runs on everything.
4. **Humans in the loop.** Annotation queues collect human ratings on real traces, and the judge-alignment table shows how well each LLM judge agrees with your reviewers, so you know when to trust the machine grade.

## Related solutions

- [Mocked environments](https://runagain.ai/solutions/mocked-environments.md)
- [Experimentation](https://runagain.ai/solutions/agent-experimentation.md)
- [Continuous improvement](https://runagain.ai/solutions/continuous-improvement-loop.md)

Book a demo: https://cal.com/tamas-szuromi/30min · Get in touch: tamas@runagain.ai

---

Markdown mirror of https://runagain.ai/solutions/agent-simulation-evals for agents and LLMs. Append .md to any runagain.ai page URL for its markdown twin. Overview: https://runagain.ai/llms.txt · For agents: https://runagain.ai/for-agents.md