# The JSON parsed. That does not mean it is right.

Structured-output evals run property checks on the answers your agents emit and watch the shape itself, so a missing field, a renamed key or a quietly narrowed enum raises an alert instead of a downstream exception.

## The problem: Schema drift breaks the system next door

The agent's output feeds code: a parser, a workflow, a database. When a model update renames a field or drops one, the agent looks fine and the consumer breaks. The error report comes from the wrong service.

## How RunAgain does it

1. **Property checks on live output.** The structured-output eval scores each sampled run's JSON answer against your property checks, with heuristic scorers like valid_json, json_diff and regex available for the simple cases.
2. **Shape-drift detection.** Beyond pass or fail, structured_output.changed watches the output shape over time and flags when it shifts, catching the model update that reshaped your data before a parser does.
3. **Scoped where structure matters.** Run strict checks only on the agents and routes that emit machine-consumed output, scoped by agent or metadata, with sampling for the rest.
4. **Gate it before it ships.** The same checks run as experiment scorers, so a prompt or model change that breaks the schema fails in CI instead of in production.

## FAQ

**Q: How do I validate that my agent always returns valid JSON?**

A: Attach a structured-output eval (or the valid_json heuristic scorer) as an online eval on the relevant agent. Every sampled run gets a score, failures alert to Slack, and the trace shows the offending output.

**Q: What is shape drift?**

A: A change in the structure of the output rather than one bad value: fields appearing or disappearing, types changing, keys renamed. RunAgain tracks the shape over time and alerts when it diverges.

**Q: Can schema checks block a deploy?**

A: Yes. Run them as scorers in a CI-gated experiment with runagain-eval and a threshold, and a schema-breaking change exits non-zero before merge.

## Built on these solutions

- [Simulation & evals](https://runagain.ai/solutions/agent-simulation-evals.md)
- [Monitoring & alerting](https://runagain.ai/solutions/agent-monitoring-alerting.md)
- [Debug AI agents](https://runagain.ai/solutions/debug-ai-agents.md)

Book a demo: https://cal.com/tamas-szuromi/30min · Get in touch: tamas@runagain.ai

---

Markdown mirror of https://runagain.ai/use-cases/structured-output-validation for agents and LLMs. Append .md to any runagain.ai page URL for its markdown twin. Overview: https://runagain.ai/llms.txt · For agents: https://runagain.ai/for-agents.md