// VERIFY · MOCKED ENVIRONMENTS

Deterministic tests. No burned credits, no side effects.

The sandbox proxy records your agent's real tool and API traffic as fixtures inside the trace. Replay them and the same input exercises the same path: flaky APIs frozen, paid endpoints free, refunds not actually issued.

THE PROBLEM

You can't unit-test against the live internet

Real agent tests hit real APIs: flaky search endpoints, metered LLM calls, tools with side effects. So test suites get skipped, and the first real test of a prompt change is production.

// HOW RUNAGAIN SOLVES IT

01Fixtures recorded from reality

Drop the transparent proxy into the agent's sandbox with one command, agent unchanged, and every outbound HTTP call is captured as a span and stored as a fixture, matched to its trace by trace-context propagation.

02Replay instead of re-call

Serve the stored responses back and a run becomes deterministic: same tools, same results, every time. Iterate on prompts and models without spending a cent on the tool side.

03Regression suites from production

Capture real traces, including the ones that failed, into versioned datasets. Your test cases are things that actually happened, not things you imagined might.

04CI gating on scores

The runagain-eval CLI runs an experiment in CI and fails the build when a scorer drops below threshold or regresses significantly against a baseline. Statistically, not vibes: McNemar-tested, so noise doesn't block merges.

01 TRACE

Every span, tool call and token, captured live.

02 EXPERIMENT

A/B prompt

Fork prompts, models and tools behind flags.

03 TEST

✓ mocks

✓ sims

Deterministic runs against mocks and sims.

04 EVALUATE

0.91

LLM-judge, rubric, assertion and dataset scoring.

05 IMPROVE

Diff regressions, fix the weak spans, ship.

06 RUN AGAIN

The loop closes; every pass raises the score.

Test is a loop stage: mocks and sims make it deterministic.

COMMON USE CASES

USE CASETime-travel debuggingread →USE CASERegression testing in CIread →

Be first to run again.

Book 30 minutes and see the loop on your own agents, or write to tamas@runagain.ai.