Want to learn evals? buildevals.com →

// USE CASE · TIME-TRAVEL DEBUGGING

Go back to the run that broke, and step through it.

Every run is preserved as a full trace: what the model saw, what each tool returned, in order, nothing truncated. Recorded fixtures let you replay the past deterministically, change one variable, and watch the same moment play out differently.

THE PROBLEM

You cannot debug what already evaporated

The agent failed yesterday. The API responses it saw are gone, the context it built is gone, and the log line says only that something went wrong. Reproducing the bug means recreating a moment in time, and normal tooling throws that moment away.

// HOW RUNAGAIN DOES IT

01The full moment, preserved

Every span keeps its inputs and outputs, with large payloads offloaded to blob storage instead of truncated. What the model saw at step twelve is exactly what you read at step twelve.

02Deterministic replay

The sandbox proxy records every outbound API and tool response as a fixture inside the trace. Replay serves yesterday's real responses back, so the run unfolds identically, every time, without touching live systems.

03Change one variable

Rerun the same moment with a different prompt or model against the same fixtures, then diff the two runs. When only one thing changed, the cause is not a guess.

04The past becomes a test

Capture the failing run into a versioned dataset and it is a regression case forever. The bug you time-traveled to fix cannot quietly come back.

00:01.2  span.start plan · model=claude-fable-5
00:01.9  tool.call web_search("return policy france")
00:02.8  tool.result 200 · 4 documents · 1.2kb
eval.inline faithfulness 0.94 ✓
00:03.1  tool.call db.query(orders.recent)
00:03.7  tool.result 200 · 12 rows
00:04.0  span.start generate
eval.inline grounding 0.41 ✗ · cited 30-day, source says 14-day
00:04.6  drift.check baseline Δ +0.9% ok
00:05.1  span.end run_a91f · 4.21s total
00:01.2  span.start plan · model=claude-fable-5
00:01.9  tool.call web_search("return policy france")
00:02.8  tool.result 200 · 4 documents · 1.2kb
eval.inline faithfulness 0.94 ✓
00:03.1  tool.call db.query(orders.recent)
00:03.7  tool.result 200 · 12 rows
00:04.0  span.start generate
eval.inline grounding 0.41 ✗ · cited 30-day, source says 14-day
00:04.6  drift.check baseline Δ +0.9% ok
00:05.1  span.end run_a91f · 4.21s total

The run as it happened, replayable line by line.

FREQUENTLY ASKED

What exactly is recorded for each run?

The full OpenTelemetry trace: every LLM call, tool call and step with timing, status, token counts, cost and complete input and output payloads. Large payloads are stored in blob storage and referenced, not truncated.

Can I replay a run without hitting real APIs?

Yes. The sandbox proxy captures outbound HTTP responses as fixtures matched to the trace. Replaying against fixtures is deterministic and free: same tool results, same path, no side effects.

How is this different from reading logs?

Logs describe what code chose to print; a trace preserves the actual data flow, ordered and complete. And logs cannot be re-executed. A trace with fixtures can, which turns debugging from archaeology into an experiment.

BUILT ON THESE SOLUTIONS

More use cases: llm cost tracking · reduce llm costs · regression testing in ci · prompt a/b testing · hallucination detection · conversation tracing · llm-judge calibration · tool-call failure analysis · structured output validation · tracing coding agents · agent & llm alerting

Be first to run again.

Book 30 minutes and see the loop on your own agents, or write to tamas@runagain.ai.