This example demonstrates using the replay provider to run scenarios against pre-recorded sessions without making any API calls.

What’s Included

Running the Example

From the repo root:

# Build promptarena (if not already built)
go build -o /tmp/promptarena ./tools/arena/cmd/promptarena

# Run the replay scenario
/tmp/promptarena run --config examples/session-replay/config.arena.yaml

Or with go run:

go run ./tools/arena/cmd/promptarena run \
  --config examples/session-replay/config.arena.yaml

Note: Run from the repo root since file paths are relative to the working directory.

Expected Output

The scenario runs against the recorded session and verifies assertions:

Running scenario: geography-replay with provider: replay-geography

Turn 1: What is the capital of France?
  Response: The capital of France is Paris...
  Assertions: PASS (content includes "Paris", "capital")

Turn 2: What is the population of Paris?
  Response: The city of Paris proper has a population of approximately 2.1 million...
  Assertions: PASS (content includes "million", "population")

Turn 3: What river flows through Paris?
  Response: The Seine River flows through Paris...
  Assertions: PASS (content includes "Seine")

Result: ALL ASSERTIONS PASSED

Use Cases

  1. Deterministic Testing: Verify prompt changes don’t break expected outputs
  2. CI/CD Integration: Run tests without API costs or rate limits
  3. Debugging: Replay problematic sessions to investigate issues
  4. Baseline Comparison: Compare new model outputs against recorded baselines

Recording Your Own Sessions

  1. Run a scenario with a real provider (e.g., gemini-flash)
  2. Export the session to a recording file
  3. Update the replay provider to use your recording

Configuration Options

The replay provider supports:

additional_config:
  recording: path/to/recording.json  # Required
  timing: instant | realtime | accelerated
  speed: 2.0  # For accelerated mode
  match: turn | content