Skip to content
PromptKit Logo PromptKit

Arena Tutorials

Step-by-step learning paths to master PromptArena from beginner to advanced.

Learning Path

Section titled “Learning Path”

Work through these tutorials in order for the best learning experience:

Beginner

Section titled “Beginner”
**[Tutorial 1: Your First Test](01-first-test)**

Install PromptArena, write your first test scenario, configure a provider, and run tests. Perfect starting point for newcomers.

Time: 20 minutes

Intermediate

Section titled “Intermediate”
**[Tutorial 2: Multi-Provider Testing](02-multi-provider)**

Test the same scenario across OpenAI, Claude, and Gemini. Learn to compare providers and optimize for cost vs. quality.

Time: 30 minutes

**[Tutorial 3: Multi-Turn Conversations](03-multi-turn)**

Build complex conversation flows with context retention. Test customer support dialogs, information gathering, and conversation branching.

Time: 45 minutes

Advanced

Section titled “Advanced”
**[Tutorial 4: Testing MCP Tools](04-mcp-tools)**

Test LLMs that use Model Context Protocol tools for database queries, API calls, and system interactions.

Time: 45 minutes

**[Tutorial 5: CI/CD Integration](05-ci-integration)**

Integrate Arena into GitHub Actions, GitLab CI, and other pipelines. Set up quality gates and deployment automation.

Time: 60 minutes

**[Tutorial 6: Duplex Voice Testing](06-duplex-testing)**

Test bidirectional voice conversations with real-time audio streaming. Learn duplex mode, turn detection, and self-play with TTS.

Time: 45 minutes

What’s the Difference?

Section titled “What’s the Difference?”

Tutorials are learning-oriented lessons that guide you through completing projects:

  • ✅ “Build your first LLM test from scratch”
  • ✅ “Set up multi-provider testing step-by-step”
  • ✅ “Create a CI/CD pipeline for LLM tests”

Looking for something else?

  • How-To Guides - Task-focused recipes for specific problems
  • Explanation - Conceptual understanding and design decisions
  • Reference - Complete technical specifications

Prerequisites

Section titled “Prerequisites”

Before starting, ensure you have:

  • Go 1.23+ installed
  • Git for cloning the repository
  • At least one LLM API key (OpenAI, Anthropic, or Google)
  • Basic command-line knowledge

Getting Help

Section titled “Getting Help”

Stuck on a tutorial?

  1. Check the How-To Guides for specific solutions
  2. Review the Reference Documentation for complete specs
  3. Visit GitHub Discussions
  4. Read the FAQ (coming soon)

What You’ll Build

Section titled “What You’ll Build”

By completing all tutorials, you’ll have:

  • ✅ A working LLM test suite
  • ✅ Multi-provider comparison tests
  • ✅ Complex conversation flows
  • ✅ Tool/function calling tests
  • ✅ Automated CI/CD pipeline
  • ✅ Voice/duplex streaming tests

Ready to get started? Begin with Tutorial 1 →