Runtime

Core framework powering all PromptKit tools

What is the Runtime?

The Runtime is the foundational layer that provides:

Who Should Use Runtime?

The Runtime is for advanced users who want to:

Note: Most users should use the SDK instead. Use Runtime directly only if you’re extending PromptKit.

Quick Start

package main

import (
    "context"
    "github.com/AltairaLabs/PromptKit/runtime/pipeline/stage"
    "github.com/AltairaLabs/PromptKit/runtime/providers/openai"
)

func main() {
    // Create provider
    provider := openai.NewOpenAIProvider(
        "openai", "gpt-4o-mini", "",
        openai.DefaultProviderDefaults(),
        false,
    )

    // Build stage-based pipeline
    pipeline := stage.NewPipelineBuilder().
        Chain(
            stage.NewProviderStage(provider, nil, nil, &stage.ProviderConfig{
                MaxTokens:   1500,
                Temperature: 0.7,
            }),
        ).
        Build()

    // Execute
    input := make(chan stage.StreamElement, 1)
    msg := types.Message{Role: "user"}
    msg.AddTextPart("Hello!")
    input <- stage.NewMessageElement(msg)
    close(input)

    output, _ := pipeline.Execute(context.Background(), input)
    for elem := range output {
        if elem.Text != nil {
            fmt.Print(*elem.Text)
        }
    }
}

Next: Build Your First Pipeline Tutorial

Documentation by Type

Tutorials (Learn by Doing)

Step-by-step guides for extending Runtime:

  1. First Pipeline - Build your first pipeline
  2. Multi-Turn Conversations - State management
  3. MCP Integration - Add MCP tools
  4. Validation & Guardrails - Content safety
  5. Production Deployment - Deploy pipelines
  6. Advanced Patterns - Complex pipelines

How-To Guides (Accomplish Specific Tasks)

Focused guides for specific Runtime tasks:

Explanation (Understand the Concepts)

Deep dives into Runtime architecture:

Reference (Look Up Details)

Complete API documentation:

Key Concepts

Stage Interface

Implement custom processing stages:

type Stage interface {
    Name() string
    Type() StageType
    Process(ctx context.Context, input <-chan StreamElement, output chan<- StreamElement) error
}

Pipeline Architecture

Streaming request processing flow:

Input → [Stage 1] → [Stage 2] → [Stage N] → Output

Each stage runs in its own goroutine, enabling concurrent streaming.

Stage Types

Different stage behaviors:

TypePatternExample
Transform1:1 or 1:NValidation, enrichment
AccumulateN:1VAD buffering, collection
Generate0:NLLM streaming, TTS
SinkN:0State save, metrics

Tool Execution

MCP tool integration:

type Tool interface {
    Execute(ctx context.Context, params map[string]interface{}) (interface{}, error)
    Schema() *ToolSchema
}

Architecture

graph TB
    subgraph "SDK Layer"
        SDK[SDK]
    end

    subgraph "Runtime Layer"
        Pipeline[Pipeline]
        Stages[Stages]
        Providers[Providers]
        Tools[Tool Execution]
        Validators[Validators]
    end

    subgraph "External Services"
        OpenAI[OpenAI]
        Anthropic[Anthropic]
        MCP[MCP Servers]
    end

    SDK --> Pipeline
    Pipeline --> Stages
    Stages --> Providers
    Stages --> Tools
    Stages --> Validators

    Providers --> OpenAI
    Providers --> Anthropic
    Tools --> MCP

    style Runtime Layer fill:#f3e5f5

Use Cases

For Framework Contributors

For Advanced Developers

For System Integrators

Pipeline Modes

Runtime supports three pipeline configurations:

Text Mode

Standard HTTP-based LLM interactions:

Message → StateStoreLoad → PromptAssembly → Provider → StateStoreSave → Response

VAD Mode (Voice Activity Detection)

For voice applications using text-based LLMs:

Audio → AudioTurn → STT → Provider → TTS → Audio

ASM Mode (Audio Streaming)

For native multimodal LLMs with real-time audio:

Audio/Text → DuplexProvider → Audio/Text

Core Types

StreamElement

type StreamElement struct {
    Text      *string
    Audio     *AudioData
    Message   *types.Message
    ToolCall  *types.ToolCall
    Metadata  map[string]interface{}
    Error     error
    Timestamp time.Time
}

Message

type Message struct {
    Role    string
    Content string
    Parts   []ContentPart
}

Provider Response

type Response struct {
    Content   string
    ToolCalls []ToolCall
    Usage     *Usage
    Metadata  map[string]interface{}
}

Design Principles

Streaming First

Runtime is designed for streaming from the ground up:

Extensibility

Runtime is designed to be extended without modifying core code:

Performance

Optimized for production use:

Type Safety

Leverage Go’s type system:

Getting Help