Telemetry

OpenTelemetry-compatible tracing for PromptKit sessions, following the OpenTelemetry GenAI Semantic Conventions.

Overview

The runtime/telemetry package integrates PromptKit with the OpenTelemetry Go SDK. It provides:

• A real-time event listener that converts EventBus events into OTel spans as they occur
• Typed spans using gen_ai.operation.name as a discriminator, per the GenAI Semantic Conventions
• TracerProvider helpers for standalone OTLP export
• Propagation setup for W3C Trace Context, W3C Baggage, and AWS X-Ray headers

Because it uses the standard OTel SDK, spans are exported through any configured SpanExporter — OTLP/HTTP, OTLP/gRPC, Jaeger, Zipkin, or custom exporters.

import "github.com/AltairaLabs/PromptKit/runtime/telemetry"

Trace Structure

Each session produces a single trace. Span names follow the GenAI SIG convention {gen_ai.system} {gen_ai.operation.name} where applicable.

promptkit invoke_agent (root, SpanKindServer)
├── openai chat (SpanKindClient)
│   ├── [event] gen_ai.user.message
│   └── [event] gen_ai.assistant.message
├── execute_tool (SpanKindInternal)
├── openai chat (SpanKindClient)
│   └── [event] gen_ai.assistant.message
├── promptkit.middleware.auth (SpanKindInternal)
├── promptkit.eval.banned_words (SpanKindInternal)
├── promptkit.eval.response-quality (SpanKindInternal, instant)
├── promptkit.workflow.transition (SpanKindInternal, instant)
├── promptkit.workflow.transition (SpanKindInternal, instant)
└── promptkit.workflow.completed (SpanKindInternal, instant)

Typed Operations

Every span carries a gen_ai.operation.name attribute (where applicable) that identifies its semantic type. This follows the GenAI Agent Spans convention.

Span Name	gen_ai.operation.name	Span Kind	Description
promptkit invoke_agent	invoke_agent	Server	Root session span
{system} chat	chat	Client	LLM provider call
execute_tool	execute_tool	Internal	Tool execution
promptkit.pipeline	—	Internal	Pipeline execution
promptkit.middleware.{name}	—	Internal	Middleware execution
promptkit.eval.{name}	—	Internal	Guardrail or eval execution
promptkit.workflow.transition	—	Internal	Workflow state transition (instant)
promptkit.workflow.completed	—	Internal	Workflow terminal state (instant)

SDK Integration

The simplest way to enable tracing is via the WithTracerProvider SDK option:

import (
    sdktrace "go.opentelemetry.io/otel/sdk/trace"
    "github.com/AltairaLabs/PromptKit/sdk"
)

tp := sdktrace.NewTracerProvider(/* your exporter */)
defer tp.Shutdown(ctx)

conv, _ := sdk.Open("./app.pack.json", "chat",
    sdk.WithTracerProvider(tp),
)

When a TracerProvider is configured, the SDK automatically wires an OTelEventListener into the EventBus. All pipeline events are converted to spans in real time — no manual wiring needed.

OTelEventListener

OTelEventListener converts runtime events into OTel spans in real time. It is safe for concurrent use and tolerates out-of-order event delivery (the EventBus dispatches events asynchronously).

Constructor

func NewOTelEventListener(tracer trace.Tracer) *OTelEventListener

Session Lifecycle

func (l *OTelEventListener) StartSession(parentCtx context.Context, sessionID string)
func (l *OTelEventListener) EndSession(sessionID string)

StartSession creates a root promptkit invoke_agent span, optionally parented under the span in parentCtx. All subsequent spans for this session are children of this root. EndSession ends the root span.

OnEvent

func (l *OTelEventListener) OnEvent(evt *events.Event)

Handles a single runtime event and creates/completes OTel spans accordingly. Pass this method to EventBus.SubscribeAll:

tracer := telemetry.Tracer(tp)
listener := telemetry.NewOTelEventListener(tracer)
listener.StartSession(ctx, sessionID)

bus.SubscribeAll(listener.OnEvent)

// ... run conversation ...

listener.EndSession(sessionID)

Handled Event Types

Session span (invoke_agent)

The root span for each conversation session.

Span name: promptkit invoke_agent

Attributes follow the GenAI Agent Spans convention:

Attribute	Source	Spec reference
gen_ai.operation.name	"invoke_agent"	gen_ai.operation.name
gen_ai.system	"promptkit"	gen_ai.system
gen_ai.conversation.id	Session ID	gen_ai.conversation.id
gen_ai.agent.name	Pack name (when available)	gen_ai.agent.name
gen_ai.agent.id	Pack ID (when available)	gen_ai.agent.id

Provider events (chat)

Event	Span
provider.call.started / provider.call.completed / provider.call.failed	{system} chat span (SpanKindClient)

Span name: {provider} chat (e.g., openai chat, anthropic chat)

Attributes follow the GenAI Client Spans convention:

Attribute	Source	Spec reference
gen_ai.operation.name	"chat"	gen_ai.operation.name
gen_ai.system	Provider name	gen_ai.system
gen_ai.request.model	Model name	gen_ai.request.model
gen_ai.usage.input_tokens	Input token count	gen_ai.usage.input_tokens
gen_ai.usage.output_tokens	Output token count	gen_ai.usage.output_tokens
gen_ai.response.finish_reason	Finish reason	gen_ai.response.finish_reasons
promptkit.message.count	Number of messages	PromptKit-specific
promptkit.tool.count	Number of tools	PromptKit-specific
promptkit.provider.cost	Estimated cost (USD)	PromptKit-specific

Pipeline events

Event	Span
pipeline.started / pipeline.completed / pipeline.failed	promptkit.pipeline span (SpanKindInternal)

Attributes: promptkit.run.id, promptkit.pipeline.cost, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens

Message events

Event	Behaviour
message.created	Appended as a SpanEvent on the active provider span

Messages are not separate spans. They are attached as span events on the currently active {system} chat span, following the GenAI Events conventions. If no provider span is active, the event is attached to the root session span.

Event name: gen_ai.<role>.message (e.g., gen_ai.user.message, gen_ai.assistant.message)

Event attributes:

Attribute	Type	Description
gen_ai.message.content	string	Text content of the message
gen_ai.tool_calls	string (JSON)	Tool calls requested by assistant (present only when non-empty)
gen_ai.tool_result	string (JSON)	Tool result for tool-role messages (present only when non-nil)

Tool events (execute_tool)

Event	Span
tool.call.started / tool.call.completed / tool.call.failed	execute_tool span (SpanKindInternal)

Attributes follow the GenAI Agent Spans convention:

Attribute	Type	Spec reference
gen_ai.operation.name	"execute_tool"	gen_ai.operation.name
gen_ai.tool.name	string	gen_ai.tool.name
gen_ai.tool.call.id	string	gen_ai.tool.call.id
gen_ai.tool.call.arguments	string (JSON)	gen_ai.tool.call.arguments (omitted when nil)
gen_ai.tool.type	string	gen_ai.tool.type — "function" for regular tools, "extension" for MCP tools

Tool execution duration is captured by the span’s start/end timestamps. Success or failure is captured by the span status code. MCP tools (prefixed mcp__) are automatically detected and tagged with gen_ai.tool.type = "extension" per the MCP conventions.

Middleware events

Event	Span
middleware.started / middleware.completed / middleware.failed	promptkit.middleware.{name} span (SpanKindInternal)

Attributes: promptkit.middleware.name, promptkit.middleware.index

Validation events (guardrails)

Event	Span
validation.started / validation.passed / validation.failed	promptkit.eval.{name} span (SpanKindInternal)

Guardrail validations are traced as evaluation spans using the GenAI Evaluation Attributes. The promptkit.guardrail attribute distinguishes guardrails from other evals.

Attributes:

Attribute	Type	Description	Spec reference
gen_ai.evaluation.name	string	Validator name (e.g., banned_words)	gen_ai.evaluation.name
gen_ai.evaluation.score	float64	1.0 if passed, 0.0 if failed	gen_ai.evaluation.score
gen_ai.evaluation.explanation	string	Error message or joined violations (on failure only)	gen_ai.evaluation.explanation
promptkit.eval.type	string	Validator type (e.g., output)	PromptKit-specific
promptkit.guardrail	bool	true — distinguishes guardrails from evals	PromptKit-specific

Eval events

Event	Span
eval.completed / eval.failed	promptkit.eval.{evalID} instant span (SpanKindInternal)

Evals (assertions, LLM judges, content checks) are traced as instant evaluation spans. They share the same GenAI Evaluation Attributes as guardrails but with promptkit.guardrail = false.

Attributes:

Attribute	Type	Description	Spec reference
gen_ai.evaluation.name	string	Eval ID	gen_ai.evaluation.name
gen_ai.evaluation.score	float64	Numeric score (omitted when nil)	gen_ai.evaluation.score
gen_ai.evaluation.explanation	string	Human-readable explanation (omitted when empty)	gen_ai.evaluation.explanation
promptkit.eval.type	string	Handler type (e.g., llm_judge, contains)	PromptKit-specific
promptkit.guardrail	bool	false — distinguishes evals from guardrails	PromptKit-specific

Passed evals have span status Ok. Failed evals have span status Error with the explanation or error message.

Workflow events

Event	Span
workflow.transitioned	promptkit.workflow.transition instant span (SpanKindInternal)
workflow.completed	promptkit.workflow.completed instant span (SpanKindInternal)

Workflow spans are instant — their start and end times are both set to the event timestamp.

Transition attributes:

Attribute	Type	Description
promptkit.workflow.from_state	string	State before transition
promptkit.workflow.to_state	string	State after transition
promptkit.workflow.event	string	Trigger event
promptkit.workflow.prompt_task	string	Prompt task of the new state

Completion attributes:

Attribute	Type	Description
promptkit.workflow.final_state	string	Terminal state reached
promptkit.workflow.transition_count	int	Total number of transitions

Error handling

When a *.failed event is received, the corresponding span’s status is set to codes.Error with the error message.

Out-of-order delivery

The EventBus dispatches each Publish() in a separate goroutine, so completion events can arrive before their corresponding start events. The listener handles this transparently by buffering early completions and applying them when the start event arrives.

Tracer

func Tracer(tp trace.TracerProvider) trace.Tracer

Returns a named tracer with instrumentation scope github.com/AltairaLabs/PromptKit (version 1.0.0). If tp is nil, the global noop provider is used.

NewTracerProvider

func NewTracerProvider(ctx context.Context, endpoint, serviceName string) (*sdktrace.TracerProvider, error)

Creates a TracerProvider that exports spans via OTLP/HTTP to the given endpoint. The caller is responsible for calling Shutdown on the returned provider. Use this for standalone applications that don’t have their own OTel setup.

tp, err := telemetry.NewTracerProvider(ctx,
    "http://localhost:4318/v1/traces",
    "my-service",
)
if err != nil {
    log.Fatal(err)
}
defer tp.Shutdown(ctx)

SetupPropagation

func SetupPropagation()

Configures the global OTel text-map propagator to handle:

• W3C Trace Context (traceparent / tracestate)
• W3C Baggage
• AWS X-Ray (X-Amzn-Trace-Id)

Call this once at application startup if you need distributed trace propagation across HTTP boundaries:

telemetry.SetupPropagation()

Relevant Specifications

• OpenTelemetry Specification — core concepts (traces, spans, resources)
• OpenTelemetry Go SDK — Go instrumentation guide
• OTLP/HTTP Protocol — wire format for trace export
• GenAI Semantic Conventions — attribute naming for LLM workloads
• GenAI Client Spans — span naming for chat completions
• GenAI Agent Spans — span naming for agents and tools
• GenAI Attributes Registry — full attribute reference
• Resource Semantic Conventions — service.name, service.version
• W3C Trace Context — traceparent / tracestate headers
• AWS X-Ray Trace Header — X-Amzn-Trace-Id

See Also

• How-To: Export Traces with OTLP — end-to-end setup guide
• How-To: Prometheus Metrics — Prometheus-based monitoring
• Logging Reference — structured logging
• SDK Observability — event system overview