Telemetry

Spans

Every provider call emits one OTel span carrying the gen_ai.* semantic conventions (model, token usage, finish reason) plus costhelm.* attributes the exporter world doesn’t have: the computed cost in USD, the principal dimensions, the routing role and tier, the confidence score, escalation count, cache-hit status, and the ledger row id — so a span links back to the exact priced row it produced.

Point it at any OTLP/HTTP collector:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318   # base URL; /v1/traces is appended
OTEL_SERVICE_NAME=costhelm-gateway
COSTHELM_TRACE_UI=http://localhost:16686            # dashboard deep-links into Jaeger

deploy/docker-compose.observability.yml starts a local Jaeger. Use port 4318 (OTLP/HTTP) — gRPC 4317 would need an exporter that isn’t a declared dependency.

Content capture is off by default

Prompt and completion text is the highest-PII payload the gateway handles. Spans always carry usage and cost; text only with COSTHELM_OTEL_CAPTURE_CONTENT=1, and only against a collector you trust.

The in-process trace ring

The observability dashboard serves traces from the gateway’s own bounded ring rather than querying Jaeger back: the page must still work when the collector is down, and the gateway knows things the exporter drops. The trace ids are the same ids Jaeger holds, so every row deep-links into the real waterfall. GET /v1/traces/recent, GET /v1/traces/{id}.

The refusal log

A budget refusal writes no ledger row (nothing billed) and opens no span (the span starts after admission) — correct behaviour, and a blind spot, because the most important thing a budget controller does is the thing you cannot see afterwards. So refusals are observed at the HTTP boundary by an ASGI middleware keeping a bounded ring of 402 responses with their envelopes intact: GET /v1/refusals.