Observability

Use local records to investigate failed runs, or connect an OpenTelemetry collector to track model usage, tool failures, and latency across runs. Local records are enabled by default; exporting requires an OTLP/HTTP endpoint. The default export is traces only.

Local records and signals

Local telemetry events are NDJSON under ~/.jazz/telemetry/events/YYYY-MM-DD.ndjson, retained for 90 days by default (expired date partitions are removed during normal writes). Operational logs live under ~/.jazz/logs/. The local event stream includes run start and terminal state, LLM usage and retries, tool outcomes, periodic process samples, and CLI command completion. It is recorded even when no OTLP endpoint is configured.

OTLP logs are telemetry event records. They are separate from the local operational log file. Routine tool diagnostics at INFO level contain IDs, outcomes, and durations, not command text, arguments, results, or error messages. Local tool audit records retain a bounded, redacted argument shape; protect ~/.jazz as sensitive data.

Configure an OTLP collector

Set a base endpoint to enable export, and select the signals you need:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

{
  "telemetry": {
    "otlp": {
      "signals": ["traces", "logs", "metrics"],
      "serviceName": "jazz"
    }
  }
}

The base endpoint gains /v1/traces, /v1/logs, and /v1/metrics. A signal-specific endpoint must include its full path. Config values take precedence over environment variables. OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES supply resource identity; set deployment.environment.name through resource attributes when useful for filtering. Jazz sets service.version from its own package version; a resource override is ignored. Jazz uses OTLP/HTTP, so point it at an HTTP receiver, normally port 4318.

An invalid endpoint or a selected signal without an endpoint produces a startup error log naming the config field or environment variable. Jazz disables OTLP export for that process while continuing the agent run. Endpoint values are omitted from the diagnostic because they may contain credentials. An invalid explicit config value does not silently fall back to an environment value.

telemetry.otlp.signals defaults to ["traces"]. With only a signal-specific endpoint, explicitly select that signal. The supported environment overrides are OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, OTEL_EXPORTER_OTLP_LOGS_ENDPOINT, OTEL_EXPORTER_OTLP_METRICS_ENDPOINT, OTEL_EXPORTER_OTLP_HEADERS, and their per-signal *_HEADERS forms. Jazz percent-decodes header values. Keep credentials in environment or a secret store rather than committing them in config.

Backend recipes

SigNoz

Point Jazz at your SigNoz OTLP/HTTP ingestion endpoint and select all three signals:

export OTEL_SERVICE_NAME=jazz
export OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.<region>.signoz.cloud:443
export OTEL_EXPORTER_OTLP_HEADERS=signoz-ingestion-key=<your-ingestion-key>

For self-hosted SigNoz, use its collector’s HTTP endpoint, commonly http://<collector>:4318. Search the service name in Services, Logs Explorer, and Metrics Explorer. SigNoz documents the endpoint and header format. Jazz does not use OTEL_EXPORTER_OTLP_PROTOCOL; its transport is fixed as described above.

Datadog

For production, point Jazz at a local Datadog Agent or OpenTelemetry Collector OTLP/HTTP receiver. Enable the Agent’s OTLP HTTP receiver, then set:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

Select ["traces", "logs", "metrics"] in Jazz config. The Datadog Agent accepts traces and metrics when OTLP ingestion is enabled; OTLP log ingestion needs separate log collection and OTLP log settings. Datadog recommends Agent or Collector ingestion for production. Direct cloud intake has signal-specific endpoints and payload limits; use those endpoints only after configuring each signal deliberately.

Prometheus

Prometheus receives metrics only. Start it with --web.enable-otlp-receiver, then configure the full metrics URL and select only metrics:

{
  "telemetry": {
    "otlp": {
      "metricsEndpoint": "http://localhost:9090/api/v1/otlp/v1/metrics",
      "signals": ["metrics"]
    }
  }
}

The Prometheus OTLP receiver is disabled by default. It accepts OTLP/HTTP protobuf at /api/v1/otlp/v1/metrics. Send traces and logs to another receiver if needed.

Langfuse

Langfuse receives traces, not OTLP logs or metrics. Configure its full traces endpoint and Basic authentication:

export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://cloud.langfuse.com/api/public/otel/v1/traces
export OTEL_EXPORTER_OTLP_HEADERS="authorization=Basic $(printf '%s:%s' "$LANGFUSE_PUBLIC_KEY" "$LANGFUSE_SECRET_KEY" | base64),x-langfuse-ingestion-version=4"
export OTEL_RESOURCE_ATTRIBUTES=langfuse.environment=production

Keep signals at ["traces"]. Use your region’s host or self-hosted host as appropriate. The root run is an agent observation, LLM calls are generation observations, and tool calls are tool observations. A conversation is mapped to the Langfuse session ID on each span; nested subagent runs share the parent trace. Token counts include cache usage when the provider reports it. Langfuse calculates cost from its own model pricing, so check your model’s price configuration before relying on cost totals. See Langfuse’s OTLP mapping.

Langfuse’s best-practices guide recommends one trace per unit of work, a session for the conversation, interleaved generations and tools, and stable observation names. Jazz follows that structure for top-level runs. It does not send input, output, or reasoning text, so content-based evaluators and dataset experiments will see empty fields. The x-langfuse-ingestion-version=4 header enables current ingestion behavior; validate the resulting trace in a non-production environment if migrating existing Langfuse dashboards or evaluators. The v4 migration guide explains the changed mappings. Set langfuse.environment as a resource attribute to keep development and production traces separate.

Trace and metric fields

Jazz uses gen_ai.provider.name, gen_ai.request.model, gen_ai.response.model, gen_ai.operation.name, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and cache or reasoning token attributes when available. langfuse.observation.type marks agent, generation, and tool spans. Jazz IDs and additional counters use the jazz.* namespace. Dashboard queries using the old gen_ai.system field should migrate to gen_ai.provider.name.

Metric names and low-cardinality dimensions are:

Metrics use cumulative temporality and a 30-second export interval by default. The service resource includes a process-specific service.instance.id unless you set one. Prometheus may normalize dots in metric names to underscores; inspect its target before writing queries.

Short-lived CLI runs export a zero jazz.agent.runs sample at run start and a terminal sample on shutdown. This gives Prometheus a before/after pair for the run counter even when the 30-second interval never fires. Query each instance’s rate before summing across instances, and use a window that contains both samples. For Prometheus’s default name translation, sum by (status) (rate(jazz_agent_runs_total{job="jazz"}[5m])) shows run activity. The model- and tool-specific counters may still have only a terminal sample in a short process; use traces or OTLP logs for exact short-run event counts. Rates for those counters are most useful when Jazz runs as a long-lived daemon. Prometheus recommends cumulative temporality for its OTLP receiver; its delta-to-cumulative mode is currently experimental.

Privacy and delivery

Jazz’s shared telemetry event stream does not contain prompt, completion, tool argument, or tool result text. String attributes are bounded and known credential-bearing fields are redacted. No setting adds that text to events or OTLP exports. Never assume an OTLP collector is a private boundary; configure only approved endpoints.

Traces and logs have independent disk-backed queues under <telemetry.storagePath>/otlp-outbox. They contain payloads but no authentication headers, use private file permissions, and retry independently of local event files. Pending payloads are bound to both endpoint and configured headers: changing credentials at the same URL leaves old entries undelivered until they expire, rather than sending them to a different account. Queue limits default to 32 MiB and seven days; oldest pending payloads are dropped with a warning when those limits are reached. Retryable HTTP responses (429, 502, 503, 504) and network failures get bounded retries, honoring Retry-After. Other HTTP failures are dropped with a warning. An OTLP partial-success response is acknowledged and reported, not retried. An ambiguous connection failure may produce a duplicate after retry or restart; consumers should use stable trace/span IDs where possible.

Metrics use the OpenTelemetry SDK’s cumulative aggregation and periodic exporter. They are flushed on clean shutdown but are not stored in the disk outbox, so a crash or prolonged outage can lose a metric interval. Logs and telemetry also flush on normal CLI shutdown. Observability failures are best-effort and do not block agent actions.

Set telemetry.otlp.maxQueuedBytes, telemetry.otlp.maxQueueAgeMs, and telemetry.otlp.metricExportIntervalMs to tune delivery. Set telemetry.otlp.enabled to false to retain local records without export, or telemetry.enabled to false to disable both local telemetry events and OTLP. Operational logs remain governed by logging config.