AI monitoring (Insights)

Insights is LLM observability built on the tracing pipeline. The official Sentry SDKs' AI-agents and MCP integrations emit OpenTelemetry gen_ai.* and mcp.* spans; Errora projects those spans into a queryable table and gives you two dashboards.

Setup

There is no separate product to enable — turn on tracing and the AI integration for your stack.

import sentry_sdk

sentry_sdk.init(
    dsn="…",
    traces_sample_rate=1.0,       # AI traffic is low-volume and high-value
    send_default_pii=False,       # set True only if you want prompts recorded
)

The Python SDK instruments OpenAI, Anthropic, LangChain and the Agents SDKs automatically once tracing is on. In JavaScript, Sentry.init({ tracesSampleRate: 1 }) plus the openAIIntegration() / anthropicAIIntegration() / vercelAIIntegration() integration does the same.

For MCP servers, the SDK's MCP integration emits mcp.* spans per JSON-RPC method — the same spans Errora's own MCP server produces.

:::caution Prompts and PII Prompt and completion text is only recorded when the SDK is configured to send it (send_default_pii=True or the integration's explicit option). Leave it off unless you have a reason — prompts routinely contain user data. :::

Manual spans

If you call a model directly, wrap it yourself:

with sentry_sdk.start_span(op="gen_ai.chat", name="summarize") as span:
    span.set_data("gen_ai.system", "openai")
    span.set_data("gen_ai.request.model", "gpt-4o-mini")
    response = client.chat.completions.create(...)
    span.set_data("gen_ai.usage.input_tokens", response.usage.prompt_tokens)
    span.set_data("gen_ai.usage.output_tokens", response.usage.completion_tokens)

The AI Agents dashboard

• Agent runs over time, with duration
• LLM calls — count and latency
• Tokens by model, split cached / not cached
• Tool calls — which tools your agent actually reaches for
• Recent runs, each opening a per-run span timeline: the agent's steps, the model calls it made and the tools it invoked, in order

The MCP Server dashboard

• Traffic over time
• By client — which agent is calling you
• By method and transport
• Most-used tools, resources and prompts

API

GET /api/v1/projects/{project_id}/insights/agents
GET /api/v1/projects/{project_id}/insights/agents/runs
GET /api/v1/projects/{project_id}/insights/agents/runs/{trace_id}
GET /api/v1/projects/{project_id}/insights/mcp

Troubleshooting

Symptom	Cause / fix
Dashboards are empty	Tracing is off, or the SDK's AI integration isn't active — check that transactions contain gen_ai.* spans
Token counts missing	The provider response didn't carry usage, or a manual span didn't set gen_ai.usage.*
Runs look truncated	The trace was sampled away mid-run; raise traces_sample_rate for AI workloads
Prompts visible when they shouldn't be	An integration option is sending PII — audit send_default_pii and per-integration flags