Observe

See what happens in production

Full visibility into every agent interaction. Every request traced, every response scored, every regression caught before your customers notice.

Conversation
MessagesDetailsReviewActivityRaw
Topics
Task
The user is asking about the status of an order.
Input
LLM

Hi! What's the status of order #12345? I placed it last week.

Assistant
get_order_status
call_abc123
Parameters
order_id

12345

Output
LLM

I found your order! Here's the status:

Order #12345

  • Status: In Transit
  • Shipped: January 20, 2026
  • Estimated Delivery: January 25, 2026
  • Carrier: UPS
  • Tracking: 1Z999AA10123456784

Your package is on its way! Would you like me to help with anything else?

What is agent observability?

Traditional monitoring tracks uptime and latency. Agent observability tracks whether LLM responses are good.

Trace every step of every LLM call: the prompt, the response, tool calls, retrieved context, and the final output. Score that traffic against quality criteria automatically, and get alerted when something drifts.

Seamless setup

Two paths to full visibility

Go from zero observability to full tracing with minimal effort. Use the gateway as your default capture layer, and the SDK for deeper context on your agent’s tool calls, planning steps, or loops.

Initialize once and every LLM call is traced automatically.
import { initLogger } from "braintrust";
import OpenAI from "openai";

// Call once at startup — all LLM calls are traced automatically
initLogger({
  apiKey: process.env.BRAINTRUST_API_KEY,
  projectName: "My Project",
});

const client = new OpenAI();
const response = await client.responses.create({
  model: "gpt-5-mini",
  input: "What is the capital of France?",
});

SDK tracing

Enable auto-instrumentation once and every call to a supported provider is logged with inputs, outputs, latency, tokens, and cost. Wrap your own functions for deeper context on tool calls, planning steps, and agent loops. Logs are buffered and flushed asynchronously to keep overhead low.

Trace LLM calls
Set the base URL to https://gateway.braintrust.dev.
import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.braintrust.dev",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

async function main() {
  const response = await client.responses.create({
    model: "gpt-5-mini",
    input: [{ role: "user", content: "Say hello!" }],
  });

  console.log(response.output_text);
}

main();

Gateway

Change your base URL to the gateway and pass your Braintrust API key. No other code changes needed. You get a unified API across OpenAI, Anthropic, Google, AWS, and other providers, with automatic caching and observability on every request.

Use the gateway
Monitoring in production

Everything you need to observe agents in production. From the first span to the final score.

Customer support agent28.62s, 5,914 tok, $0.009
Understand request3.12s, 812 tok, $0.001, GPT-5.5
search_orders0.86s
Ask for missing detail4.74s, 1,106 tok, $0.002, GPT-5.5
track_package1.18s
Explain shipping status5.88s, 1,442 tok, $0.002, GPT-5.5
create_followup_ticket1.34s
Confirm next steps6.27s, 1,779 tok, $0.003, GPT-5.5
Inline prompt6.46s, 672 tok, $0.001
Conversation quality scorer5.97s, 672 tok, $0.001, GPT-5.5
Topics0s
Pipeline193.65s
Facets192.34s, 7,446 tok, GPT-5.5
Embeddings0.04s, GPT-5.5
Topics0s

Traces

Each request becomes a tree of nested spans: LLM calls, tool invocations, and retrieval steps. See the full chain from input to output, with latency, token counts, and cost attached to every step.

Log your first trace
Spans

Dashboards

Track latency, cost, quality, and volume in real time. Break down LLM spend by model, feature, or team, and build views for engineering, product, or leadership.

Explore monitoring
Latency

Alerts

Set thresholds on any metric, like score drops, latency spikes, or error rate changes. Get notified before your customers do.

Set up alerts
Created
Input
Factuality
Helpfulness
Sentiment
1
Jan 25 2:34 PM
Hi! What's the status of order #12345?
94%
91%
82%
2
Jan 25 2:31 PM
Can I return my order from last week?
91%
88%
75%
3
Jan 25 2:28 PM
What are your shipping rates to Canada?
88%
86%
79%
4
Jan 25 2:25 PM
I need to change my delivery address
96%
93%
85%
5
Jan 25 2:22 PM
Do you have any discounts for bulk orders?
89%
87%
88%
6
Jan 25 2:19 PM
My package arrived damaged
92%
84%
45%
7
Jan 25 2:16 PM
What payment methods do you accept?
97%
92%
81%
8
Jan 25 2:08 PM
Where is my refund? It's been 2 weeks!
78%
55%
32%
9
Jan 25 2:01 PM
How do I track my order?
95%
90%
77%
10
Jan 25 1:58 PM
Can I pick up my order in store?
93%
89%
84%
11
Jan 25 1:55 PM
When will order #88219 ship?
91%
87%
71%

Online scoring

The same scorers you use in offline experiments run automatically on production traffic. Every conversation is scored as it happens.

Score production traffic
Customer stories

How the best AI teams run in production

Kylie Czajkowski, Agent Experience Manager

“We get to iterate directly in development and then ship it to production.”

Fred Zhao, Software Engineer

“This discipline is what Braintrust was built for.”

Allen Kleiner, AI Engineering Lead

“Loop helps us understand trace details that would be impossible to scan manually.”

Start building

Create a free account and start monitoring in minutes.