Use Braintrust with your coding agent

With a coding agent, you can write and ship code quickly. But to improve what you're building, you need to incorporate evidence from production, like what's failing, which patterns keep coming up, and whether your evals are looking good.

The Braintrust Model Context Protocol (MCP) server connects the coding agent you already use to the production traces, experiments, datasets, prompts, scorers, and project configuration in your Braintrust organization. The MCP can create and test evaluators, build datasets, run evals, configure Topics, build monitor views, and set up alerts without moving data between tools or sharing context manually. Move from investigating a problem to acting on what it finds, all from your terminal.

Use production traces to improve your agent

Suppose a product support chatbot starts returning claims about your product that are untrue. You can ask your coding agent to:

1. Query recent production traces and identify examples of the behavior.
2. Create an evaluator that detects those unsupported claims.
3. Test the evaluator against representative traces and refine it.
4. Add the failing cases to a regression dataset with corrected responses as expected values.
5. Run an eval that compares the current behavior with a proposed change.

Each step is connected to the underlying Braintrust objects. In your terminal, you'll get permalinks to inspect and share the traces, datasets, scorers, and experiments it creates.

Large SQL query results can be returned through a signed URL instead of being placed in the model context. This lets your agent work with production-scale results without filling its context window.

What you can do with the MCP

Investigate production behavior

Query experiments, datasets, and production logs with Braintrust SQL. Your agent can infer schemas, inspect common field values, summarize an experiment, and compare results with a baseline.

Example prompts:

• "Show me the last 10 logged requests with errors."
• "Compare accuracy, latency, and cost between these experiments."
• "What metadata fields are available in this dataset?"

Build eval workflows

Create versioned prompts and evaluators, test evaluators against representative traces, edit datasets, and run evals. Evaluators can return numeric scores or categorical classifications.

Example prompts:

• "Create a scorer that detects unsupported claims, then test it against these traces before saving it."
• "Add these failed traces to my regression dataset and use the corrected responses as expected values."
• "Run an eval comparing the current prompt with this proposed version, then summarize the quality, latency, and cost differences."

Running an eval may execute code or call an AI provider, so it can incur compute and model usage. Dataset deletions are permanent and should be reviewed before approval.

Configure observability and automations

Your coding agent can configure Topics, build monitor views, and manage automations from the same conversation it uses to investigate production behavior.

For Topics, it can create and test preprocessors and facets against real traces, enable Topics for new traffic, and process historical data with a rewind. For monitoring, it can preview charts and create or update saved views. It can also configure alerts for matching logs, environment changes, and thresholds over recent project data, or schedule recurring investigations.

Example prompts:

• "Write a preprocessor that extracts the customer request and agent response from this trace shape, then test it on 20 traces."
• "Create a monitoring view with daily request volume, error rate, latency, and cost."
• "Alert the team when the error rate exceeds 2% over a one-hour window."

Instrument your application

Ask your coding agent to:

Install the Braintrust SDK and add tracing to my app.

The agent can identify the language and frameworks in your project, install the appropriate SDK, configure auto-instrumentation, run the application, verify that traces reach Braintrust, and provide a permalink to view them.

Installation guidance is available for Python, TypeScript, Go, Java, Ruby, and C#.

Search documentation and follow guided workflows

Search Braintrust documentation without leaving your coding agent. Braintrust MCP also includes workflow skills for Topics, evaluators, and automations.

Tools define what an agent can do. Skills provide the sequence, validation steps, and decision points needed to complete the work reliably. Additional resources cover SDK installation, SQL, URL formats, and experiments.

Choose between MCP and the CLI

MCP connects Braintrust directly to an AI tool, while the CLI makes Braintrust available inside a shell. Choose between them based on what your AI tool can access and where the workflow needs to run.

Use MCP when your AI tool can connect to Braintrust but does not have access to an authenticated shell, as is often the case with chat applications. It's a good fit when you want an agent to reason over Braintrust data and take several connected actions in a conversation without installing and maintaining the bt CLI in its execution environment.

For repeatable work in scripts, CI, local files, and shell pipelines, use the Braintrust CLI. The bt CLI provides deterministic commands for instrumentation, data syncing, and eval execution. Coding agents with shell access can call those commands too.

If your AI tool supports both, either can work. Choose the interface that is more reliable and efficient for the task and best fits where the workflow needs to run.

Connect your MCP client

Braintrust MCP works with Claude Code, Codex, Cursor, and other MCP-compatible clients.

The server uses streamable HTTP and supports API key and OAuth 2.0 authentication. Write tools use the permissions of the authenticated Braintrust account. Configure your MCP client to require confirmation before it runs write operations.

Organizations on the US data plane connect to https://api.braintrust.dev/mcp.

Organizations on the EU data plane use https://api-eu.braintrust.dev/mcp. Self-hosted organizations can use the MCP URL shown in their data plane settings.

Get started

Connect your preferred coding agent with the Braintrust MCP setup guide. If you need more help for your specific use case, book a demo.