Setup
Install Claude Desktop
If you haven’t already, download and install Claude Desktop.
Add the Braintrust MCP server
Follow the Claude Desktop documentation to create a custom connector with the following details:
- Name:
Braintrust - URL:
https://api.braintrust.dev/mcp
Claude Desktop uses OAuth 2.0 for authentication. You don’t need to provide an API key in the connector configuration - you’ll authenticate when you first use the server.
Usage
Once configured and authenticated, Claude can access Braintrust data through the MCP server. You can fetch experiment results, query logs, log data, and more. See MCP documentation for details. Example prompts in Claude Desktop:- “Show me my recent Braintrust experiments”
- “Query the last 10 logged requests with errors in my production logs”
- “What’s the average latency for the summarizer prompt today?”
- “Summarize my latest A/B test results and create a permalink to share”
Troubleshooting
OAuth flow doesn't open
OAuth flow doesn't open
- Ensure you’re using the latest version of Claude Desktop
- Check your default browser settings
- Try clicking the authentication link manually if it appears in the chat
Authentication fails or gets stuck
Authentication fails or gets stuck
- Clear your browser cache and try incognito mode
- For SSO users: ensure you’re logged into your SSO provider first
- Verify you can access Braintrust in your browser
- Try removing and re-adding the connector
Connector not appearing in conversation
Connector not appearing in conversation
- Verify the connector is saved in Settings > Connectors
- Check that the URL is exactly
https://api.braintrust.dev/mcp(no trailing slash) - Restart Claude Desktop
- Try starting a new conversation
Access denied errors
Access denied errors
- Verify you can access the project in the Braintrust web interface
- Check that you’re authenticated as the correct user
- Try re-authenticating through the OAuth flow
Next steps
- Run evaluations: Check out the evaluation guide to learn evaluation patterns.
- Explore MCP tools: See the MCP documentation for all available commands.
- Query with SQL: Learn how to query with SQL for complex data analysis.