Set up the MCP server
Braintrust provides a hosted MCP server with OAuth 2.0 authentication. Add it to your AI tool’s configuration:Claude Code
Cursor
Click to automatically add Braintrust: Add to Cursor Or manually add to.cursor/mcp.json:
VS Code
- Open Command Palette (Ctrl+Shift+P or Cmd+Shift+P)
- Run MCP: Add Server
- Select HTTP and enter:
- URL:
https://api.braintrust.dev/mcp - Name:
Braintrust
- URL:
- Click Add
Windsurf
Add tomcp_config.json:
Claude Desktop
- Open Settings (gear icon)
- Navigate to Connectors > Add custom connector
- Enter:
- Name:
Braintrust - URL:
https://api.braintrust.dev/mcp
- Name:
Authenticate
The MCP server uses OAuth 2.0 with PKCE for secure authentication. On first connection, the OAuth flow opens automatically:- Your AI tool triggers the OAuth flow
- A browser window opens asking you to log into Braintrust
- After authentication, review the consent screen showing the redirect URL
- Click Yes, I trust this URL - Proceed
- Return to your AI tool
Available tools
The MCP server provides these tools to your AI assistant:Search documentation
search_docs - Find guides, API references, and code examples:
- “How do I create a custom scorer?”
- “Show me examples of BTQL queries”
- “What’s the difference between experiments and project logs?”
Resolve objects
resolve_object - Convert names to IDs and vice versa:
- “Find the ID for my ‘sentiment-analysis’ experiment”
- “What’s the name of experiment abc123?”
List recent objects
list_recent_objects - Browse projects, experiments, and datasets:
- “Show me my recent experiments in the ‘chatbot’ project”
- “List datasets in the recommendation engine project”
Infer schemas
infer_schema - Understand data structure:
- “What fields are available in my experiment data?”
- “Show me the schema for production logs”
Query with BTQL
btql_query - Execute BTQL queries:
- “Compare accuracy scores between my GPT-4 and Claude experiments”
- “Show experiments where factuality score was below 0.7”
- “What were the costs for my recent chatbot experiments?”
Summarize experiments
summarize_experiment - Get performance summaries:
- “Summarize the results of my latest A/B test”
- “Compare my experiment against the baseline”
Generate permalinks
generate_permalink - Create shareable links:
- “Create a link to share my experiment results”
- “Generate a permalink to the customer-reviews dataset”
Example workflows
Analyze experiment results
Ask your AI assistant:Debug production issues
Share findings with teammates
Self-hosted instances
For self-hosted Braintrust, use your API URL:Set the
MCP_SERVER_URL environment variable to your API URL for OAuth discovery. Terraform deployments handle this automatically.Troubleshooting
OAuth fails or gets stuck: Clear browser cache and try incognito mode. For SSO, ensure you’re logged into your SSO provider. Invalid client errors: Verify the URL is exactlyhttps://api.braintrust.dev/mcp (no trailing slash).
Connection timeouts: Check internet connection. Corporate networks may need to allowlist api.braintrust.dev and *.braintrust.dev.
MCP server not appearing: Restart your AI tool and verify JSON configuration syntax.
Access denied: Verify you can access the project in the Braintrust web interface.
BTQL queries failing: Test queries in the web interface first. Use infer_schema to verify column names.
Next steps
- Use the Data API for programmatic access
- Query with SQL for complex data analysis
- View logs in the web interface
- Run evaluations and analyze with MCP