Setup
Install VS Code
If you haven’t already, download and install Visual Studio Code.
Install an AI assistant extension
VS Code requires an AI assistant extension that supports the Model Context Protocol (MCP). Popular options include:
- GitHub Copilot
- Continue
- Other MCP-compatible extensions
Add the Braintrust MCP server
Add the Braintrust MCP server to your VS Code settings, either in workspace settings or user settings:
-
Workspace settings - Create or edit
.vscode/mcp.jsonin your project: -
User settings - Add to your VS Code user settings (
Cmd+,/Ctrl+,→ Search for “mcp”):
YOUR_BRAINTRUST_API_KEY with your actual API key.VSCode also supports OAuth authentication. If you omit the headers field, VSCode will prompt you to authenticate via OAuth when you first use the server.Usage
Once configured, your AI assistant can access Braintrust data through the MCP server. You can fetch experiment results, query logs, log data, and more. See MCP documentation for details. Example prompts:- “Show me my recent experiments”
- “Query the last 10 logged requests with errors”
- “What’s the average latency for the summarizer prompt today?”
Troubleshooting
MCP server not appearing
MCP server not appearing
- Verify the JSON configuration syntax is correct
- Check that you’re using an AI assistant extension that supports MCP
- Try restarting VS Code completely (not just reloading the window)
Authentication fails
Authentication fails
- Verify your API key is correct (no extra spaces or quotes)
- For OAuth: Clear browser cache and try incognito mode
- Ensure you can log into Braintrust in your browser
Connection timeouts
Connection timeouts
- Check your internet connection
- Corporate networks may need to allowlist
api.braintrust.devand*.braintrust.dev - Verify the URL is exactly
https://api.braintrust.dev/mcp(no trailing slash)
Next steps
- Run evaluations: Check out the evaluation guide to learn evaluation patterns.
- Explore MCP tools: See the MCP documentation for all available commands.
- Query with SQL: Learn how to query with SQL for complex data analysis.