Devin Desktop - Braintrust

Devin Desktop is Cognition’s AI-native IDE, formerly known as Windsurf. The Braintrust MCP server connects to Devin Desktop, giving its coding agent access to your Braintrust projects, experiments, and logs.

Setup

Install Devin Desktop

If you haven’t already, install Devin Desktop.

Add the Braintrust MCP server

Edit ~/.codeium/windsurf/mcp_config.json and add the Braintrust server:

{
  "mcpServers": {
    "braintrust": {
      "serverUrl": "https://api.braintrust.dev/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_BRAINTRUST_API_KEY"
      }
    }
  }
}

Replace YOUR_BRAINTRUST_API_KEY with your actual API key.

Restart Devin Desktop

Close and reopen Devin Desktop to load the new MCP server configuration.

Usage

Once configured, Devin Desktop’s agent can access Braintrust data through the MCP server. You can fetch experiment results, query logs, log data, and more. See MCP documentation for details. Example prompts:

“Show me my recent Braintrust experiments”
“Query the last 10 logged requests with errors”
“What’s the average latency for the summarizer prompt today?”
“Summarize my latest A/B test results”

Troubleshooting

MCP server not appearing

Verify the JSON configuration syntax is correct (no trailing commas, proper quotes)
Check that the file path is exactly ~/.codeium/windsurf/mcp_config.json
Try restarting Devin Desktop completely

Authentication fails

Verify your API key is correct (no extra spaces or quotes)
Ensure you can log into Braintrust in your browser
Check that the Authorization header is properly formatted

Connection errors

Verify the URL is exactly https://api.braintrust.dev/mcp (no trailing slash)
Check your internet connection
Corporate networks may need to allowlist api.braintrust.dev and *.braintrust.dev

Configuration not loading

Ensure the JSON file is valid (use a JSON validator)
Check file permissions: chmod 644 ~/.codeium/windsurf/mcp_config.json
Look for error messages in Devin Desktop’s developer console

Next steps

Run evaluations: Check out the evaluation guide to learn evaluation patterns.
Explore MCP tools: See the MCP documentation for all available commands.
Query with SQL: Learn how to query with SQL for complex data analysis.

VS Code

Codex

⌘I

​Setup

​Usage

​Troubleshooting

​Next steps

Setup

Usage

Troubleshooting

Next steps