Skip to main content
The AI Proxy provides unified access to AI providers through a single interface. Configure provider API keys and custom endpoints at the organization level to enable access across playgrounds, experiments, and production deployments.

​
Add provider API keys

Configure API keys for standard providers:
  1. Navigate to Settings > Organization > AI providers.
  2. Click the provider you want to configure.
  3. Enter your API key.
  4. Click Save.
Configured providers are immediately available throughout your organization without users needing individual API keys.

​
Supported providers

Standard providers include:
  • OpenAI (GPT-4o, GPT-4o-mini, o4-mini, etc.).
  • Anthropic (Claude 4 Sonnet, Claude 3.5 Sonnet, etc.).
  • Google (Gemini 2.5 Flash, Gemini 2.5 Pro, etc.).
  • AWS Bedrock (Claude, Llama, Mistral models).
  • Azure OpenAI Service.
  • Third-party providers (Together AI, Fireworks, Groq, Replicate, etc.).
See the full list in Use the AI Proxy.

​
Add custom providers

Configure custom models or private endpoints:
  1. Navigate to Settings > Organization > AI providers.
  2. Click + Custom provider.
  3. Enter provider details:
    • Name: Display name.
    • Model name: Identifier for code (e.g., gpt-4o-acme).
    • Endpoint URL: API endpoint.
    • Headers: Optional authentication headers.
    • Flavor: Chat or completion.
    • Format: OpenAI, Anthropic, Google, Window, or JS.
    • Streaming: Whether endpoint supports streaming.
    • Multimodal: Whether model accepts images.
    • Input cost: Price per million input tokens.
    • Output cost: Price per million output tokens.
  4. Click Save.

​
Templated headers

Headers support Mustache templates with these variables:
  • {{email}}: Email of the user making the request.
  • {{model}}: Model name being requested.
Example header: 
X-User-Email: {{email}}
X-Model: {{model}}

​
Non-streaming endpoints

If your endpoint doesn’t support streaming, disable Endpoint supports streaming. The proxy will convert responses to streaming format for compatibility with playgrounds.

​
Load balance across providers

Configure multiple API keys for the same model to automatically load balance requests:
  1. Add your primary provider key (e.g., OpenAI).
  2. Add Azure OpenAI as a custom provider for the same models.
  3. The proxy automatically distributes requests across both.
Load balancing provides:
  • Resilience if one provider is down.
  • Higher effective rate limits.
  • Geographic distribution.

​
Set up for self-hosted

For self-hosted deployments, configure proxy URLs:
  1. Navigate to Settings > Organization > API URL.
  2. Enter your URLs:
    • API URL: Main API endpoint.
    • Proxy URL: AI Proxy endpoint (usually <API_URL>/v1/proxy).
    • Realtime URL: Realtime API endpoint.
  3. Click Save.
Test connectivity with the provided commands.

​
Access the proxy

Users and applications access the proxy through configured endpoints: 
import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "https://api.braintrust.dev/v1/proxy",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

// Use any configured model
const response = await client.chat.completions.create({
  model: "claude-sonnet-4-5-20250929",
  messages: [{ role: "user", content: "Hello!" }],
});
The proxy automatically uses organization-configured API keys and routes requests to the appropriate provider.

​
Monitor proxy usage

Track proxy usage across your organization:
  1. Create a project for proxy logs.
  2. Enable logging by setting the x-bt-parent header when calling the proxy.
  3. View logs in the Logs page.
  4. Create dashboards to track usage, costs, and errors.
See Use the AI Proxy for detailed logging configuration.

​
Next steps