> ## Documentation Index
> Fetch the complete documentation index at: https://www.braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Self-host the Braintrust Gateway

> Deploy the Braintrust Gateway in your data plane, restrict organization access, and configure API connectivity and private quarantine routing.

Run the [Braintrust Gateway](https://www.braintrust.dev/docs/deploy/gateway.md) in your data plane to serve model requests and cache completions on your own infrastructure, using provider credentials managed in Braintrust.

> [!WARNING]
>
> Braintrust Gateway self-hosting is in [public preview](https://www.braintrust.dev/docs/feature-lifecycle.md) and can change before reaching general availability.

## Traffic and data handling

Running the Gateway yourself keeps LLM traffic on your own infrastructure: requests egress directly from your VPC to providers, completions are cached (encrypted) in the data plane's Redis, and internal features like [Topics](https://www.braintrust.dev/docs/observe/topics.md) route through it too. When the Gateway is disabled, the data plane does not fall back to the Braintrust-hosted Gateway. The API service calls providers directly.

On AWS, quarantine LLM calls route through your deployment's AI proxy on Terraform module v6.5.0 and later. The [Loop runtime](https://www.braintrust.dev/docs/admin/self-hosting/configure/loop-runtime.md) does the same on v6.5.2 and later. See [quarantine routing](#route-quarantine-llm-calls-through-privatelink) for the network paths and PrivateLink setup.

> [!NOTE]
>
> Even with the Gateway enabled, two paths still leave your data plane:
>
> * **Built-in models** are served by Braintrust, so requests for them reach the Braintrust-hosted Gateway at `gateway.braintrust.dev` rather than your own. Self-hosted organizations have [built-in models](https://www.braintrust.dev/docs/admin/ai-providers.md#manage-built-in-models) disabled by default and opt in explicitly.
> * **Operational telemetry** (status, metrics, usage, and optional logs and traces) can still be sent to Braintrust's control plane, the same as the rest of the data plane. See [Telemetry and data retention](https://www.braintrust.dev/docs/admin/self-hosting/configure/telemetry.md). This is monitoring traffic, not LLM traffic.

## Deploy and enable the Gateway

> [!NOTE]
>
> For Terraform module versions before v6.7.2 and Helm chart versions before v6.17.0, before you deploy the Gateway, [contact Braintrust](mailto:support@braintrust.dev) to enable it for your organization. Newer module versions do not need this enablement.

**AWS**

Terraform module v7.0.0 and later deploys and enables the Gateway by default. It runs as an ECS Fargate service behind an internal load balancer. To upgrade from v6, follow [Upgrade to Terraform module v7](https://www.braintrust.dev/docs/admin/self-hosting/upgrade/v7.md).

Three variables control the Gateway, so you can provision the infrastructure before routing traffic to it:

* `create_ai_gateway` creates the private Gateway infrastructure (an internal ALB and the Gateway ECS service).
* `enable_ai_gateway` sets `GATEWAY_URL` on the API ECS service, and on the APIHandler and AI Proxy Lambda functions when they exist, so internal data plane traffic routes through the Gateway. It requires `create_ai_gateway`.
* `use_private_ai_gateway_origin` points the public CloudFront `/v1/proxy` endpoint at the private Gateway (a VPC origin on its internal load balancer). It requires `create_ai_gateway`.

All three variables default to `true` on v7. To run without the Gateway, set all three to `false`. Setting only `create_ai_gateway = false` fails the plan.

An enabled Gateway route requires `ai_gateway_min_capacity` of at least `1`. Terraform waits for the Gateway service to reach steady state before routing the API, Loop runtime, CloudFront, or quarantine PrivateLink traffic to it. If the initial deployment fails, those callers keep their previous configuration.

> **Enable the Gateway on Terraform v6**
>
> Requires Terraform module v6.5.0 or later, which keeps quarantine LLM calls in your account. All three Gateway variables default to `false` on v6.
>
> **Existing deployment**
>
> Apply each step separately and verify it before continuing:
>
> 1. Provision the infrastructure without changing traffic and apply:
>
>    ```hcl theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
>    create_ai_gateway = true
>    enable_ai_gateway = false
>    ```
>
> 2. Route internal traffic through the Gateway. Once the Gateway is healthy, set `enable_ai_gateway = true` and apply:
>
>    ```hcl theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
>    enable_ai_gateway = true
>    ```
>
>    Public `/v1/proxy` traffic still uses its existing origin.
>
> 3. Cut over the public endpoint and apply:
>
>    ```hcl theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
>    use_private_ai_gateway_origin = true
>    ```
>
>    CloudFront then routes `/v1/proxy` and `/v1/proxy/*` to the private Gateway.
>
> **New deployment**
>
> On a new v6 deployment, you can set `create_ai_gateway` and `enable_ai_gateway` together in the first apply, then cut over the public endpoint with `use_private_ai_gateway_origin` once the Gateway is healthy.

**GCP / Azure**

Requires Helm chart 6.6.0 or later.

1. The Gateway and routing through it are disabled by default. To deploy it, add the following to your `values.yaml` and apply with `helm upgrade`:

   ```yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
   aiGateway:
     enabled: true
   ```

2. Verify the Gateway pods are running before routing traffic to them.

   > [!WARNING]
   >
   > Enabling routing before the Gateway is ready causes API requests to fail.

3. Once the Gateway pods are running and ready, set `aiGateway.useGateway: true` to route API traffic through the Gateway, then apply again with `helm upgrade`:

   ```yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
   aiGateway:
     enabled: true
     useGateway: true
   ```

Optionally, set `aiGateway.region` to your cloud region (for example, `us-east-1`). It defaults to an empty string and sets `GATEWAY_REGION` when provided.

## Size and upgrade the Gateway

**AWS**

* **Task size:** Set `ai_gateway_cpu` and `ai_gateway_memory`.
* **Autoscaling:** Set `ai_gateway_min_capacity` and `ai_gateway_max_capacity`. An enabled Gateway route requires a minimum capacity of at least `1`.
* **Load balancer access:** Set `ai_gateway_authorized_security_groups` to allow additional security groups. The API and Brainstore security groups are authorized automatically.

The Terraform module pins the Gateway image to its data plane release. Upgrade the module to upgrade the Gateway. Leave `ai_gateway_version_override` unset unless Braintrust instructs you to pin a specific tag.

**GCP / Azure**

| Setting | Default | Purpose |
| - | - | - |
| `aiGateway.replicas` | `2` | Number of Gateway pods. |
| `aiGateway.resources` | `cpu: 2`, `memory: 4Gi` | Pod resource requests and limits. |

The Helm chart pins the Gateway image to its data plane release. Upgrade the chart to upgrade the Gateway.

## Restrict organization access

Data plane v2.21.0 and later enforces organization restrictions on Gateway requests and AI provider credentials. This version ships in AWS Terraform module v7.1.0 and Helm chart 6.26.0.

> [!WARNING]
>
> Terraform module v7.1.0 and Helm chart 6.26.0 do not copy the API's organization restrictions to the Gateway. Configure the Gateway separately to restrict its requests.

Set these environment variables on the Gateway:

* `ALLOWED_ORG_IDS`: A comma-separated list of organization UUIDs to allow. Defaults to empty. When `ORG_NAME` names a specific organization, that organization and its credentials are already allowed by name, so list only additional organizations here.
* `ORG_NAME`: An organization name to allow in addition to the listed IDs. Defaults to `*`. When set to `*` or empty, only `ALLOWED_ORG_IDS` restricts access. If that list is also empty, all organizations are allowed.

**Apply the settings:** On AWS, add string entries to the module's `ai_gateway_extra_env_vars` map. On GCP or Azure, add `name` and string `value` entries to Helm's `aiGateway.extraEnvVars` list. Apply the configuration to redeploy the Gateway.

The Gateway checks authentication and provider credentials separately:

* **Authentication:** Any organization associated with the API key can satisfy the restriction, even if a different organization was selected for the request. Otherwise, the Gateway returns HTTP 403: `Organization is not allowed to use this data plane`.
* **Provider credentials:** Credentials owned by disallowed organizations are filtered out. If all returned credentials are filtered out, the Gateway returns HTTP 403: `No provider secrets are allowed for this deployment`.

## Set the Gateway's API endpoint

Set `OVERRIDE_BRAINTRUST_API_URL` when the Gateway needs to reach the API at a different URL, such as an internal endpoint. This requires data plane v2.21.0, shipped in AWS Terraform module v7.1.0 and Helm chart 6.26.0.

> [!WARNING]
>
> The override applies to every organization served by the Gateway. Use an endpoint that serves all intended organizations and is reachable from the Gateway. An incorrect endpoint can break project registration, project-level secret fetching, trace logging, and token-budget queries. API-key login and organization-secret fetching use `BRAINTRUST_APP_URL`.

1. Set the API base URL on the Gateway:
   * **AWS:** Add `OVERRIDE_BRAINTRUST_API_URL` as a string entry in the module's `ai_gateway_extra_env_vars` map.
   * **GCP or Azure:** Add an entry with `name: OVERRIDE_BRAINTRUST_API_URL` and the URL as its string `value` in Helm's `aiGateway.extraEnvVars` list.
2. Apply the configuration to redeploy the Gateway.

The override takes precedence over `BRAINTRUST_API_URL` and organization-specific API URLs, including cached ones. It applies to project registration, project-level secrets, trace logging, and token-budget queries. API-key login and organization-secret fetching use `BRAINTRUST_APP_URL`.

The override is unset by default in Terraform module v7.1.0 and Helm chart 6.26.0. Without it, calls that resolve an organization's API URL use that URL, falling back to `BRAINTRUST_API_URL`. Helm's `aiGateway.braintrustApiUrl` sets the default URL, not an authoritative override. It defaults to an empty string, which uses the in-cluster API service URL.

This setting does not create a private API endpoint or change client connectivity. Configure [Gateway organization restrictions](#restrict-gateway-requests) separately.

## Route quarantine LLM calls through PrivateLink

This section applies to AWS deployments.

By default, LLM calls from user-authored code in the quarantine environment reach the Gateway through the AI proxy Lambda function URL. On Terraform module v7.0.0 and later with `enable_ecs_api = true`, that function doesn't exist, so the calls go through the quarantine VPC's NAT gateway to the deployment's CloudFront `/v1/proxy` URL, and CloudFront serves them from whichever origin `/v1/proxy` uses:

* If you restrict CloudFront with a WAF or a source IP allowlist, allow the quarantine VPC's NAT gateway address.
* With `use_global_ai_gateway_origin = true`, these calls leave your data plane for the Braintrust-hosted Gateway.

On Terraform module v6.7.0 and later, route these calls over AWS PrivateLink to keep them inside your VPCs:

```hcl theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
use_private_gateway_quarantine_proxy = true
```

The module connects quarantine to the Gateway through a network load balancer, VPC endpoint service, and interface endpoint, then configures the API ECS service to use that endpoint. This requires `create_ai_gateway` and `enable_quarantine_vpc`. If quarantine is disabled and you haven't set `quarantine_proxy_url`, Terraform fails rather than silently falling back.

### Use existing VPCs

On Terraform module v7.1.0 and later, the module creates the PrivateLink path whether it manages each VPC or you supply it with `existing_vpc_id` or `existing_quarantine_vpc_id`. When you supply a VPC:

* Both VPCs must be in the same AWS account and region as the provider.
* The three private subnets in each VPC must be distinct and in distinct availability zones, and each quarantine subnet's availability zone must also have a main VPC private subnet. Terraform validates subnet membership and availability zones.
* The module doesn't change your routes, network ACLs, or DNS. DNS in the quarantine VPC must resolve the endpoint hostname, and network ACLs must allow the connection and its return traffic.

### Use your own endpoint

On earlier module versions, the module creates no PrivateLink resources if you supply either VPC. Create the network load balancer, endpoint service, and quarantine interface endpoint yourself, then point the deployment at your endpoint with `quarantine_proxy_url`:

```hcl theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
quarantine_proxy_url = "http://vpce-0abc123.vpce-svc-0def456.us-east-1.vpce.amazonaws.com/v1/proxy"
```

`quarantine_proxy_url` always takes precedence over the automatically selected URL. The `quarantine_proxy_url` output reports the value the deployment ended up using. When `use_global_ai_gateway_origin` is `true`, `use_private_gateway_quarantine_proxy` has no effect and no PrivateLink resources are created.

### Upgrade or disable PrivateLink

> [!NOTE]
>
> The network load balancer and interface endpoint are billed separately, even when `quarantine_proxy_url` routes traffic elsewhere. To use your own endpoint without these extra resources, set `use_private_gateway_quarantine_proxy = false`. If you supply either VPC and set both options, make this change before upgrading to v7.1.0 or later.
>
> When `use_private_gateway_quarantine_proxy` is `false`, quarantine LLM calls reach the Gateway through the AI proxy Lambda function URL, or through the CloudFront `/v1/proxy` URL on v7.0.0 and later when `enable_ecs_api` is `true`. To send them somewhere else, such as the Braintrust-hosted Gateway, set `quarantine_proxy_url`.

> [!WARNING]
>
> If you use the module-created path with your own VPCs, set `use_private_gateway_quarantine_proxy` to `false` before you downgrade to a module version earlier than v7.1.0. Before you turn the option off, make sure quarantine can reach the fallback URL.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.