Upgrade a self-hosted Braintrust data plane on AWS, GCP, or Azure
Upgrading to data plane v2.0? It requires a multi-step migration with irreversible infrastructure changes. Follow the Upgrade to v2.0 guide instead.
AWS
GCP
Azure
This guide shows the routine process for upgrading the Braintrust data plane in your AWS account.
Upgrading from AWS Terraform module v5.x to v6? It requires a two-step apply to move the API from Lambda to ECS. Follow the Upgrade to v6 guide instead.
On AWS, the Braintrust data plane runs on ECS and EC2 (Terraform module versions before v6.0 run the API on Lambda). The data plane version is bundled into the Terraform module — each module release pins specific versions of the Braintrust API and Brainstore in VERSIONS.json. When you update your module source to a newer version and run terraform apply, Terraform automatically deploys the corresponding data plane.To check which data plane version you’re currently running, go to Settings > Data plane in the Braintrust UI.
Check the Self-hosting releases page for your target data plane version to find the minimum Terraform module version required. Then update the ?ref= in your module source:
Upgrade to v6.5.0 or later. On v6.4.0 and earlier, running the API on ECS also sends LLM calls from user-authored scorers and tools to the Braintrust-hosted Gateway, outside your AWS account.
v6.7.0
Your configuration sets enable_brainstore
Delete the line, or the plan fails with an unsupported argument error. Brainstore is always enabled.
v6.9.0
The identity that runs Terraform uses a scoped IAM policy
Grant lambda:InvokeFunction on the <deployment_name>-BrainstoreDeployment function, which the apply invokes.
As of v5.6.0, the modules/services sub-module requires the hashicorp/http provider ~> 3.3. If your .terraform.lock.hcl pins an older 3.x version, run terraform init -upgrade before terraform plan or terraform apply.
Review the planned changes before applying:
terraform plan
If the plan deletes or replaces a database, S3 bucket, or other resource you didn’t expect, don’t apply it. Applying it can cause data loss. Contact Braintrust for help.
Some module versions move resources to new Terraform addresses, so downgrading past them can destroy and recreate those resources. Review the plan before any rollback:
v6.8.0: API ECS listener rules, which can disrupt traffic. See the v6.8.0 release notes.
v6.9.0: The Loop runtime’s database, Redis, Brainstore, and Gateway ingress rules, if your deployment runs the Loop runtime.
Apply the changes:
terraform apply
This updates both the infrastructure and the data plane components in a single apply.On module v6.9.0 and later, the apply waits for the Brainstore reader and fast reader fleets to finish their instance refreshes before it updates the API ECS services and the API, catchup ETL, automation, and billing Lambda functions, so new API versions don’t serve traffic before Brainstore is ready. The apply can take noticeably longer as a result. During the refresh, reader capacity can temporarily double. Writer nodes refresh independently and don’t hold up the apply.
If the apply fails while waiting for Brainstore
Terraform waits up to 14 minutes for the refreshes, then up to 14 more minutes in the same apply. The apply fails, and API updates stay blocked, if a refresh fails, health checks fail, or the second wait also runs out.Fix the reported failure and rerun terraform apply. The apply checks live AWS state and resumes an active refresh. Progress and refresh IDs are logged to the /braintrust/<deployment_name>/<deployment_name>-BrainstoreDeployment CloudWatch log group.
This guide shows the routine process for upgrading the Braintrust data plane in your GCP account.
GCP deployments have two independently upgradeable components:
Terraform module (terraform-google-braintrust-data-plane): provisions your cloud infrastructure — the GKE cluster, Cloud SQL database, Redis, and storage buckets. You upgrade this by changing the ?ref= version in your module source and running terraform apply.
Helm chart: deploys the Braintrust application (API and Brainstore containers) onto your GKE cluster. You upgrade this by running helm upgrade with a new --version.
The Helm chart --version is the chart version, not the data plane version. The data plane version is set separately via image tags in your values.yaml. Upgrading the Helm chart alone does not change which data plane version is running — you must also update the image tags.
To check which data plane version you’re currently running, go to Settings > Data plane in the Braintrust UI.
Before upgrading, check the Self-hosting releases page for your target data plane version. Each entry specifies its requirements:
“Requires: Helm X.Y.Z+” — you must upgrade the Helm chart to at least that version before or alongside updating your image tags. Update Terraform first if infrastructure changes are also listed.
No requirements listed — you can upgrade by updating image tags in values.yaml and running helm upgrade. No Terraform or chart version change needed.
Helm-only release (“No data plane version change”) — update only the Helm chart version. No image tag change needed.
Run this step when a release lists infrastructure requirements, or when you want to apply infrastructure changes independently.
terraform apply
Carefully review the output of terraform plan before applying any changes to your deployment. If you see something unexpected, like deletion of a database or storage bucket, contact Braintrust for help.
To pin to a specific Terraform module version, update the ?ref= in your module source:
Run this step to deploy a new data plane version or apply a Helm chart update.
1
Set the data plane version
Set the data plane version by updating the image tags for the API and Brainstore in your values.yaml. Refer to the Helm chart values reference for the exact field names.
Replace <chart-version> with the Helm chart version required by the release (or your current chart version if no chart upgrade is needed).
When a release requires both Terraform and Helm changes, apply Terraform first, then run the Helm upgrade.
Helm chart 6.20.0 and later enable the Brainstore startup gate by default. Before you upgrade, confirm that:
api.image.tag is a release tag such as v2.17.0, or api.brainstoreStartupGate.minimumVersion is set. Otherwise, the chart fails to render.
Every API pool uses the RollingUpdate strategy with maxUnavailable: 0 and a nonzero maxSurge. Otherwise, the chart fails to render.
brainstore.image.tag is a release tag. If it’s a commit SHA, a digest, or a pre-release tag, new API pods don’t start. To deploy such an image, set api.brainstoreStartupGate.enabled: false.
Azure deployments have two independently upgradeable components:
Terraform module (terraform-azure-braintrust-data-plane): provisions your cloud infrastructure — the AKS cluster, Azure Database for PostgreSQL, Redis, and storage. You upgrade this by changing the ?ref= version in your module source and running terraform apply.
Helm chart: deploys the Braintrust application (API and Brainstore containers) onto your AKS cluster. You upgrade this by running helm upgrade with a new --version.
The Helm chart --version is the chart version, not the data plane version. The data plane version is set separately via image tags in your values.yaml. Upgrading the Helm chart alone does not change which data plane version is running — you must also update the image tags.
To check which data plane version you’re currently running, go to Settings > Data plane in the Braintrust UI.
Before upgrading, check the Self-hosting releases page for your target data plane version. Each entry specifies its requirements:
“Requires: Helm X.Y.Z+” — you must upgrade the Helm chart to at least that version before or alongside updating your image tags. Update Terraform first if infrastructure changes are also listed.
No requirements listed — you can upgrade by updating image tags in values.yaml and running helm upgrade. No Terraform or chart version change needed.
Helm-only release (“No data plane version change”) — update only the Helm chart version. No image tag change needed.
Run this step when a release lists infrastructure requirements, or when you want to apply infrastructure changes independently.
terraform apply
Carefully review the output of terraform plan before applying any changes to your deployment. If you see something unexpected, like deletion of a database or storage account, contact Braintrust for help.
To pin to a specific Terraform module version, update the ?ref= in your module source:
Some Terraform Azure module upgrades include one-time breaking changes, such as renamed node pool variables, an azurerm provider upgrade, or a Terraform state migration. Before bumping the module across major versions, review the Terraform Azure module releases entries on the Self-hosting releases page for the versions you’re crossing.
Run this step to deploy a new data plane version or apply a Helm chart update.
1
Set the data plane version
Set the data plane version by updating the image tags for the API and Brainstore in your values.yaml. Refer to the Helm chart values reference for the exact field names.
Replace <chart-version> with the Helm chart version required by the release (or your current chart version if no chart upgrade is needed).
When a release requires both Terraform and Helm changes, apply Terraform first, then run the Helm upgrade.
Helm chart 6.20.0 and later enable the Brainstore startup gate by default. Before you upgrade, confirm that:
api.image.tag is a release tag such as v2.17.0, or api.brainstoreStartupGate.minimumVersion is set. Otherwise, the chart fails to render.
Every API pool uses the RollingUpdate strategy with maxUnavailable: 0 and a nonzero maxSurge. Otherwise, the chart fails to render.
brainstore.image.tag is a release tag. If it’s a commit SHA, a digest, or a pre-release tag, new API pods don’t start. To deploy such an image, set api.brainstoreStartupGate.enabled: false.
AWS
GCP
Azure