Documentation Index Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
Use this file to discover all available pages before exploring further.
Use datasets as the data source for evaluations. You can pass datasets directly or convert experiment results into dataset format.
Pass datasets directly Pass datasets directly to Eval():
import { initDataset, Eval } from "braintrust" ; import { Levenshtein } from "autoevals" ; Eval ( "Say Hi Bot" , { data: initDataset ( "My App" , { dataset: "My Dataset" }), task : async ( input ) => { return "Hi " + input; }, scores: [Levenshtein], });
Assign to environments Environments are attached to dataset snapshots , not datasets directly. To assign a snapshot to an environment:
Go to
Open the dataset.
Click Snapshots in the toolbar.
Find the snapshot you want to assign, then select Environments .
Toggle the environments in the submenu.
# 1. Create a snapshot curl -X POST https://api.braintrust.dev/v1/dataset_snapshot \ -H "Authorization: Bearer $BRAINTRUST_API_KEY " \ -H "Content-Type: application/json" \ -d '{ "dataset_id": "DATASET_ID", "name": "prod snapshot", "xact_id": "DATASET_VERSION_OR_XACT_ID" }' # 2. Upsert the environment association curl -X PUT https://api.braintrust.dev/environment-object/dataset/DATASET_ID/ENVIRONMENT \ -H "Authorization: Bearer $BRAINTRUST_API_KEY " \ -H "Content-Type: application/json" \ -d '{ "object_version": "SNAPSHOT_XACT_ID" }'
Once assigned, load the dataset for that environment in your evals:
import { Eval, initDataset } from "braintrust" ; // Load by environment Eval ( "My App" , { data: initDataset ({ project: "My App" , dataset: "My Dataset" , environment: "production" }), task : async ( input ) => { /* ... */ }, scores: [], }); // Load by version name Eval ( "My App" , { data: initDataset ({ project: "My App" , dataset: "My Dataset" , version_name: "version 1" }), task : async ( input ) => { /* ... */ }, scores: [], }); // Load by xact_id (less readable, but precise) Eval ( "My App" , { data: initDataset ({ project: "My App" , dataset: "My Dataset" , version: "8234923849293849..." }), task : async ( input ) => { /* ... */ }, scores: [], });
Convert experiment results Convert experiment results into dataset format using asDataset()/as_dataset(). This is useful for iterative improvement workflows where you want to use the results of one experiment as the baseline for future experiments:
import { init, Eval } from "braintrust" ; import { Levenshtein } from "autoevals" ; const experiment = init ( "My App" , { experiment: "my-experiment" , open: true , }); Eval < string , string >( "My App" , { data: experiment. asDataset (), task : async ( input ) => { return `hello ${ input }` ; }, scores: [Levenshtein], });
Next steps 
