Manage projects

​

Create a project

import * as braintrust from "braintrust";

// Get a handle to the project (creates if it doesn't exist)
const project = braintrust.projects.create({ name: "my-project" });

// Use the project to create functions
project.prompts.create({...});
project.tools.create({...});

import * as braintrust from "braintrust";

// Creates "my-project" if it doesn't exist
const experiment = braintrust.init("my-project", {
  experiment: "my-experiment"
});

​

Configure AI providers

• Different projects need separate billing or rate limits
• You want to isolate API usage by project
• Projects require different provider accounts or credentials

1. Navigate to your project.
2. Go to Configuration.
3. Under Project, select Project AI providers.
4. Click the provider you want to configure.
5. Enter your API key for that provider.
6. Click Save.

​

Add custom providers

​

Add tags

1. Go to Settings.
2. Under Project, select Tags.
3. Click Add tag.
4. Enter tag details:
   • Name: Tag identifier.
   • Color: Visual indicator.
   • Description: Optional explanation.
5. Click Save.

​

Configure human review

1. Go to Settings.
2. Under Project, select Human review.
3. Click + Human review score.
4. Enter a name and description for your score. Descriptions support Markdown.
5. Select a score type:
   • Categorical score: Predefined options with assigned scores. Each option gets a unique percentage value between 0% and 100% (stored as 0 to 1). Use for classification tasks like sentiment or correctness categories. Also supports writing to the exptected field instead of creating a score.
   • Continuous score: Numeric values between 0% and 100% with a slider input control. Use for subjective quality assessments like helpfulness or tone.
   • Free-form input: String values written to the metadata field at a specified path. Use for explanations, corrections, or structured feedback.
6. Click Save.

​

Create aggregate scores

1. Go to Settings.
2. Under Project, select Aggregate scores.
3. Click Add aggregate score.
4. Define the aggregation:
   • Name: Score identifier.
   • Type: Weighted average, minimum, or maximum.
   • Selected scores: Scores to aggregate.
   • Weights: For weighted averages, set score weights.
   • Description: Optional explanation.
5. Click Save.

​

Set up online scoring

1. Go to Settings.
2. Under Project, select Online scoring.
3. Click Add rule.
4. Configure the rule:
   • Name: Rule identifier.
   • Scorers: Select which scorers to run.
   • Sampling rate: Percentage of logs to evaluate (1-100%).
   • Filter: Optional SQL query to select specific logs.
   • Span type: Apply to root spans or all spans.
5. Click Save.

​

Configure span iframes

1. Go to Settings.
2. Under Project, select Span iframes.
3. Click Add iframe.
4. Configure rendering:
   • Field path: Which field to render (e.g., output.html).
   • iframe URL: Template for the iframe src attribute.
5. Click Save.

​

Set comparison key

1. Go to Settings.
2. Under Project, select Advanced.
3. Enter a SQL expression (default: input).
4. Click Save.

• input.question - Match by question field only.
• input.user_id - Match by user.
• [input.query, metadata.category] - Match by multiple fields.

​

Edit project details

1. Navigate to your project.
2. Click Edit project in the top-right.
3. Modify name and description.
4. Click Save.

​

Delete a project

1. Navigate to Configuration.
2. Scroll to the bottom of the page.
3. Click Delete project.
4. Confirm by typing the project name.
5. Click Delete.

​

Next steps

• Control access to projects with permission groups
• Set up automations for project-specific alerts
• View logs filtered by tags and metadata
• Run evaluations on project datasets