Deploy functions

• Prompts: LLM prompts with model configuration and templating (see Deploy prompts)
• Tools: General-purpose code that LLMs can invoke to perform operations or access external data
• Scorers: Functions for evaluating LLM output quality (returning a number from 0 to 1)
• Workflows: Chains of two or more prompts for multi-step workflows

​

Composability

• A vector search tool that embeds a query, searches for relevant documents, and returns them
• A system prompt with instructions for how to retrieve content and synthesize answers using the tool

​

Deploy tools

import * as braintrust from "braintrust";
import { z } from "zod";

const project = braintrust.projects.create({ name: "calculator" });

project.tools.create({
  handler: ({ op, a, b }) => {
    switch (op) {
      case "add":
        return a + b;
      case "subtract":
        return a - b;
      case "multiply":
        return a * b;
      case "divide":
        return a / b;
    }
  },
  name: "Calculator",
  slug: "calculator",
  description: "A simple calculator that can add, subtract, multiply, and divide.",
  parameters: z.object({
    op: z.enum(["add", "subtract", "multiply", "divide"]),
    a: z.number(),
    b: z.number(),
  }),
  returns: z.number(),
  ifExists: "replace",
});

npx braintrust push calculator.ts

from typing import Literal

import braintrust
from pydantic import BaseModel, RootModel

project = braintrust.projects.create(name="calculator")

class CalculatorInput(BaseModel):
    op: Literal["add", "subtract", "multiply", "divide"]
    a: float
    b: float

class CalculatorOutput(RootModel[float]):
    pass

def calculator(op, a, b):
    if op == "add":
        return a + b
    elif op == "subtract":
        return a - b
    elif op == "multiply":
        return a * b
    elif op == "divide":
        return a / b
    else:
        raise ValueError(f"Unsupported operation: {op}")

project.tools.create(
    handler=calculator,
    name="Calculator",
    slug="calculator",
    description="A simple calculator that can add, subtract, multiply, and divide.",
    parameters=CalculatorInput,
    returns=CalculatorOutput,
)

braintrust push calculator.py

​

View and manage tools

​

Add tools to prompts

​

Call tools directly

import { invoke } from "braintrust";

const result = await invoke({
  projectName: "calculator",
  slug: "calculator",
  input: {
    op: "add",
    a: 5,
    b: 3,
  },
});

console.log(result); // 8

​

Deploy scorers

​

Deploy workflows

1. Navigate to Playgrounds
2. Click + Workflow
3. Add prompt nodes by selecting + in the comparison pane
4. Use template variables to pass data between prompts:
   • {{dataset.input}} - Access dataset inputs
   • {{input}} - Access previous prompt’s output
   • {{input.field}} - Access structured output fields
5. Save the agent

​

Invoke functions

• Slug: The unique identifier within a project for any function type (e.g., slug: "calculator")
• Global function name: Built-in Braintrust scorers only - globally unique functions like Factuality from autoevals

import { invoke } from "braintrust";

const result = await invoke({
  projectName: "my-project",
  slug: "my-function",
  input: { query: "hello" },
});

import { invoke } from "braintrust";

const score = await invoke({
  globalFunction: "Factuality",
  functionType: "scorer",
  input: {
    output: "The capital of France is Paris",
    expected: "Paris is the capital of France",
  },
});

​

Handle dependencies

• TypeScript: Uses esbuild to bundle code and dependencies (excludes native libraries like SQLite)
• Python: Uses uv to cross-bundle dependencies to Linux (supports most binary dependencies)

​

Version functions

import { invoke } from "braintrust";

const result = await invoke({
  projectName: "calculator",
  slug: "calculator",
  version: "a1b2c3d4", // Pin to specific version
  input: { op: "add", a: 5, b: 3 },
});

​

Use the REST API

curl https://api.braintrust.dev/v1/function \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BRAINTRUST_API_KEY" \
  -d '{
    "project_name": "calculator",
    "slug": "calculator",
    "input": {
      "op": "add",
      "a": 5,
      "b": 3
    }
  }'

​

Function features

• Well-defined parameters and return types: Type-safe interfaces using Zod (TypeScript) or Pydantic (Python)
• Streaming and non-streaming invocation: Handle real-time and batch operations
• Automatic tracing and logging: All function calls are traced in Braintrust
• OpenAI argument format: Prompts can be loaded directly in OpenAI-compatible format
• Version control: Functions are automatically versioned with each deployment

​

Organize functions

import * as braintrust from "braintrust";

// Get a handle to the project (creates if it doesn't exist)
const project = braintrust.projects.create({ name: "my-project" });

// Use the project to create functions
project.tools.create({...});
project.prompts.create({...});
project.scorers.create({...});

​

Next steps

• Write scorers to create custom evaluation functions
• Deploy prompts that use your tools
• Monitor deployments to track function performance
• Manage environments to version functions across environments