OpenAI Agents SDK

The Braintrust SDK provides trace processors for the OpenAI Agents SDK that send traces and spans to Braintrust for monitoring and evaluation.

For TypeScript, install the Braintrust OpenAI Agents integration package:

pnpm add braintrust @braintrust/openai-agents @openai/agents

trace-agents.ts

import { initLogger } from "braintrust";
import { OpenAIAgentsTraceProcessor } from "@braintrust/openai-agents";
import { Agent, run, addTraceProcessor } from "@openai/agents";
 
// Initialize Braintrust logger
const logger = initLogger({
  projectName: "openai-agent",
});
 
// Create the tracing processor
const processor = new OpenAIAgentsTraceProcessor({ logger });
 
// Add the processor to OpenAI Agents
addTraceProcessor(processor);
 
async function main() {
  const agent = new Agent({
    name: "Assistant",
    model: "gpt-4o-mini",
    instructions: "You only respond in haikus.",
  });
 
  const result = await run(agent, "Tell me about recursion in programming.");
  console.log(result.finalOutput);
}
 
main().catch(console.error);

The constructor of the tracing processor can take a braintrust.Span, braintrust.Experiment, or braintrust.Logger that serves as the root under which all spans will be logged. If not provided, the current span, experiment, or logger is selected automatically.

OpenAI Agents SDK Logs

The Agents SDK can also be used to implement a task in an Eval, making it straightforward to build and evaluate agentic workflows:

eval-agents.ts

import { Agent, run, addTraceProcessor } from "@openai/agents";
import { OpenAIAgentsTraceProcessor } from "@braintrust/openai-agents";
import { Eval } from "braintrust";
 
// Set up the trace processor
const processor = new OpenAIAgentsTraceProcessor();
addTraceProcessor(processor);
 
async function task(input: string) {
  const agent = new Agent({
    name: "Assistant",
    model: "gpt-4o-mini",
    instructions: "You only respond in haikus.",
  });
 
  const result = await run(agent, input);
  return result.finalOutput;
}
 
Eval("openai-agent", {
  data: [
    {
      input: "Tell me about recursion in programming.",
    },
  ],
  task,
  scores: [
    // You can use autoevals or custom scoring functions
    {
      name: "haiku_check",
      scorer: async ({ output }) => {
        // Custom scoring logic for haiku validation
        const lines = output.split("\n").filter((line) => line.trim());
        return lines.length === 3 ? 1 : 0;
      },
    },
  ],
});

OpenAI Agents SDK Eval

OpenAI Agents SDK

On this page