Skip to main content
Groq provides ultra-fast inference for open-source language models including Llama, Mixtral, and Gemma models. Braintrust integrates seamlessly with Groq through direct API access, wrapper functions for automatic tracing, and proxy support.

​
Setup

To use Groq models, configure your Groq API key in Braintrust.
  1. Get a Groq API key from Groq Console
  2. Add the Groq API key to your organization’s AI providers or to a project’s AI providers
  3. Set the Groq API key and your Braintrust API key as environment variables
.env
GROQ_API_KEY=<your-groq-api-key>
BRAINTRUST_API_KEY=<your-braintrust-api-key>

# If you are self-hosting Braintrust, set the URL of your hosted dataplane
# BRAINTRUST_API_URL=<your-braintrust-api-url-here>
API keys are encrypted at rest using transparent data encryption with a unique 256-bit key and nonce.

​
Use Groq with Braintrust AI proxy

The Braintrust AI Proxy allows you to access Groq models through a unified OpenAI-compatible interface. Install the braintrust and openai packages. 
# pnpm
pnpm add braintrust openai
# npm
npm install braintrust openai
Then, initialize the client and make a request to a Groq model via the Braintrust AI Proxy. 
import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "https://api.braintrust.dev/v1/proxy",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const response = await client.chat.completions.create({
  model: "openai/gpt-oss-120b",
  messages: [{ role: "user", content: "Hello, world!" }],
});

​
Trace logs with Groq

Trace your Groq LLM calls for observability and monitoring. When using the Braintrust AI Proxy, API calls are automatically logged to the specified project. 
import { OpenAI } from "openai";
import { initLogger } from "braintrust";

initLogger({
  projectName: "My Project",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const client = new OpenAI({
  baseURL: "https://api.braintrust.dev/v1/proxy",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

// All API calls are automatically logged
const result = await client.chat.completions.create({
  model: "openai/gpt-oss-120b",
  messages: [{ role: "user", content: "What is machine learning?" }],
});
The Braintrust AI Proxy is not required. For more control, learn how to customize traces.

​
Evaluate with Groq

Evaluations distill the non-deterministic outputs of Groq models into an effective feedback loop that enables you to ship more reliable, higher quality products. Braintrust Eval is a simple function composed of a dataset of user inputs, a task, and a set of scorers. To learn more about evaluations, see the Experiments guide. 
import { Eval } from "braintrust";
import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "https://api.braintrust.dev/v1/proxy",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

Eval("Groq Evaluation", {
  data: () => [
    { input: "What is 2+2?", expected: "4" },
    { input: "What is the capital of France?", expected: "Paris" },
  ],
  task: async (input) => {
    const response = await client.chat.completions.create({
      model: "openai/gpt-oss-120b",
      messages: [{ role: "user", content: input }],
    });
    return response.choices[0].message.content;
  },
  scores: [
    {
      name: "accuracy",
      scorer: (args) => (args.output === args.expected ? 1 : 0),
    },
  ],
});
To learn more about tool use, multimodal support, attachments, and masking sensitive data with Groq, visit the customize traces guide.