Alerts

Alerts let you trigger a webhook when conditions are met on new logs in Braintrust. This is useful for triggering alerts in tools like Slack or PagerDuty, or for kicking off custom workflows.

Creating an alert

To create an alert, navigate to Configuration > Alerts in your project.

Alert configuration

• Alert name: A descriptive name for your alert
• Description (optional): Additional context about the alert's purpose
• Event type: Currently supports Log event
• BTQL filter: Filter logs using BTQL syntax (if empty, matches all logs)
• Interval: How frequently the alert should check for matching events
• Webhook URL: The endpoint that will receive the alert context as a payload

Webhook payload

When a webhook alert is triggered, it sends a JSON payload to your webhook URL with the following structure:

{
  "organization": {
    "id": "org_123",
    "name": "your-organization"
  },
  "project": {
    "id": "proj_456",
    "name": "your-project"
  },
  "automation": {
    "id": "c5b32408-8568-4bff-9299-8cdd56979b67",
    "name": "High-Priority Factuality",
    "description": "Alert on factuality scores for logs with priority 0 in metadata",
    "event_type": "logs",
    "btql_filter": "metadata.priority = 0 AND scores.Factuality < 0.9",
    "interval_seconds": 3600,
    "url": "https://braintrust.dev/app/your-organization/p/your-project/configuration/alerts?aid=c5b32408-8568-4bff-9299-8cdd56979b67"
  },
  "details": {
    "is_test": false,
    "message": "High-Priority Factuality: 5 logs triggered alert in the last 1 hour",
    "time_start": "2025-05-12T10:00:00.000Z",
    "time_end": "2025-05-12T11:00:00.000Z",
    "count": 5,
    "related_logs_url": "https://braintrust.dev/app/your-organization/p/your-project/logs?search=..."
  }
}

Testing alerts

Before saving or updating an alert, you can test it to confirm behavior with the Test alert button. Braintrust will trigger the alert as if the intiating event occurred, running it through the BTQL filter on recent data. If matching logs are found, a test payload will be sent to your webhook URL.

If no matching logs are found, you may need to adjust your BTQL filter or the alert interval.