Skip to main content
Applies to:
  • Plan -
  • Deployment -

​
Summary

Issue: Automations configured to run scorers stop firing silently, with no UI error. Playground tests return Braintrust gateway error: Unable to detect source format, and backend logs show unknown model or provider not configured for Gemini models such as gemini-2.5-flash, gemini-2.5-pro, or gemini-3-flash-preview. Cause: The Gemini model configured in the scorer is unavailable or not routable through the currently enabled AI providers in your Braintrust org. Resolution: Enable the Vertex AI default registry and update affected scorers to use a valid Vertex model ID.

​
Resolution steps

​
Step 1: Enable the Vertex AI default registry

  1. Go to Settings → AI providers in your Braintrust org.
  2. Locate the Vertex AI provider and enable the default registry.

​
Step 2: Update the affected scorer

Open each affected scorer and change the configured model to one available in the Vertex registry (for example, publishers/google/models/gemini-2.5-flash or gemini-2.5-flash).

​
Step 3: Verify the automation fires

Trigger a manual test of the automation or apply the scorer directly to a log entry. Confirm that scores are written and no gateway error is returned.

​
If you prefer to avoid Vertex AI

​
Step 1: Switch to a different model provider

Update the scorer to use a model from a provider already configured and verified in your org (for example, an OpenAI or Anthropic model). Retry the automation test to confirm it runs successfully.

​
Notes

  • Silent failures occur because scorer LLM call errors at the gateway layer do not surface as visible errors in the automation UI.
  • If multiple automations stopped firing around the same date, check whether a model was removed or a provider configuration changed around that time.
  • If the playground returns Unable to detect source format, treat this as a signal of a misconfigured or unavailable model rather than a data formatting issue.