Skip to main content
Applies to:


​
Summary

Issue: Evaluations fail with various errors or produce incomplete results when running with unlimited concurrency. Cause: By default, maxConcurrency is undefined, allowing unlimited concurrent tasks which can exhaust system resources or hit external service limits. Resolution: Set maxConcurrency to 10-20 to limit concurrent execution and prevent resource exhaustion.

​
Resolution Steps

​
For Python SDK

​
Step 1: Add maxConcurrency parameter to Eval

Set max_concurrency between 10-20 when calling Eval(). 
from braintrust import Eval

Eval(
    "my-project",
    data=my_dataset,
    task=my_task,
    scores=[my_scorer],
    max_concurrency=10  # Limit concurrent execution
)

​
Step 2: Verify issues are resolved

Run the evaluation and confirm all tasks complete without resource or service limit errors.

​
For TypeScript SDK

​
Step 1: Add maxConcurrency parameter to Eval

Set maxConcurrency between 10-20 when calling Eval(). 
import { Eval } from "braintrust";

await Eval("my-project", {
  data: myDataset,
  task: myTask,
  scores: [myScorer],
  maxConcurrency: 10  // Limit concurrent execution
});

​
Step 2: Verify issues are resolved

Run the evaluation and confirm all tasks complete without resource or service limit errors.

​
Additional Information

  • Start with 10 for most use cases
  • Increase to 20 if performance is adequate and no errors occur
  • Reduce to 5 if still experiencing resource issues

​
When concurrency limits are needed

  • API rate limits: OpenAI/Anthropic APIs have RPM limits
  • Memory constraints: Large documents/images per task
  • Database limits: Connection pool exhaustion
  • Cost control: Spread API usage over time
  • Self-hosted infrastructure: CPU/memory resource limits
  • File descriptor limits: System resource exhaustion

​
Common symptoms requiring concurrency control

  • Evaluations with incomplete or failed tasks
  • System resource exhaustion errors
  • Timeout errors during task execution
  • API rate limit (429) errors
  • Out of memory errors with large datasets

​
Reference