Skip to main content
Applies to:


​
Summary

Goal: Structure dataset rows with varying system and user prompts for multi-model comparison in Playground. Features: Datasets with chat message arrays, Playground model comparison, role-based message formatting.

​
Configuration Steps

​
Step 1: Structure dataset rows with chat message arrays

Each dataset row should contain an array with system and user messages using role-based formatting. 
{
  "input": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the capital of France?"}
  ]
}

​
Step 2: Add rows with varying system prompts

Create multiple rows where system prompts differ while maintaining the same structure. 
{
  "input": [
    {"role": "system", "content": "You are a creative writer."},
    {"role": "user", "content": "Write a short story."}
  ]
}

​
Step 3: Enable “Append dataset messages” in Playground

In Playground Advanced settings, enable “Append dataset messages” and set the path to your messages array field (e.g., input).

​
Step 4: Configure model comparison

Add multiple model configurations in Playground to test each dataset row across all models with their respective system prompts.

​
Key Notes

  • Each row represents one test case that runs across all configured models
  • System prompts are included in the dataset row and vary per input
  • All messages in the array (system and user) are appended when using “Append dataset messages”
  • Alternative: Save fixed system prompts as reusable custom prompts and store only user content in datasets