Experiments

An experiment runs your task over a dataset, records each output, and scores it — so you can compare versions and catch regressions.

​

Create an experiment

In Experiments → New experiment, choose a dataset and set:

• Prompt template — e.g. Answer concisely. {{input}}
• Task model — e.g. claude-opus-4-8
• Scorers — the evaluation methods to apply.

​

Run it

Trigger the run from the experiment page. The eval worker executes the task for each dataset item, writes a run item (input, output, latency, tokens, cost, and any error), and applies the scorers.

​

Compare

The experiment view shows per-scorer averages and counts. Set a baseline experiment on the project to diff a new run against it — e.g. “faithfulness 0.94, −1 case vs main”.