Metrics
Score the project against a small hand-labelled gold set and see every model call: the developer view of how the AI behaves.
Where Sidebar → Metrics
What you see
- ScoreHow many targets the latest run met, a bar with one mark per target, when it ran and which gold set it used.
- Latest runEach measure with its target, what was measured and Met, Missed or Info.
- Earlier runsPast runs with their met and missed counts.
- Model callsEvery call to the model: time, purpose, model id and prompt version (for provenance), inputs, result and latency. Prompts, keys and full documents are not stored.
Run the evaluation
- Load Broken Tide or The Ashbell Pass into a project of its own (the evaluation scores those samples and the Sherlock Holmes sample; the Oz books and Alice have none).
- Review candidates, scan conflicts and decide them as you would normally.
- On Metrics, choose Run evaluation.
- Read the score and the Latest run table; it scores the data as it is now.
What the score means
The gold set is small and hand-labelled. It checks that the system behaves as designed on the sample story; it is not evidence of real-world accuracy. In a project without a gold set the page says there is nothing to measure and links to Open projects.
Who can do what
- Viewer
- Read the scores and the model calls.
- Editor, admin, owner
- Also Run evaluation.
Roles are per studio. Roles and permissions has the full list.