Skip to content
The Internet Compass

AI

Model Evaluation

Also known as: Evals

Model evaluation is the practice of measuring output quality against a fixed dataset and defined criteria, using automated metrics, model-based grading or human review, so that changes can be compared objectively.

A useful eval set is small, representative and stable: a hundred well-chosen cases beat ten thousand arbitrary ones, because they can be inspected when scores move.

Without evaluation there is no way to distinguish a prompt change that improved output from one that shifted failures somewhere less visible.