Definition
Ground truth is the set of verified, authoritative correct answers against which AI agent outputs are measured. It serves as the "answer key" for evaluation: by comparing agent outputs to ground truth, teams can compute accuracy, precision, recall, and other quality metrics. Ground truth can take many forms depending on the task: correct document classifications, accurate extracted entities, verified Q&A pairs, approved generated text samples, or expert-validated decision outcomes.
Engineering Context
Ground truth quality is the bottleneck of any eval system. A golden set with bad ground truth produces meaningless metrics. Collection approaches: (1) domain expert annotation (highest quality, expensive), (2) programmatic extraction from existing correct decisions, (3) LLM-assisted annotation with human review. For production AI systems, ground truth comes partly from human review decisions—capture these systematically to continuously expand your eval set. Establish inter-annotator agreement metrics when using multiple human annotators: low agreement signals ambiguous tasks where ground truth itself is unclear. Version control your ground truth sets alongside model and prompt versions for reproducibility.
Related Terms
Building production AI agents?
We design and implement deterministic AI agent systems for enterprise teams.
Start Assessment