Bias Audit

A controlled evaluation of differences in quality, error, or outcome across groups, languages, regions, and counterfactual variations of the same case. The protocol must state the population, analyzed attributes, metrics, sample sizes, and cost of each error type. Without that scope, an aggregate disparity may reflect uneven data coverage, task difficulty, or genuinely discriminatory behavior without distinguishing among them.

The result provides evidence for investigation and risk decisions. Conclusions about fairness or discrimination require causal context, applicable policy, and domain expertise. Useful audits preserve uncertainty, show segment-level distributions, and become versioned regression cases.