AI bias auditing is the systematic process of testing AI systems for discriminatory outcomes across protected characteristics — race, gender, age, disability, religion, national origin, and other attributes protected by law. In 2026, bias auditing is moving from voluntary best practice to legal requirement. NYC Local Law 144, effective since July 2023, requires annual independent bias audits of automated employment decision tools with public results disclosure. The Colorado AI Act (SB 205, effective 30 June 2026) requires deployers of high-risk AI to conduct impact assessments that include evaluation of algorithmic discrimination. The EU AI Act mandates that high-risk AI systems include processes for bias monitoring and mitigation throughout the system lifecycle. Organisations that wait for enforcement action to begin bias testing face significantly higher costs and reputational damage than those that build testing into their governance frameworks now.
What bias auditing actually involves
A bias audit is not a single test. It is a structured evaluation that examines whether an AI system produces different outcomes for different demographic groups at rates that cannot be explained by legitimate, non-discriminatory factors. The core methodology involves defining the decision being made (hiring, credit, pricing, service allocation), identifying the protected characteristics relevant to that decision, measuring outcome rates across demographic groups, calculating adverse impact ratios or statistical parity metrics, investigating root causes of any disparities found, and documenting findings and remediation actions.
The most common statistical measure is the four-fifths rule (also called the 80% rule): if the selection rate for a protected group is less than 80% of the rate for the most-selected group, adverse impact is indicated. NYC Local Law 144 uses this framework. However, statistical parity is only one measure — equal opportunity, predictive parity, calibration, and individual fairness are alternative metrics that may be more appropriate depending on the context. No single metric captures all dimensions of fairness, and the choice of metric itself is a governance decision that should be documented and justified.
Legal requirements by jurisdiction
NYC Local Law 144 requires employers using automated employment decision tools to obtain an annual independent bias audit conducted by an external auditor, publish the results on their website, and provide notice to candidates that the tool is being used. The Colorado AI Act extends obligations beyond hiring to credit, insurance, housing, and government services — requiring deployers of high-risk AI to conduct impact assessments, implement risk management policies, and provide notice to individuals. The EU AI Act requires high-risk AI system providers to implement measures to identify, prevent, and mitigate bias, and to use training, validation, and testing datasets that are sufficiently relevant, representative, and free of errors.
Practical implementation
Build bias testing into the AI lifecycle — not as an annual compliance exercise, but as a continuous monitoring process. Before deployment: test the AI system against historical data to identify baseline disparities. During deployment: monitor outcomes in production across demographic groups on a rolling basis. Regularly: conduct formal bias audits at intervals appropriate to the risk level and regulatory requirements. Remediation: when disparities are identified, investigate root causes (biased training data, proxy variables, feedback loops) and implement corrections. Documentation: maintain records of all testing, findings, and remediation actions — this is the evidence that regulators and auditors will request.
Further reading: NIST AI RMF | OECD AI Principles
