What Is an AI Audit?
An AI audit is a structured, independent assessment of an AI system's design, development, deployment, and outcomes against defined governance standards, regulatory requirements, and performance benchmarks. AI audits evaluate whether the system operates as intended, whether it produces fair and accurate outcomes, whether it complies with applicable regulations, and whether adequate controls and oversight mechanisms are in place. AI audits can be internal (conducted by the organisation's own audit function) or external (conducted by independent third parties). NYC Local Law 144, effective since July 2023, requires annual independent bias audits for automated employment decision tools — the first jurisdiction to mandate AI auditing by law.
AI Audit — an independent examination of an AI system to assess whether it meets defined criteria for performance, fairness, safety, regulatory compliance, and governance.
AI audit sits within the third line of defence in the standard three-lines model. It is distinct from AI testing (which is a first-line activity) and AI monitoring (second line). High-quality AI audit covers governance documentation, model performance and stability, data lineage, bias and fairness testing, security controls, and operational integration. NYC Local Law 144 is the first mandate of its kind globally; expect more.
Source: NYC Local Law 144 of 2021; ISO/IEC 42001 Clause 9
Why it matters for governance
APRA's April 2026 letter observed that risk and audit teams often lack sufficient AI expertise and that point-in-time, sample-based assurance is inadequate for AI systems that learn, adapt, and degrade over time. AI auditing requires skills that traditional audit teams may not have: understanding of model performance metrics, bias testing methodology, data quality assessment, and the ability to evaluate AI-specific controls. Building AI audit capability is now a regulatory expectation, not a best practice.