What Is...
Governance Concept
What Is AI Explainability?
AI explainability is the ability to explain, in human-understandable terms, why an AI system produced a particular output. It is distinct from interpretability (understanding the model's internal workings) and is a legal compliance requirement in multiple jurisdictions.
Legal requirements by jurisdiction
GDPR Article 22
EU / UK
Meaningful information about the logic involved in automated decisions with legal or significant effects. Must be specific to the individual decision.
EU AI Act Article 13
EU
High-risk AI systems must be transparent — enabling deployers to understand how the AI works, interpret its outputs, and implement appropriate oversight.
ECOA / Reg B
US
Specific principal reasons for adverse credit decisions — algorithmic scores without reasons do not satisfy the obligation.
Privacy Act / APPs
Australia
Right to access personal data including AI-generated assessments; right to seek correction of inaccurate information.
Common explainability techniques
SHAP values
Assigns each feature a contribution score for a specific prediction. Most widely used in regulated industry contexts.
LIME
Local Interpretable Model-agnostic Explanations — fits a simple interpretable model around a specific prediction.
Counterfactual explanations
"What would need to change for the outcome to be different?" — most useful for actionable explanations to individuals.
Attention visualisation
Shows which parts of the input the model weighted most heavily — common for text and image models.
Key distinction: Explainability techniques explain specific decisions to specific people. Interpretability means understanding what a model computes internally. Both matter for governance — they are not interchangeable.