AI Governance in Insurance: Underwriting, Claims, and the Fairness Problem

Why insurance AI governance is different

Insurance is built on discrimination, in the technical sense. The entire business model depends on differentiating risk. Pricing a 22-year-old driver differently from a 55-year-old driver is not unlawful discrimination; it reflects actuarially justified risk differentiation. The problem with AI in insurance is that modern machine learning models find correlations that humans cannot see, and some of those correlations map onto protected characteristics in ways that are not actuarially justified, they are proxies for race, disability, or socioeconomic status rather than genuine predictors of risk.

This is the central governance challenge. The line between legitimate risk differentiation and unlawful indirect discrimination is not always visible in an AI model's outputs, and it requires active governance to find and manage it.

Where AI is being used in insurance

AI applications in insurance now span the full value chain. Underwriting AI assesses risk and sets initial pricing. Claims AI triages, assesses, and in some cases settles claims without human involvement. Fraud detection AI flags suspicious claims patterns. Customer segmentation AI identifies cross-sell and retention opportunities. Telematics and behavioural data AI continuously reprices motor and life products based on real-time data.

Each of these applications carries distinct governance obligations. Claims AI that denies or delays payment is subject to different regulatory scrutiny than underwriting AI that sets initial pricing, but both require governance frameworks that address fairness, explainability, and human oversight.

The indirect discrimination problem

AI pricing models in insurance regularly incorporate data signals that are legitimate risk proxies on average but that correlate with protected characteristics in their practical application. Postcode data is the clearest example. Using postcode to proxy flood risk or car theft risk is actuarially defensible. Using the same postcode data in a context where it serves primarily to proxy for the demographic characteristics of residents, rather than genuine geographic risk factors, creates indirect discrimination exposure.

The legal test is not whether discrimination was intended, but whether the outcome is discriminatory and whether the differentiating factor can be objectively justified. AI models that produce pricing differentials correlated with race, disability, or other protected characteristics will face regulatory scrutiny regardless of whether the model was designed to discriminate. The burden of demonstrating objective justification falls on the insurer.

Governance requires proactive testing: regular audits of AI pricing and underwriting decisions, stratified by the protected characteristics relevant to the jurisdiction, with documented analysis of whether observed differentials are actuarially justified or whether they represent unjustifiable proxy discrimination.

Claims AI and the human oversight obligation

Automated claims processing offers substantial efficiency gains. It also creates significant governance exposure when claims are denied, delayed, or underpaid on the basis of AI decisions that policyholders cannot understand or challenge.

In most jurisdictions with mature insurance regulation, policyholders have rights that extend to AI-driven claims decisions. In Australia, the Insurance Contracts Act and the General Insurance Code of Practice create obligations around claims handling that apply regardless of whether the decision was made by a human or an algorithm. In the EU, the AI Act's human oversight requirements for high-risk AI: which includes AI in access to essential services, require that policyholders be able to request human review of AI-driven decisions.

Governance requires: a documented process for policyholder requests for human review; training for claims staff on how to interpret and if necessary override AI recommendations; clear criteria for when AI claims decisions must be escalated to human review regardless of the AI system's confidence; and incident reporting when AI claims decisions are subsequently overturned on review.

Fraud detection: the false positive problem

Fraud detection AI is among the highest-stakes insurance AI applications. When it works, it protects the insurance pool from fraudulent claims. When it produces false positives, classifying legitimate claims as fraudulent, it causes serious harm to policyholders who are denied rightful payment, sometimes at their most vulnerable.

The governance challenge is that fraud detection models trained on historical fraud data reflect historical fraud patterns, which may be unevenly distributed across demographic groups. Models that identify high-fraud-risk individuals based on patterns that correlate with demographic characteristics produce discriminatory false positive rates. A model that is 95% accurate overall but produces a 12% false positive rate for one demographic group and a 2% false positive rate for another is not equally fair in its outcomes, even if the overall performance metric looks strong.

Governance requires demographic stratification of fraud detection model performance, not just aggregate accuracy metrics. False positive rates must be monitored across relevant groups, and remediation required when material disparities are found.

Regulatory landscape

The UK Financial Conduct Authority has been explicit about its expectations for insurance AI. The FCA's pricing practices rules, combined with its Consumer Duty, create obligations around fair value and fair treatment that apply to AI-driven pricing. The FCA has indicated it will scrutinise AI models that produce outcomes inconsistent with these obligations.

In Australia, ASIC's regulatory guidance on digital advice and automated decision-making, combined with the Australian Financial Services Licence obligations, creates a framework that applies to insurers using AI in product design and distribution. APRA's guidance on model risk applies to APRA-regulated insurers.

For insurers with EU exposure, the EU AI Act's Annex III explicitly lists AI used in access to essential services: which includes insurance, as high-risk. The compliance obligations are substantial: documented risk management, data governance, conformity assessment, human oversight mechanisms, and registration in the EU AI database.