Governance Risk

What Is Algorithmic Bias?

Algorithmic bias occurs when an AI system produces systematically unfair outcomes for particular groups of people — based on characteristics such as race, gender, age, or disability. It is one of the most legally significant AI governance risks, because biased AI outcomes can breach anti-discrimination law regardless of whether bias was intentional.

Intent does not determine liability

The most important thing to understand about algorithmic bias in a legal context: discrimination law in Australia and most comparable jurisdictions covers discriminatory outcomes, not only discriminatory intent. An AI system that systematically produces worse outcomes for women, or for people of a particular ethnic background, may breach the Sex Discrimination Act or the Racial Discrimination Act — regardless of whether the organisation intended to discriminate, and regardless of whether they designed the AI tool themselves or purchased it from a vendor.

The vendor liability point

Purchasing a biased AI tool from a vendor does not transfer the legal liability. The organisation deploying the tool is the one making the decision that affects the individual. If that decision discriminates, the deploying organisation is the respondent — not the tool vendor.

Where bias enters AI systems

Training data bias

If training data reflects historical discrimination — e.g. hiring data from an era when women were excluded from senior roles — the model learns those patterns. Outputs perpetuate historical inequity.

Proxy discrimination

A model may not use a protected characteristic directly but use a correlated variable — postcode, school attended, language patterns — that acts as a proxy, producing discriminatory outcomes by another route.

Measurement bias

The metric used to define "good outcomes" in training may itself be biased. Criminal recidivism tools trained on arrest rates embed policing bias; healthcare risk tools trained on spending embed access inequality.

Feedback loops

When a biased model is deployed and its outputs influence future data — e.g. a hiring tool that depresses applications from certain groups — the bias compounds over time as new training data reflects the model's own biased choices.

Australian law: what applies

Legislation	AI application scope
Sex Discrimination Act 1984 (Cth)	AI in employment decisions, service delivery, education
Age Discrimination Act 2004 (Cth)	AI-driven hiring, credit, and service access decisions
Disability Discrimination Act 1992 (Cth)	AI systems affecting access for people with disability
Racial Discrimination Act 1975 (Cth)	AI that produces race-correlated disparities in outcomes
State anti-discrimination laws	Equal Opportunity Acts in each state add further obligations

EU AI Act: bias as a high-risk AI issue

The EU AI Act requires high-risk AI systems — those in employment, education, credit, essential services and law enforcement — to be tested for bias before deployment, with ongoing monitoring and corrective action required where biased outputs are identified. Bias testing is a conformity assessment requirement, not an optional best practice.

AI bias governance guide AI for HR teams in Australia