Why public sector AI governance is different
The core principles of AI governance, accountability, transparency, human oversight, fairness, apply to both public and private sector AI use. What differs is the accountability structure those principles must serve. A private organisation using AI is accountable to its regulators, its customers, and its shareholders. A government agency using AI is accountable to citizens, to the public as a whole, and to the democratic oversight mechanisms that give government its legitimacy.
This difference has concrete implications for governance. Government AI decisions can affect citizens who have no alternative provider to turn to. Government AI decisions are backed by coercive state power in ways that private decisions are not. And government AI decisions carry constitutional and administrative law constraints, natural justice, procedural fairness, reasons for decisions, that do not apply to private actors in the same way.
Due process and automated government decisions
When a government agency uses AI to make or inform decisions about individuals, welfare eligibility, visa processing, tax compliance, social housing allocation, the affected individuals typically have rights to procedural fairness that must be designed into the AI governance framework.
Procedural fairness in automated government decision-making requires, at minimum: that individuals know AI is being used in decisions about them; that they can access the reasons for an adverse decision in terms they can understand; that they have a genuine opportunity to challenge the decision and have that challenge assessed by a human capable of exercising independent judgment; and that the AI system's criteria are applied consistently and without discrimination.
These requirements are not hypothetical. Australia's Robodebt Royal Commission examined an AI-assisted government debt recovery program that used automated data matching to raise debt notices against welfare recipients. The Royal Commission's findings, that the program was unlawful, that it caused serious harm, and that governance failures at multiple levels enabled its continuation, provide a detailed case study in what happens when public sector AI governance fails.
The Robodebt lessons
The Robodebt program used income averaging to calculate welfare overpayments, generating automated debt notices to approximately 443,000 welfare recipients between 2015 and 2019. The legal problem was that income averaging was not a valid method for calculating debts under the relevant legislation, the program was unlawfully raising debts it could not legally establish.
The governance lessons extend beyond the legal error. The program was implemented without adequate legal review of the automated methodology. Concerns raised by frontline staff and external lawyers were not escalated effectively. The volume of complaints was treated as a system management problem rather than a signal of systemic error. And the burden of proof was effectively reversed, recipients were required to disprove debts that had been algorithmically generated without lawful basis.
For public sector AI governance, the Robodebt Royal Commission established critical principles: automated government decision methodologies require independent legal validation before deployment; error rates in automated systems must be treated as a governance signal, not a customer service problem; and individuals must not bear the burden of challenging automated government decisions without adequate support and access to information.
EU AI Act and public sector obligations
The EU AI Act takes particular note of government AI use. AI systems used in law enforcement, migration and border control, administration of justice, and management of critical infrastructure are classified as high-risk, subject to the Act's most demanding compliance requirements. AI-powered real-time biometric identification in publicly accessible spaces is prohibited outright, with narrow and strictly regulated exceptions for law enforcement in specific circumstances.
These provisions apply to government agencies directly and to private companies supplying AI systems to government. A technology vendor supplying an AI-powered border control system to an EU member state is subject to the provider obligations under the Act, regardless of where the vendor is based.
Democratic accountability mechanisms
Private sector AI governance is primarily about managing risk and satisfying regulatory requirements. Public sector AI governance has an additional dimension: democratic accountability. Citizens, through their elected representatives, have a legitimate interest in knowing what AI systems government is using, on what basis, and to what effect.
This requires governance mechanisms that go beyond internal corporate governance structures. Algorithmic impact assessments, structured evaluations of the potential effects of AI systems on individuals and communities, should be publicly available for significant government AI deployments. Public registers of AI systems in use by government agencies, with sufficient information for meaningful public scrutiny, are increasingly considered a democratic accountability baseline. Parliamentary or legislative oversight mechanisms for high-risk government AI use provide democratic checks on executive discretion that internal governance cannot substitute for.
