Why AI incidents are different from other technology incidents
AI incidents have characteristics that distinguish them from traditional technology incidents and that require specific response protocols. First, scale: an AI model failure can affect thousands or millions of decisions before it is detected, because automated systems process at a speed and volume that human review cannot match. Second, ambiguity: it is often not immediately clear whether an AI is failing or performing as designed but producing outcomes that are unacceptable — this ambiguity creates delay in the response that worsens the outcome. Third, causation complexity: establishing what went wrong in an AI system — whether it is the model, the data, the deployment, the monitoring, or the business logic — requires technical expertise that may not be immediately available. And fourth, regulatory multiplicity: an AI incident may simultaneously engage privacy law, sector-specific regulation, consumer law, and employment law — the response must coordinate across all of these simultaneously.
The 72-hour response framework
Hours 0-4 (Contain and Assess): The immediate priority is to understand whether the AI system is currently causing harm and whether continued operation will cause additional harm. If the answer to either question is yes, the first decision is whether to suspend operation. Suspending an AI system is often commercially costly — but continuing to operate an AI system that is causing harm at scale is almost always more costly, in regulatory, legal, and reputational terms. The contain and assess phase ends with a clear factual statement: what happened, when, what is the current scale of impact, and what is the trajectory if the system continues to operate.
Hours 4-24 (Investigate and Notify): The investigation phase establishes the root cause of the AI incident with sufficient confidence to brief regulators and affected parties. Parallel to investigation, the privacy breach assessment must be completed: does this incident involve personal information? Is there a risk of serious harm to affected individuals? If yes to both, the Privacy Act notification clock is running — you must complete an assessment within 30 days of suspecting an eligible data breach, then notify as soon as practicable. For APRA-regulated entities, the CPS 230 operational incident reporting assessment must also be completed in this window: is this a material operational incident that requires APRA notification?
Hours 24-72 (Remediate and Communicate): By 72 hours, the organisation should be implementing remediation of the AI system failure, communicating with affected individuals if notification obligations apply, briefing relevant regulators, and preparing for external scrutiny. The communication to affected individuals is often the most challenging element — it must be honest, specific, and actionable, without creating additional legal exposure. Legal review of all external communications is essential before release.
OAIC notification: what triggers the obligation
The Notifiable Data Breaches scheme under the Privacy Act requires notification to the OAIC and affected individuals when an "eligible data breach" occurs. An eligible data breach is an unauthorised access to, or disclosure of, personal information, where a reasonable person would conclude that the access or disclosure is likely to result in serious harm to the affected individual. AI incidents can trigger this obligation in several ways: an AI system that exposes personal data through its outputs (for example, an AI that returns other users' data in responses), an AI system that is compromised through adversarial attack, or an AI system whose training data is exfiltrated. The assessment of whether serious harm is likely must be made promptly — the 30-day notification clock runs from when the organisation becomes aware that an eligible data breach has occurred.
