AI in Your Supply Chain: Managing Third-Party AI Risk

The third-party AI risk gap

Most AI governance frameworks focus on AI systems that an organisation develops or directly controls. This focus is misplaced for most organisations. The majority of enterprise AI risk does not arise from bespoke AI systems built in-house, it arises from AI embedded in the commercial software, platforms, and services that organisations purchase and use.

Your HR platform may use AI to score job applicants. Your ERP system may use AI to flag anomalous transactions. Your customer service platform may use AI to route and respond to customer inquiries. Your email system may use AI to filter, prioritise, and suggest responses. In each case, the AI is a component of a third-party product, not a system you built, not a model you trained, not a dataset you curated. But the outcomes of that AI are your organisation's outcomes, and the governance obligations are yours.

Deployer obligations under the EU AI Act

The EU AI Act makes clear that deployer obligations cannot be contracted away. An organisation that uses a high-risk AI system built by a vendor carries direct regulatory obligations as a deployer: implementing human oversight measures, monitoring AI system performance, conducting fundamental rights impact assessments where required, and informing affected individuals of AI use. These obligations exist regardless of whether the organisation had any role in designing or building the AI system.

This creates a direct dependency: deployers cannot satisfy their regulatory obligations without adequate information from their AI vendors. A deployer cannot implement appropriate human oversight of an AI system whose decision logic is opaque. A deployer cannot monitor AI performance without access to performance metrics. A deployer cannot assess fundamental rights impacts without understanding what the AI system does and how it does it.

Third-party AI risk management is therefore not just prudent governance, it is a prerequisite for regulatory compliance.

The AI vendor due diligence framework

Assessing AI vendors requires a different due diligence framework than traditional technology vendor assessment. Standard vendor due diligence focuses on security posture, financial stability, and service reliability. AI vendor due diligence must additionally assess AI governance maturity.

Technical documentation: Does the vendor provide adequate technical documentation of the AI system, training data sources, model architecture, performance metrics, known limitations? Under the EU AI Act, providers of high-risk AI systems are required to provide deployers with sufficient information to implement the system correctly. Vendors who cannot or will not provide this information are either not compliant with their own regulatory obligations or are operating systems that cannot be adequately governed.

Bias testing: Has the AI system been tested for bias across relevant demographic groups? What were the results? What remediation has been applied? Vendor assurances that their AI is "fair" without supporting testing methodology and results are not adequate evidence of fairness.

Data processing: What data does the AI system process? How is that data used: specifically, is it used to train or improve the model? Where is it processed and stored? What are the vendor's data retention practices? These questions are essential for understanding both privacy risk and the potential for your data to improve a competitor's model.

Incident response: How does the vendor identify and communicate AI system failures or performance degradation? What is the notification timeline for incidents that may affect your organisation? What remediation capabilities exist?

Contractual protections

Vendor contracts for AI systems should address governance explicitly. Standard technology contracts are not designed for AI and will not capture the governance obligations that AI creates. At minimum, AI vendor contracts should include: representations about AI governance practices and regulatory compliance; audit rights allowing the deploying organisation to assess AI system performance and governance; incident notification obligations with defined timelines; data processing terms that address AI-specific risks including training data use; and provisions addressing material changes to the AI system or its underlying model.

Change management provisions are particularly important. An AI vendor that updates the underlying model powering a system may materially change the system's behaviour, its performance, its outputs, its bias characteristics, without the deploying organisation's knowledge. Contractual requirements for notification of material model changes, and the right to assess those changes before they affect production systems, are a basic governance requirement.

Concentration risk

Organisations that rely on a small number of AI platform providers for critical functions face systemic concentration risk. If a major AI platform experiences a significant outage, changes its terms of service, or faces regulatory action, organisations that depend on it face operational disruption without an adequate alternative. This risk is analogous to other forms of third-party concentration risk but is more acute because AI capabilities are currently concentrated among a small number of providers.

Governance should assess AI concentration risk as part of broader operational resilience planning, identifying critical dependencies on AI platforms, evaluating the availability of alternatives, and considering whether concentration in any single provider has reached a level that represents unacceptable operational risk.