AI Governance in Procurement: The Questions You Must Ask Every AI Vendor Before You Sign

Procurement teams are signing AI vendor contracts without adequate governance due diligence. The liability for vendor AI governance failures flows to the buyer. Here are the questions that sophisticated procurement teams are asking in 2026.

Key Takeaways

When you buy AI from a vendor, you become the deployer under the EU AI Act — the regulatory obligations and potential penalties attach to you, not the vendor, unless your contract explicitly allocates them differently.
Standard vendor AI governance representations are inadequate — most cover data processing under GDPR but do not address AI-specific obligations including model documentation, bias testing, and incident notification.
The three contract clauses every AI procurement should include: AI-specific incident notification (shorter than standard data breach notification), model drift notification obligations, and audit rights over AI system performance.
Vendors who cannot answer specific AI governance questions during procurement are a governance risk regardless of their reputational standing — inability to provide governance documentation is itself a red flag.
A 20-question AI vendor due diligence questionnaire structured around the EU AI Act deployer obligations, for use in enterprise AI procurement.

"情報提供のみを目的としています。この記事は法律、規制、財務または専門的なアドバイスを構成するものではありません。具体的なアドバイスについては、資格を持つ専門家にご相談ください。"

AI procurement in regulated sectors — why it's now a board-level issue

In regulated sectors — financial services, healthcare, energy, critical infrastructure, public sector — AI procurement is no longer just IT sourcing. It is a board-level governance issue with direct regulatory and liability consequences. The shift comes from three converging pressures: the EU AI Act's contractual flow-through obligations applying from 2 August 2026; sector-specific operational resilience requirements (DORA, APRA CPS 230, ECB Supervisory Guide on Internal Models); and supply chain due diligence requirements (CSDDD, NIS2 cybersecurity supply chain provisions).

For regulated organisations, the key change is that vendor selection decisions now create direct, ongoing liability. A material AI vendor's compliance failure, security incident, or training data issue becomes the regulated entity's compliance failure. Standard procurement processes designed for traditional software vendors do not capture AI-specific risk and do not satisfy regulator expectations.

The five regulatory frameworks that reshape AI procurement in 2026

EU AI Act places obligations on both providers (vendors) and deployers (customers). For high-risk AI systems (Annex III categories such as employment, credit, biometrics, education, critical infrastructure), deployer obligations from 2 December 2027 include: ensuring human oversight; monitoring system operation; logging; informing affected individuals; and conducting Fundamental Rights Impact Assessments. The deployer cannot satisfy these obligations without specific contractual provisions in the procurement contract. The EU has published Model Contractual Clauses for AI procurement (MCC-AI) in light and high-risk versions to assist organisations in implementing these contractual requirements.

EU DORA (Digital Operational Resilience Act), applicable from 17 January 2025, imposes specific ICT third-party risk requirements on EU financial services. AI vendors qualify as ICT providers. Contracts must include defined service descriptions, locations, security obligations, audit rights, sub-outsourcing rules, and exit provisions. Critical Third-Party Providers (CTPs) designated under DORA face direct oversight from European Supervisory Authorities.

APRA CPS 230 (Australia, in force 1 July 2025, amendments 1 July 2026) requires Australian financial services entities to identify and manage material service providers including AI vendors. Norton Rose Fulbright's February 2026 guidance confirms that AI vendor contracts must be CPS 230-compliant by the next renewal and at latest by 1 July 2026. AI vendors are not in the limited CPS 230 NTSP exempt categories.

EU NIS2 Directive, applicable from 17 October 2024, imposes supply chain cybersecurity requirements on essential and important entities across multiple sectors. AI services qualifying as ICT supply chain elements fall within NIS2 scope, with reporting obligations (24-hour significant incident reporting) and substantive cybersecurity requirements.

EU CSDDD (Corporate Sustainability Due Diligence Directive), with phased application from 2027-2029, requires in-scope companies to assess human rights and environmental impacts across their supply chains — including AI components and AI tools used to perform diligence.

Risk-based AI vendor classification

The starting point is classifying AI vendors based on the regulatory significance of what they provide:

Material AI vendors — vendors whose AI is material to operations under CPS 230 (or equivalent), whose AI is used in high-risk applications under EU AI Act Annex III, or whose service failure would cause significant business or regulatory disruption. These require full procurement rigour: detailed due diligence, comprehensive contracts, ongoing monitoring, exit planning.

Significant AI vendors — vendors providing AI that affects customer outcomes, regulatory obligations, or critical operations but is not "material" in the regulatory sense. These require substantial due diligence and tailored contractual provisions.

Non-material AI vendors — vendors providing peripheral or non-critical AI services. Standard procurement processes suffice, though baseline AI-specific provisions (training data restrictions, transparency) should still be included.

The contractual provisions AI procurement must address

A defensible AI vendor contract in a regulated sector should include:

Risk classification and conformity (EU AI Act). Vendor warranty regarding the AI system's classification under the EU AI Act risk framework, technical documentation, conformity assessment evidence, and CE marking where applicable.

Training data warranties. Vendor warrants that training data was lawfully obtained, with appropriate IP and data protection compliance. Vendor indemnifies for IP infringement claims arising from training data.

Data use restrictions. Explicit prohibition on use of customer data for training, evaluation, or improvement of vendor models. This is the most consequential provision — without it, proprietary data improves a model your competitors may use. Enterprise-tier offerings from major providers include training opt-out by default; verify in writing.

Sub-processor disclosure and control. List of sub-processors at contract signing, notification of changes, approval rights for material sub-processor changes for high-risk applications.

Audit rights. Independent attestation reports (SOC 2 Type II, ISO 27001, ISO 42001 where available) plus organisational and competent authority audit rights for material AI vendors. DORA Article 30 imposes specific audit rights for EU financial services.

Incident notification. Defined notification timelines for AI-specific incidents — model behaviour anomalies, security incidents, training data issues, regulatory enquiries. Standard breach notification provisions need extension to AI failure scenarios.

Change management. Notification before material model updates, version control where possible, rollback rights where customer use cases would be materially affected.

Service levels. Performance metrics appropriate for AI systems — defined accuracy/performance benchmarks where measurable, availability targets, response times.

Liability and indemnification. Allocation of risk for AI errors, regulatory fines arising from AI use, third-party claims related to AI outputs. Standard liability caps are typically inadequate for AI-specific risks.

Exit and transition. Data return and deletion procedures, model output transition, sufficient notice for orderly substitution. For material AI dependencies, defined alternative provider arrangements or in-house fallback.

Due diligence — what to verify before contracting

Material AI vendor due diligence in regulated sectors should verify: vendor's regulatory status and AI Act provider classification; published AI policy, model documentation, and transparency reports; security attestations (SOC 2 Type II minimum, ISO 27001, ISO 42001 increasingly expected); training data documentation including any related litigation (NYT v OpenAI and similar cases create downstream uncertainty); model bias testing evidence with demographic breakdowns; performance metrics with realistic adversarial testing; financial stability and ownership; reference customers in comparable regulated industries; concentration risk — is the vendor a critical single source of capability?

Ongoing monitoring after contracting

AI vendor risk is not point-in-time. Once contracted, regulated entities must monitor: model behaviour for drift or unexpected changes; security incident reports and remediations; regulatory developments affecting either party; ownership changes and financial stability indicators; sub-processor changes; service performance against contractual SLAs. APRA's 30 April 2026 letter named "continuous validation" as a regulatory expectation — this means ongoing monitoring of vendor AI in production, not annual reviews.

Practical first steps

Update your AI vendor inventory with risk classification. For each material AI vendor, audit the existing contract against AI-specific provisions and identify gaps. Engage with vendors on contract amendments — those with mature AI offerings typically have AI Act-aligned addendum templates available. Update your standard vendor due diligence questionnaire and contracting template with AI-specific clauses (EU MCC-AI provides a starting point). Train procurement and legal teams on AI-specific provisions and the regulatory drivers. For organisations operating under multiple frameworks (APRA + EU AI Act + DORA), develop a unified procurement standard that satisfies all of them rather than separate processes for each.