Technology companies — particularly SaaS providers, platform companies, and B2B software vendors — face AI governance obligations that operate at two levels simultaneously. First, internal AI use: AI in engineering (code generation, code review), sales (lead scoring, deal intelligence), marketing (content, personalisation), operations (analytics, automation), and customer support (AI agents, knowledge bases). Second, AI embedded in products sold to customers — AI features, AI-powered workflows, AI assistants, and AI APIs that customers integrate. The combination creates distinctive governance demands that pure-internal AI governance frameworks do not address. This guide covers the operating model for technology companies governing AI on both sides.

1. ISO/IEC 42001 as the de facto B2B standard

ISO/IEC 42001:2023 has emerged as the de facto AI governance standard for B2B technology providers. Enterprise customers in regulated industries — financial services, healthcare, public sector, professional services — increasingly require or expect ISO 42001 certification or equivalent evidence of governance maturity from their AI vendors. The certification process typically takes 6-18 months. Annex A controls cover policy, leadership, planning, support, operation, performance evaluation, improvement, and AI-specific concerns (risk assessment, impact assessment, data management, lifecycle management). The investment is significant but increasingly necessary for AI-enabled B2B providers selling into regulated markets. For technology companies not yet on the certification path, NIST AI RMF implementation provides an alternative reference, and credible roadmap-to-certification is typically acceptable to enterprise buyers.

2. EU AI Act provider obligations

The EU AI Act distinguishes provider obligations from deployer obligations. Providers — entities that develop AI systems and place them on the market — face the most demanding obligations for high-risk AI: risk management system, data governance, technical documentation, record-keeping, transparency, human oversight design, accuracy/robustness/cybersecurity, conformity assessment, registration, CE marking, post-market monitoring, serious incident reporting. Annex III high-risk obligations apply from 2 December 2027 under the Digital Omnibus delay. Annex I (embedded in regulated products) high-risk obligations from 2 August 2028. GPAI providers face Article 53 obligations from 2 August 2025: technical documentation, downstream provider documentation, copyright compliance, training data summary. GPAI providers of systemic-risk models face additional obligations: model evaluation, systemic risk assessment, incident reporting, adversarial testing, cybersecurity protection.

3. Enterprise customer governance expectations

Enterprise customer expectations of AI vendor governance have crystallised. The current baseline: training data exclusion — explicit contractual commitment that customer data is not used to train models, including fine-tuning or model improvement; model card documentation — purpose, capabilities, limitations, performance evidence, intended use, known failure modes; bias testing evidence — methodology, demographic coverage, results, remediation; incident notification — defined timelines and content for AI-related incidents; audit rights — including third-party audit and AI-specific audit; IP indemnification — for AI-generated content where applicable; data residency and sovereignty — where data is processed and stored; exit and portability — data return and transition support. For APRA-regulated customers, additional CPS 230 material service provider obligations apply.

4. Product-level AI governance that scales

The challenge for SaaS providers is scaling AI governance across product features and customer use cases. Manual review per use case does not scale beyond a handful of customers. The patterns that work: policy-as-code — AI usage policies encoded into platform controls that customers can configure; structured documentation — model cards, capability statements, and use case guidance generated and maintained alongside the product; customer-facing controls — administrators can configure AI behaviour for their organisation (which AI features are enabled, data handling, retention, audit logging); automated compliance — features like training data exclusion, audit logging, and incident detection built into the platform rather than implemented per customer.

5. AI vendor management within technology companies

Technology companies are themselves AI buyers — typically of foundation models (OpenAI, Anthropic, Google), specialist AI services (vector databases, AI safety providers, evaluation platforms), and infrastructure (AWS, Azure, GCP AI services). Vendor management responsibilities propagate from the foundation model provider, through the technology company, to the enterprise customer. Documentation of upstream vendor relationships, data flow mapping, and incident response coordination across the chain are essential. Customers in regulated industries increasingly ask vendors to disclose upstream AI providers and the governance arrangements for those relationships.

The operating model

A defensible technology company AI operating model includes: internal AI governance (policy, inventory, risk classification, training, monitoring) for AI used in operations; product AI governance (model documentation, bias testing, accuracy monitoring, incident response, customer transparency) for AI in products; ISO 42001 readiness or implementation as the integrating framework; customer-facing governance documentation (trust centre, security and AI documentation, model cards) accessible to customers and prospects; upstream vendor management for foundation model and AI service providers; regulatory engagement as the framework evolves.

Useful third-party resources

Related reading on AIRiskAware