Retail AI is mainstream, governance is not
Artificial intelligence in retail and e-commerce is not an emerging technology, it is the operating infrastructure of modern retail. Recommendation engines determine what products customers see. Dynamic pricing algorithms set prices in real time across millions of SKUs. Demand forecasting AI drives inventory and supply chain decisions. Customer segmentation AI shapes marketing and loyalty programs. Fraud and returns detection AI influences which transactions are approved and which customers face friction.
What is emerging, and emerging fast, is regulatory attention to this infrastructure. Consumer protection authorities in Australia, the EU, the UK, and the US are increasingly focused on how retail AI affects consumers, particularly in relation to personalised pricing, manipulative design patterns, and discriminatory outcomes. Governance that was sufficient two years ago is increasingly insufficient today.
Recommendation engines and personalisation
Recommendation engines are among the most commercially significant and least regulated AI applications in retail. They determine, for each customer, which products appear in search results, which promotions are shown, and how the retail environment is configured. The personalisation is based on inferred characteristics, purchase history, browsing behaviour, demographic signals, that the customer may not know are being used.
The governance questions for recommendation engines: What data is being used to personalise recommendations? Is the personalisation creating a genuinely better customer experience, or is it primarily designed to increase basket size regardless of customer benefit? Are recommendations personalised in ways that create discriminatory outcomes: for example, showing higher-margin products to lower-income customers, or showing different product ranges based on demographic characteristics?
The EU Digital Services Act, which applies to large online platforms, requires algorithmic transparency, users must have access to a recommendation system that is not based on profiling. While this primarily targets very large platforms, it signals a regulatory direction that retail AI governance should anticipate.
Dynamic pricing: the discrimination risk
Dynamic pricing AI adjusts prices based on demand signals, competitor pricing, inventory levels, and customer characteristics. The efficiency benefits are real: better inventory management, competitive pricing, and margin optimisation. The governance risks are also real, and they are not always visible in standard pricing analytics.
The primary risk is price discrimination based on demographic characteristics, intentional or as an emergent property of the pricing algorithm. If a dynamic pricing system produces systematically higher prices for customers in lower-income postcodes, older customers, or customers with certain browsing patterns that correlate with demographic characteristics, it may be creating outcomes that breach anti-discrimination law regardless of whether discrimination was intended.
In Australia, the ACCC has pursued cases involving algorithmic pricing practices that were found to be misleading or unconscionable. In the EU, consumer protection authorities have investigated dynamic pricing in e-commerce for compliance with the Consumer Rights Directive and Omnibus Directive. Governance requires: documented analysis of pricing algorithm outcomes stratified by relevant customer characteristics; a process for identifying and addressing discriminatory pricing patterns; and customer-facing transparency about how pricing is determined.
Dark patterns and manipulative AI
The EU AI Act's prohibition on AI systems that use "subliminal techniques beyond a person's consciousness" or that "exploit any of the vulnerabilities of a specific group of persons" applies to retail AI that is designed to manipulate purchasing behaviour rather than facilitate it. This is a meaningful constraint on some e-commerce practices that have become industry standard.
Countdown timers that show false scarcity, social proof notifications that exaggerate purchase frequency, recommendation systems designed to exploit identified anxieties or impulse purchasing patterns, all of these may fall within scope of AI manipulation prohibitions as they are applied and clarified by regulators.
Australia's Australian Consumer Law prohibitions on unconscionable conduct and misleading representations create domestic law constraints on manipulative retail AI that predate the EU AI Act but are increasingly being applied to digital retail contexts. Governance requires a review of customer-facing AI for compliance with evolving dark pattern standards.
Demand forecasting and supply chain governance
Demand forecasting AI (models that predict future sales to drive inventory, procurement, and supply chain decisions) creates a governance challenge that is different from consumer-facing AI but equally significant. Single algorithmic failure points in demand forecasting have caused supply disruptions that affected retailers' ability to serve customers, particularly during demand shocks.
Governance for demand forecasting AI: documented model performance standards with monitoring and escalation processes; human oversight of algorithmic procurement decisions above defined thresholds; scenario testing for demand shock conditions; and supplier agreement provisions that address algorithmic procurement, some suppliers now require contractual protections against purely algorithmic procurement decisions without human review.
Returns and fraud detection
AI systems that identify potentially fraudulent returns or flag transactions for additional friction are among the highest-impact retail AI applications from a customer experience perspective. A customer whose legitimate return is declined or whose transaction is delayed because of an AI fraud flag has a materially negative experience that may result in lost loyalty, complaints, or, in egregious cases, regulatory complaints.
Governance requires: bias testing of fraud and returns detection models, stratified by demographic characteristics where data permits; a clear and accessible appeals process for customers whose transactions are affected by fraud AI; monitoring of false positive rates and their distribution across customer segments; and limits on automated action, very high-confidence fraud flags may warrant automated intervention; lower-confidence flags should require human review.
