The disclosure obligation: you must say you are AI

The EU AI Act's Article 50 transparency requirements prohibit AI systems that interact with natural persons from concealing that they are AI when the person sincerely wants to know. This means: when a customer asks "am I talking to a real person?" or "are you a chatbot?", an AI customer service system must disclose that it is AI — it cannot deny being AI. This prohibition applies in the EU and, through the extraterritorial reach of the EU AI Act, to any organisation whose AI customer service interacts with EU residents.

Beyond the EU AI Act's specific prohibition, many jurisdictions' general consumer protection law prohibits misleading conduct — and an AI system that creates the impression of human interaction when it is not is arguably engaging in misleading conduct. The practical standard: deploy AI customer service with clear identification that it is AI, at the start of interactions and when sincerely asked. This is not only legally required but commercially sensible — customers who discover they were deceived about whether they were talking to a human have stronger negative reactions than those who knew they were interacting with AI from the start.

Accuracy and consumer law

AI customer service systems that make incorrect statements about products, pricing, service terms, or entitlements create consumer law liability for the business. The AI system is acting as the business's agent in the interaction — its statements are the business's statements for consumer law purposes. An AI chatbot that incorrectly states a product has a warranty it does not have, that quotes an incorrect price, or that makes a commitment the business cannot fulfil has made a misleading representation. The consumer's remedies (refund, replacement, damages) apply against the business, not the AI provider.

The accuracy requirement for AI customer service is therefore higher than many businesses realise. Accuracy must be verified before deployment, monitored in production, and maintained as products, prices, and policies change. AI customer service systems are often updated less frequently than the business information they represent — creating growing accuracy gaps over time. Quality management for AI customer service must include a process for updating AI systems when business information changes and for monitoring accuracy of AI responses in production.