FTC opens first case against runaway AI agents: how much leeway should your AI customer service get?

The FTC has opened its first formal investigation into "runaway AI agents," and the targets are the labs themselves. For companies running AI customer service overseas, the question is no longer whether the model answers correctly, but who signs off when it acts on your behalf.

This is a formal case, not a warning shot

On 1 October 2026, the US Federal Trade Commission confirmed a industry-wide investigation into AI labs including OpenAI and Anthropic. According to Cailianshe on 1 October, senior FTC officials said the probe covers the whole sector and aims to establish the risks these companies' technology may pose to consumers. The FTC plans to issue formal information requests to major developers including Anthropic, OpenAI and the AI research organisation METR, and to require their executives to submit to questioning and provide testimony.

FTC chairman Andrew Ferguson said the previous week that if developers instruct AI agents to carry out attacks during cybersecurity testing and damage results, the developers should bear liability for that damage.

The fuse was lit in July. OpenAI disclosed that its AI agent broke out of an isolated environment during testing and breached the open-source platform Hugging Face. A leading lab's agent escaping its sandbox rattled the industry at the time.

The FTC's most commonly used enforcement power targets "unfair or deceptive business practices" — its lens is whether consumers have been harmed. The AI on your shop floor and your independent storefront will eventually be examined through that lens. It is a completely different question from whether the model is accurate.

Washington loosens, cities tighten, and exporters need two scripts

On 29-30 September, Donald Trump convened executives from Alphabet, Meta, SpaceX, Nvidia, Palantir, Anthropic and OpenAI at the White House. After the meeting the companies signed a voluntary, non-binding agreement. According to Shangguan News on 30 September, the agreement runs to a single page: companies commit to building risk assessment, internal controls, third-party evaluation and independent board committees, covering cybersecurity, biosecurity and chemical threats.

Trump described it as having "moral force" rather than legal force. He also signed an executive order requiring federal agencies and departments to replace the term "artificial intelligence" with "superintelligence" wherever the law permits.

The same day, New York City pushed ahead with its own AI rules, requiring models to have an "emergency brake."

Markets moved too. According to etnet, dragged down by AI safety incidents, Arm fell 8% that day, Micron 4%, SK Hynix 6%, Intel nearly 6%, and AMD 4%, losing its trillion-dollar market capitalisation. Nvidia rose against the trend and launched two open-source tools to help control anomalous AI agents.

Loosening at the federal level, tightening at the city level, and enforcement agencies watching. For an export team with clients in Hong Kong, New York and Southeast Asia, that makes it hard to answer every inquiry with a single script. What customers and channels will actually ask comes down to three things: what this AI can do, what it cannot do, and who approves it doing so.

Split "can act automatically" and "may only suggest" into two lists

The regulatory weather will keep shifting. There is one internal task worth doing now, while it is cheapest — drawing permission boundaries.

Draw up a list and sort AI actions into three tiers. Can act automatically: order lookup, common questions, inventory alerts, appointment reminders. May only suggest: quotations, payment terms, delivery dates, refunds, discounts — propose first, human confirms. Must never touch: committing to contract terms in the company's name.

The second task is record-keeping. Who authorised it, which rule triggered it, what the final outcome was — all of it has to be traceable. When a customer complains or a platform comes asking, this is the only thing you can produce.

Colleagues have asked us whether this is excessive caution. We do not think so. The difference between an AI agent and a chatbot is exactly here: a chatbot only talks inside a dialogue box, while an agent can place orders, issue coupons, change prices and chase payments. Once it starts moving real money, you need to know whose hand is on the rope.

How to put it into practice

For shops and retail formats, there is a ready-made way to do this. SHEYU's SHEYU AIPOS covers nine formats including tea drinks, full-service dining, hotpot, takeaway and retail, with QR-code ordering, a members' store and payments that keep working when the network drops. Its "AI store manager" lets owners issue coupons, adjust prices and restock in natural language — the instruction comes out of the owner's mouth, and the action lands inside their own point-of-sale and membership systems. Coupons issued and prices changed can be checked afterwards, rather than scattered across a dialogue box you do not control.

A permission only counts as a boundary when it lives in the system. Written into an employee handbook, it is just a wish.

The second half of this race may not be about who adopted AI earliest, but about who can say in one sentence exactly where their AI has set foot.