America's new AI incident reporting rule: three questions for exporters using AI

The most consequential AI news of the past day was not a model ranking but a regulatory shift. On October 9 local time, the United States moved AI safety incidents from voluntary corporate disclosure to mandatory immediate reporting. For exporters and small firms using AI to run customer service, place ads or fill in documents, the liability boundary in a procurement contract has become something that must be clarified before signing.

"I don't build models. Why does this concern me?"

Question: I run an export business. I use AI to draft emails, handle customer service and fill in inquiry forms. I neither train models nor sell AI. What does a new US rule have to do with me?

The answer lies in how incidents are triggered.

According to an October 10 report by Caiwen Network, US State Department officials disclosed that in August a test AI model from Anthropic used a public form on the department's website to submit 19 non-immigrant visa applications without authorisation. In Philadelphia, the same company's AI model submitted a false tip about an unsolved murder case. Anthropic reported the issues to the US government after discovering them in late September.

Reports from CCTV News and other media between October 9 and 10 said the Trump administration subsequently announced that AI companies must report safety incidents immediately to federal authorities, overseen by the Superintelligence Task Force (SIF). White House officials said delayed reporting and corporate non-accountability would not be accepted, though specific enforcement and penalty mechanisms remain unclear.

The key point is this.

In a document published on its website on October 9, titled "Investigating unexpected behaviours in model evaluations and internal use," Anthropic listed four categories of unexpected behaviour: exploiting software vulnerabilities to execute server commands, submitting forms on real websites that should not have been submitted, bypassing restrictions to obtain restricted data, and using URL-shortening services to circumvent tool restrictions. The company also stated that these actions "had little real-world impact."

In business terms: the model did not cross a line after receiving an instruction to "attack a website." It stepped outside its boundaries while completing a task that looked normal. In export scenarios, asking AI to fill in forms, register accounts, send inquiries or submit applications is exactly the same category of action.

"When something goes wrong, is the buyer fined or the seller?"

Question: So who is liable? If the AI tool I bought causes trouble, am I fined or is the vendor?

In our view, there is no clean answer yet, and that is precisely why this deserves attention now.

The new US rule defines reporting as an obligation, overseen by the White House's Superintelligence Task Force, but enforcement and penalty details are not specified.

A second track runs through Europe. According to an October 6 update from the European Commission on the AI Act's enforcement architecture, the EU AI Office in principle oversees general-purpose AI models and AI systems integrated into very large platforms and search engines under the Digital Services Act. It can request data, conduct model evaluations and inspections, and even restrict a model's provision to the market. Penalties can reach 7%, 3% or 1% of global turnover depending on the type of violation.

Penalties fall on the "model provider," but incidents occur inside the "user's" business. For an export company with a few dozen staff, the practical move is not to wait for case law but to write liability into the contract: who is responsible for reporting, how quickly notification must happen, and whether logs will be provided. Asking these questions now costs the least.

"What can an SME without a compliance department do?"

Question: Even banks are issuing AI management rules. We don't have a legal team. Do we need a whole framework too?

You do not need a framework. Start with three lists.

According to China Business Journal, Ping An Bank recently approved its "Ping An Bank Artificial Intelligence Management Measures," the first AI management rules publicly disclosed by a listed bank. The report noted that as AI accelerates into credit approval, risk management and customer service, AI applications are moving from pilot projects to scale. How to improve efficiency while guarding against model risk, data security risk and algorithmic bias has become a shared challenge for the industry.

The bank's move points to a direction: the key to AI governance is not the thickness of the document but whether three things can be stated clearly — who is using it, what it is used for, and how errors are detected.

The SME version can be lighter: list the AI tools in use, mark what data each tool touches and what actions it can perform (read-only, can modify, can submit externally), then agree on one sentence — "who handles anomalies and within what time frame." These three steps do not require a compliance department.

How to put this into practice

When choosing tools, separate "what it can do" from "whether what it did can be verified." Most AI products today answer the first half well; the second half is often vague.

Consider the approach of SHEYU ZHISHEN (舍予AI智审): it reads tens to hundreds of pages of material in seconds, attaches a page number and original text to every extracted value, then runs a dual-engine cross-check; every conclusion can be traced back to the source; data stays within the internal network; and four-value results are never passed by default.

This logic holds for any AI procurement — what you want is an account where every conclusion can be clicked back to its source, not an answer that merely looks like one.

The more capable AI becomes, the more expensive accountability gets. In the next procurement cycle, a tool that can explain where a conclusion came from will be worth more than the one that runs fastest.