FTC is Investigating OpenAI and the Impact of its AI Hallucinations

The U.S. Federal Trade Commission (FTC) has opened an investigation of OpenAI, largely to determine whether its AI “hallucinations” harm individuals by propagating potentially dangerous misinformation.

“[The] subject of [the] investigation [is to determine] whether [OpenAI], in connection with offering or making available products and services incorporating, using, or relying on Large Language Models [LLMs] has engaged in unfair or deceptive privacy or data security practices, or engaged in unfair or deceptive practices relating to risks of harm to consumers, including reputational harm, in violation of Section 5 of the FTC Act, and whether Commission action to obtain monetary relief would be the public interest,” an FTC Civil Investigative Demand (CID) sent to OpenAI, and obtained by the Washington Post, reads.

As a result of this CID, OpenAI must suspend its routine document destruction operations and take measures to preserve any relevant documentation. OpenAI has 14 days to schedule a meeting with the FTC to ask questions and make available employees related to document retention. The FTC will then use any documentation it obtains from OpenAI as part of its investigation.

Among other things, the documents that the FTC wants are related to OpenAI’s LLM-based products, their users and use cases, how Open markets these offerings, and so on. But most importantly, the FTC would like to understand, in detail, how or if OpenAI has tried to determine what impact the accuracy of its products has on users, how it retains and uses information that it collects, and, how OpenAI has trained these LLMs.

And that’s the tricky part, as OpenAI has been very vague on that last point. The FTC wants to know how OpenAI obtains its data, what all the sources of data are, the extent to which public website data constitutes the overall data collected, exactly which websites it has sourced, how it determines which data sources to use, and which various languages are used in its data sources.

Regarding the spread of misinformation, the FTC is looking to see which steps OpenAI has taken to determine the accuracy of its results, especially with regard to individuals who may be slandered by erroneous results, whether they are public figures or not. And the FTC clearly reads the news, as it references a March 2020 OpenAI disclosure in which it stated that bugs allowed individuals to see others’ chat history titles and payment-related information; it would like a lot more information about that and other incidents.

Given my stance in the FTC’s case against Microsoft’s attempted acquisition of Activision Blizzard, you may be surprised to know that I support this investigation. Regulatory bodies like the FTC are mandated to guard against consumer or competition harm from monopolies or other dominant market parties (antitrust), of course, but they are also tasked with protecting consumers more broadly. And it is rare to see a regulatory body go after a quickly emerging and potentially groundbreaking technology like AI. This is just prudent.

It will be interesting to see what comes of this.