Misinformation researcher admits ChatGPT added fake details to his court filing

A misinformation expert accused of using AI to generate a legal document admitted he used ChatGPT to help him organize his citations, leading to “hallucinations” that critics said called the entire filing into question. Jeff Hancock, the founder of the Stanford Social Media Lab who wrote the document, says the errors don’t change the “substantive points in the declaration.”

Hancock submitted the affidavit in support of Minnesota’s “Use of Deep Fake Technology to Influence an Election” law, which is being challenged in federal court by Christopher Khols — a conservative YouTuber who posts under the name Mr Reagan — and Minnesota state Rep. Mary Franson. After discovering that Hancock’s filing seemed to contain citations that didn’t exist, attorneys for Khols and Franson said it was “unreliable” and asked that it be excluded from consideration.

In a subsequent declaration filed late last week, Hancock acknowledged that he used ChatGPT to draft the declaration but denies he used it to write anything. “I wrote and reviewed the substance of the declaration, and I stand firmly behind each of the claims made in it, all of which are supported by the most recent scholarly research in the field and reflect my opinion as an expert regarding the impact of AI technology on misinformation and its societal effects,” Hancock wrote.

As for the citation errors, Hancock explained that he used Google Scholar and GPT-4o “to identify articles that were likely to be relevant to the declaration so that I could merge that which I knew already with new scholarship.” Hancock says he used GPT-4o to create a citation list, not to write the document, and didn’t realize the tool generated “two citation errors, popularly referred to as ‘hallucinations’” and added incorrect authors to another citation.

“I did not intend to mislead the Court or counsel,” Hancock wrote in his most recent filing. “I express my sincere regret for any confusion this may have caused. That said, I stand firmly behind all the substantive points in the declaration.”