A partner at Sullivan & Cromwell, a notable Wall Street law firm, has formally apologized to a federal bankruptcy judge after realizing that several legal citations and other details in court documents were unintentionally fabricated by AI.
A senior partner of the firm reached out last week to Manhattan Chief Justice Martin Glenn, admitting that their past filings included incorrect quotes along with what he referred to as AI ‘hallucinations.’ These documents were submitted on behalf of Prince Global Holdings, which is currently in bankruptcy and is being represented by Sullivan & Cromwell.
Andrew Diedrich, who co-heads the global finance and restructuring team at the firm, outlined the issue in his letter. He described ‘hallucinations’ as instances when AI tools create fictitious case citations, misrepresent legal authorities, or generate nonexistent legal resources. “We deeply regret that this occurred,” he stated.
The letter included a diagram specifying the errors in the motion, which involved incorrect names, case numbers, and quotations that appeared to be entirely made up rather than drawn from actual legal precedents. Such inaccuracies pose a significant breach of the standards expected in federal court filings where precise citation of legal authority is crucial.
The oversight wasn’t caught internally within Sullivan & Cromwell; it was flagged by an attorney at Boies Schiller Flexner, another law firm representing creditors in bankruptcy cases. Diedrich expressed gratitude to the other firm for identifying the errors and extended an apology for the lapse.
Sullivan & Cromwell has been around for 140 years and has more than 1,000 attorneys, securing its place as one of the leading law firms on Wall Street. Diedrich noted that the firm follows strict policies on the use of AI in its legal practices and has put safeguards in place to avoid such missteps. However, he admitted that those procedures were not adhered to in this case, leading to failures in the citation review process.
Previously, it was reported that the head of Morgan & Morgan, another law firm that recently faced issues with AI-generated inaccuracies, labeled the risks posed by AI in the legal field as “sickeningly terrifying.” In an internal communication shared during a court proceeding, Morgan & Morgan’s chief transformation officer cautioned over 1,000 lawyers that relying on fake AI-generated cases in legal documents could have serious ramifications, including potential termination. This followed an incident where a lead attorney cited eight non-existent lawsuits in a case against Walmart, which were later found to be created by ChatGPT, an AI chatbot.
Concerns are rising about the escalating reliance on AI tools within the legal sector and the dangers of using them without adequate verification. Walmart’s legal representatives are urging the court to impose sanctions against Morgan & Morgan based on the argument that the referenced cases “don’t seem to exist outside of the realm of artificial intelligence.”
As law firms grapple with fake case citations in their filings, it seems like a larger awareness is dawning across America about the dual nature of AI—capable of offering immense benefits yet equally fraught with risks. Wynton Hall, a social media director for Breitbart News, has authored a book titled Code Red: Left, Right, China, and the Race to Control AI, exploring how the MAGA movement can adopt a productive stance on AI that serves humanity without ceding power to Silicon Valley or allowing China to dominate.
