Beyond the Mirage: Beware of Generative AI and Hallucinations; New York State Bar Association (NYSBA), June 26, 2026

Cynthia Feathers , New York State Bar Association (NYSBA); Beyond the Mirage: Beware of Generative AI and Hallucinations

"The work of attorneys can be arduous. With demanding caseloads comes an openness to tools that can help us do our jobs more efficiently. The advent of online legal research was a game changer for attorneys decades ago. In recent years, generative artificial intelligence – AI that can create original content such as text, images, video, audio or software code in response to a user’s prompt or request – has begun to revolutionize legal practice.[1] This development has included the integration of AI into legal research and writing.

The focus here is on the risks inherent in popular generative AI models used to complete such tasks: They are prone to producing false legal information, so-called hallucinations, including false case citations and false reasoning, quotes and holdings. A nationwide epidemic of cases involving such fabrications has made the risks of unverified AI use well known. Hundreds of decisions have touched on this issue.[2] Thus, the legal profession has been alerted that blind faith in generative AI results is misplaced.

This article makes no attempt to be exhaustive as to the rapidly unfolding case law but does seek to highlight some emblematic decisions issued by state, federal, trial and appellate courts throughout the country over the last two years and to bring attention to the dangers of failing to check AI results...

Evolving technology is seductive in creating the illusion that it can save us from the hard work. But our ethical duties to our clients and the courts still require that we rigorously verify every case cited. When generative AI output becomes more reliable, new questions will arise about how far we can go in abdicating our lawyerly judgment to new technology.[20]

For now, New York attorneys should be aware of a new rule on AI adopted by the New York State Unified Court System. Effective June 1, Part 161 of the Rules of the Chief Administrator of the Courts permits the use of AI tools in preparing submissions to a court and does not require the disclosure of such use. However, Part 161 sets forth a model rule that does require attorneys using such tools to “carefully review” each submission and “independently ensure” that they do not contain “fabricated or fictitious cases, statutes, or other material.”[21]  Individual judges retain discretion to implement their own AI-related rules, adopt the model rule or impose no additional requirements through their part rules.

Perhaps soon we will see more standing orders on AI use and updated ethics rules nationwide targeting AI issues.[22]  In the meantime, in the use of AI, we can be guided by the new court rule and longstanding mandates regarding competence, diligence, accuracy and candor and the supervision of lawyers and nonlawyers."

Ethics journal retracts paper by high school student for AI, peer review manipulation; Retraction Watch, July 6, 2026

 Retraction Watch; Ethics journal retracts paper by high school student for AI, peer review manipulation

"The Journal of Medical Ethics has retracted a paper on the use of AI in the pharmaceutical industry for containing references that don’t exist. The article’s sole author: a high school student. 

The paper, which argues biased algorithms can exacerbate inequities in health care, was published in September. The author, Irfan Biswas, listed his affiliation as Shrewsbury Public Schools in Massachusetts.

According to the May 28 retraction notice, an investigation by the journal found Biswas used generative AI to “identify and understand referenced sources” and did not verify the references prior to submission. 

“The journal investigated concerns about the quality of the work and the accuracy of the references, including concerns that several references did not exist,” the notice reads. 

Last year another ethics journal made a similar retraction after a reader found fabricated references in a paper on whistleblowing. The Biswas article joins the estimated one in 277 papers indexed in PubMed with fabricated references, a phenomenon that came about with the rise of AI tools like ChatGPT.

The retraction notice for the latest paper also cites “evidence of peer review manipulation.”

Caroline White, media relations manager for BMJ Group, which publishes the journal, declined to elaborate on the problematic peer review. Biswas confirmed to the journal he was a high school student, and agreed to the retraction, White said. 

Biswas did not respond to our emails or LinkedIn message asking for clarification on how the references ended up in the paper. 

In an August 2025 paper in Frontiers in Genome Editing, “Ethical dimensions and societal implications: ensuring the social responsibility of CRISPR technology,” Biswas listed affiliations with the University of Massachusetts Medical School in Worcester and the University of Rhode Island in Kingston. A representative from UMass told Retraction Watch they have no record of Biswas attending the school. URI said a student by the same name is enrolled as an undergraduate, but said they were “unable to confirm whether or not it is the same individual.” 

AI in the Trenches and on the Bench; American Bar Association (ABA), April 8, 2026

 Paulette Marie Rodriguez Lopez, Richard Platkin, Jeffrey Huang, Alina Lee, and Bradford Newman , American Bar Association (ABA); AI in the Trenches and on the Bench

"Ethical considerations require legal practitioners to balance innovation with professional responsibility across multiple dimensions. The duty of competence demands that lawyers understand the capabilities and limitations of AI tools they employ, while communication obligations require transparency with clients about AI use in their matters. Confidentiality concerns arise when sensitive client data is processed through third-party AI platforms, particularly when terms of service remain ambiguous about data retention and use. The duty of candor to tribunals has taken on new significance as courts grapple with AI-generated hallucinations, exemplified by high-profile sanctions in cases where lawyers submitted fabricated citations. Supervisory duties extend to ensuring that junior attorneys, paralegals, and staff use AI tools appropriately and in compliance with firm policies.

Courts have responded with divergent approaches: Some have issued standing orders prohibiting AI use in court filings entirely, with potential sanctions ranging from striking pleadings to contempt citations and case dismissal, while others have declined to adopt special AI rules, instead emphasizing that reliance on AI will not excuse otherwise sanctionable conduct. These varied judicial responses reflect ongoing uncertainty about how to balance technological advancement with the integrity of legal proceedings.

Despite these risks and varied judicial approaches, AI tools offer significant potential for enhancing legal practice when used responsibly. Use cases range from chambers research and case management to in-house contract review, legal research, document drafting, and litigation support at law firms of all sizes."

Navigating Today’s AI Landscape with an Ethical Polestar; American Bar Association (ABA), April 30, 2026

 Afton Pavletic ,  American Bar Association (ABA); Navigating Today’s AI Landscape with an Ethical Polestar

"On November 30, 2022, OpenAI released ChatGPT to the general public. The model, along with other generative AI (GenAI) technologies, flooded headlines and seized the spotlight. Less than six months later, the story of Mata v. Avianca, Inc.  —a case in which the plaintiff’s attorneys filed a brief featuring ChatGPT-hallucinated case citations—went viral and fired an emergency flare in the legal community. With a growing awareness of AI’s potential to transform the profession, the legal field responded with a cascade of AI guardrails, guidelines, and task forces. Judges targeted GenAI usage with standing orders, at times outright prohibiting the use of the technology   ; state bars and legal associations issued opinions and recommendations tailored to the Rules of Professional Conduct  ; law firms and government agencies crafted AI-use policies for their offices  ; and the list goes on.

So where are we today? With avoiding ethical missteps as its compass, this article seeks to map out the current AI legal landscape. After launching from the shores of the Rules of Professional Conduct, it will discuss prevalent technologies in use by legal practitioners, the risks associated with the current wave of AI, how courts have responded to AI-related issues, and ways to steer your practice’s AI policy to avoid troubled waters."

Can we trust AI models? Yale researchers explore the roots of chatbot errors; YaleNews, June 12, 2026

 Mike Cummings, YaleNews; Can we trust AI models? Yale researchers explore the roots of chatbot errors

"The rapid rise of artificial intelligence (AI) has inserted a new character into people’s lives: the chatbot. 

Individuals now engage with agentic AI chatbots to perform a growing number of tasks; They can help a person shop for a new laptop, manage email, or plan a vacation.   

And while these interactions can save time and increase productivity, they also carry risk. Large language models (LLM) — the AI systems trained on massive datasets to generate human-like text — are imperfect. They hallucinate. They misinterpret. They make mistakes. 

Two multidisciplinary teams of researchers associated with the Center for Algorithms, Data, and Market Design at Yale are pursuing projects that aim to balance the capability and safety of AI systems and improve interactions between users and AI models. 

Yale News recently spoke with members of both teams about their research projects."

More courts are coming down on ‘non-offending counsel’ for AI missteps; ABA Journal, June 10, 2026

 AMANDA ROBERT , ABA Journal; More courts are coming down on ‘non-offending counsel’ for AI missteps

"Amid the proliferation of cases involving artificial intelligence-generated hallucinations, more judges are expressing frustration not only at the attorneys who make the mistakes but at opposing counsel for not pointing it out. 

In the past year, courts have admonished attorneys for failing to identify and report fake citations in their opponents’ court filings. In at least two cases, judges refused to award attorney fees or grant relief to counsel who didn’t bring AI-induced errors to their attention."

Federal judge removes 4 plaintiff and defense attorneys over AI errors; ABA Journal, June 10, 2026

 AMANDA ROBERT, ABA Journal ; Federal judge removes 4 plaintiff and defense attorneys over AI errors

"A federal judge in Mississippi on Monday disqualified the plaintiff counsel and the defense counsel after both parties filed briefs with artificial intelligence-generated mistakes in a dispute over attorney fees.

“This case presents the court with an unusual scenario—attorneys for both litigants engaged in similar sanctionable conduct,” said Senior U.S. District Judge Sharion Aycock of the Northern District of Mississippi in her June 8 order...

Aycock revoked Wilson’s and Williams’ pro hac vice statuses and barred them from appearing in the Northern District of Mississippi for two years. The judge also ordered all four attorneys to pay monetary sanctions, ranging from $1,000 each for the local counsel to $2,500 for Williams and $3,500 for Wilson."

We Asked the Future of Truth Author to Explain How He Used AI. It Didn’t Go Well; Wired, May 29, 2026

 Kate Knibbs, Wired; We Asked the Future of Truth Author to Explain How He Used AI. It Didn’t Go Well

"EARLIER THIS MONTH, WIRED published an excerpt from Steve Rosenbaum’s buzzy new book, The Future of Truth, which looks at how artificial intelligence warps people’s sense of reality. Shortly thereafter, The New York Times reported that the book contained over a half-dozen made-up or misattributed quotes. In a statement, Rosenbaum, who has a master's degree in "truth" from New York University, admitted that he had accidentally included “a handful” of “improperly attributed or synthetic” quotes. In an ironic twist, the veracity of a book about how AI impacts truth was now under intense scrutiny because of how its author had used AI."

Florida Makes New AI Rule: Check Your Damned Work Or Else!; Above The Law, June 1, 2026

Chris Williams, Above The Law; Florida Makes New AI Rule: Check Your Damned Work Or Else!

"No matter how much the marketing department leans in on an LLM program being ready to help you with your law practice, you’re still responsible for the work product because you’re the with the actual license to practice. Florida’s Supreme Court made some changes to remind lawyers that the responsibility falls on them, not the black box they’re prompting their legal questions into. ABA Journal has coverage:..

My soft spot for the rule update is for pro se litigants. Despite all of the equality under the law trimmings we pretend to believe, fighting in court is a monied person’s battlefield. Pro se litigants are often in that position because they cannot afford a lawyer due to limited resources. Getting help from Claude could be the closest thing they have to getting a legal opinion. If and when they screw up these rules, I hope that the court shows as much grace as it can toward them. Even the annoying sovereign citizen variety."

Book on Truth in the Age of A.I. Contains Quotes Made Up by A.I.; The New York Times, May 19, 2026

 Benjamin Mullin, The New York Times; Book on Truth in the Age of A.I. Contains Quotes Made Up by A.I.

"The author of a nonfiction book about the effects of artificial intelligence on truth acknowledged on Monday that he had included numerous made-up or misattributed quotes concocted by A.I.

The author, Steven Rosenbaum, whose book “The Future of Truth” was released this month to great fanfare, incorporated more than a half-dozen misattributed or fake quotes in sections of the book reviewed by The New York Times.

The Times asked Mr. Rosenbaum about the quotes on Sunday and Monday. On Monday night, Mr. Rosenbaum acknowledged in a statement that the book had “a handful of improperly attributed or synthetic quotes” and said that he had started his own investigation.

He said that the inclusion of the incorrect quotes was an accident and that he had “no intention of fabricating any viewpoints” while writing the book.

“As I disclosed in the book’s acknowledgments, I used A.I. tools ChatGPT and Claude during the research, writing and editing process,” Mr. Rosenbaum said in the statement. “That does not excuse these errors, of which I take full responsibility. I am now working with the editors to thoroughly review and quickly correct any affected passages; any future editions will be corrected.”

“The Future of Truth” was published by an imprint of BenBella Books and distributed by Simon and Schuster. BenBella Books, which operates independently of Simon and Schuster, did not respond to a request for comment. Simon and Schuster declined to comment."

AI won’t replace lawyers. It will create more of them.; The Washington Post, May 17, 2026

Damien Charlotin , The Washington Post; AI won’t replace lawyers. It will create more of them.

"The replacement story often rests on a particular picture of what a lawyer does: reading documents, applying rules and producing text. Since AI can read, apply rules and produce text, the argument goes, lawyers are cooked. That picture is not entirely wrong, but it is the perception engineers have always had of the legal domain: Feed in the facts, apply the rule, return the output. Yet the reason lawyers exist (and command high prices for their services) is that law is shot through with ambiguity. If the rules ran themselves, no one would need us. Every step in the chain — reading, applying, producing — involves choices, some of which are genuinely difficult.

A better way to think about jobs is as bundles of tasks. Some bundles are loose: A job composed of a handful of discrete, repetitive, well-specified tasks can be peeled apart and the tasks automated one by one. Other bundles are tight, because the tasks reinforce one another and cannot be cleanly separated. The key example here is offered by radiologists, long predicted to be facing extinction due to AI. Despite the dire forecasts, their numbers keep growing, and they keep commanding ever-higher salaries.

Legal work is also hard to neatly separate. For instance, doing legal research and evaluating an argument are, for an experienced lawyer, often the same mental activity: A lawyer checks the argument by writing it. Pull those tasks apart, hand the writing to a machine, and verification suddenly becomes a separate, deliberate, expensive act — at least if you want to avoid landing in my database of courts sanctioning parties for filing “hallucinated” material. In fact, an irony is that automating the easy parts of a job often makes the hard parts harder, not easier."

Law Schools Implement AI to Focus on Ethics and Technology; Los Angeles Times, May 17, 2026

 David Nusbaum, Los Angeles Times; Law Schools Implement AI to Focus on Ethics and Technology

"Over the last two years, Loyola Law School in Downtown Los Angeles has incorporated AI into six courses. It’s a sign of a growing trend where law firms are looking for attorneys who can utilize the technology to improve efficiency. While law schools have constantly looked to update coursework to keep curriculum updated as laws are updated, the application of generative AI to the practice of law is the biggest change that has happened in generations, according to Rebecca Delfino, associate professor of law at Loyola Law School...

Delfino is one of several professors who have integrated AI into their coursework. She is involved with two courses specifically focused on the ethical implications of generative AI and the legal practice.

In a first-year civil procedure course, students are divided in half, with one group an analog approach that relies on textbooks and class notes while the other half uses generative AI technology. The results are compared to see where the technology is effective and ineffective. The goal is to use AI as something that is additive rather than giving over too much authority and power, according to Delfino. For many exercises, there are six or seven AI models that are tested and compared.

Students understand that they need the AI skill set to make themselves a more attractive candidate, no matter what area of law they practice. It can be used to draft documents, conduct legal research and assist with discovery. Chatbots are tested for hallucinations, and the drawbacks are identified."

Most U.S. doctors are quietly using this AI tool. Few patients know about it.; NBC News, May 13, 2026

  Jared Perlo, NBC News; Most U.S. doctors are quietly using this AI tool. Few patients know about it. 

"Almost two-thirds of physicians — or roughly 650,000 doctors — in the U.S. actively use OpenEvidence, while another 1.2 million use it internationally, OpenEvidence representatives said. With its quick and tailored replies, OpenEvidence has become an AI-era equivalent of consulting a colleague for their expert opinion, though the software can also write patient discharge notes and provide custom study tools for doctors’ medical exams.

“Sixty percent of all the searches are about how to make clinical decisions,” said Jena, who is currently examining 90 million OpenEvidence queries submitted since 2024 as part of a new research project. “The physicians are asking: For this particular patient, or with this profile, this condition, maybe other comorbidities that they have, what’s the right treatment?”

Yet with OpenEvidence’s skyrocketing popularity, some experts worry about potential hallucinations or incomplete answers, a lack of rigorous scientific studies on the tool’s patient impact, and the potential for doctors’ critical thinking and evaluation skills to erode with increased OpenEvidence use and dependence.

But many in the medical world see OpenEvidence as a time-saving tool that can improve patient care."

Google’s AI Summary Invents State Ethics Rules… And It’s Not A Hallucination Problem; Above The Law, May 6, 2026

  Joe Patrice , Above The Law; Google’s AI Summary Invents State Ethics Rules… And It’s Not A Hallucination Problem

"If you’re a Pennsylvania lawyer wondering whether you need to disclose AI use in your court filings, Google’s AI summary has an authoritative answer for you. It’s a wrong answer, mind you. But authoritative!...

If one opens an incognito window and searches Google for “Pennsylvania AI disclosure lawyers,” the AI-generated summary will explain that “Key developments include mandatory disclosure of Generative AI (GAI) in court filings.” Throw in “August 2024” because you vaguely remember seeing something about AI on that date and the result reads “As of August 2024, Pennsylvania mandates explicit disclosure of AI use in all court submissions, making transparency a mandatory filing requirement.”

None of that is true.

The Legal AI Governance tracker, an invaluable tool maintained by Brian Alenduff of Desired Path Consulting, provides a comprehensive rundown of Pennsylvania’s AI rules. There are standing orders in some courtrooms, and the state supreme court issued a rule governing court personnel only, but as for the state of Pennsylvania writ large, there is no statewide rule as of now. The tracker notes that what Pennsylvania does have is Joint Formal Opinion 2024-200, a 2024 advisory ethics opinion from the Pennsylvania Bar Association and the Philadelphia Bar Association flagging AI as a competence issue under existing rules. But that opinion explicitly states that it is “advisory only and is not binding.” The ABA’s own 50-state survey classifies Pennsylvania as “court dependent.”...

Hallucinations are all the rage right now, but over the long haul the greater AI risk will be an unfailingly credulous bot elevating and validating mistakes until the error gets picked up as reality."

Copyright Infringement Suits Loom With Unchecked AI Vibe Coding; Bloomberg Law, April 29, 2026

Christopher Suarez, Bill Toth, Anthony Pericolo, Bloomberg Law; Copyright Infringement Suits Loom With Unchecked AI Vibe Coding

"Deferring the job of software coding to artificial intelligence doesn’t immunize that code from copyright risk—it could even increase it, if the person directing the coding has limited oversight over the result.

This is particularly true with “vibe coding,” where developers use high‑level natural language prompts to generate code using AI models, often with limited manual review or modification of the resulting code.

Just as lawyers should check for “hallucinated” citations when writing with large language models, engineers and software development managers need to have human and technical monitoring protocols to account for infringement and licensing risks."

A.I. ‘Hallucinations’ Created Errors in Court Filing, Top Law Firm Says; The New York Times, April 21, 2026

  Santul Nerkar, The New York Times; A.I. ‘Hallucinations’ Created Errors in Court Filing, Top Law Firm Says

Sullivan & Cromwell apologized for submitting a court document that had fake citations created by artificial intelligence.

"An elite Wall Street law firm has apologized to a federal judge for submitting a court filing replete with errors created by artificial intelligence, including “hallucinations” that fabricated case citations.

The A.I.-generated errors came in a recent motion in U.S. Bankruptcy Court in Manhattan and were discovered by lawyers from an opposing firm, Andrew Dietderich, a partner at Sullivan & Cromwell, wrote in a letter to Judge Martin Glenn on April 18."

AI Is Getting Smarter. Catching Its Mistakes Is Getting Harder.; The Wall Street Journal, April 14, 2026

  

Katherine Blunt , The Wall Street Journal ; AI Is Getting Smarter. Catching Its Mistakes Is Getting Harder.

As chatbots and agents grow more powerful and ubiquitous, recognizing the moments when they go rogue can be tricky

"Chad Olson was confused when his Gemini artificial-intelligence chatbot told him he had a family reunion planning session marked on his calendar."

Sanctions ramping up in cases involving AI hallucinations; ABA Journal, April 2, 2026

 AMANDA ROBERT, ABA Journal ; Sanctions ramping up in cases involving AI hallucinations

"The use of monetary sanctions against attorneys is seemingly on the rise as courts continue to address artificial intelligence-generated hallucinations in case documents."

The dictionary sues OpenAI; TechCrunch, March 16, 2026

Amanda Silberling, TechCrunch; The dictionary sues OpenAI

"Encyclopedia Britannica and Merriam-Webster have filed a lawsuit against OpenAI, alleging in its complaint that the AI giant has committed “massive copyright infringement.”

Britannica, which owns Merriam-Webster, retains the copyright to nearly 100,000 online articles, which have been scraped and used to train OpenAI’s LLMs without permission, the publisher alleges in the lawsuit.

Britannica also accuses OpenAI of violating copyright laws when it generates outputs that contain “full or partial verbatim reproductions” of its content and when the AI lab uses its articles in ChatGPT’s RAG (retrieval augmented generation) workflow. OpenAI’s RAG tool is how the LLM scans the web or other databases for newly updated information when responding to a query. Britannica also alleges that OpenAI violates the Lanham Act, a trademark statute, when it generates made-up hallucinations and attributes them falsely to the publisher."

AI legal advice is driving lawyers bananas; Axios, February 9, 2026

 Emily Peck, Axios; AI legal advice is driving lawyers bananas

"AI promises to make work more productive for lawyers, but there's a problem: Their clients are using it, too.

Why it matters: The rise of AI is creating new headaches for attorneys: They're worried about the fate of the billable hour, a reliable profit center for aeons, and are perturbed by clients getting bad legal advice from chatbots.

Zoom in: "It's like the WebMD effect on steroids," says Dave Jochnowitz, a partner at the law firm Outten & Golden, referring to how medical websites can give people a misguided understanding of their condition."