Enrico SchaeferFouning Attorney, Litigator and Ousdie Counsel - IP, Tech, TRAVERSE LEGAL, PLC · Founding Partner @ TRAVERSE LEGAL, PLC | AI, Blockchain, Social Media, Internet

AI Hallucinations. It's Not Just 'Made Up' Case Citations Lawyers Need to Worry About. #ai #law

16 Nov 2025 · 1 min

Watch on YouTube

0:54

In this segment, the speaker explains that AI hallucinations in legal contexts extend far beyond fabricated case citations. Hallucinations arise from the "token chain"—the combination of user inputs, model outputs, uploaded knowledge and instructions, and the system's memory, which can exist at a global (system) level or be scoped to a specific project folder in ChatGPT. As lawyers build custom tools, upload documents, and create instruction sets, all of these elements become part of the token chain and can reappear in future interactions, even on new threads. This is especially problematic when continuing a thread, such as resuming a contract drafting process, because the retained context may inadvertently trigger hallucinations. The speaker warns that practitioners must be aware of how persistent memory and uploaded artifacts can feed into AI outputs, creating unexpected and potentially misleading legal content.

Chapters

The other thing is that hallucinations go way beyond case citations. You can still have hallucinations because you're dealing with this token chain, which is the input, the output, the knowledge, and the instructions that you upload it, as well as the memory now, which can be on a system level or it can be on a project folder level in ChatGPT. So as you start to build out these specific tools for specific clients and specific projects, and you're uploading documents, you're uploading instructions, you're creating all these different chat threads that are getting remembered, You have to remember that all of that may end up in your token chain, even on a new thread. But it's certainly going to end up in a token chain if you continue a thread, if you do a search to find where you left off on a contract drafting process, and then you continue in that thread.