AI Lies to You, Here's How

We all know that AI doesn't always give you exact answers. But how much is AI really making up? Well, the truth is that people make stuff up on the Internet all the time, and AI is no different because content is what it's trained on. It's called AI hallucination, and here's what you need to know about it. If you're new here, I'm Jim Carter. Welcome to the prompt. When an AI writes text, it is not checking facts or pulling from a trusted database. It is making the best guess for the next word based on patterns it learned during training. It works like supercharged Autocomplete. Sometimes the guess matches real life, sometimes it does not. The tone stays confident. Either way, these hallucinations show up in a few common ways. First are basic factual mistakes. Think wrong dates, wrong names, wrong numbers, that kind of thing. Second are made up sources. We're talking fake research papers, fake court cases, and made up stats that sound real. Third are logic jumps. The AI connects ideas that do not belong together and builds a conclusion that sounds right but has no real support. The last two are the most concerning because without some serious Internet literacy, it's easy to believe what AI has told you. The stats are honestly a little alarming. Even top models hallucinate more than 15% of the time on structured fact tests. When researchers tested AI tools on citing news sources, the results varied a lot. Perplexity had a 37% hallucination rate, which was the lowest in that test. ChatGPT was at 67%. Gemini was at 76%. Grok3 was at 94%. Almost almost every citation it gave was wrong or made up. Fact checking helps, but the problem runs deeper. Hallucinations happen because of how these systems are built. They are trained to produce likely text, not to verify truth. When you ask a question, the model aims for the most likely answer pattern from training, even when the facts are missing. This matters because AI is being used in high risk places. 77% of companies surveyed say hallucinations are their top concern. With AI. In healthcare, a false claim can lead to a wrong diagnosis. In law and finance, false claims can create major risk. One law firm was fined over $100,000 after filing AI written briefs with fake citations. The training data adds to the issue. These models learn from huge amounts of Internet text that includes old info, mixed claims and junk. When the model hits a gap, it tends to fill it with something that sounds believable. Long chains of reasoning make this worse. Even each step can add an error and those errors stack up. Prompt pressure makes it worse, too. If you tell an AI to answer as an expert or punish it for saying I don't know, you push it to invent an answer. Studies show this kind of prompt can raise error rates by 15 to 25%. But there are ways to lower the risk. One big method is called Retrieval Augmented Generation, or rag. Instead of letting the model write from memory, you give it specific documents and tell it to use only those sources. This this can cut hallucination rates by 50 to 70%. Cross checking with more than one model helps, too. Ask the same question in different tools and compare the answers. It is not perfect, but it can cut errors by around 30%. Some teams add guardrails, so basically they add checks that block claims with no support before a user sees them. There are also tracking tools like Vectara's Hallucination Leaderboard that measure how often models make things up. The most useful change is how you write prompts. Make room for uncertainty. Tell the AI to use only the sources you provide. Ask it to flag low confidence areas. Require quotes that support each key claim. Make it acceptable for the AI to say I don't know. These changes can cut hallucinations by 25 to 40%. Here's a prompt I use a lot, using only the information in the context below. Answer the question. If the context doesn't contain the answer, say the context doesn't provide enough information to answer confidently. After each factual statement, include the supporting quote in parentheses. Don't use outside knowledge and don't guess another one I like. Answer in three sections. First, claims supported by widely documented facts. Second, ideas that seem possible but are uncertain clearly marked. Third, items you cannot answer. If you are not highly confident something is documented, put it in section 2 or 3. I use AI every day. It helps a lot when you treat it like a tool that guesses likely text, it does not verify what is true. If you trust it without checks, it will eventually burn you. The tech is improving, but it is not solved yet. For now, hallucinations are part of the deal. The fix is knowing that upfront and building a workflow that expects mistakes. If you want to learn more, join me and other builders in my Slack community Control Alt build. We talk about AI safety, practical workflows, and share what works in real projects. I'll drop the link in the show notes. If you understand how AI is built and use prompts that reduce error, it makes AI a lot more trustworthy. Hope this helps and I'll catch you in the next episode.