I Took Notes on Karpathy's 3-Hour LLM Masterclass — Here's Everything | Part-5 #machinelearning
In this Hindi monologue, Akansha Yadav reviews key takeaways from Karpathy's LLM masterclass. She explains that human‑generated training data is imperfect and that conversational LLMs like UltraChat 1 are needed to generate coherent answers. She describes the problem of model hallucination—confident but false responses—illustrating how to observe it on Hugging Face inference pages. She outlines two primary mitigation strategies: (1) prompting the model to retrieve up‑to‑date information via a web search, and (2) teaching the model to refuse when it lacks knowledge. She then shifts to architectural considerations, emphasizing the importance of the context (working) window measured in tokens rather than characters, and how this limits the amount of text a model can process at once. Finally, she touches on the distinction between knowledge stored in model parameters (the "WAG collection") and the model's short‑term working memory, noting how retention of previously learned facts can be confusing for users.
