Anthropic discovered that large language models develop internal structures to count characters and track page boundaries without being taught to do so. The models create spiral-shaped representations that space out numbers to avoid interference, similar to place cells found in animal brains. This emergent ability suggests AI systems spontaneously build tools for understanding their own outputs.
What would you see if your whole world was made out of numbers? Yes, like in The Matrix movie. A bat sees with sound. A reindeer shifts its vision into ultraviolet for the winter. But an AI system does not have eyes. All it sees is a flood of numbers. So how come it can understand your questions? How can it pass the bar exam? I mean, what is going on here? You see, today's AI systems are able to turn a bunch of input tokens into geometry. Yes, it quietly builds little curved shapes inside. They denote areas, concepts, and decision boundaries. And if we do this at a really large scale, these systems have properties that almost feel like magic. Getting a gold medal performance at the International Mathematical Olympiad, No problem. Now check this out. Let's ask a simple question. Little AI, if I add the word aluminum here, does it fit the page, or does it go over? Well, this simple task is surprisingly difficult for it. Why? Because the model gets no character counts and no page width. Yet, it still answers correctly. Wait, what? It does not have eyes, so how come it can judge that? Even the input tokens it gets do not come with character count information. So, before it can count characters, it has to learn to count by itself. Then, figure out how wide the page is, and then subtract. Okay, okay, now I hear you asking, Károly, who cares? Why does this problem matter? Well, it matters because of two things. One, whenever we give an AI a task it has seen or done before, it usually does pretty well. But if we give it a fundamentally new task it hasn't seen before, what happens then? Of course, it fails, right? No, not necessarily. In these cases, during its training step, it recognized that it needs a tool for line lengths, and invented that tool from scratch. Nobody told it to do that. And with this paper, we are now able to crack it open and see the tool sitting inside. Ooh, that is incredibly amazing. And I am thinking, perhaps that is a spark of intelligence. Second, earlier, researchers put a mouse in a box and recorded its neural brain activity as it wandered around for delicious cheese. And they found something that's wild. A few neurons that fired only when the mouse stood in a particular spot. Move to another spot, another neuron fires up. But only then. They call these neurons place cells, and it's kind of like a GPS built into the mouse's brain, which the brain builds on its own. It also has boundary cells that fire up when the animal is near a wall. Now hold on to your papers, Fellow Scholars, because the AI grows the same two ideas on its own. It has neuron-like features that fire depending on how far along the line it is, or when it is near the end of the page. When scientists talk, what I just said in the paper is the following: character counts are represented on low-dimensional curved manifolds discretized by sparse feature families analogous to biological place cells. Yes, nobody built this into the system. It found that it has to learn this by itself. As the mouse feels there is a wall nearby, the AI made neurons recognize that it is near the end of the page. And that is why this whole experiment matters. Not because of this stupid aluminum, no! But now that we see a bit of the insides of an AI system, scientists found even more surprising things sitting in there. One, it does not count numbers. No! It counts tokens. And multiplies by roughly 4 characters each. Like when we count books on the shelf and assume each of them is about 1 inch thick. 2. Now get this. The counter is not one number. It is a rippling spiral. What does that even mean? And why? What is that good for? Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. This is the most challenging part of the paper, so I'll try my best to explain it. Imagine turning on an old radio. You turn the dial, and if two stations are very close together, their sound bleeds into each other. Hmm, if only we could space the stations further apart, that would be perfect. Wait, we can actually do that! That is what these spirals do. This way, every number gets its own channel, but further apart. And once again, no one asked for this. The AI found during its training by itself that it has to do this to become more reliable. That is absolutely incredible. Wow. And if it built secret spirals to find numbers, I wonder what else is hiding in there that we haven't looked for yet. I feel like we are becoming the biologists of a new kind of mind. Maybe call it robopsychology like Asimov did 70 years ago? Either way, what a time to be alive! And now, let's get Doom to run on it. I use Lambda to reproduce AI research papers, often in minutes. It's also great to train your own models, or fine-tune an existing one. Run inference, or text to image or video? Easy peasy. Running a Deepseek chatbot or agent? Super fast, super reliable. Lambda gives you powerful NVIDIA GPUs to run your own experiments. I test ideas from the papers I cover, and moments later, Results. Love it. Seriously, try it out now at lamda.ai/papers.
