The Research Behind Thinking Machines Lab's Interaction Models-Explained

So in this video, I'm going to do a research preview breakdown of interaction model research by Thinking Machines. And this research got me very excited and interested because I've been working on like voice projects or BAI, and then I always keep hitting this roadblock and constraint that the conversation never felt very natural. There's usually some kind of a latency or this does not feel right. In a voice-focused product, that natural conversation is the key. And it's just not the problem I noticed that Orbee AI project that I'm hitting roadblocks on, but also most of the voice-based project, it's been a big problem. At the same time I was going through this challenges of that natural interaction between the user and the agent, I came across this research by Thinking Machines, and it's about the interaction model. And I think it's a very interesting research finding which I want to share it with you. So they kind of take a very different approach how they solve it, and they show some of the demos and things which are very interesting to see, which I'm gonna walk you through this video. So this research paper This paper was published on May 11th, 2026, and May 2026. So when you talk to a ChatGPT voice or Claude or any other agents, when you talk to them, you see that you need to finish a conversation and then the agent will talk back to you. So this is something called turn models. So the machine and the human, they take turns. That's why they're called a turn model. So what happens is like, that's not usually how a human communicate from one person to another person. Feels very unnatural. And also sometimes when you are talking, we are thinking stuff. Right now the agents can't detect or understand that you are thinking and things or like jump in between you, ask you follow-up questions. They wait for you to finish your thought and then they talk. That's not natural and it breaks the entire experience of conversation between human and agents. I also, while reading this paper, I thought like agent can switch the context very fast. And I did an experiment. I took Claude and opened the voice and started talking with it. And I kept switching context very fast of like talking about cars and then cats and skies and things like that. But the model is able to do that. That's very interesting. But it did say that like, oh, you are keeping on my toes. It did say that. But I think that's never— that is not a problem that I have. Problem was like the mid-thought. If you are like having a thought and then mid-thought you have something else, Capturing that thought is very hard for agents right now. It also can't do like, you know, two contexts at the same time. For example, let's say you're vibecoding, then you're talking about Skies and Karts. It cannot do that. It needs to do one thing to finish another. Current limitations of this model right now. So most of the companies, what they do is they use this hardness method. If you don't know what hardness is, I created another video on hardness engineering. So you can look at that. They create some kind of infrastructure and hardness and ways around, or even making like UX trade-offs, right? Sometimes the agent will like guide you back or the designs are done in such a way, the conversation that it's around the limitations of the models, right? And there are designed like guardrails and things like that for the agent. But that's, so this research paper, it argues that creating harnesses and infrastructure like that is backward approach. And the best approach is to bake in that capability within the model itself. What it does, it like building harnesses and infrastructure, it's a very expensive thing, right? And this voice industry is like $22 billion industry and most of the companies have this problem. Just imagine the scale of this problem, right? And even if you build a voice-based product, you'll start hitting this problem that how I kept noticing while building OddB AI. So one of the interesting thing that this research was calling out was like most of this agents are how companies are kind of like building with building agents is like to keep less human in loop collaboration and more like autonomous. I mean, it makes sense that You don't want to keep, you know, babysitting the agents and stuff. But this, I think that's the best way to work with the agencies with that human collaboration, human-in-loop, more of that for the judgments and things. And this kind of paper focuses on bringing more human-in-loop with this agent systems and stuff. And it tries to solve this problem in multiple levels, which I'm gonna walk you through. First approach they propose or they're exploring is MicroTURNS. So like I said, human and the agent, they take turns, right? But with micro turns, instead of finishing that complete turn in midway, they take like micro turns. So that way it helps to switch context or the agent to jump in between and ask you follow-up questions and things. That's a way kind of one of the things approaching. So here you can see the example. Let's see one of the examples here.

I'm going to tell you sorry. And whenever you hear that word, please Gotcha. I'll count them out as you go. Let's hear it. Okay, last weekend I drove down to South Bay to visit a fan.

So here you can see the agent is like continuously listening and then it recognizes the user is like thinking and taking time. So it's waiting for the users to finish, right? It has that context around that.

3. I really wish I saw a capybara instead. 4. You said deer once, sheep once, coyote once, and capybara once.

You see, it's joining in the conversation and counting as the user talks. It's not waiting for the users to finish the thought. It's counting in.

Cherry. 2. Then on the way back, we saw a coyote. 3. I really wish I saw a—

So it's joining in the conversation. That's what makes it a very interesting one. And the second thing is like they try to solve it also through multimodal processing, a unique approach. When humans, we talk to each other, we just don't talk through voice. We have our body language or visuals, expressions, and things like that. Right now, the audio, video, text processing, doing all that at the same time is harder. This paper focuses on doing that real-time processing. It can do multimodal processing at the same time. So here you can see an example.

And you'll be golden. You're starting to slouch forward. Try pulling your shoulders back.

So the agent is understanding the visual cues as well and interrupting or jumping in and telling the user.

Try keeping it level. You're upright again.

Yeah, so it works off— works from the visual cues. And then they also have this experiment where people will start talking in like different languages at the same time and it switches. And this is really good, right? Like just imagine in a room of people, you are brainstorming with a group of people. Right now you need to finish your talk. In order for the agent to talk back to you, right? Then that breaks the entire collaboration approach. But now it is listen and it will jump in between the conversation and talk to you. So it fosters more human and agent collaboration. The third method they take, the second method is that, you know, native multimodal processing. And the third one is split architecture. So the way that works is The user talks with the interaction model, which they're introducing, and the reasoning and the context and the tool calling is given to the other model, the background model that works behind processing reasoning and everything. And the interaction model talks to the users directly, making that conversation feels more natural. So if you are building voice-based product, you'll definitely start hitting this walls and this pain point and that it keeps breaking that experience and that conversation does not feel natural. Sometimes I was first experiment with Orbea. I was making it like a voice-based to-do list. It takes time, especially when I asked for, give me a summary of my to-do list, and you select this latency and time and stuff and having, having, trying to have a conversation with it. Now I'm pivoting that Orbea AI more towards a thought capture tool. I feel like that's the biggest use case that I'm seeing. And I'll publish once that app is ready. I'll definitely share this in my portfolio, the side project that I'm building. But this interaction model is very interesting, and I think it also goes to show that, you know, the hardness method versus the model capability method, where this is heading. And yeah, it's— I just wanted to— yeah, and yeah, I wanted to share this research breakdown with you. And then yeah, if you find any interesting part of this research or want to, um, or want more research like that, let me know in the comments. And yeah, do subscribe to this YouTube channel