#351 Will World Models Bring us AGI? with Eric Xing, President & Professor at MBZUAI

It's the first time that you see a frontier model which is able to be open-sourced, both in model weights but also the data, the training recipe, and so forth, so that people can really take it away and reproduce it.

Large language models have revolutionized many industries already, but there's a growing sense that in order to make the next leap in intelligence, AI needs to gain an understanding of the world. Consequently, World models are being hyped as the next big thing. I want to know what the use cases are and where you can benefit from using them.

So world model is trained on more than textual data, but also image data and other sensory data. It's actually a simulator of the real world, of the physical and social world where agents can learn autonomously and perpetually. We are not building a predictor. World model is not a predictor. World model is a simulator.

Our guest is AI research royalty. Eric Xing is president at Mohammed bin Zayed University of Artificial Intelligence. He's also a professor at Carnegie Mellon University. Previously, Eric's had visiting professorships at Stanford University and Facebook. Eric's AI research focuses on three strands. Firstly, his team's created the PAN world model. Secondly, he's been a prominent open source contributor, having created K2Think v2, the world's most powerful, completely open source LLM. And thirdly, Eric's latest work involves creating digital organisms. Naturally, I want to learn about all three things.

Data right now is a commodity. You actually sell data. The real problem is the technical limitation of long horizon consistency and long horizon planning.

Let's see what Eric has to say. Hi Eric, welcome to the show.

Hi Richie, nice to meet you. Thank you for having me.

Yeah, great to have you here. Now, I want to start off by talking about world models, and there have been a few prominent experts like Yann LeCun have suggested that these the next big thing after LLMs? So world models going to be the thing that takes us to AGI?

First of all, AGI is a very ambiguous buzzword that many people refer to different things. So I'm not sure I know the answer that you ask, that world model alone is going to take us to AGI. In a recent conversation in Web3 Bubbles, I kind of break down intelligence into multiple levels. And where you can see word models as well in the middle. So what currently the large language models is delivering, which we see to be amazing, like solving very fancy math problems, engaging in very intense and clever conversations, so forth. I call all those intelligence to be book intelligence, which basically draws knowledge and insights from written materials. Produced in history, but also deliver that kind of outcome in text. But on the other hand, if you want to do real-world things like pour a coffee into this cup and navigate in the real environment and plan for complex missions and so forth, you already go beyond book knowledge because they need to put those plans and reasoning into action. That requires what we call physical intelligence, right? And obviously, world model is right there to deliver that type of intelligence because the world model is trained on more than textual data, but also image data and other sensory data. So that level of intelligence is stronger, including a superset of book knowledge. But then beyond that, why we need a world model? Well, it's actually a simulator of the real world, of the physical and social world, where agents can learn autonomously and perpetually by just like you sent a pilot into a flight simulator. So world model is basically providing that. It is not going to deliver you all you need, but it will give you the experiences that an agent can learn experiences from. So that level of intelligence is a different one, which I call social intelligence, because now agents need to know skills, but also need to know boundaries of their skill. Who am I? Who are you? How can we collaborate? How can we divide the job among different people so that we can together jointly finish something bigger than every individual? So that level of intelligence is what I call social intelligence, which is really behind a new generation of agent models that is different from what we call now the agents, which are autonomous scripts accomplishing to fix tasks. After you have the agentic intelligence, there is yet another level which I call the philosophical intelligence, which means that the agents start to develop their own curiosity, ask their own problems, and have this drive and maybe even the reward of solving those problems like we do as a scientist in discovery and so forth. And that requires all the levels of intelligence as I just said before. Maybe after philosophical intelligence, I kind of can see what maybe an AGI would look like. So, you know, does world model leads to AGI? Well, I believe it is an essential step, but still not enough.

Well, that's a really great hierarchy. I would say it's something I've not heard before, but certainly the idea of being book smart is, I guess, it's a common plotline. I think in Hollywood movies, you have someone who's read a lot of books and then you throw them into an adventure and they can't cope.

Exactly, exactly. Yeah.

So that's interesting. So the next level up is physical intelligence. So I presume, is this mostly robots are we talking about, or there are other use cases here?

Yeah, robot is a major platform, but you can imagine even, you know, developing personalized games and, you know, metaverse experiences also requires that level of physical intelligence because you need to simulate the physical world very faithfully and creatively and also with infinite complexity so that without going to the physical world, you can already start experiencing all the possibilities. I call this an environment for thought experiments. In fact, we people do reasoning very oftentimes not through optimization but through thought experiments. We just imagine different scenarios. We have our mental world model. That could give you the outcome of a complex sequence of actions that you would like to perform in physical world, but you don't need to go there. That's why we're training the bots which are supposed to be landed on Mars without going to Mars. So the world model basically gives you that possibility. It is a simulator of all possibilities conditioned on actions in all situations. So it could be in a digital virtual cyber environment, also could be in a physical environment. I think a robot to really develop their true general purpose physical intelligence needs a world model. But even people, when they want to train themselves for a complex environment, they would also use a world model without going to the real environment.

Okay, that's interesting. So I mean, you got the sort of the real world physical environment, that's the robots, and then you got the virtual environments, that's games, that's the metaverse, things like that. So there's a choice there. What sort of state, like what does the cutting edge look like? What's the sort of fanciest world model capable of at the moment?

Yeah, so you mentioned about Yann LeCun's model and there are a couple of others out there like the model behind the GPT-4 is also in my opinion a world model. All these world models give you the impression that they can generate scenarios in a physical world. Some through pixel manipulation and you don't actually need to understand the world. You can still generate pretty videos, give you the feeling of having the physical knowledge. If you want to put actions in the middle and start to steer this physical world, you will tell the difference. Some world models are simply not able to take in arbitrary actions in a steerable way and leads you to the outcome that caused by your action. Some are able to. I would say right now there are still a lot of disagreements even on what is a world model. Most of the world model, the so-called world model out there only generates videos and also short videos which are maybe 60 seconds or 2 minutes and so forth. It's not because of a memory boundary or compute boundary. The real problem is the technical limitation of long horizon consistency and long horizon planning. Because for example, you can play with the existing VideoJam models. You say, I look straight ahead and in 5 minutes I turn 360 degrees and show me what you see. You may see a different thing already from what you start with because to maintain consistency in 5 minutes when you turn around is actually an architectural problem. In your architecture, you need to have the representations which carries that information consistently without disruption, without distortion, and also ingest any perturbations and the inputs. And the right architecture right now are still being developed. You saw, for example, one proposal proposed by Yang Kun called JEPA, the Joint Embedding Predictive Architecture. I recently had a debate with him, right? Because I have my reservation about that architecture. That architecture basically says that you need to do reasoning only in the latent space. And also you verify your reasoning also in the latent space, meaning that you only do thought experiment without ever emerging into the real world, which to me is a very, very problematic architecture. The real world gives you the opportunity to correct, to calibrate. But why is that the case? Because coming back to the real world needs a cost. It increases the computational cost whenever you reconstruct what you see in the real world. Especially, there has been argument that it is not necessarily even possible to reconstruct every detail in the real world. And therefore, joint embedding is a cheaper and cost-effective way of doing something. But on the other hand, Is there a different way of doing a closed-loop inspiration that you take information from real world and go back, but without necessarily all the pixel-level details? That basically leads to a representational question. How do you represent real-world knowledge using proper encodings? The architecture that we propose, called the generative latent prediction architecture, is using a generative framework which allows different levels of abstraction. And the granularity to be reconstructed and compared to the real world. I'm not saying this is the right model, but I want to say that the world model debate and development are far from being settled yet. It is just the start of this new movement, which is already going beyond large language models. One thing we all agree who work on world model is that, yes, large language model is not going to be sufficient. To get to that level of intelligence. But then what is to happen next? We'll see. I think in this year, many teams are already in the game to push for their in parallel, basically different architectures. We recently released a PaLM model which actually uses a mixed backbone architecture that combines the symbolic inference capability of a large language model, which allows it to do very long-term inference. Because once you go symbolic, maintaining long horizon consistency is cheap, but on the other hand, it doesn't give you the pixel-level resolution fidelity. So on top of that, we have a diffusion-based backbone that is going to do this more reflective, more high-resolution, short-range consistency inferences alongside the symbolic inferences. So this is a joint architecture that is very different from what we see in other papers, which of course makes the computation a little bit more difficult. But on the other hand, there are additional approaches to reduce the cost through approximation or through infrastructure innovations and stuff like that.

The fundamental problem that you're describing, how do you represent the world? How do you represent knowledge? And I feel like this is something philosophers have been arguing about, I guess, from Plato onwards. So it's interesting that it's still an unresolved debate on how do you best encode this stuff.

I think this is a question that needs to be asked every time you face a new problem, because every time you face a new problem, that means you are solving the problem in a new space of data and possibility. And therefore, the information naturally poses you the question about how to represent that. For example, in the JEPA framework, the proposal is to use only continuous representation in the latent space, which basically rules out this symbolic knowledge that is already dealt with in high efficiency with large language models for a good reason, because continuous information allows you to play with gradient-based calculation, which is efficient. Basically, the assumption is that maybe our brain is only doing gradient descent for learning and for action. But on the other hand, I think we do not only gradient-based computation. We may do nearest neighbor search. Or comparison or indexing-based retrieval. So these are typical operations on symbolic discrete knowledge. So once you have that representation, you open the space of one more modality of computational inference. The interesting part of the research, which goes very technical, different specialists, different innovators really are playing with this very, very big space of all possibilities of innovations. To find both a high-fidelity, high-quality inference engine, but also computationally affordable and efficient. There's obviously a trade-off over there.

Okay, so there's essentially lots of different ways of doing it, and maybe it's going to be different representations for different use cases, and I guess a lot of research to be done. You mentioned that you have your own world model, PaN. Just maybe let's take it a step back. Why did you decide to build this? What were the goals of the project?

The goal of the project is to really try out this new idea and also to allow a more general-purpose world model that is serving our needs. Because in our definition of the world model, it is way more than video generation. If you look at driving, for example, a visually impaired person can also drive very well even though they don't have high-resolution picture knowledge, pixel knowledge of the world. Right. So you actually need the model at its necessary resolution to do reasoning. And that's basically a guiding principle in our world model architecture design. So we design a world model that is allowing you to do thought experiment, to do simulation or reasoning. And we found that there has been no such instances out there trying these ideas. So In the PaLM model, we want to prove, first of all, this is the necessary pathway toward more general intelligence and inference, which allows you to do what people called system 1 reasoning and system 2 reasoning. One is reflective and intuitive. The other is more deliberate and purposeful symbolic reasoning. And these two are both needed. One is called maybe muscle memory. You don't need to think and you can still act, but the other requires you to do very, very abstract thinking so that you can do long-term planning. I think PaLM is the only architecture out there that allows both types of reasoning to be facilitated. And also our PaLM model benefits from our very own development of the large language model. So different from the Yann LeCun's architecture, we don't necessarily dismiss large language models. We think large language model is not enough, but it's also useful because we use that as a building block inside the world model to cover a part of the reasoning task, which is long-term symbolic reasoning. Therefore, we actually have our own backbone large language model trained from scratch. From other projects. So we have a very good ecosystem unlike many other developers where we can be self-sufficient in providing all these different building blocks. And when we find deficiency and flaw, we can go back to this model and retrain or augment. This is very different from taking, for example, a QoEM model or some other open-weight model, plug them in and just use as what it is. In our case, we have a more systematic and holistic approach and opportunity to get everything right.

Okay, I like what you said about there being a difference between muscle memory and having to think abstractly about and reason about things. I suppose that's the way humans do this as well. It's like sometimes you can do a task and you don't need to think about it, it's just something you've learned and you're straight away solving it. Okay, so do you want to talk me through more about like what goes into this model then? So you said you've got a large language model in there. What sort of data goes into to train a world model?

So it is not trained at this point monolithically in one shot, right? So the model divides into different modules. You have a visual language model as an encoder to basically ingest the real-world signals, be it visual signals, or textual prompts. They basically turn these signals into some latent representations, discrete or symbolic, that can be entering the reasoning backbone. The reasoning backbone itself is a large language model, but augmented with a bigger vocabulary because you actually get a lot of new vocabulary by discretizing the visual signals. Therefore, you actually are really influencing not in the traditional textual dictionary, but also more abstract discrete representations representing physical phenomenons and behaviors. And then the last stage is a diffusion-based decoder which allows you to, in a sense, produce the outcome of the action and the reasoning in the way that can be compared with what you actually see in the real world. So these three components are trained separately based on either pure text data or pure visual data or the combination of visual and text data. In fact, we also have in the future placeholders for sensory data, audio data, and other modalities. And once these building blocks are separately produced into some satisfactory quality, you put them together to do the joint tuning and the fitting. It's like building an aircraft, you know, parts, you know, built separately, but then there's an assemblage phase where you need to do a lot of tests, you know, holistically on everything.

Okay. So it sounds like the sort of the end goal for this is you've got lots of different models sort of working together in harmony and maybe some sort of top-level controller to decide which bit, which models need to work overall. And then the output is some sort of a diffusion model. So is this, this is video generation? That's always the output, is it?

The output is importantly a video generation, but more than that, video generation is usually a pragmatic need because I need to convince people that I'm doing something reasonable. Therefore, the best way to convince people is to let them see it, right? But our internal evaluation is actually not only based on the video data. The video quality is an important but inadequate metric of modeling, of measuring the quality of the data because they only measure pixel continuity, visual consistency, action generation, and so forth. But our goal for the world model is to solve long-term reasoning problems. Therefore, the ability to solve actual inference problems, for example, can you put objects into the right bin? For example, do you also have a long-term consistency in the order of minutes and hours after you take the first signal? So these type of measurements of performance are not done in the video space because you cannot generate hours of videos. In fact, it's useless to generate that many. We actually directly take latent representations and decode them in proper form to measure the quality.

Okay, I see. So you mentioned the idea of putting objects in bins. It's like a common factory task. You've got some robot arm has to put an object somewhere. Is that the most common use case you think? It's going to be used in factories, or do you have anyone using it for real tasks at the moment?

We haven't had anyone using it, but that's actually the set of benchmarks that we're developing right now. The Bing object sorting is one simple task that people can understand because you can basically visually see, I have this path to put the object in this way and condition on the intermediate results. My next move will be so on and so forth. Or the other branches. There's a branching simulation. That can happen in parallel so that your thought experiment can happen. But again, video in this case is an auxiliary window for you to peep into the activity rather than the outcome of the simulation, right? You talk about utility, we imagine now two possible utilities already in preparation. One is indeed a robotic platform. We want to equip a robotic platform with this kind of a world model so that they can autonomously accomplish certain mission without I program the robot or train the program at the robot to do exactly that. They should be able to do their own thought experiment using the world model and figure out what to do. The other actually is the game. We actually have a number of potential parties reaching out. I can name two scenarios. One is the sports game. How can you basically even capture all the moments and maneuver and tactics when you play soccer or other fancy games. So using the world model, you can actually have infinite possibilities of simulating all these scenarios. So the gamers would be given a lot more possibilities of playing their different experiences. The other use case that we already were approached with is the autonomous driving. Again, when you want to train very, very safe autonomous driving agents, you don't necessarily have all the videos you need, especially those videos with accidents, because in real world, such things happen very rarely. You just don't have enough videos for those. But in your world model, you can simulate all imaginable accidents the way they could happen and then turn it into videos. And then these videos will be given to the driving agents to learn policy offline. By the way, this is another differences of our world model. In using world model, many of the current systems are using an algorithm called MPI. I think it's called multiple projection. I don't know. I forgot the name for that. Basically, it's an optimization problem. You simulate all the scenarios. Conditions, then you solve a sequence of actions that optimize certain scores. In our case, we also allow reinforcement learning to be used to learn policies offline. When you are not doing any task, you can play with the world model like a chess player play with himself based on their world model of the chess. In this case, the world model of the word to learn policies of different conditions. And then when you are in real action, you first scan your brain and look for those policies. Maybe they already solve the problem you face, then you directly use it very, very quickly. And if not, then you run the real-time online inference to solve the problem. So the world model really gives you multiple different operational modes for System 1, System 3, or System 3 inferences.

Okay, so that's very interesting. It seems like, so you talked about factory examples, you talked about computer game examples, the self-driving car examples, and that last one particularly, I like the idea that you can simulate crashes without having to go and like crash a real car somewhere. It seems like a much cheaper or much safer approach anyway.

If you go to the Penn, you know, we have a website called pennworld.ai. You actually will see some demos about how those accidents are simulated. And we actually will be very soon opening up a playable site. People can prompt the system with their own scenarios and also in real time steer the model to generate different scenarios. Okay.

And so for some of these other use cases, like we have factories that kind of work already, computer games exist already, what do you expect the impact to be? Like, how do you measure the success of PaLM or these world models? Like, is it about making things cheaper or is it about exploring things that you couldn't do before? What's the sort of the impact you're striving for?

Multiple impacts, right? So, you know, the game market, you know, first of all, is already, you know, a very lucrative and promising, you know, consumer space, you know, for generating revenue because you can now generate games a lot faster and cheaper and also interactively and personalizable, you know, with the user. So this is a space that we definitely would look into. But another low-hanging fruit for the value of the world model is the data generation. Data right now is a commodity. You actually sell data. But with the world model, you don't have to basically send the crews to collect the data from real world. You do that, but not necessarily for everything. Your world model can actually generate data on demand. And then giving more complex scenarios. Again, this word, how word model operating is fascinating, right? It's not like you need to know all the physics and all the theory before you create what you actually see as an outcome of this theory. Because look at, for example, how people generate the AlphaGo scenario, right? The rule of the AlphaGo is very simple. You have a few rules of playing the Go or playing the chess.. But the combination of that in simulation gives you infinite complexity. Therefore, you can get huge complexity out of a very, very simple set of principles. That's basically what models are. The world model itself doesn't have to be infinitely complex, but the way world models are used can generate infinite complexity scenarios. And that's actually where I saw a lot of potential because without, for example, going to Mars, going to the moon, going under the water to see everything that you could possibly see. You already could extrapolate from the training data to generate those experiences.

That's very cool. So do you think there can be some scientific breakthroughs aided by world models then?

Actually, that's a very good question. So I talked about the 4 levels of intelligence. As now the world model is a basis for you to create a simulation of the real-world possibilities. And one of these possibilities that we are also exploring in another project is the possibility of life. Imagine how do you want to design a medicine? Why people require drug trials of multiple phases? Because you don't have a simulator of life so that you have to basically test your drug in a wet lab. On maybe animal models. But imagine that you have a virtual cell or a virtual organism, which is actually a world model, which will simulate back to you the potential outcomes and responses to drug or genetic perturbation. Then you actually could shortcut a lot of the wet lab and the physical trial and experiments for drug validation. So this is where I see an immediate possibility of a breakthrough in the science because you dramatically reduce the risk and the cost of scientific experimentations.

Okay, that seems very cool. I mean, certainly there are so many different simulation-type problems where the computational space is just incredibly huge and having these world models, I guess if you can reduce the time to run these things or the cost to run these things, that's definitely going to help with breakthroughs. Now, you mentioned AlphaGo a few minutes ago. Now I remember there was AlphaGo was the first sort of champion Go player model, but then there's also AlphaZero, which was kind of the same, but it wasn't taught the rules of Go. It learned them itself. And I'm curious with the world models, do you have to feed the rules of physics into them or do they learn the rules of physics themselves? So is it like the AlphaGo or the AlphaZero?

I think the world model as of now, is at the AlphaGo phase, which is purely draw connectivities and learn embeddings and manifolds from data so that those manifolds can be used to extrapolate from visible data and then get the possibilities beyond the seen data to be simulated. But I think that's the first step toward AlphaZero. Because once you have this data or this capability of simulation, you can always go backward from what we simulated and start to trace off how those outcomes are produced and start to examine basically the thinking path in those models, and some of which may actually reflect a physical law. So I think there is a potential to reverse engineer. Some of the simulation results within a world model and then hopefully helping you to discover the real physical laws out of it. Again, I'm not sure, I'm speculating on that, but I think this is a natural move people would do next time.

Okay. All right. So an evolving field then, and we're kind of gradually moving towards learning the universe from first principles, I suppose. That's like the long-term thing.

Okay.

All right. So I'd like to talk about your LLM as well. So of course you've got an LLM called, let me get the name right. So it's Kato. K2, not to be confused with KIMI K2 Thinking. The names are very close, so don't get those two confused. Anyway, tell me about K2 and what the goals were around this and how it's different from other large language models.

Yeah, so first of all, we started the K2 series even before KIMI started their K series because they already are going beyond K2 and call it K2.5 or 3 or whatever because the K stands for differently, right? In our case, the K2. Refers to that mountain, that very difficult mountain in Kalakandrin. Originally, the goal is to build a very, very open implementation of the large language models in the academic way so that the community can actually reproduce our results and can study that and then promote the field of large language model to be more transparent and also more rigorous and maybe more safe. Because once you expose all these issues, there are more opportunities for people to look into it and make it close the loopholes and discover any problems. But as we do more and more on this, we also realize that on the one hand, you need to be scientifically rigorous, transparent, and willing to share. But on the other hand, you need to first and foremost be performant. If you have a weak model that is not as good as the industrial frontrunners, nobody is going to use it, even though it is academically interesting project to play with. And that basically leads to continuous versioning of the K2 model. We took two parallel approaches. One is, of course, to keep building the base model, which needs to be done from scratch. And it's a long-term, expensive, and also not necessarily a high-reward project because beating those frontier models, which are orders of magnitude bigger than us, is very difficult. We don't have that much resource. All we can do is to do a mid-size model that is adequate for that level of task. On the other hand, there are special interests in the UAE community and also in my university on advanced reasoning such as mathematics, such as IQ games and so forth, where you need to use strong base model but also do a lot of post-training, long train of thoughts, reinforcement learning, and in the inference time optimization, many other techniques that is specially designed on strong data to enhance the post-training capabilities. So we want to establish also our track record in both tracks. And also to test some of our new ideas in both pre-training and post-training. So these two things go in parallel. Our K2-Think last year first release was actually based on a commodity open-weight model, which is the Q1 model. We use it and we do post-training on that, which bump up the score 30 or 40 points, which get us into the first tier comparable to the DeepSeq models and a few other frontier models. And then later last year, when we finished our base model training of the K2, we actually replaced the QoWen with our own base model. And now the K2 Think V2 is a 100% sovereign, in-house built model from head to tail with our own data and also our own capability. I think this is very, very important because it is the first time that you will see a frontier model which is able to be open-sourced both in model weights but also the data, the training recipe, and so forth, so that people can really take it away and reproduce it. I think this is important because otherwise we don't even know. We were actually accused by someone about data contamination, which is a very, very, how shall I say, a serious issue because The base model that we took, we don't actually know what data they use in training their data. And it is not impossible there are contamination. Now that we replaced all this model with our own self-made model with the data that we collect, we actually have a stronger guarantee on the performance and also on the right implementation so that the score are more reliable and more reproducible. I have to say that many of the strong scores that you see on AA can be very difficult to reproduce because the test scores, the data all over the place, you actually could achieve a high score just by contaminating your training data with those testing data. And people cannot tell because you don't publish your data. In our case, we publish data out there. People can help us to find out whether there are contaminations and other issues which actually could really tangibly leads to meaningful improvements of the training strategy because it is a transparent game that everybody can play by themselves.

Well, so that last issue is fascinating because I think it's been a well-established problem in machine learning where you have data leakage and information from your test set gets into your training set and you think your model's amazing, you put it in production, then it performs terribly. So I hadn't really thought about what happens in the large language model case once you're training these huge models. How do you even prevent that happening?

Yeah, it is, you know, I don't think it is really an ill-intended kind of mistake. You know, people sometimes just, you know, unintentionally, you know, get unfortunately bad data, or maybe they just didn't pay enough attention to scrub their data, you know, for potential overlapping of the testing dataset. It requires additional effort, and not many people are doing that. So I'm not saying that people are playing game to get better score. It's probably just an inevitable phenomenon that's hard to avoid. But in our case, because we are opening the data to the public, we had the opportunity to invite the public to be the gatekeeper to really, if they knew what we don't know, say a particular dataset actually is present in the testing dataset, we could be informed or they could be informed, and then we can take that data out. I think this is the better way of doing science. Again, academic research really champions not just performance, but also rigor and reproducibility, which is very different from the corporate practice where the performance and also the confidentiality is the key, right? I think both are interesting, both are necessary to make sure that the results the progress that we make are authentic and also are sustainable.

Absolutely. So this real strong push for openness, it's pretty unique. But do you want to talk me through who might want a fully open model here? Like, who's it important for?

I already hear complaints in the Valley and in the US ecosystem about the concerns of those open-weight models from China, right? There's geopolitical concerns for sure, but also there are safety concerns because you don't really know what training data has been used in producing all those. We are hoping the K2 series provides a very solid alternative to those open-weight models so that for the ecosystem startups and so forth, who are building their business and operation on those free, open models can now have additional options which could be safer and also more reproducible. So I think there are this type of angle and opportunity for us to really start maybe a third quarter in the landscape that is promoting 100%, you know, 360 openness. And then on the other hand, the academic to us is a very, very important user. They don't pay us and we are not expecting any pay, but having the academic adoption of this model obviously gives us the right feedback, which allows us to do our science even better and also to move the field further. I have to say that there is a problem right now with academic AI globally in terms of competitiveness and relevance. Somehow, because of resource reasons, because of many other economical and political reasons, academic research in frontier AI is marginalized. We want to use our effort and results to maybe promote or stimulate a comeback that we want to be at the forefront of AI research in terms of innovation and performance, not just constantly chase after the big pack to do very, very small incremental work. But on the other hand, are we philanthropists that we are just spreading money and for no return? Well, of course not. We build those models. Our students and faculty are given opportunity to take this further into their own startups and spinoffs. In fact, at MBCI, we have an ecosystem incubating startups which could actually be directly built on those apps and models that we developed. Well, being the builder of those models, you have the firsthand knowledge. Of doing the next step. For example, company may be willing to share with their prepared data for you to develop agents and other utilities using the language model as a base. Then all engineers and developers are in a good position to actually make the best use of those open models for commercial applications. So those applications of course may or may not be open sourced for good reason., but the financial kind of, you know, reward hopefully will be, you know, substantial, you know, from those activities.

Yeah, it's very interesting in the AI space. I think a lot of other technical spaces, open source has been very dominant. It's kind of what you think about the operating system space, the database space, like all the most popular products are completely open source. But in the AI space, I guess the cost of building these things, it's sort of, it's inverted the economics somehow.

Yeah, I think this is maybe because of a cost issue and also the culture issue, because for building large language models or maybe world models and the future generation of foundation models is not a classical traditional practice of lone wolves and the solo heroes. It's a big science problem. You need to have a large team and a lot of resources. Therefore, there has to be the right culture. To allow such activities to take place. But I think still it is important. To me, open source, you know, I said about science, I said about safety and other things, but also there is an open question about who makes the standard of next generation of AI infrastructure, right? And I don't think the standard should be made by an adversary that is not willing to open or by, you know, a corporate that has its own financial interest built in, right? Having a true open environment allows the community and the dynamic to be part of, you know, this standard-making practice and which will lead to obviously better and more robust standard. Absolutely.

And I suppose there are ways of, well, one of the most common use cases for doing things with your own AI is going to be things like fine-tuning and adding in your own datasets, adapting existing AI models. And if there's that with an open-source model, that's going to be a lot easier to tune to your own needs, whether they're commercial or academic.

Yeah, yeah, yeah, absolutely. That's also why, in a way, intentionally started from mid to small size models. Which is a lot easier for people to adapt and to take further for the next phase of post-training and fine-tuning. But that said, we actually are in the process of rolling out more powerful, bigger models that is on par with some of the frontier models that we see out there, just because the science in making those models are even less understood and nobody actually see how those people, those things are made. In our case, we do see the opportunity and the need to first of all, learn this experience based on our own exercise and the production, but also to share this knowledge to invite kind of more activities and study into this topic.

Okay. And how competitive are you with these closed-source frontier models? Like how close can you get to that top performance?

I'm very optimistic that whichever model we're going to release, there will be the SOTA or close to SOTA in that size band. For example, you know, our K2 model, K2V2 that we released in December is a 70 billion parameter model. It is one of the best 70 billion parameter models and maybe sometimes even better than the bigger models, slightly larger. The next model we're going to release likely to be a 400 billion parameter model. I think it will be among the best in that size band. Again, the recipe of training models in a sense is known to many people already. What is not known are the secret tricks and the engineering kind of practices that you need to put there to make it happen. I think this is something required a more rigorous and holistic study. And that's actually where, you know, our release hopefully can expose and also draw attention from the community.

Absolutely. So I want to ask you what your secret sauce is to getting the high performance for K2-Think, but it's not very secret because it's all open source. So what tricks have you used to get that high performance out of your models?

Oh, it's— we actually published a very, very detailed technical report for each of our releases already. It's There's no single silver bullet that you can use to make this performance better. The way you pick the hyperparameters, the way you deal with those bad data points to remove it, how to program the curriculum in the right order to promote the right kind of convergence curve. And how much you want to give in your post-training fine-tuning in the reinforcement setting for specific task-oriented tasks. There are a lot of little tricks that require both experiences and also a good feeling about the dynamics of the curve evolution. So that's exactly why my team alone isn't enough to extract the rigorous science out of it. All we can do is to now publish all these logs and checkpoints and intermediate results in public so that the whole community can jump onto it and study it.

That's so fascinating. It sounds like there are still maybe more optimizations you can make to your existing model. It just needs people to dive in and make changes. So for anyone who's interested in working on this, like, What can they do?

The LM production right now, it's still like the old school workshop violin makers. Different masters have their own tricks. It's not an open, transparent science yet. I think we still have a lot of room to understand how the scaling law behaves, when it stops, and when to put more data and change architecture. This study right now isn't active enough or rewarding enough to produce guidelines and principled kind of recipes for the training. And so I think in the next few years, when the spearhead is moving toward world models and the fancier agent models, the understanding of the LLM model, you know, LLMs won't stop. They will actually produce more fundamental principle knowledges.

Okay, so it sounds like there's lots of work to be done. If people are interested in contributing to this, is there a way they can get involved?

I think, you know, by making the model open, you know, the involvement is already made easier. You know, if they have a reasonable amount of resources, definitely they can just download those models and download those checkpoints and study the data or maybe to build their own application on top of it. And more kind of proactive stimulation or promotion of the involvement while still looking into the opportunities. I know that multiple university institutions are now, including us, are in discussion of maybe creating a consortium and maybe seeking sponsorship and resources to create some kind of computing resources that allow researchers and students to bring into their own contribution and proposals to this type of study. But again, I don't think there is a systematic framework for doing that at this point. And I think the awareness needs to come first, that people know that there are these kind of open models that is able to be studied. And then they are also good and easy to play with. I think this year we will see multiple efforts toward promoting this kind of awareness and also more general practice. And we'll see whether there are additional donors, you know, or government efforts, you know, putting into this effort.

Okay, wonderful. So it sounds like at the moment it's just a case of try it out and see what you feel, see if you get good answers and maybe give some feedback to you or to your team. All right, now before we wrap up, I'd love to talk a bit about your third strand of research, which is around creating digital organisms. Can you tell me what that involves?

So I briefly mentioned that just now, right? So The digital organism could be viewed as a special type of world model that is built on biological data with the goal of simulating biological possibilities, not physical world possibility. Well, of course, biological world is a part of the physical world, but very specialized. So our approach is that we want to come up with a new type of architecture, which are more appropriate to deal with biological data with all their special properties. Right now, if you look at the so-called foundation models for biology, they are still borrowing ideas from large language models and treating the biological information as a special form of natural language. And then let's hope for the best, say having a DNA model, having a structure model. Which actually is working to some degree. If your goal is simple, say predicting the structure of the protein from a sequence, this is a reasonable task that is built on linear sequence information of the data. And then your model, your technique, draw from the large language models and transformers may be adequate. But if your goal is to simulate for example, cellular behavior or network behavior as a result of gene knockouts, mutation, and the small or big molecule medicine. These are by definition not sequential data. They are higher dimensional spatial and temporal data that is going beyond what the current transformer architecture is able to offer. So our AI-driven digital organism is exploring new architectures which allow us to integrate these different data modalities in a holistic way so that representations of different entities, be it genes, proteins, and cells, and the cellular environments, can actually be convergent on a particular information space that they can talk to each other and create responses. So in that sense, we are trying to do something very different from what you see out there, be it the perturbation prediction model of the virtual cell or drug development models and so forth, which is built on this old school machine learning mentality that What I need to do is a functional approximation. I built a big model that allows you to predict from A to B. And this model used to be a linear regression, now becomes a fancy transformer. But the spirit is to make a prediction, right? In our case, we are not building a predictor. Word model is not a predictor. Word model is a simulator. Therefore, what it generates is to take the input as the prompts. And then it creates a distribution of all possibilities coming out of the prompt. So the idea is by definition stochastic and also open, right? It is not an A to B mapping, but an open space simulation. Why is it important? So I give people this example. When you design complex solutions, or outcomes, say design an aircraft, design a rocket, your mental model is not that of an A/B test. You have two hypotheses, one is working, one is not working. You don't A/B test to do aircraft design, right? And you actually simulate because it's so complex. You simulate all the possibilities until satisfaction, and then you build a real thing and you basically launch it on a pad. In medicine, strangely enough, people are still using this A/B test, a cohort study for medical practices. But drug or tissue or organ or individual is something that is not resolvable by two hypotheses, A and B. That's why our approach is to build this digital organism using the world model principle so that it takes data from all different modalities, sequence, structure, network, and image so that they can give you a holistic representation of what a cell, what a tissue should truly be behaving. And then based on that, simulate the outcome with whatever prompt people may come up with.

Okay, that's really interesting. I had not really thought about the idea of, I suppose, like when you think about drug development, it's either the drug works or the drug doesn't kill people, then it gets approved or not. So it's still that A/B testing mentality. But actually, once you start, if you're involved in drug development, you probably want to start thinking about simulation possibilities.

Yeah, I actually, I used to tell people, I used to tell people a funny story about what we see as the future of drug development versus what is happening now, right? So yeah, people are now using AlphaFold and others, to already develop protein drugs. But then what happens to the design that they enter back into the older pipeline of trials and so forth, right? This whole multi-stage trial process is actually a very ancient practice developed some decades ago before we even know DNA, before we even know AI. So in a sense, you imagine that a horse wagon, which is an old antiquity transportation tool, Now you replace the horse with a Boston Dynamics bot. Well, you have a fancy thing in the middle, which is the AlphaFold, but the whole thing is still a horse wagon. I think the right thing to do is to not ask whether this whole framework, this whole process is even correct. Because if you have the ability to simulate a digital organism or a cell, then your trial shouldn't even happen in wet lab or in test tube. Your trial should primarily happen in a computer with a simulator. That's how people develop, for example, nuclear power plants and the silicon and the chips and so forth. But in biology, that level of simulation is not happening yet. So that's where we see the virtual cell digital organism project that we're pursuing in GenBio.AI. Is truly disruptive. It's very, very different from any of the existing drug design companies, you know, which is to put their Boston Dynamics dog in the middle of the horse wagon to replace one slice of the process. We are trying to now reimagine the whole process.

Wow, that's a hugely ambitious goal. I guess the question is like, how close are we to having all this work? Like, what sort of simulations work?

What's still to be done? It seems to be complicated, but you can start from something modest. For example, simulating a single one cell, right? And that is something you have to do anyway, because with all these technologies, it is the best way to put them together to basically demonstrate what is possible. Our plan, and in fact, we are in the middle of developing a first version of the virtual cell, which hopefully will happen in the next couple of months. Which is, you can imagine that I call it to be the Wright brothers' aircraft. Well, the Wright brothers' aircraft is really, really primitive. It is using very, very cheap material. It is only taking off and fly for like, I don't know, half a second, half a minute and 100 meters. But the moment people see it, people know that It is not a car. Okay? It is not a horse wagon. It is something flying in the air. Then only a few years later, people are already using the aircraft to do their battle in World War I. So once you see the prototype, the next iteration will be extremely fast. So this year we're trying to roll out the first prototype, which is a virtual cell that is able to give you limited simulation of the outcome based on, let's say, 20 prompts that you can come up with, knock out a gene, put a little compound and maybe heat things up. And let's just ask what happened to the cell and then also make the cell very simple. How about just a liver cancer cell or brain cell? Just very limited application. But that's already different from predicting what is the N-1 gene look like. Where you knock out one gene. That's the older paradigm people are trying to work on. But we work on a new paradigm, which is primitive but different. And then we believe that you have one cell, you can put two cells together and multiple things together to make a tissue. And once you have the tissue, you can go all the way up into an organ and so forth. So there is a clear roadmap that needs to be started from a single virtual cell.

That's amazing. And I love the idea of you've simulate things at a very small scale. So you're simulating a single cell, which I guess even in itself is just incredibly— it's a complicated thing to do. And once you do that, then you're sort of gradually scaling things up. And one day we get our digital organism and that's gonna revolutionize healthcare. I love it. Okay, so just to finish up, I always want more people to learn from. So talk me through who's at work are you most excited about right now?

It's a very difficult question. I can be very, very candid. Among all the people I talked to, the one that excited me the most is a conversation I had a few, I would say 2 or 3 months ago with Demis, the head of Frontiers Math, right? Because I was very, very happy but surprised to find that we almost have perfect alignment on what is a world model. And also what is a virtual cell, which is very different from what we hear from the public in terms of the fanfares and also the hypes. He had a very, very grounded but sophisticated view, technical view about how they should be built and also how to test them. I actually feel very excited about his vision. And also feel quite intimidated by that vision because we need to also move fast. Otherwise, we are not going to be able to make our impact if we cannot deliver our results fast enough.

Absolutely. So it's interesting that you're kind of aligned with Demis Hassabis from DeepMind about the state of biology, of course, he won the Nobel Prize for his work with AlphaFold and the protein folding.

So his vision and his team is ahead of the time compared to many of the other. They don't say much. And I believe, again, my personal opinion is that what they show, what people see is already quite a few months, if not years behind what they actually have in their chest. So I wouldn't be surprised that they have something fancier and disruptive coming out in the next few months.

Okay. All right. Well, I guess if you're trying to compete with that, then it's a big challenge.

I wouldn't put myself in that position. I think it's almost like a confirmation about the way that we are pushing. It's good to have multiple teams on very similar topics., and we can compare notes and also we can, you know, cross-check each other.

That's very exciting. Absolutely. I mean, certainly it's such an exciting time at the moment with all these kind of new developments. So it's been a real pleasure to talk to you, Eric.

Thank you. It's a pleasure.