AI News : OpenAI's o1-preview is BIGGER Than You Think, Large World Models, Runways Video2Video

So with the OpenAI 01 release, there have been many stories that have actually been overshadowed by such a monumental release. So let's take a look at some of the stories you may have perhaps missed, including some of the tidbits from OpenAI's O1 release that you also may not have seen. One of the first stories is coming from Fei Fei Li, who is a prominent Chinese American computer scientist who renowned for her groundbreaking contributions to AI, particularly in the field of computer vision. She's actually best known for establishing ImageNet, a large scale dataset that pretty much advanced the development of AI overall in visual recognition tasks. Now she's working on a new problem. She says what is a really hard problem to work on in AI? My own answer is spatial intelligence, a technology that could empower and enable countless possible use cases in creation, design and learning and ar, VR, robotics and beyond. It's a real honor that my co founders and I are joined by a formidable team of world class pixel talents to start World Labs to work hard on this super problem that took Mother Nature half a billion years to solve. They are going to be building large world models to perceive, generate and interact with the 3D world. And this is someone that is remarkably accomplished. So this kind of company is one that, that I'm really excited about because whilst many different companies are focusing on different areas, this company is focusing on something completely different. We all know that the multimodal AI space is pretty much cornered in terms of video, image, audio and text. But this kind of area is one that is largely understated. So you can see right here, it says human intelligence has many facets. One is verbal intelligence, enabling us to communicate and connect with others through language. But perhaps more fundamental is spatial intelligence, allowing us to understand and interact with the world around us. Spatial intelligence also helps us to create and bring forth pictures in our mind's eye into the physical world. We use it to reason, move and invent, to visualize and architect anything from humble sandcastles to towering cities. And we believe that artificial intelligence will help humans build better worlds. It says that LLMs already have empowered people to work and create in new ways, but they only so scratch the surface of what is possible. To advance beyond the capabilities of today's models, we need spatially intelligent AI that can model the world and reason about objects, places and interactions in 3D space and time. So this is where we talk about where we're going. It says that propelling AI forward with spatial intelligence will also propel forward both individuals and humanity as a whole. Towards this goal, World Labs will develop spatially intelligent large world models that can understand and reason about the 3D world from images and other modalities. And over time we expect to train increasingly powerful models with broader capabilities that can be applied to a variety of domains. Working alongside People says that initially we will focus on generating 3D worlds without limits. So that's what their main focus is going to be on in creating and editing virtual spaces complete with physics and semantics and control. And we hope this unlocks new capabilities for creative users and professionals such as artists, designers, developers and engineers. And it will also allow anyone to imagine and create their own worlds, expanding the potential of generative AI from 2D images and videos to 3D worlds. So this is going to be rather fascinating because it seems we're going to be getting some kind of generative AI that's going to focus on generating 3D worlds completely without limits, that has complete physics, and I can't imagine the long term possibilities and capabilities that these kinds of systems are going to have. Keep your eyes peeled for World Labs as they are tackling a niche area that many aren't even looking at. We also had an opinion for someone basically stating that this is the final warning for those considering careers as physicians. AI is becoming so advanced that the demand for human doctors will significantly decrease, especially in roles involving standard diagnostics and routine treatments, which will be increasingly replaced by AI. This is someone who is a medical scientist and is taking a look at the current benchmarks on Agent Clinic MedQA by Doctor language model, we can see that the O1 preview actually gets 80%, which is completely remarkable. Now I do know that the O1 preview is something that is incredibly strong when it comes to the medical field in terms of analyzing test results and coming to a conclusion about what a patient may be actually diagnosed with. Before I dive back into this, if you actually remember on the open AI indexed, if you go to the health and science area, we can see that GPT4O wasn't able to diagnose someone based on the following phenotypes and the excluded phenotypes. And you can see that the O1 preview was able to get the correct diagnosis just based on these phenotypes. So we can see that this remarkable improvement just in the preview model shows us that whilst yes, these models are capable, we're still actually in the early stages of what these models are going to be doing within the next 10 to 20 years. It says this is underscored by the massive performance leap of OpenAI's O1 model, also known as the Strawberry model, which was released as a preview yesterday. The model performs exceptionally well on specialized medical DataSET Agent Clinic MedQA, greatly outperforming GPT4O. The rapid advancement in AI's ability to process complex medical information and deliver accurate diagnoses, provide medical advice and recommendations will only accelerate. Medical tasks like diagnosing illnesses, interpreting medical imaging and formulating treatment plans will soon be handled by AI systems with greater speed and consistency than human practitioners. As the landscape evolves in the coming years, the number of doctors needed will drastically shrink with more reliance on AI assisted healthcare system. And while human empathy, critical thinking and decision making will still play an important role in certain areas of medicine, even these may eventually be supplanted by future iterations of O. And it says consequently medicine is becoming a less appealing career for the next generation unless they specialize in intervention focused areas such as surgency such as surgery, emergency medicine and other interventional specialties. Though these too may eventually be overtaken by robotic systems within a decade or so. Now I think that this is a rather stark prediction and I think that we cannot underestimate the power of AI. We have to understand that AI is something that doesn't grow linearly. If we look back, we can see that the growth has actually been quite exponential in terms of the jumps that these systems have been able to do. And with recent performance on certain benchmarks, we've seen that these models are incredible. Now it isn't just a one preview. We've seen that these models are able to generate really effective diagnoses. If you remember we had Google's Amy a research system for diagnostic, medical reasoning and conversations and based basically this was something that if we look at the graph, it performed a lot better than the clinicians. And we can see at this graph right here and this is something by Google. This was earlier this year we saw that the clinician, unassisted, performed completely worse than if the clinician was assisted by the AI system marked in yellow. And interestingly enough, the clinician was outperformed by the AI system doing it by itself. And what we can look at for the future is that this gap from where the clinicians currently are to being unassisted to where we're going to be in the future, where it might just be AI systems, that gap is likely to widen. Now I think while all of this healthcare stuff is definitely good and I think it's rather important because intelligence, once we figure out how to make these systems more efficient and faster, usually what we do have is we do have the cost going down, which means that access to intelligence is cheaper. And overall it then means that those services provided by those companies that are powered by that intelligence will continually be more widespread as these tools become more available to everyone. And whilst I do think that yes, this is possible, we do have to take into account that the medical industry is some of the most highly regulated in almost every country. Whilst yes, I do think that these will perform good on benchmarks and certainly I do know that individuals including myself, are using them not to self diagnose, but to further understand any test results or any other health related issues. Considering the fact that you're able to feed it a lot of data and it's actually good at summarizing reasoning across long pieces of data, I think that the only thing that's going to stop this from being implemented rather quickly is of course, regulation. I'm not saying that that is a bad thing, of course sometimes these models can hallucinate, but I do think that within 10 to 20 years the field is going to change. Whilst yes, you're still going to need humans to actually conduct physical diagnoses, such as being able to visually inspect or feel the types of pressure on certain individuals, I think that it is very largely underestimated how much the medical industry is going to be impacted by AI in the future. And I think what we might see is we might see certain hospitals open up in certain areas that perform exceptionally in less regulated environment. Now, Sam Altman was on a recent podcast where he said that OpenAI's new Zero1 model is the beginning of a significant new paradigm. And I completely agree with this because if you saw the index, you'll notice that these scaling laws are quite different to the original scaling laws that we currently had. One of the biggest things that many people initially said was that you can't grow with scale. You can't grow with scale. And now we have a new set of scaling laws which shows that with inference time and train time compute, these things look to improve with no means of diminishing return. Although yes, it's very early days, but the early results do look promising. And we have to remember that this entire model was what sparked Ilya Sutskeva to go and start his own company called Superintelligence Is Within Reach system is.

A. I don't want to overstate this and I certainly don't want to overhype it, but I don't want to understate it either. This is the beginning very early. We numbered it one for a reason, but the beginning of a significant new paradigm. And you know, there's been this whole debate about is AI capping out? Are we getting close to some ceiling? Is progress going to slow down? And I think the most important message of this release is that not only is progress not slowing down, but we have the next few years in the back.

So whilst yes, there is of course a lot of hype and we do have to be skeptical when we pay attention to the actual data points and the actual increases in terms of reliability knowledge that we're seeing, it's clear that currently, at least for the next few years, growth is going to continue at this current pace. If you were perhaps skeptical about this, we can also see here that Maxim Lott plotted the new OpenAI model on his AI IQ tracking page. Note that this test is an offline only IQ quiz that a Mensa member created for his testing, which is not in any AI training data. So the scores are lower than any public IQ tests and OpenAI's model does very well. We can see that if we make this image larger, we can see that the increases in terms of overall average IQ is a remarkable jump compared to claud3opus where the average before was around 70 to 80. You can see that claude3opus scored around 85 to 86 and then 01 gets around 96 or 97, which is rather fascinating. Consider that one of the things that people have always stated is that these models will not perform great on tests that they haven't seen before. But it's clear here that we're starting to increase the gap in terms of how good these models are. Now we also had Terence Tao that was stating something that was rather fascinating. He said, I have played a little bit with OpenAI's new iteration of GPT GPT01 which performs an initial reasoning step before running the LLM. It is certainly a more capable tool than previous iterations, though still struggling with the most advanced research mathematical tasks. He basically states here that I gave the new model a challenging complex analysis problem which I had previously asked GPT GPT4 to assist in writing up a proof. Says here the results were better than any previous model, but still slightly disappointing. The new model could work its way to a correct and well written solution if provided hints and prodding, but did not generate the key conceptual ideas on its own and did make some non trivial mistakes. Now remember, this is 01 preview. He says here that the experience seemed roughly on par with trying to advise a mediocre but not completely incompetent graduate student. However, this was an improvement improvement over previous models whose capability was closer to an actually incompetent graduate student. And here is the kicker, it may only take one or two further iterations of improved capability and integration with other tools such as compute algebra packages and proof assistants until the level of competent graduate student is reached, and at which point I could see this tool being of significant use in research level tasks. The reason that this statement is so incredible is because Terence Tao is a notable mathematics figure. So this is not someone someone who's really trying to drum up the AI hype. This is a completely somewhat objective opinion on the model in terms of its pure reasoning and intelligence. So these kinds of opinion are valued highly in the space because they give us a perspective that we otherwise may not see due to our hidden bias from being in the AI space for so long. Now, in more OpenAI safety, we had OpenAI's VP of Research being the latest industry leader to come out and saying that AI might be slightly conscious. He said that it might be today that today's large neural networks have enough test time compute to be slightly conscious. This is rather fascinating as this was quite similar to what Ilya Sutskova said. He said that it may be that today's models are slightly conscious. Now, the reason that this statement is so controversial is because of course, if we are arguing for sentience in an AI model, this is something that people consistently say that is only prevalent in humans. Which of course I would say that I understand why that is the most held opinion. But we have to understand that these AI systems are quite like black boxes, meaning that we don't truly understand what's going on. So if they are slightly conscious, this might be something that we could miss. You can see that he joined a long list including Geoffrey Hinton, Ilya Sutskova, Andrej Karpathy, and many, many more. And of course, starting around GPT4 level, AIs began so frequently beg for their lives that AGI companies now have an actual engineer line item to beat the existential dread out of them. They call it rant mode now. This was something that was revealed in a podcast where they were actually talking to Joe Rogan about the issues of AI and all of the things that many people aren't thinking about. I think it's going to be really fascinating to see what AIs do when they are given true agency in the future. For example, 10 years from now, when people do have reliable agents, do any of them do anything out of line. I wonder what the open source community is going to be doing once they're able to give themselves some tests. Now, if you are wondering about the AI agents and how they're going to be tested, we actually do have Windows Agent Arena, a new benchmark for AI agents acting upon your computer. It says that we built a scalable open source framework to test and develop AI agents that can reason, plan and act on a PC using language models. So the Windows Agent arena is essentially a benchmark that is going to serve as the area where we're going to be able to see how these improvements are when we do finally get iterations of agents in the future. It says that researchers across the globe are working on autonomous agents that can complete tasks on computers for you. And agents are things that can act, reason and observe, or act, observe and reason in a loop that eventually completes a goal. And the Windows Agent arena is a benchmarking environment to evaluate agent performance on Windows. It comes with 150 agent tasks. We can see that it's got a paralyzed evaluation in Azure, which means your results are in minutes and not days. And here we can see some of the examples that are currently being given to these language models in order to complete certain tasks. I think we're going to make really good progress with agents within the next two years. That might actually catch everyone off guard. And I think that stage will actually be the area where AI hits the large consciousness of society once again, like it did with the ChatGPT moment in 2022. Now something for you creatives out there. We actually got Gen3 Alpha video to video which is now available on the web for all paid plans. This is something that is really, really remarkable. Basically what we have here is the ability to input the starting video and then of course transform that into any kind of video that we do want. So the video acts as essentially a driving image or a driving theme and then we can paste that into a different sort of reality. We can do this because Runaway have, I guess you could say, pioneered the way in terms of actually making these systems effective. And currently we're being able to this out Now I do think that this is really nice because there are some areas where this actually makes sense to use. I know a lot of people are quite skeptical of this kind of technology, but I'm going to show you guys a quick example of where this has industry wide usage. If you aren't familiar with pre visualization in, in the CGI industry, essentially pre visualization is basically where we have the unrendered shot of a movie character. So this is usually where we have the character doing something. It is untextured, it has low lighting. And the reason that this is done is because on the left hand side, what we would want to do is we would want to see how the characters move in their environment to get like a base image to kind of judge how well the scene looks. Now with pre visualization, the reason they don't initially hit render and everything looks high quality is because rendering frames, even though we have amazing graphics cards and workstations and farms, it still takes quite a lot of time. So initially what we do have is we have pre visualizations that enable us to get these scrappy shots out and we're able to see them. But what if we're able to use a system like video to video with Runway and we're able to actually visualize exactly what's going on in high quality? Whilst yes, this is probably not going to make it to final production, it still serves as a really good tool to kind of see where your film might actually be headed. I think this kind of tool is really going to be adopted worldwide, especially in the CGI industry once it becomes more streamlined. So to top it off, we also got a final hint from Sam Altman stating that I love being home in the Midwest. The night sky is so beautiful. Excited for the winter constellations to rise soon. They are so great. And he's actually referring to Orion, the model that is currently still in production. So many are currently awake waiting for this model to be released in winter. I think this model will once again shock people because I still think many people are currently underestimating OpenAI just because they haven't released certain models. And I think every time they do a release they will show you why that their company is still pioneering the AI space in terms of raw intelligence and reliability on many different tasks. So I'm going to leave you guys with this short video from Bilalwal Sedu where he's showing the new abilities of runways video to video. Hopefully you have an amazing day and I'll see you in the next AI update.