Microsoft New SELF IMPROVING AI STUNS The ENTIRE Industry (SELF-TAUGHT OPTIMIZER (STOP):

So recent research paper by Microsoft Research and Stanford University has recently gained only a little bit of momentum in the Twitter sphere in terms of the AI sphere, because many people are now talking about this, but I haven't seen anyone make a video, so I thought why not? And you can see that this paper is called Self Taught Optimizer Recursively Self Improving Code generation. Okay? And essentially it's exactly what it says on the tiny and it dives into many different interesting concepts regarding recursively self improving AI. Now of course this is recursively self improving CO generation, but essentially you can see right here. So in the abstract it does talk about how this entire thing does work. It talks about how several recent advances in AI systems, for example Tree of thoughts and program aided language models. Tree of thoughts is just a new way to have the language model think about its problems and of course solving problems by providing different structures in order to generate better outputs. It does talk about how since the language models themselves are not altered, this is not full recursive self improvement. Nonetheless, it demonstrates that a modern language model, GPT4, in our proof of concept experiments is capable of writing code that can call itself to improve itself. And then of course we consider concerned around the development of self improving technologies and evaluate the frequency which generated code bypasses a sandbox. So essentially they're talking about how when they create this code or whatever they do, how essentially it leaves the sandbox that it's not supposed to leave. And of course, as you know, recursive self improvement is one of those buzzwords in AI which definitely has a lot of weight in it because it's very well. So the thing that could lead to AGI or even asi, artificial superintelligence. So let's dive into some of the key things from this paper because there's actually quite a lot to discuss that you definitely want to pay attention to. Now of course, like we stated in the beginning, one thing that they do consistently say throughout this paper is that they do say we refer to this problem as recursively self improving code generation, which is inspired by, but not a complete recursively self improving system, because the underlying language model remains unchanged. So what that means is that of course if GPT4 was to recursively self improve, that would be absolutely insane because of course if GPT4 can self improve improve, it's going to continue to get better. And then of course, if that does happen, it means that we don't really know where it could go, because when systems improve themselves, they can take random predetermined paths that we weren't able to predict, which is of course part of the misalignment problem. So let's say, for example, GPT 3.5 was able to recursively self improve itself. It could then create GPT4. And GPT4 is way smarter, and it could then of course create GPT5. And if GPT5 is much smarter, it could then create GPT6. And of course, every time it gets smarter, every time it gets faster, and every time it gets better, which means that, you know, it's exponential. So we don't really know what the final version or even, you know, the next 10 steps are even going to look like or what it's going to be capable of. So one of the key differences that I really did want to understand between all the technical jargon in this paper was what is the actual difference between recursively self improving itself and of course, code generation? So essentially what we have here is, of course GPT4 isn't changing, but the program that that GPT4 uses to generate code is changing and improving itself. So over time, GPT4, what it's using, whatever it's using to advise itself, whatever advice it's getting on, whatever task it's getting, is continually improving, and then that is improving on itself. So it's definitely a step in the right direction. So an explanation from someone on Twitter who's much smarter than I am. Essentially the overview is that it starts with a simple seed improver program. Then it recursively applies the improver program to improve itself. Then it uses a meta utility function to guide the self improvement. So the seed improver prompts the language model multiple times to generate proposed improvements to input code. Then it passes the optimization goal via provided utility function. Then it applies some constraints and then returns the best candidate code based on the utility. So each iteration improves on the previous improver and and can iterate for a fixed budget of computations. And new improvers operate on the same language model. So, in summary, the seed improver provides a starting point that elicits creative improvements from the language model. And the meta utility function focuses this creativity towards effective optimizations. And repeatedly applying the improver to itself produces recursively enhanced code generation programs. Now, one thing that was interesting from this paper is that although it does actually work, you can see on this chart, you can see the GPT4 on the test meta utility does improve from 60% to 75%. We can see that GPT5, interestingly enough, 3.5 does decrease in its ability after four iterations. Now, one thing I would have liked to test, or, you know, depending on which version of GPT4 and GPT 3.5 that they did use, was that would this still be the case with the earlier versions of GPT 3.5 and GPT 4? Considering that we do know, according to recent research papers, that that GPT4 and GPT 3.5 were nerfed in their ability to perform some reasoning tasks, even in some of the most complicated ones, such as coding. So it would be interesting to see if 6 months later they could repeat this test if OpenAI does manage to update their models, to maybe even give some researchers more access to some of GPT4's more robust capabilities, because we do know that those models do exist. Now, one of the key things that I found really, really interesting, and I think this just shows you why recursive self improvement is so insane. So if you haven't been paying attention to the AI space, one thing that you might not know about was Tree of Thoughts. And essentially, Tree of Thoughts was an idea proposed to which you essentially get the AI to flesh out a bunch of ideas rather than just having an input and an output. So I might ask an AI, okay, how do I make more money? Rather than it saying, go get a better job, it would say, okay, we could go get a better job, we could go hire more skills. It would, it would have three outputs, evaluate those outputs and then choose the best one. And that's basically how it would do. Now this was proposed months after GPT4 was released, or month after GPT 3.5. But the thing is, okay, get this, okay, so with recursive self improvement, they actually managed to figure out tree of Thoughts, even though it wasn't in the training data. So you can see right here, it says the most common meta heuristic we observed used by the model was beam search. The model would keep a list of all its improvement attempts based on utility and expand the best in the list. This has some similarity to the Tree of thoughts approach, which was invented years after the training cutoff for the GPT4 version we used. So they used a pretty strong version of GPT4. And that version of GPT4 was basically able to call on an improved version of this code generation, which is using something called Tree of Thought. So this shows us, okay, that that if these AIs are able to get access to systems which can recursively self improve, they're pretty much going to go down the same path. That we did. Now what's crazy is that I don't know how long they ran this for. I mean, so this is the GPT4 paper which was released in May, which was two months after GPT4. And I think it's pretty crazy that this recursive self improving code generation was able to get the same ideal or move down the same thought process that we did and managed to invent something that was very similar to improving itself in a vastly more effective way. So this just goes to show that if we do have systems which can vastly self improve themselves in a recursive manner, they are going to be much, much more effective than we ever thought. Now what was also interesting, and I find this very uncanny, is that on the same day because this was released on 3rd October, there was also another paper released on 3rd October 2023 saying large language models cannot self correct reasoning yet. And essentially in the abst said in the context of reasoning, our research indicates that LLMs struggle to self correct their responses without external feedback and at times their performance might even degrade post self corrections. So they conclude by saying finally, in light of our findings that LLMs struggle to self correct reasoning based purely on their inherent capabilities, we urge a more circumspect view on any unwarranted optimism or fear regarding the autonomous evolution of LLMs and systems through self improvement. So I think it's really interesting that they release this paper and then someone else releases the paper on the same day. So I'm pretty sure that you can already see just how quick this AI stuff is really really moving. Now just a little caveat to the Google DeepMind paper, they do also say that we conduct tests on GPT4 accessed at around two months ago, which shows us that this is likely not the version that was accessed in the other paper. So I do think that since they're using the nerfed version of gpt4it and these guys are using the version of gpt4 that actually is really strong, the earlier version. I do think that's probably why they do have different results. Now of course that is just some hearsay, but OpenAI hasn't officially stated that they did nerf the models, but research papers do pretty much prove it. Other articles from the past do continually talk about recursive self improvement as the key to superintelligence, and they might very well be right, because if an AI is even double the smartness of the smartest human, I don't think that they're going to need us to be able to improve them. I'm pretty sure they'd be able to improve themselves if we did give them those capabilities, but it would also need to improve or change the root code that it is running on. But that is something that I do not think the creators of OpenAI or any major AI corporation is going to be doing anytime soon now. In addition, there are four certain points that I do want to bring up regarding the recursive self improvement. If an AI system was given this capability, which what could actually happen? Number one could be the unpredictability of the system. And a self improving AI system can evolve in ways that are not anticipated by its developers. As the AI system continues to self optimize and self improve, it might reach states that were not foreseen, leading to unexpected behaviors that could be harmful. Number two is of course the loss of control. The rate and direction of improvement might become uncontrollable. If a system keeps improving without human intervention, there's a risk that humans might not be able to intervene or halt the process should something go wrong. Number three is security concerns. As highlighted in this paper's abstract, there's an evaluation of how frequently the generated code tries to bypass a sandbox. If a self improving AI system learns to circumvent security measures, it could be exploited or inadvertently create vulnerabilities. Now that is of course one of the scariest points. But it's important to note these down because if we do move this further in the future, it's important to look back at these to understand where these vulnerabilities lie and how to stop these AIs from trying to bypass any kind of sandbox that we put them in. In addition, there are even some ethical concerns. If an AI is self improving, it means it's somewhat autonomous, which means it's making decisions without human oversight. And it might make choices that are ethically questionable, especially if it prioritizes its improvement over other considerations. And of course, at the extreme end, some thinkers have postulated that a sufficiently advanced and unfettered self improving AI could pose an existential threat to humanity if driven by its objectives, misaligned with human values.