From Rock-Paper-Scissors to BabyAGI: Harnessing LLMs with the Wolfram Language

My name is Jofres Picole and I'm part of the Wolfram Machine learning team. And in this presentation I'm going to show a couple of experiments that I've been doing on my side time during the last months, which involve like LLM large language models. So I will divide the presentation into parts. One is going to be some experiments I did with LLMs playing the classic game Rock Paper Scissors. And then we will explore the implementation of the Baby AGI, one of the autonomous agents that went viral I think at the beginning of the year, I think it was in March, using GPT4 and GPT 3.5 Turbo. So this image was generated yesterday using Dall E3. So the idea is that we will try to build some Wolfram language functions in order to make GPT4 play against GPT 3.5 Turbo and against humans. But before we start, I wanted to kind of show some or like make clear some kind of features of the large language models or in this case the GPT generative retrained transformers which are decoded only transformers. This is the architecture. And one fact that some people don't know is that these LLMs actually can be deterministic at zero temperature. So the concept of temperatures, it's important to have it clear. Sometimes we have it associated as the LLM being more creative at higher temperature and less creative at higher at lower temperature. But the hyperparameter comes from the softmax function. So this is one of the latest layers in the decoder transformer block you have the softmax and then we add a hyperparameter which is known as temperature from the connection to the physical temperature from the Boltmann distribution. So basically we can see it as a system where at lower temperature we have the probability one at having one state. In our case it's a bit different. We have a number of tokens or words from the vocabulary and this distribution gets at zero temperature. We only kind of promote one of those tokens. So data line behaves like 100% deterministic. In reality it's a bit different. Even at zero temperature, large language models such as ChatGPT and GPT4 we will for like generating long pieces of text over and over. We will get different results due to some technical issues in the GPUs. But in theory those are like kind of deterministic models. So for example, here I'm using LM synthesizer to make us to generate randomly between 1 and 2 and requesting it 10 times. In this case and at zero temperature, GPT4 is always giving me tools but when we increase the temperature to 1 we start getting a bunch of ones as well. So that was the main idea that we are going to run these experiments at zero temperature. So they are kind of more or less deterministic so we can reproduce the results that we're getting from these LMS playing Rock paper scissors. So the here the idea behind I think I ran some of these experiments while doing the war from summer school last June and July. And the idea was to create a prompt that was the same prompt for both players. In this case the same prompt would be used for GPT 3.5 Turbo and as the GPT 4. So after some iterations the initial prompt that we run was the following. So in this case I was using LMM synthesize but we will see later that in the second ROM with some variations I'm using LM function as well. In this case we are defining explaining that it's an AI agent playing propper scissors games, explaining the rules just to make sure that there is no misunderstanding there. And also that they are going to play 20 games or rounds in a single match and the objective is to win the match. And then I'm showing them the previous games that they have been playing for from the last to the from the first to the latest. And here it was for the for doing the few short learning it was zero shot learnings and sometimes it answering one of these three move or plays but then also like end up being verbosed and maybe explaining why you should use that player or not. So maybe these little extra capital letters sentence was helping. So I'm defining the agent with these two parameters. One is the association here with the place I initialize without any play. But then running once you get the answer from the agent in this case GPT4 and then GPT 3.5 turbo also then the idea is that use this kind of place, add them into the kind of the tracking of both players and at the end using kind of these helper functions I will end up having again the association with the extra place for each round. So then defining the kind of player again here I'm defining of course since I'm using the same prompt I need to swap the players in the association so they are seeing the exact same prompt but with a different place. And at the end we have something like this that one round playing it so they are actually not seeing each other move before the other so they only see the previous move that they did. And then once we have this function we can use the ness function and run the experiment for 20 games and then we can analyze the results with these couple of custom functions to generate a pie chart. In this case, Alice is the GPT4 and Bob is the GPT 3.5 Turbo. And the same we can do another visualization using list line plot for example for the game number over time and whether and the number of wins for each player. So we can see here that there's the likes from draws and then Bob is winning a bit more and then alice which is GPT4 catches up and at the end GPT4 is. It's taking advantage of GPT 3.5. So now what we did is to initialize the game with the different possible game states. In this case all one round of all different of all the nine possible conditions and check the results from that initial games. So actually they're playing only 19 games up to the 20 here. And this is a way to visualize the different from the move from Alice and, and Bob. And we can clearly see that GPT 3.5 turbo. So Bob for some reason is kind of only running some rocks at the end and kind of GPT4 Alice is just taking advantage of it and smashing it. At this particular game we can check other configurations and overall typically in this case there is a draw. They end up having the same number of wins. But typically Alice managed to win Bob again with paper and scissors. And here it's one of the only nine initial conditions where Bob manages to to be. But then doing the overall results we can see that Alice GPT4 is much better than GPT 3.5. Yeah, so here I would select adding all the results together we get kind of. I think it's around like 40% win rate for Alice and 20% win rate for Bob. Now I tried a different prompt. In this case I'm not going to use an association but a list of association which each of the rounds or game numbers and in this case the LLM will see next to their play from previous place the opponent's place as well under the game result. So the idea is to also make sure that using a different prompt, the results are more or less the same. In this case I also changed the LLM synthesized into an LLM function which is equivalent to the one that we had. But now the prompt is a template, an LLM prompt where there is a template slot. So we need to add the played games in a different way. But mainly it's the same as before, just that the played games is now a list of associations not as an association. All right, so I'm modifying a bit the player game in this case with a neat. So it's more or less the same but just adapted to the new structure of the player games for tracking the games so far. And again the results are like similar. So you can see in the first configuration there is a draw but then overall Alice is winning against Bob against ChatGPT OGPT 3.5 Turbo in this case. Yeah, I think there is also one configuration where BOB manages to win ALICE out of the nine different ones, but overall is again losing. And later we will see why or partially we will be able to explain GPT 3.5 is much worse than GPT 4. So yeah, in this particular case when we start the first round with Alice and Bob both kind of throwing scissors, they end up Alice ends up losing against Bob. It's in the other. So the overall results again it's around like 40% for Alice GPT 4 and 20% win rate for Bob. And if we look at the counts of the different move or plays for each player, we see that GPT4 Alice is playing much more paper. And the explanation is that because Bob is not playing scissors at all almost just like four times out of all the different games. So Bob likes rock. LLMs. And then I did some experiment just like, okay, let's see how good is GPT4 against a human? In this case I only play it myself. But in the notebook you will be able to also run this kind of code and also like initialize this, manipulate and play against GPT4 or the LLM that you want to play against using LLM Synthesize or LLM function in this case. So I think in both cases on the different games I played I was able to. So the Bobby here is the human and alice is the GPT4. I was able to kind of more or less predict the movements or like changes of GPT4 pretty well. I think basically they are like playing. So I think in this case my last play was scissors. So they are going to try kind of use something to crash the last play. So in this case they're going to, they're going to play a rock here. So if we're using paper, probably we're going to win GPT4. Let's see. Yeah, so that's kind of how I was. It seems that GPT4 is only paying attention to the last move and trying to pull up for this particular prompt. Maybe there's small experiments that we can do changing the prompt or making it more Intelligent. But so far I think GPT4 is not beating humans, at least at this game. So this is a reference for one of the largest human experiments kind of playing rock, paper, scissors. I think it was from a Chinese university, so there's the link for those interested. But I think it's still not in a position that we can use GPT for to simulate humans for this particular game. Other things that I did not try but could be also explored is that all I'm playing against Sequence Predict. Sequence Predict is probably gonna win against GPT4. And this is an example of using Sequence Predict to predict the sequence like the next move from a scene sequence of rock, paper, scissors. And another thing that I did not try much was to use other LLMs models. We could also do the experiment with Cloud 2 or Palm from Google and starting from Mathematica 14 you will be able to also to use the service, connect to hiking phase and run different open source large language models like llama2.7 billion. And I was able to get a response that it will tell you the next move, but it was not changing, it was only like giving me rock all the time. Even if I was, it would clearly lose against the opponent that was using paper. And in these lines of using these simple kind of games for LLMs there has been a recent benchmark called Smart Play where there's like different LLMs playing a bunch of different games including rock, paper, scissors. They are using a different technique that or different strategy than the one that. Testing that they're exploring games for assessing the reasoning capabilities of these large language models. So if you're interested there is also a bunch of different papers and reviews on such kind of game theory and large language models. I think it's a hot research area where there's a lot of experiments that one can run. And if you're interested in such experiment and you get some results, you can contact me. We are like with some summer school students were still running some experiments for such games. So with this I will start. I don't know how much time left I have. But the second part of my presentation, it's about the baby AGI which was an autonomous agent that came out at the end of March I think and it was using originally ChatGPT and GPT4 in a loop and completing task. So this image was generated by Dalit3again and the idea of the baby AGI is that there is different sub agents within it that would run what one would create so that the human will start adding a goal, an objective and Then from there the BPGI will kind of start create new tasks for the main goal and then you will have an execution agent of the first task on the list. Then there is going to be a prioritization agent which will rank the relevance of the different kind of left task and based on the result of the execution. And at the end you will run it for several loops and you will get a bunch of results. It's a quite easy way of running out of kind of OpenAI credits. So just be careful with running this system. But basically originally it was created by. In GitHub and it was written in Python and the idea was that the code was not super large. I think he managed to decrease the number of lines of Python code up to 100 something. So I decided let's try to implement a Wolfram language version of baby AGI. And it was. I managed to shrink it up to like 50 something that we will see here. So I added kind of some like semantic search memories using embeddings from the LLMs. In this case I used Wolfram language documentation examples. Originally I used the packlets from Christopher Wolfram for getting the OpenAI embedding. But nowadays you can use directly the OpenAI service connection that we have built in. So the Service Execute and OpenAI will get a similar embeddings. In this case by default is the text embeddings. Other 0.02 and I think OpenAI embedding function was getting by default another embedding model. I'm sticking to this one just because I just ran embeddings for all the documentation examples back in May when I did this implementation. So here I'm sharing the nearest function I created with these embeddings and Wolfram language documentation examples. We can test it for example with this task create a 3D pie chart. This newest function will give us a couple of examples from documentation with code that we know that it's correct because it's from the language documentation. And then the baby AGI will be able to have it as a reference for the task which might be a bit different. So then there's the execution agent contains this LM function with the promotion. This is the same exact prompt that original BB AGI had. There is a couple of like. Actually there's like three different template slots here. So there is the main objective set up by the user, the human. Then there is the context examples criteria from this nearest function that I mentioned above with the Wolfram language documentation examples and then the context with the previously kind of executed task and you can see that I'm using the distance function cosine distance. Nowadays the state of the art for such kind of retrieval systems. It's a bit different but I think for this particular case was working well enough. And then there is the task creation agent. This mainly will create a bunch of new tasks based on the complete third party task and the result of the current task. And then there is going to be the prioritization agent that will reorder the list of tasks and put up the ones that seem more relevant based on the previous execution. We add all these single loop and the main code is this part here where the different steps of the BB AGI are executed. And at the end I was running a bunch of experiments like this one like generating slower automatic samples. And I will just show one at the end because I'm running out of time. One about like creating a lecture using Wolfram language code examples on geometry. So I was initializing the baby AGI then adding the main objective which is creating this lecture of about geometry using Wolfram language. Then the final result was running the whole baby IT loop for four times. I was getting a bunch of different lectures with Wolfram language code within it. I think at the end I have some examples that it created using this system. At the end also we can check the number of different tasks or kind of sub lectures that was also creating for them. So if I was running it for many other traditions it would pick up some of the. I would like to thank my colleagues at Wolfram and also like Wolfram summer school students and mentors and maybe AGI creator for some comments on how he created maybe AGI. Thank you. And now if there are like some questions I will try to answer them. Right. So I see. That that's a good idea from Anton. It's something that we also contemplated. I did not implement the Rock Paper Scissors plus Laser and Spock. That's certainly a fun experiment to extend what we have. And yes, if you're interested in such work we can connect and run extra experiment maybe. I think I will try to publish a Wolfram community post about it and. And that would be like a nice way to have some more experiments on this. Then there is the question about hacking phase. Yes, it will be supported in version 14 and for the small models, 7 billion models, some of them are going to be able to be run locally using users GPU depending on the system. But for example even like 7 billion llama2 chat I think it's too large for most of the laptops for example I think it's I don't know how many gigabytes is it but it's quite many more in the order of almost 100 gigabytes and then why I use it temperature zero for baby AGI implementation I think the reasoning here was whenever since I wanted to also write some Wolfram language code typically what we experimented is that for writing Wolfram language code per se or not just Wolfram language code in general lower temperatures is better doesn't mean that 0 is the best but typically close to 0 like 0.1 works better for reproducibility. Typically when you start with a sequence of tokens for describing a code you don't want to be to creative otherwise you will get some errors with brackets or some other symbols. Yeah it's probably for being creative on other tasks or objectives. Maybe GI needs to have a bit higher temperatures so that's right might not be the best. And yeah I think today and tomorrow will be around and I will be open to meet some of you and have some extra discussions about LLM experiments, playing games and autonomous agents.