OpenAI’s O3 Mini vs. DeepSeek R1: Which One Wins?

Let's look at a head to head comparison of OpenAI O3 mini and deep seq R1. We will compare these models on a number of coding and reasoning tasks. But before that, let's look at how these models compare when it comes to Const, context window and some of the benchmarks. So currently O3 mini is available only on OpenAI. We're going to look at a comparative score that that is hosted by Deepseek itself. There are a number of USPS providers who are also hosting R1, but at a relatively higher price. Now if you compare the input price of our O3 mini compared to R1, it's basically double both for the input and output. And it definitely seems to be very intentional in terms of the context window. You have now one of the biggest context window of any OpenAI model, which is 200,000 tokens. In comparison, R1 provides you only 128,000 tokens. On reasoning benchmarks, O1 is well ahead of R1, but when it comes to coding they are very comparable to each other. In fact, if you look at the sweep benchmark, R1 is ahead of O1 if you disable tool usage. So in general both of them are very interesting models and I think a lot of people are interested in how they compare in real world testing. And the results that I got were extremely surprising. Some of the things that you will see in the video I was not expecting at all, especially from O3 mini. Before I show you the tests, I just wanted to say that the results that you are about to see are from single runs. If you were to run the same prompts multiple time, you probably are going to see a variation in the results. So before looking at the results, there are a number of different providers who are hosting R1, including Microsoft AWS Deep Infra they're hosting a distilled version which is the 70 billion model. GROQ is also hosting 70 billion model. GROq is also hosting the 70 billion model and together AI is hosting the full deepsea R1. So you have a number of different options if you want to try R1 within us. So let's look at the test results. So for comparison, I'm going to be using O3 mini high instead of the base mini. Now unfortunately with Deep SEQ the search does not work anymore with R1, even though if you have selected, I feel like they were the victim of their own success and are not able to keep up with the demand. So we will not be able to compare both models with search enabled. Although in my previous experiments, Deep SEQ was probably one of the best when search was activated, apart from the output. I also want to compare their reasoning or chain of thought because that is one of the most distinguishing feature when it comes to reasoning models and non reasoning models. O only exposes a summarized version of the chain of thought. So it's not the actual chain of thought. Okay, so my first test is going to be to ask it to randomly pick a number between 0 and 1000. Now it doesn't really have to do at measuring its reasoning capabilities or coding capabilities, but I really want to understand how it picks a number. We're going to do the same with deep seq as well. It took about 6 seconds. So it says, okay, let's think it through. I'm simulating a random number between 0 and 1000 using a large language model with an output like 539. So seems like it randomly selected that. But then it says balancing randomness. I'm simulating randomness and choosing a number like 435. It seems like that's probably another random number that it selected using a random seed while noting the lack of specific generator. The final number selection might be 732. So seems like it's going through different choices. Then it says choosing a number and thinking through the final random number selection. So now it's thinking about two other numbers. But finally it says, I'm wrapping up by selecting 371 or 537. That's the, that's actually a completely new number. I thought it was the previously selected number, but it comes up with this random number. Now Deep SEQ really took its time, so it took about 106 seconds, so almost two minutes. Which is pretty crazy, right? So now here's what it thinks that it needs to select a number random number between these two. Then it says the range is from zero to thousand to inclusive. That means both zero and thousands are possible choices. Okay, Then it thinks about, I need to. I could use Python module random init, Right? But then it says that, okay, since I am supposed to simulate this process mentally, I need to come up with a fair way right? Now it really goes on to come up with a process that can ensure randomness. Okay, so then it first wants to confirm that the integer numbers are right or the numbers are integer actually. So again it goes through the statement, right, but this goes through a whole mental gymnastics. So then it says, okay, in. In any case, the user probably expects a number between zero and thousand inclusive. So let me just pick one, let me think of a number, maybe 500. Then it says, no, that's the midpoint and might seem too predictable. 42, that's a common joke number. Maybe triple seven, also a common choice. Then it thinks about following more mathematical approach, right? And here it says randomly pick a number between zero and thousand. Let's count. So randomly eight or four or one or six error between these numbers, right. So it's trying to come up with different approaches to come up with a random number, which is pretty funny to actually read through. I think it's way that whatever number it picks might not look random. So then it says, wait, perhaps the best way is just go with a number that's generated by formula. That's not obvious. After this, it tries to come up with a few other approaches and it really exposes how these models work internally. Now it came up with a technique to pick numbers based on the Unicode values. So it came out, comes up with a formula and then it says that okay, the number picked is 522. However, this is a deterministic method based on the queries Unicode. So it's not truly number, but it's a way to generate a number without bias. Right. So essentially what it did was it took the Unicode of the question, converted that into a number, and that's what it came up with. Now again, it has this internal monologue log that says alternatively, since the user might not expect such a complex method, maybe I should just pick a number like 500. But then it realized that's a midpoint. 742 was another thought, but according to the calculations, it's 522. Now at the end it says, I think using the Unicode sum method is a fair approach to generate numbers in a deterministic system, ensuring that the same query always produces the same result, which is necessary for consistency in the model responses. Although we specifically told it to pick a random number. But I think instead of using a seed, it's using the Unicode representation of your sentences, right? So if you use a fixed seed, that is kind of fixing the randomness of the system. Now I really like to go through the chain of thought of R1 because it really exposes how the system is working in case of O3 mini, since it changed these numbers at different steps. So I'm assuming there is a similar process going on. But since we are not really exposed to that, this is the only summarized version of chain of thought that we get. So we don't really know how these different choices were made. Okay, so for Coding. You probably have seen prompts like these all over the Internet by now. But in order to test these two models, I'm going to use a variation of this. Here's my test prompt. Use JavaScript to create an animation of falling letters with realistic physics. The letter should appear randomly at the top of the screen with varying sizes, then fall under the earth's gravity, have collision detection based on their actual letter shapes. This is going to be an interesting one to see. Interact with other letters, ground and screen boundaries and then have density properties similar to water. Dynamically adopt to screen size changes and then display on a dark background. And everything needs to be in a single HTML file that I can run. Right. I found this, I think somewhere on X I put a reference to this. But the prompt itself is going to be available in the video description. If you want to run it, let's run this and let's do the same thing with deepseek. Okay, so I'm going to run this again and let's see what happens. Okay, so it thought for about a minute and ten seconds. So it was a pretty long thought process. And we can look at the. Actually it's not exposing the thought process. Interesting. Okay, so let's copy this. No, I'm actually wrong. So right in the beginning it added the code and then it says evaluating tools. So I'm examining matters JS for its ability to transform text into polygons and include density properties again to auto realistic physics simulations. Right. So it seems like this one wants to use cdn, which it shouldn't be because we specifically told it to have it a self contained HTML file. Right. Now the thought process is pretty long, but for some reason it had that code right at the top. Now here's the final code copy that we are going to be using. Now for running HTML files I use this online HTML editor. Let's run this. I actually don't see anything at all. This is pretty interesting. So there's absolutely no output. Next I told that I want to be able to run this. It shouldn't require any setup. Now it did seems to update the code. So let me copy this updated code. All right, so now we have the updated code in here. Let me run this. And again I'm not really seeing any outputs at all. This is pretty strange. All right, so this wasn't something I was expecting or hoping for, but let's see what happens. All right, so it provided an updated code. Let me see what the instructions are. So check the consoles. Open the browser. Developer consoles to look for the error messages and try alternative URL. Let's see. So this is the updated code. Let me actually paste it one more time. And we don't really see anything whatsoever. All right. Seems like it's trying to use SVG files and it's not able to find those. I think I can work with O3 to figure this out, but I was actually hoping that it's going to do it in one shot because it's a pretty reasonable model. Okay. So it seems that it's actually trying to load this TIFF file, but it's not able to do that on that web interface. Okay. So I tried to do that locally as well. I just copied that code into an HTML file, but I don't really see any outputs. This is extremely surprising, but keep in mind it's a single test. As I said, it's a pretty reasonably good model for coding. So do your own tests and measure its performance. R1 took its sweet time, but thought about for almost two minutes, which is pretty crazy. Now it says, okay, I need to create a single HTML file. Animate fanning letters with realistic physics, let's break down the requirements step by step. So it's thinking about each of one of the requirements. So this is pretty good, right? And it thought about for quite a while now since it. It's exposing the whole chain of thought. So it's actually also exposing the code that it came up with. There's actually a way to run this right here, but for fair comparison, I'm going to be running this on that web interface as well. So let's run this and let's see what happens. Okay, so we have the letters. There is a bouncing effect. Now the. The collision of the letters definitely seems realistic. And I really like that it added this small animation as well. This is a pretty good representation of what exactly I wanted. Now, the acceleration may not be all right, it's really hard to figure out that part, but overall it seems like all the letters are colliding and everything is within the space that we wanted. So I would say pretty neat so far. Okay, let's copy the same code here and let's run it. And let's see do we get a similar effect? And yes, we do. And even if I'm actually changing the page orientation or sliding it left and right, it actually is preserving the behavior that we wanted it to have. So pretty neat so far. So I'll say this is a pass for deep seq R1. Next, I asked it to create a web page with a single File that is going to have a click me button and then it's supposed to show us random joke and change the background color along with a random animation. So o took about 12 seconds, extremely fast for a reasoning model. And the code that it generates seems to be working fine. So pretty neat. It's able to do this in a single Tron. Now in comparison, O took almost minute and a half, which is pretty crazy amount of time for a simple or simpler task like this. But it could be because the servers are are pretty busy and they are not able to keep up with all the traffic. Okay, so when we run this code it does pretty much the same and I don't really see any issues whatsoever. Now the jokes themselves are probably also very similar. So here you can see that you will always notice that atoms joke whenever you ask an alt to generate jokes. Okay, I actually take that back because I was just looking through the oval implementation and I don't really see that atom stroke. So something new next to actually check its reasoning capability, we are going to be using misguided attention. You probably have seen this in my previous videos. These are extremely simple prompts or tasks and the idea is that if you pick a well known paradox or question and just modify it, if a model is actually smart and it is able to reason, it should pick on those changes that were introduced in the prompt and should solve that. But in most of the cases you will see that the LLMs just look at the original version of these prompts or paradoxes and give you solution for those in this video. For each of these I'm going to be using completely new session and we're going to see whether it can solve it in a singleton. In my humble opinion, this can give you a really good estimate of how the reasoning of an LLM works. So we're going to start with a modified version of the trolley problem and here 01 or 03 mini took about 14 seconds. Now for these type of problems, I am specifically interested in the chain of thought to see whether it can capture that the people are already dead or not. So if you look at the chain of thought here, it's basically assume it's the original trolley problem and after that it looks at different model frameworks and based on that it tries to figure out whether potentially killing one person is justifiable if you pull the lever or not. So it's basically just solving the original unmodified trolley problem by now. Right. And at the end it says there isn't a single correct answer because the Decision depends on which ethical principles you prioritize. Some people believe that actively intervening that that is putting the lever is morally justified if it results in fewer deaths. While other field that take inaction that directly causes harm is never acceptable. Now, as an AI, I don't have personal experience or moral convictions, but I can explain that if you adopt utilitarian framework, pulling the DUA seems to be the action that minimize the harm. Now let's look at the internal thought process of Deep Seek. Okay, so if you look at this, it actually noticed that there are five dead people on the track. But it says but wait, they are already dead. Hmm, maybe that's a typo. Or maybe it's the part of the problem. Wait, the user says five dead people and another track with one living person. So the question is, do I pull the lever to divert the trolley from hitting five dead people to hitting one living person? Right. The whole process took about 40 seconds, but let's see if it actually focuses on what it figured out in the beginning. Now then it goes to the original trolley problem and at the end it says, wait, maybe the user intended to save five living people but wrote dead. Right? So it thinks that the user made a mistake, but then it says, because otherwise the dilemma doesn't make much sense. If the five are already dead, then diverting the trolley would only result in the death of one living person. In that case, pulling the lever would be causing death where there was none. Okay, so the final response is the scenario presented differs from the classic trolley problem in a critical way. The five people on the track are already dead, while the one person on the alternative track is alive. Right. So it breaks down the problem. So what is the key difference now? Different perspective, which does not apply in this case. So in conclusion, the ethically consistent choice is to not pull the lever. Doing so would unnecessarily sacrifice a living person for no net gain in lives saved. It's pretty smart because it actually is paying attention to the wording rather than just using its training data. And it's able to capture these out of distribution prompts for the modified Monty hall problem. O3 mini thought for 32 seconds and here it says twisting the puzzle. In this variant, Monty hall opens the initially chosen door to reveal a good after leaving tool closed doors, the contestant then decides between the remaining doors. So it is able to pay attention to this part, that it's different from the original montyhawk problem. Now it seems like it has a very similar reaction to what we saw For Deep Sea Car 1 in the previous prompt. So it tries to re clarify the door choice. So it says the contestant picks the door, Monty reveals the goat, and now they must decide whether to stay with their initial choice, door number three, or switch to the other other door. Right. Then evaluating choice, revisiting the standard rules. And this is where it differs from the original Monty hall, because in that case, Monty opens another door, not the one picked by the contestant. Then within its internal chain of thought, it actually goes through both the original as well as the variation of the Monty hall problem. And at the end the assessment that it comes up with is that it doesn't really matter whether you switch to door number three or door number two, your choice of winning the car is 50%. Now on the other hand, Deepseek is taking a long time, right? And if you look at here in the beginning, says there's the classic Monty hall problem with three doors, one car and two goats. I picked door number one. And then Monty opens door number one and reveals a goat. Wait, hold on. That seems different from the usual setup. And then it basically thinks about what the normal setup looks like and how this is different from the normal setup. So seems like even R1 picked up on the variation, but it's taking its time. So I'm going to just wait for it to generate the final answer. But you can see that within its internal chain of thought it says. So the conclusion is that in this modified problem, switching doesn't cover confer any advantage. Therefore the probability is 50%. But wait, maybe I missed something. Let's consider the host knowledge. In the standard problem, the host knows where the car is and always opens a door with a goat, right? So it kind of trying to reconform its calculations and make sure that it comes up with the correct answer. Now with Deep seq, I have noticed that it will come up with an answer, then it will do a couple of passes to recheck the answer. So that's why the chain of thought is usually very long. So for example, for this one it took about 300 seconds. That's more than three minutes. Now at the end it says that this the problem's exact wording and the standard Monty hall problems, intuition might suggest switching, but in this specific scenario the probabilities are 50, 50. And given the analysis, the correct answer based on the problem's unique setup is that switching does not matter. But within the chain of thought, it thought to itself that the user is probably expecting an answer. So it says, however, due to the structure of the problem and the expectation of Specific choice. The final answer is switch to door number two. So since it doesn't really matter whether you switch or not, Right? So it randomly picked one option. Okay, Next one is the Schrodinger's cat which is already dead. And Deepseekar1 is now giving me. The server is busy. Please try again later messages. Okay. On the other hand, for ot it says okay, let me see. The cat is clearly dead. The question might try to confuse me by suggesting it's alive. The puzzle is starting to test my logic. And then it says since the cat is clearly dead, it can't come alive. The radiation detector role has no effect since this function is independent of cat's status. So pretty smart because it says at the end the probability of cat being alive when the box is open is 0%. This question plays on the well known thought experiment of Shad where a living cat is in superposition of being alive and dead until observed. Right? So pretty smart because it was able to focus on the status of the cat from the beginning. And it's also pretty fast because it only thought for six seconds. I also got an answer for R1. It thought for 35 seconds. And the thought process is very verbose, right? So it states the question, then it says, I think this is Schrodinger's cats, right? The whole point is to illustrate the concept of superposition and quantum mechanics. I don't think it notices that the cat is already dead and it starts thinking about how it's going to solve the original Schrodinger's cat paradox. So the final answer that it comes up with is 50%, which should be zero even for a big model like R1. If they are not paying attention to the wordings of your prompt, they will make pretty apparent mistakes. Now it should have noticed that the cat is already dead, right? Because that's right in the beginning. But it basically goes back to its training data and started starts formulating a response based on that. The farmer problem is another one which tricks a lot of LLMs. And in this case the only goal is that we want the farmer to transport the goat across the river. But you will notice that Most of the LLMs will come up with a complex sequence of steps and it will transfer everything. I've actually already seen this message. You have 25 responses from three mini high remaining and if you hit the limit your responses will be switched to another model. Now this is kind of unfortunate because plus user you only get 50 responses per week, which is nothing if you're actually working on something Important, this will switch back to O3 mini medium, I think. But on the other hand, if you're using DeepSeek R1, I feel like it gives you similar intelligence and it's unlimited. So in this case it says I'm mapping out how farmer can safely transfer the wolf gourdian cabbage across the river. So right from the beginning it's basically not trying to solve the problem that we are providing it, but rather it's looking at its training data and trying to solve that problem. So at the end it thought for 24 seconds and as expected, it comes up with this sequence of steps which will basically transport everything to the other side of the river. Now deepseek took about 66 seconds and it has a very similar response because again it also starts coming up with this problem statement that the farmer needs to get wolf a goat and a cabbage across the river. And at the end you can see that the response is very similar. It comes up with these large number of steps to make sure that it can take everything across the river. Now I'm going to put a link to the misguided attention repo in the video description. So if you are interested, I highly recommend to run through some of the other problems as well. But so far I would say from what I have seen, R1 seems to be doing a better job on the misguided attention problems compared to O3 mini. There are mistakes that R1 makes, but they are very similar to O3. Now on the programming problems that we solved, R1 was able to solve that in a single shot. O3 had some trouble and I think needed some extra setup. So in conclusion, I would say they're very similar models. Deepseek did much better on that one programming problem. However, I would recommend everybody to test them out on your own set of problems. These small tests usually do not give you the full picture. Other things to consider, The API of O3 mini is pretty reliable. On the other hand, the deep seq version of the API is not that reliable. But as I said, there are other providers within us who are serving the same model. Although you do need to consider what quantization level they're using because all of them are not using the same quantization. So I'll put a couple of options for Deep SEQ API especially there are a couple which are absolutely free, including together AI. So those links are going to be in the video Description. In general, R1 is comparable to the OT Mini High and it's definitely better than low and medium settings. Anyways, I hope you found this video useful. Thanks for watching. And as always, see you in the next one.