Gemini 2.5 Pro's: The Best Coding Model Just got Better

Okay, so Google just released a major Upgrade to Gemini 2.5 Pro which is supposed to improve its already excellent coding capabilities with a special focus on web development. So now it's supposed to create much better looking web apps. At the time of the recording of this video on Web Dev Arena Leaderboard Claude 3.7 Sonnet is top rank model. However, I suspect with this new update, Gemini is going to be leading by a big margin. I haven't seen the leaderboard scores, so I'm just speculating at this point. But we can test this model against the other Gemini models and see how it behaves. I can see the model as Gemini 2.5 Pro preview in early access. So I'm going to use the compare feature in AI Studio to compare this model with the previous Generation which is 2.5 Pro Preview0325.AI Studio in itself is an excellent product if you are trying to tinker with Gemini models. I recently created a detailed video on highlighting different features. I'll highly recommend to check that out. Link is going to be in the video description. Okay, so to get started we're going to test it on a very simple prompt. On the left hand side we have Gemini 2.5 Pro. On the right hand side we have the new updated 2.5 Pro. And I'm asking both models to create a fantasy sports league manager dashboard. Implement everything in a single file and let's compare how the generation looks like from these both models and as well as the final output. So right now both of the models are creating their traces of chain of thought. One thing I really like about Gemini models is that within the chain of thought they are specifically thinking about how they are going to implement the code. And sometimes you can actually see the code snippets. Okay, so everything for both of these models were default. The newer updated version took about 60 seconds while the previous generation took about 50 seconds. And the chain of thought in both cases looks very similar. Okay, so first we're going to try the previous Gemini 2.5 Pro. I am going to be using this HTML editor. So here we have the code on the right hand side we're going to see a preview. Here's the preview of the dashboard. So we have the standings of the different teams, then roster for one of the teams and the matches. So what the schedule looks like, what were the recent activity? It looks pretty decent. It's not bad at all. Now let's look at Gemini 2.5 Pro updated. Okay, so very similar actually even the name is the same. I guess like it looked at the emoji that I provided and now you can see a lot more details. So it added multiple different tabs. So this is the different matches, the schedule with the free agents. Right. I don't think it's going to be working, but overall the layout is definitely improved here. Now, I'm curious, even the theme looks very similar. It might be inspired by that emoji that we provided. So let me remove that and let me rerun this and let's see what happens. Okay, so this time again, 2.5 Pro was much faster or relatively faster with 53 seconds. The updated version took about 60 seconds. Okay, so let's try the same prompt without that emoji and I want to see what the difference in the outputs are going to look like. I am going to test these models on much harder prompts, but I just want to see how the visual differences look like in the beginning. And keep in mind, these are my first tests, first run, and I'm just telling you my initial impression. Okay, so in terms of speed, again, 2.5 Pro, the existing version is taking about 50 seconds. The updated version is taking about 64 seconds. Now, I thought the updated version is going to take relatively a shorter amount of time because I don't think a lot of people are currently using this, but they may not have allocated enough resources. Okay, this is going to be the current generation with the updated prompt. And we can see it came up with a very similar design. Wise, it moved around the pieces a little bit more, but you have the league standing and the theme also changed. So I do think it was influenced by that emoji. Okay, let's copy the 2.5 Pro updated. Okay, so the design is different or the theme is different, but overall it definitely looks much better and much more detailed. Again, it decided to add multiple tabs, and even within the tabs you can see that it added the different matches, the final scores, the teams, they're playing against each other. So pretty neat. Now, for an application like this, I'll definitely go for this newer version compared to the previous version. Okay, we're going to create one more simple web app. So create a simple encyclopedia of the first 25 Legendary Pokemon. Include their types, load snippets and images, and everything again is supposed to be within a single HTML file. So we are running 2.5 pro against 2.5 pro updated. Okay. This time they took very similar time to generate the output. So 62 seconds versus 68 seconds. Actually, let's look at the number of tokens so the new version created about 700 tokens or 7,000 tokens and it's very similar. Okay, interesting. I think it's exactly the same number of tokens. Alright, so this is interesting. It is probably a bug because I don't think it's the exact same code. Okay, so we're going to start with the existing version of 2.5 Pro. So let me run this and it looks pretty neat, right? It did add these small animations, so if you hover, you can see the card is moving. So this is pretty neat. Now what did the updated version come up with? So let's replace the code. And yeah, it looks way similar. I don't really see a lot of differences, but the fact that we provided very specific details. So I think that's the reason you see very similar outputs. Okay, so here's what I'm going to do. I'm going to ask it to code a modern landing page using HTML CSS JS and put everything in a single file. Again, I just want to see if you don't provide specific instructions what exactly it comes up with. All right, so this is the current version of Gemini 2.5 Pro. Seems to be pretty working and it's a decent looking landing page for a modern SaaS company. Here's the one from the updated version. Very similar in functionality and even in the design. Although I think it tried to add an emoji or picture here which is missing. Okay, now let's try it on some more complex things. And going to be using is this. So, so for this prompt, not only the coding capabilities comes into play, but also the creativity of the model. We are asking it to code a TV that lets me change channels using number keys 0 to 9, come up with an idea for the channel for all numbers. Inspired by classic genre of TV channels. Show detailed, interesting animations for content and create name for channels on screen. The name needs to be pretty creative and it's supposed to create a P5JS sketch, no HTML. Make sure the content of all the channels stays masked to the TV screen and put everything in a single file. Okay, again we're going to just run this prompt through both of the models and let's see what the output looks like. So chain of thought in both cases looks very similar. I don't really see a lot of differences between the chain of thoughts. Now this is potentially the summarized version of chain of thoughts, so that's why probably we're not seeing much differences. Okay, again, the current version of 2.5 Pro is much faster compared to the updated version. And yeah, we are seeing 9,865 tokens on both cases. Okay, this is the current version. Seems like it's able to fulfill our requirements. This was channel number zero, then 1, 2, 3, 4, 5, 6, 7, 8 and 9. Okay, so the code is about 700 and 570 lines of code. In this case, the code is about 570 lines. Now let's copy this one and see what type of output we actually get. Okay, so this is the updated version. Let me just run this. And we ran into an issue. This is not what I was expecting. Okay, what I'm going to do is I'm going to just copy this. Now the problem here is that it's going to send it to both 2.5 Pro, the current version and the newer version. But let's see what both of these models do with this error before even testing this updated code from the 2.5 Pro. Updated, I want to actually show you how these models think internally. And it's really fascinating. Right, so this is 2.5 Pro 0325. Now, the code generated by this model did not have the error, right? So it says analyzing the error. Okay, so this is the error message. It says look at the offending lines. So it looks at that specific line. Okay, so I think it looks at that function, but it says the error message mentions line 115 and it wants to double check that. Okay, Then it realizes that it says, wait, the error mentioned this specific line, but the code that I provided was from the other file. So it's not really the same code. So it really goes into an investigation mode and starts looking at different possibilities. So since it's not able to find that specific piece of code, it says maybe the line number reported isn't exactly P5JS source line, but relates to the internal canvas calling that react function. Now the problem is that even though it did not generate the error, it's not able to find the bug. But still make some adjustments to how that canvas is rendered. Alright, so let's copy the updated code and let's hope it actually works. Okay, so we do see an output here. This is channel 0, channel 1, 2. Actually, the animations I think looks comparatively better to the previous generation. I would say not by a huge margin. This one definitely looks much better. This looks like a real animation, but I would say they are pretty close, not drastically different. I do want to see what happened to the code that was generated by the 0325 version. Okay, I Think it just adjusted the code did not really introduce any new functionality. Now by looking at this again, I feel yeah, the newer version definitely created much better visuals and interestingly the channels that it came up with and even the animations, especially for this one or I think we had this jackpot. They look very similar. The ideas are very similar it seems like because they are using the exact same base model and even probably in exact same fine tuned or instruct version with some additional post training. Okay, we're going to try a couple of other prompts. So this one is JavaScript Animation Letters that are falling under realistic physics and we want to see how it's able to deal with this complex situation. Again, within the traces of the chain of thought, I don't really see huge differences. Both of them seems to come up with a very similar chain of thought. The final implementation varies a little bit, but not by a huge margin. Now for this one, the existing version took just a couple of seconds longer. Okay, so I'm going to copy this code. Okay, so I rerun the same prompt because I want a single HTML file this time. 2.5 Pro took 82 seconds versus 55 seconds from the updated version. So let's actually copy this. Okay, we're going to paste this and we see letters falling down. It's under different shapes and the reason is that I think it draws a box around the letters. So all I need to do is just to tell the model that the letters are not showing up. But let's see if it fulfills the other requirements. Seems to be working fine now. Before doing that, let's check the newer version. Okay, so this is the newer version. I'm going to run this and yeah, it's actually able to produce the letters correctly. So this is pretty neat and seems like they have collision detection. Let's see if the behavior changes as I change the screen area. Seems to be working fine. Okay, we're going to just run this prompt that I'm seeing rectangular and circular objects instead of the actual letters. This should be able to easily recognize what exactly is happening. Now this one produced the correct code, but if you look at this, it says. Okay, apologies for that. You're right. If you see the default rectangular circular objects, it means that the custom text rendering part is not overriding the default physics body shape as intended. Okay, so I'm going to update the code. Let's see if. Yep, I think we have a working code now. Although again it still kept the rectangular boxes around letters. But now at least we can see Letters and the behavior is what we expected. Now, the previous version of Gemini 2.5 Pro is a very capable coding model. Now, there is this one prompt which even the previous version of Gemini 2.5 Pro had a lot of trouble with. All right, and the prompt is this. We wanted to write an HTML code which will show 20 by bouncing balls within a spinning heptagon. So this is a variation of the viral hexagon prompt. But now we want to have 20 balls instead of single ball. They need to have the same radius, they need to be numbered, and they should all start from the center. And again, they need to have collision detection both with the walls as well as with each other, and a number of different other conditions that these needs to fulfill. I think so far I have seen that Claude 3.7 sonnet gets this consistently. None of the other models that I have tested actually are able to do it consistently. So let's see what happens. In this case, the original version took 66 seconds, whereas the updated version took 73 seconds. And I suspect both of them are going to show the same number of tokens. Okay, let's run the code and see what happens. Okay, this is the previous version. Let's run this. As you can see, it's not able to do it. All right, the balls have been created, but they're outside of the heptagon. Now. How about this one? Okay, so this is the new version. Wow. I think it got it right. Let me run this again. So it starts from the middle, which is really nice, and then they are colliding with each other. The heptagon is spinning. So this is a pretty huge upgrade. And actually one of them bounced off. So pretty nice. The animation looks interesting and more realistic compared to the previous version. Now, I do want to try it one more time to see if this behavior remains consistent or none. Okay, so this is the previous generation. Let me just copy this. Much better this time, but still not really what I was hoping for. Now the newer version is stuck in a loop. I feel like. All right, so it's been more than 170 seconds, and I feel like it's just generating gibberish at this point. Okay, I'm going to kill this process because it's being more than 220 seconds. But something I want to show you, and it's this. You can actually see that within the code it starts generating the MIT license for one of the packages it's using. Anyways, this was a quick video on testing this new model. It definitely seems to have improved when it comes to building visual web apps and even for problems like this seems to be working better than the previous version. One more thing which I want to point in this video is that most of the tests that you're going to see in my videos or any of the other testing videos on YouTube, they are not really representation of real world use cases. You're going to have a lot bigger code bases that you need to work with and Gemini models in general shines because of their long context windows. And just to show you a real world example. So I'm currently working on a RAC pipeline which I'd probably showcase in one of my upcoming videos. This code base is a few hundred thousand lines of code and only Gemini 2.5 pro experimental in cursory is able to keep that whole context. The other models like 3.5 sonnet or 3.7 sonnets, they're incredibly great models. But when the context or the code base becomes too long, these models are not able to keep up with these huge code bases. So do test them out in your real world clean challenges and see which model actually holds up. These leaderboards are not that great representation of what exactly happens in real life. I recently created a video highlighting some of the limitations of leaderboards, specifically the Chatbot arena leaderboard where we covered a paper from Cohere. So do test this model out and let me know in the comment section below whether you see a huge difference in coding capabilities compared to the previous generation or previous version of Gemini 2.5 Pro. This video contain just some of my initial impressions of testing the model. Let me know what you think about it. I hope you found this video useful. Thanks for watching and as always, see you in the next one.