Kyle Balmer reviews GPT-4o Mini, which costs 20 times less than GPT-4 and matches it on some benchmarks while beating Claude Sonnet 3.5. Math performance drops significantly, and he still prefers the full GPT-4.
Questions this video answers
2024-07-24
About 20× cheaper
Chapters
Is the new ChatGPT model actually good? So we've just received GPT4O mini, which is their newest model, and the early benchmark tests are coming in showing us how it stacks up against other models. So the big selling point of GPT4O mini. God, that's a mouthful. Is how cheap it is. This doesn't actually matter if you are just using the app or the website version of ChatGPT to ask, you know, one off questions. But if you develop apps either for yourself or for clients, this is a huge deal because the new mini is 20 times cheaper than the other models. Here's one benchmark which is showing the Mini as being just as good as full version of GPT4 and actually better than Claude Sonnet 3.5, which is surprising. It's not as good in mathematics in particular. It drops way down the rankings compared to other models, but overall it seems to be just as good as the other models out there. But it's 20 times cheaper. Personally, I've used it a few times and I'm not finding it that impressive. I prefer 4.0. What about you? Have you used it yet and what do you think?