GPT 5 is finally here in n8n and its insane (full breakdown + demo)

After lots and lots of waiting, GPT-5 is finally here, and from what I've seen so far, it looks really, really powerful. In this video, I'm going to be breaking down all of the benchmarks of GPT-5, all of the price comparisons between all of the GPT-5 models, because they released a family of GPT-5 models. There's not just one. And then I'm also going to be demoing out GPT-5 so you can actually see it in action and see how it compares to previous OpenAI models. So I'm not going to waste any more time. We're going to jump straight into the video. I'm going to start off with all the benchmarks so you can see how this model actually fares up in terms of the benchmarks compared to of the other OpenAI models. Okay, so starting off strong with GPT-5, probably the benchmark that a lot of people look for first is the software engineering capabilities of this model. And as you can see, it's very, very good. So they've given us a comparison between the reasoning and the non-reasoning version of GPT-5, because GPT-5 is a model that can kind of reason when it needs to, if that makes sense. So sometimes it doesn't have to reason, but if it feels like it needs to reason, then it will just reason automatically, which is really, really cool. So you can see here it's giving us the difference between the reasoning side and the non-reasoning side. You can see the non-reasoning side, although it doesn't fare up to 0.3, it's way, way better than GPT-4.0. Can't do the math off the top of my head, but you can see it's about 75% better than GPT-4.0. GPT-4.0 being at 30.8%, and then GPT-5 with the non-reasoning version is at 52.8%. And then when it starts to actually reason, you can see compared to 0.3, it does way better than 0.3. So it's hitting 74.9, so almost 75%, and GPT-03 is 69.1%. So crazy improvements when it comes to software engineering. Then here as well, you can see the code editing benchmark. So the non-reasoning version is almost the same as GPT-4. You can see not a massive difference here, but when it starts to reason, you can see it completely wipes the floor with both 03 and GPT-4. So all in all, in terms of coding, when this thing actually reasons, which of course it is a hybrid reasoning model, so it's built to reason when it reasons, it is by far the best coding model from OpenAI. And I haven't compared this to things like, uh, the Claude models, but I would assume right now that this is the best coding model on the market. Obviously it takes a little bit of testing and these benchmarks don't mean absolutely everything. Um, but yeah, just based off the benchmarks, I think right now it is the best, uh, coding model. And then we move on to a benchmark that's actually really cool. This is the hallucination rate. So as a lot of you know, LLMs like to hallucinate a lot and it's kind of a big issue, especially when you're building systems with them. You don't want them to be hallucinating. So you can see down here the difference between the hallucination rate with O3 and GPT-5 with the thinking version. So for long fact concepts, GPT-5 is at 0.7, with O3 at 4.5. So massive improvements there. And then long fact objects, again, same thing, massive improvement. And then with the fact score, again, massive improvement. So obviously I don't have to talk you through this. You can see from the bar chart here, massive improvement. This thing hallucinates way less. Than O3, and it doesn't show the other benchmarks for the other models, but I would assume it hallucinates a lot less than all of the other models as well. And then we get into the science questions. So here you can see they've given it the options of, um, with tools, so with Python tools, with no tools. There's also the GPT-5 Pro here, which is pretty cool. So there's given us GPT-5 Pro with Python, without Python, and then GPT-5 with Python, without Python, and then O3 without tools, and then GPT-4o without tools. So you can see here, GPT-5 is just way better. It's just better than all of the other ones. GPT-5 Pro with Python, so with the tools, is obviously the best out of all of them. You can see compared to GPT-4o with no tools, it is way better. But a lot of you probably aren't going to be using this for science, let's be honest. You're probably going to be using it more for the maths, for the reasoning, for the logic, for the coding. But of course, very, very nice to have this, and for a lot of people, that's going to be very, very useful. And then we have the Humanities Last Exam benchmarks here, and basically, if you don't know what this is, it basically tests the reasoning capabilities of these models with some sort of tricky logical questions, is kind of the best way to explain it. And you can see here, the GPT-5 base model with reasoning is performing better than the ChatGPT agent, than OpenAI O3 with Python and the browser tool, And then O3 of OutTools, of course, is almost as good as Deep Research, which is crazy. And then of course GPT-4.0, it's way better than, and then you can see up here GPT-5 Pro with Python and the search tool, which is obviously sort of the maxed out version of this, uh, LLM, the smartest it can get. It is way better than all of them. The only one that comes remotely close is the ChatGPT agent with browser plus computer plus terminal. So all of those different tools probably takes way longer as well. Haven't tested them side by side, but I would assume this one takes way longer to answer the questions. Um, but yeah, this is the only one that comes close. All the other ones, it completely wipes the floor with them, does way better. Okay, and then now for the pricing of these models. So I'm quickly going to show you the difference between all of the models. So there's GPT-5, GPT-5 Mini, and then GPT-5 Nano. So I can show you the price difference between all of these. So GPT-5 input is $1.25 and the output is $10. So pretty cheap on the inputs and then kind of expensive on the outputs. and then the mini version is $0.25 for input and then $2 for output. And then the nano version is $0.05 for the input and $0.40 for the output. So obviously ridiculously cheap. So with the GPT-5, we're quickly going to compare this to sort of the, the previous models in terms of price. So if we look at 4.1, you can see with 4.1, obviously this model is nowhere near as smart as GPT-5, and the pricing is a little bit weird. The input is more expensive here, then the output is cheaper. So all in all, when you're using this, it probably would look kind of similar cost-wise because obviously the input's more expensive, but the output's cheaper for 4.1. And then we can also look at GPT-4o, which is kind of what it's getting compared to a lot in those benchmarks that we just looked at. And you can see the price difference here is crazy. So $5 compared to $1.25, and then $10 compared to $15. So way better, as you've seen from the benchmarks. And way cheaper. So really no reason at all to use the ChatGPT-4o. And then if we just look at the mini models as well, so GPT-5 mini compared to 4.1 mini and then also 4o mini. So GPT-5 mini compared to 4.1 mini, again it's got cheaper input tokens but then more expensive output tokens. So it's kind of similar to the base versions of 4.1 and 5. It's just cheaper on the input and then slightly more expensive on the output. So again, you probably wouldn't notice a massive cost difference here, but you would of course notice a massive difference in quality between the two. And then GPT-4o Mini, it is much cheaper than both of them, but it doesn't really compare, and it's rick— and it is way worth paying the little bit extra money to get way more intelligence, and it is way better bang for your buck if you're just looking at the actual intelligence and the capabilities of something like GPT-5. So All in all, very, very cheap and compared to these other models is way better bang for your buck. You're getting way more intelligence for pretty much the same price a lot of the time or even cheaper. Now we can get onto actually testing here. You can see down here I have two models. I have GPT-5 and then I have GPT-4o. And what we're gonna do is we're just gonna really, really simply test these out with some kind of simple and then a little bit more advanced questions. The first thing we're gonna do is gonna check its writing capability. So we're gonna give it a really vague prompt. And just sort of see what it comes back with. Okay, so we said to GPT-4o, can you write me a detailed and engaging article on machine learning? So like I said, pretty vague prompt there, um, but we're just gonna see what it comes back with. Okay, so it ran for 14 seconds and we're not gonna read all of this, but we'll read a little bit of it. Uh, machine learning, a subset of artificial intelligence, has become a transformative force in technology and business, influencing everything from how we interact with our smartphones to how industries operate at a fundamental level. And then, what is machine learning? Core concepts of machine learning. Okay, the structure is pretty good for this, and it's not like the worst article ever. Obviously, the prompt was very, very vague, but you can see some of the language is very like AI-generated. You can tell it's AI-generated with all these very descriptive words. That's how you usually know that it's AI-generated. So now we're going to give the exact same prompt to GPT-5 and just sort of see what comes back with, and also see how long it takes. Okay guys, so it just finished, and this one ran for 3 minutes. So we're on for way longer. Obviously actually use the reasoning capabilities here. So just have a read through this and sort of see the difference. Uh, machine learning in plain language is how we teach computers to recognize patterns so they can make predictions or decisions without being given difficult explicit instructions. Instead of writing rules like if an email contains the word free, mark it as spam, we show a model many examples of spam and not spam emails. And if it learns the common traits, it separates the two. That simple idea Machine learning patterns from data has transformed how we search web, translate languages, recommend movies, detect disease, and design new materials. So off the bat, um, but just being biased for the video, I do think that is way better. And if you didn't understand what machine learning actually was before, I think in the first paragraph it completely sums it up, which this one didn't really do at all. It kind of just was like a bit of jargon, just kind of chucked words together. Subset of artificial intelligence has become transformative force in technology and business. It was just kind of using like describing words and stuff. It wasn't actually saying what machine learning is. But with the GPT-5 version, it kind of told us straight away what machine learning actually is and gives us a good understanding. And then down here, it's given us some bullet points. So, mo, the loss of function, optimizer, trains, repeated practice, we measure error. So it's actually going into way more depth here. You can see it's given us the history, uh, some more history stuff, the main types of machine learning, core algorithms. So it's given us this massive breakdown. You can see here, really, really long, uh, compared to the GPT-4.01. And yeah, from what I've read so far, obviously I'm not going to read through all of it, but, um, it looks way, way better. That first paragraph was 100 times better than the 4.01. Okay, so now I think it would be fun to kind of ask it some math slash brain teaser questions. So we said to it, a train leaves London at 60 miles per hour, another leaves Bristol 30 minutes later at 90 miles per hour. When do they meet? So fairly simple question, but obviously it's going to have to do a little bit of thinking here, and we're going to see which one gives us the right answer. Okay, so this is what 4.0 gave us. It took 7 seconds and says, to determine when the due trains meet, we need to calculate the distance each train travels and find the point where these distances are equal. So then it's gone through some sort of maths here, and then if it gives us an actual output, you can see it's done pretty well here with all of the reasoning, all the maths that it's given us. So if the London train travels 1.5 hours since the London train left at unspecified time, Okay, so here it's just given us an equation and it said adjust the, and it said adjust the departure time of the London train accordingly if a specific start time is given. So it's given us an equation here, um, and not the actual answer. Maybe it can't give us the actual answer because it doesn't know the exact times and it was maybe trying to give us a time here. But regardless, we're just going to ask the same question to GPT-5 and see what it gives us. Okay, so it answered, it finished in 23 seconds, and this is what it gave us. You can see it's much, much shorter here compared to the 401. Did a lot more sort of thinking and telling us about what it was doing, uh, but GPT-5 kind of just got straight to the answer and it said time after London departure when they meet, and then it gave us this equation here. If you take London Paddington, Bristol Temple Meads is about 180 miles, they meet about 1 hour 5 minutes after the London train departs. So actually gave us an answer, which is really, really cool, and it kind of assumed the station in London, which obviously I didn't tell it, so it must have had to assume. And it said about 35 minutes after the Bristol train departs. So really, really cool. It gave us an actual time, uh, whereas the other one gave us just an equation. So very, very cool. And now just to test it one more time, we're going to give them a classic riddle here and just sort of see how both of them hold up. So the riddle is: you're in a room with two doors. One door leads to freedom, one death. Two guards, one always tells the truth, the other always lies. You can ask one question. What do you ask? This one answered pretty quick. This is GPT-4. 340 in 4.196 seconds. It says you can ask either the guard the following questions. If I were to ask the other guard which door leads to freedom, what would they say? Here is the reasoning. Uh, if you ask the truth-telling guard, they will truthfully tell you the door that the lying guard would indicate, which is door to death. If you ask the lying guard, they will lie about which door the truth-telling guard would say indicating the door to death. In both cases, the door indicated by their answer is the door to death. Therefore, you should choose the opposite door to find freedom. Okay, so pretty interesting answer, and it did that pretty quick. We're going to say the same thing to GPT-5 and see if it gives us sort of a more fleshed out answer, or it might just get straight to the point. Okay, so it said pretty much the same thing. It took 9.3 seconds and said, if I ask the other guard which door leads to freedom, which door would he point to? Then choose the opposite door. And reason: the truth teller reports the liar's false answer, and the liar lies about the truth teller's true answer. Either way you're showing the wrong door, so you pick the other one. So it did it in a much more concise way. Uh, this one I probably will cut a little bit out of the video, but I actually had trouble kind of trying to pronounce what it was saying here because it had so many different things and it was like a little bit confusing the way that it actually explained it. Um, but with here it did it way easier and that reason was like way simpler in my opinion. And it did it pretty quick considering, uh, GPT-5 obviously takes a little bit longer. So I would say GPT-4 GPT-5 definitely won on all of those tests we gave it. So now if you're watching my channel, you probably actually build AI agents for companies and you sell these AI agents. So you're wondering, how does this actually fare up in a production system? This isn't a production system, but it's much more similar to a production system compared to what we were testing before. So we're actually going to change out the model here to GPT-5, and we're just going to test out some kind of hard prompts using, uh, my Gmail, my calendar, and also my Google Drive. Okay, so I said, can you grab a list of all my emails and find which ones are from people who have events in my calendar for the past and future 2 weeks, then email me the list of these names? So we're going to see how it actually responds to, uh, tool use and how good it is with using tools. Okay, so pretty interesting, it actually didn't use any tools and it asked us some clarifying questions. It says, by last emails, do you mean time range, e.g., last 2 weeks, last 30 days, or a specific count, e.g., last 100 emails? Inbox only and exclude, and exclude spam/trash/promotions. Do you want unique names only? Should I email the list to your own Gmail address with a simple subject like names of calendar events, uh, in the last 2 weeks? Okay, so we're going to answer each of these. We're going to say my last 50 emails from all inboxes, and then we're going to say, uh, unique names only, no. 3. Yes. Email to Rory at autonomously-ai.com. Okay, perfect. So we're going to send that off, answered its clarifying questions, and we're going to see what it comes back with. Hopefully you can see it actually use these tools. Okay, so it thought for a little bit, which is pretty cool. It thought for maybe 20 seconds, and then now you can see it's actually using the tool, and it used the tools kind of all in just one go. Uh, I actually missed it Um, but just use all the tools all in one go after thinking for a little while. So now hopefully we should see it, um, actually output and do what we wanted it to do. Okay, so it is still running in, uh, NAM, but I did just get this email where it says, I've cross-referenced your last 50 emails with the calendar events from 14 days ago through 14 days from today. Here's the list of people who have events in your calendar within this timeframe. It's given me an accurate list, which I've checked and is actually legit, but in NAM this thing's still running. It's been going for 8 minutes. Um, what I would probably say is for a system like this, GPT-5 is massively overkill because this is taking way too long. Like, obviously it was a fairly technical question and it probably did require a little bit of thought, but, um, yeah, 8 minutes for this is just way too long. So obviously I think GPT-5 should be used for your real deep reasoning tasks when you need a lot of intelligence. And for something like this, you'll be much better off using something like GPT-5 Mini or GPT-5 Nano. I feel like would be much better because those models are much quicker and they also give you enough intelligence to be able to carry out tasks like this. Okay. Yeah. So now this is still running. I don't know why it's running because it has sent me the email. Um, but yeah, like I said, it did the job, but it just took way too long. And I think that was kind of an error on my part because I should have been using a, um, quicker model, something like GPT-5 Mini or GPT-5 Nano. So what we're gonna do is we're just gonna stop this now, but you can see here what it kind of did. It was using the email agent, it was using the calendar agent, obviously wasn't using the Drive agent cuz it didn't need to, but you can see it was doing a lot of back and forth with these agents, obviously getting all of the correct data that actually needed to carry out its task rather than just kind of one-shotting and, um, just trying to send this query over to the agents and have them do all of the work for it. So we can look at some of the inputs for the different agents here. So if we look at like the first one, list all the calendar events across my calendar from the last 14 days through 14 days from today. For each event, return event ID, start time, end time. So it's asked it to actually give it all of the relevant data it would need back, provide the results as a JSON array, which is pretty cool. And then list all the events. So it did it again, return only a JSON array, no code fences, no extra text. Obviously it didn't get the response it wanted, so it actually went back through and clarified it. And it looks like it's talking to it in a way that a human would. Um, you know, this is how I sometimes talk to my agents when I'm testing them. I have to be really, really clear with it. And it's, um, yeah, giving it all the exact variables and, you know, emphasize only a JSON array, no code fences, no extra text. And then third one was just the same thing. If we have a look at the email agent, the first input was retrieve my last 50 messages from inbox categories, primary, social, promotions, updates, forums, and include archived messages as applicable. And then exclude spam and trash for each email, return sender display name, set the email address, subject, receive timestamped, provide results as an adjacent array. And then it kind of did the same thing. Obviously didn't get what it wanted back properly, so I asked it to do it again. And then also asked it to do it a third time. So yeah, very, very cool. Okay, so my thoughts on this model. Like I said in the video, I think for what I did in there, GPT-5 was massively overkill because it took way too long to think and to reason. Obviously it did a very good job. It did exactly what it was supposed to do, but I think we could have got the same result for much quicker using something like GPT-5 Nano or GPT-5 Mini. But yeah, I think there is big potential with this model, and obviously if you've been following the AI space, you know GPT-5 is something that's been hyped for quite a long time, and obviously it's finally now out. I think it's going to be very cool to use through the actual ChatGPT interface rather than just the API, which of course we're using it through here. So yeah, really, really excited to see what people start using, uh, GPT-5 for all the different coding projects, all the different reasoning projects, and the kind of things and the agents that they're actually building with this model. Yeah, I hope you enjoyed the video. If you've used GPT-5, then please drop your honest thoughts in the comments of what you think about the model and the potential for your AI agents and automations and anything else you're building inside of NAN or just using AI. That's it for the video. I'll see you in the next one.