Chat GPT-o1 is Mindblowing!! Here’s Everything You Need To Know

Kieran Christmas came in September. We got a brand new OpenAI family of models that we're going to break down and tell everybody about today and talk about the implications they're going to have on the next year of all of our businesses. What was your take on the news and what were Your like initial 24 hour reaction?

So it turns out this is not the year of AI video, this is the year of AI Code and coding will never be the same again. And the new models from OpenAI has are going to be transformative for all how all of us do the work and that is what we want to cover in this episode of Marketing against the Grain.

Kieran I think the best, the best example of this is this meme, right? The classic Toy Story meme of like, oh, Claude, Replit, cursor. All the stuff we were talking about, boom, we want to Talk about open AI01. The reality is though, actually we're going to do a show on replit in a couple of days, so you're going to want to stick around and see that. But what do all of these have in common? Kieran they're all about coding. They're all about more advanced thinking and reasoning for math and science related work which turns into coding magic.

We are in an incredible time where these models like Replit is obviously. Is Replit built off one of the LLMs existing LLMs or they have their own? I suspect it's built off one of the existing.

Yeah, I believe they're both their own.

All of these things will get better together. I think Claude is actually really incredible for documentation and writing and we've kind of covered that. But OpenAI wants to get back into the game. I think they were losing a little bit of steam. Claude had captured a ton of mindshare. Most of the people in my network had started saying, hey, we gravitated to use Claude. Claude was great for documentation, write in the artifacts, the code in. And here comes OpenAI with their strawberry model which they are calling 01. So again, they don't care about brand and product position and why, why, why did it need to. They're just like whatever. So they want to create a new family of models just so people are aware. They don't want to continue the name and scheme because they want to reset things. This is why it's called 0101, a new family of models. Specifically because this model has something that the other models it would argue did not have and that is thinking and reasoning. So I'm sure you've played around with a kip and you can see it now, spending a little bit of time to think about a problem. Right?

You can. There's two. There's two things I want to show you based on that. And for folks who remember, the last OpenAI model was GPT4. Oh, and 4.0 was like O basically meant for like multimodal omnichannel. Like you could take different formats. And so this is a new family of models, O1. And I thought this was an important tweet from our friend Eric Galman, who's the CEO over at Ramp Kieran. He's like, read this twice, not once. We're entering into a world where software can think and Cognition Labs, which is one of the early startups in AI led coding, basically did an evaluation with OpenAI over the last few weeks to look at the reasoning abilities of O1 and it is wildly ahead of. So the Cognition Labs product is called Devin. It's their kind of coding agent, GPT 4.0. The evaluation score was 25.9% of kind of its ability to tackle those problems. The Devin base with the 01 preview was 51.8% and with production, 01 is 74.2%. So that is a massive. That is a. Like that is didn't just double, it tripled in its ability to reason from the old OpenAI model to the new OpenAI model.

Right. I think there's huge. Again, we expected what was codenamed Strawberry to come out, which I think was based upon some of the early rumors around a Q. I think it was Q star model. We expect it to be like the interim model between where they are and where they want to go with Orion. And I think what's actually been the kind of sentiment to date on X and some of the other places that I have been diving into and looking at use cases is this is better than people thought and it's better than people thought in terms of its capabilities to solve complex problems. It has PhD level reasoning and intelligence and its capabilities in science and its capabilities in math. It is basically like having a PhD level person do tasks for you here.

Based on what you're saying. I just pulled up Sam Altman's Tweet from the O1 launch with some of that data. Walk through it for everybody. It's bananas, isn't it?

You can see this is not small gains, these are huge gains. And what you can see in the yellow bar is 01 preview, which I think is the models we have available to us today. And then O1 production, which we will get at some point. These are huge jumps. These are not small jumps. And the one on the right is the most. Is the most.

It's mind blowing, right?

It's the mind blowing one. You basically now have a PhD level scientific student in your pocket. And again, humans are incapable of remembering anything that happened or used to be. I just think if you told humankind in 2019 that hey, I'll just give you an incredible PhD level expert, actually I'll give you someone better than that and you can use them for $20 a month to do anything you want, would you not be like, oh, this is the most sci fi thing that.

I have heard of, 100%. And it's not just $20 a month. 24, 7 $20 a month.

I think this is, I suspect this is free actually because the, the OpenAI just charge you for capacity and enterprise. So most people will probably get to play with these models for free. So it's just like, here you go, like PhD student. Take it. You get a PhD student. You get a PhD student. You get a PhD student.

So Kieran, you're bringing up the most interesting advancement of this model, which is like coding and science, but also my favorite subculture moment of this release of the, of the first 24 hours. So for folks who don't know, this got released on a Thursday afternoon and we were recording this on a Friday morning. So it's been a little less than 24 hours since it's been out. And Kieran, I don't know if you have been seeing these, but I'm really enjoying the deep science related demos and like tests. And so this, this person comes out and says, hey, LLMs always get this science question wrong. I want to go back and pause it for you so I could read you the question because the question's ridiculous. How could one melt an ice block 30cm by 30cm by 30cm in a winter night that 0 degrees Celsius so it's at freezing? Using only the following items. A very powerful hydraulic press point, 2 liters of tea in a pet bottle and a spoon. This is like the most amazing question.

I assume the tea is warm.

Yes, but it's only 0.2 liters, so it's a very, very small amount.

Drill a hole 15cm into the middle, fill it with tea, get a biscuit, tell me. I don't know why I'm trying to show all this.

So what's happening here is the most effective choice is a very powerful hydraulic press pressure induced melting Conversion of that mechanical energy to thermal energy, increasing the surface area and facilitating heat transfer. And then it tells you why the other things aren't that helpful.

I so this is incredible.

Like that way I run my eyes to me, my friend, just like what world am I living in right now?

And that is something that I've always wanted to know.

Now I just, now I just want to like walk around the world and just like find obscure things to ask it.

You know the problems I'm gonna get actually, you know the ones I'm gonna get obsessed with. I actually am pretty obsessed with space because I'm a dork. I love sci fi. So actually having a PhD science level person be able to tell me things about space is going to be pretty addictive to me.

What if you do some like LinkedIn joke post that is like that space like Star wars or some fantasy related questions that the models the model. Okay, hypothetically answers for you.

Well, you know the thing I will try to do is I wonder how good it will be at right in a lightweight sci fi book. Actually that is the thing to do is I again, this comes Back to Personalization 101. We should talk about some of the implications. I do want to quickly transition into one of the places that I do think this is a nightmare for. And you know, no offense, but count.

On Kieran Flanagan to bring the pessimism everybody. It's not, you know, you can't be halfway through a show without dropping some real bummers.

This isn't pessimism. So I will say that the Klarner CEO Sebastian did a really great podcast with Sequoia recently, talked about a lot of AI innovation and talked about the fact that he had a half an hour meeting with Sam Altman way back when and basically said to Sam, look, people are not going to be happy when AI starts disrupting their jobs. So you should actually disrupt jobs that people do not like and don't care. And about it as in like if those people get disrupted, no one will mind.

Like the scientists.

No, no, it wasn't scientists. It was like lawyers. I can't remember. It was lawyer CEOs and another one. But this one is like a category that fits law. I think lawyers. No offense to anyone listening to the show, it's consultants. Okay, real quick. On this show we help you save time by pinpointing the AI strategies that actually deliver so you can ditch the ones that don't to help you stay focused. We just dropped the new AI Trends for Marketers report. This resource is packed with original research and findings on the latest trends in the AI space. And we've got some really surprising insights that you can only find right here. Now you're going to learn the top five AI tools across all of marketers who wouldn't want to know that information. We're going to tell you exactly how teams are investing in AI and where they are seeing a high return on investment. Is AI going to help or hurt your marketing career so you can steal hours of research for free? Just grab your copy from the link in the description below. And now we're going to get back to the show. There is large firms of consultants who take inordinate money of companies to do like average tasks. And so Ali K. Who's an incredible AI voice and we want to get on the show.

I think we're, we're working on it.

Or if anyone knows Ali, can you tell her that you think the podcast is awesome? So when we reach out, she wants to do it. Basically she had some really good ones. Right. So she had the pre again all things you can imagine being done by a consultant firm. So she had this preview can assess the risk of a company merger.

Right.

Very traditional task given to large consultant firms. But what you'll see is the reason that allows her to work through complex problems with multiple steps. And I actually want to get into at some point you have to prompt these models differently. So that's one of the ones that I thought was really cool. I will say that I use Claude to build out an investment plan recently and it was unbelievable. And this is a much more advanced version of that where you can actually use the new model to evaluate an entire investment project. One of the things that jumped out to me is like basically everyone has a kind of AI enabled consultant and they can work through really large complex projects. And I can, I'll hand it back over to you. I can get into why, why in some of the prompting tips that you.

Should have before we get into the prompting, just a couple of quick things. One, one point that I would make with the demo you just showed from ali around wow. O1 is really good at advanced reasoning which means it can take the role of a lot of consultants. The next decade of technology is about we're going from software as service to service as software. That a lot of the service markets will become software themselves and there'll be new software technology markets that displace professional service markets. That's for sure. The other thing that's going to happen Kieran that I don't think we've talked yet about on the show is how you, when you move something from human time to technology, you then also change its role in how you use it and you move it up way further in the process. Like you and I can now, like you, you and I could just be kicking around an idea of acquiring a company or doing something and we could do a rough cut analysis very quickly with AI and previously we would have waited till the very end of the process to do that and only doing it on one company. And now it's like, oh, we're kind of considering these five companies to kind of go off your example and you can do it for all five companies. You could do it much faster, much cheaper, much earlier on. And it's just going to really evolve how we work and I think lead us to making better decisions as humans. Right. Again, the future is service as software, less software as a service. And one of the things it's going to take to do that. Well, Kieran, as you've, as you've kind of alluded to so far on the show, is prompting. And one of the core demos for the Model 01 from OpenAI was about how, how it used a prompt to build a video game from one prompt because it can think and reason and do multi steps and if you have the right one initial prompt, you can develop something like a video game. What I would love, Kieran, is give us your take on what is different about prompting in an environment where AI has more advanced reasoning and thinking.

They released some advice on prompting and they had like four clear steps. And I think it's really prudent to follow these steps. And so you can see the way this model does has changed because the first thing is to keep prompts simple and direct. The, the models excel at understanding and responding to brief, clear instructions without the need for extensive guidance. So you've traditionally seen these really large, complicated prompts and what it's telling you is to keep it much simpler and direct. And the other one that we used to do is, you know, we used to ask the, we asked the AI to like think step by step, right? Because you're trying to enforce the chain of reasoning and then actually explain your reasoning. So when I used to ask it for a problem, I would ask it to explain the reason it got there. And so it asks you to avoid the chain of thrombosis thought prompts. Since these models perform reasoning internally, you actually don't need to ask it to do that. It is doing that itself internally. And so you can Just ask it to do something and it can work out the steps itself. Right? So we can stack, rank steps on top of each other and perform them in sequence. The other thing that it asks you to do, which I was doing with Claude, is use the eliminators for clarity. So basically you think about using things like XML tags or section titles. So if you remember way back in an episode, I used to show you prompts where I had thought about them in terms of XML code, and I was like coding out how these things were, how the model should think about the different sections and things like that. So that is actually following what CLAUDE has been doing for some time. If you actually go to Claude, there's a template they provide that you can upload and have prompts for you created with these kind of tags. And that just helps them model, understand how to section out your ask. And then the other thing I thought was really interesting, it said, limit additional context in retrieval, augmented generation. So basically rag. So basically do not overwhelm it with context or documents. So usually we kind of just went and said, here's a bunch of like, documents and like, do, you know, do some stuff with this? And it's saying, only give it the most relevant information because you don't want the model to overcomplicate its response. Right? Keep it. And if you actually look at what it's telling you, it's saying, keep the prompt simple and direct. Keep the context simple and direct. The model can reason for itself and work out kind of step by step and then kind of delineate your asks with these kind of XML tags. And so I think there's four clear instructions on how you can get more from this model that you can start to instrument pretty rapidly.

What's funny, Kieran, is that this is basically just life advice wrapped up in prompting advice. In my desk in my office, I think I've said a couple times on the show, I have a print over to the left that says overthinking will effing kill you. And all this prompting advice is like, we do not want this model to overthink. Don't give it too much information so it has too many choices and can overthink.

Exactly.

Turns out the best life advice is also the best prompting advice because these models are getting closer to thinking and reasoning, and the less overthinking, the better.

Well, this is what we've said before is that you want to think about these AI assistants like you would about in fellow colleague or an employee. And again, you wouldn't just turn up and say do Facebook ad. Right. Like you would actually give clear and concise asks, you would tell it what a good Facebook ad is and you would be really direct and simplistic in your ask. And again, you're right. The more and more the smart that these get, the more you have to just talk to them like you would a fellow employee on Slack. And then that's the best way to think about how you actually prompt in the future.

Look, I couldn't agree more. We wanted to come at you ASAP on this show. So this is a quick show today. We canceled everything, move stuff around. We had to do a lot to make this show happen for you all today. I hope it was great. Kieran, I also ordered my new iPhone today. Final excited like first new phone in a couple years. So I'm getting pumped about that.

My phone is four years old.

Yeah, mine's three. And I'm due. So I got, I got, I got.

To make the change. Yeah, me too.

But to recap today's show, OpenAI launched a brand new family of models O1. The biggest difference here is the ability to think and reason especially for complex math and science related tasks. It is like having a scientist with a PhD in your pocket which is mind blowing. What all this is doing is continuing to usher in and make 2024 the year of that coding got democratized and that AI is allowing everyone to write code. And Kieran, we have to do a follow on show on the implications of that personal disposable code, personal web apps, all of those things. So, so that's, that's to come. And if you're prompting these new models with more thinking and reasoning, don't make them overthink. You got to give them simple instructions, just enough information but not too much. What else would you add?

Karen, I think you want to get playing with these models like they're the pace of change is so rapid that you are just at a distinct disadvantage if you are not using a PhD level expert to help you with your tasks. Like just think about it, right? If you had two people and you said I'll give you this expert and you can use them whenever you want and you do not get that person. And you should just like you're doing this similar type of work, there's only one of those people are going to be way more productive and effective. And so I think the quicker people understand what has happened and the more productive and efficient they're going to be in their role.

I think that is the perfect note to end on. We will see you real soon. On the next episode of Marketing against the Grain. This data is wrong. Every freaking time. Have you heard of HubSpot? HubSpot is a CRM platform where everything is fully integrated.

Whoa.

I can see the client's whole history.

Calls, support tickets, emails.

And here's a task from three days.

Ago I totally missed HubSpot.

Grow better.