EP 441: OpenAI’s o1 Pro: What is it and is it worth $200 a month?

This is the Everyday AI show, the Everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

There's a new breed of large language models that you probably haven't used. Even the dorky among us that follow generative AI every day. So, so many people haven't really used or figured out or found use cases for these new reasoning models like OpenAI's 01 and 01 Pro. So today I'm going to be specifically going into 01 Pro, telling you exactly what it is, telling you who it's for, how it works, and ultimately if it's worth the price tag. Yeah, you seen the price tag on this new 01 Pro? $200 a month. There's other things that are included in that, but I think we've kind of been spoiled by how cheap and available large language models have been, especially as prices continue to plummet. And then you see something like a $200 a month subscription for a model like 01 Pro, and you're like, what is this? And is it worth it? All right, we're going to be tackling that and hopefully a lot more on today's edition of Everyday AI. What's going on, y'? All? My name is Jordan Wilson. I'm the host of Everyday AI. This thing's yours. This is your cheat code. This is a daily live stream, podcast and free daily newsletter helping us all not just keep up with AI, but how we can actually use it to grow our company and our career. This is how you become the smartest person in your company, in your department, at AI. All right? And we all need it, whether you know it or not, right? Been saying this for many years. Even if you don't think that you're a dorky or a techie person, we all have to learn how to get the most out of AI. All right? And where you start doing that is, well, here you're listening, but also@youreverydayai.com that is our website. There you need to go sign up for the free daily newsletter. Each and every day we recap our show for the day, as well as a lot of other information in that daily newsletter. You know, the latest news, trends, fresh finds from across the Internet, tutorials, everything you need to know. It is your guide every single day. Go read. It doesn't take long. Also on our website, there's more than 430 episodes that you can go listen to from the world's leading experts, all for free, sorted by your category. So go click on that AI learning tracks or that episode episodes on our website. Go find whatever you care about. It can be marketing, legal, tech, governance, I don't care. It's all there for you for free. So make sure you go check that out. Speaking of checking Things out, Monday, January 20th mark your calendars y'. All. We are next week going to be doing five episodes. These are not just our 2025 AI predictions, but they are more like a road map on how to deal with everything that's coming, you know. Yeah, I literally spent thousands of hours in 2024 talking to smart people, thinking about AI, reading about it, writing about it. This is the culmination of all that. You are not going to want to miss it. Get your board together, get your team together. You need to tune in. All right, before we get into today's show, I'm excited about it. Let's first go over the AI news for the day. Live stream audience, thanks for joining. Got a couple questions for you. Let me know. All right, so first, Google is partnering with the Associated Press to enhance Gemini with real time news updates. So Google's AI chatbot Gemini is set to integrate real time news from the Associated Press, marking a pretty significant collaboration between the tech giant and a major news publisher. So the deal allows AP to deliver a continuous feed of real time information to the Gemini app. AP's Chief Revenue Officers emphasized the importance of this collaboration, highlighting a commitment to non partisan reporting and accurate journalism. Financial terms of the agreement have not been disclosed, raising questions about compensation and how AP's content will be credited within the Gemini app. This is another piece of news related to this. This Google and AP deal comes literally at the same time as OpenAI announcing a new partnership with Axios to expand local newsrooms. So OpenAI will fund Axios's local newsroom expansion into a couple C cities, Pittsburgh, Kansas City, Boulder and Huntsville, marking the first time. And it has directly funded newsrooms in a publisher deal. So this three year partnership allows OpenAI to use Axios journalism for chat GBT responses while providing Axios access to AI tools for content creation and distribution systems. Yeah, we're, I'm actually excited. We're going to have a specific episode coming up very soon, so make sure you tune in on AI's impact, specifically on journalism. I was a former journalist so you know it's going to be a conversation near and dear to my heart. Last piece of AI news Microsoft has unveiled new usage based AI powered copilot chat for corporate users. So Sadia Nadella Microsoft's chairman and CEO introduced a new category of PCs and of Copilot with built in generative generative AI tools at a recent event. So the newly launched Microsoft 365 Copilot chat offers an alternative to the existing Copilot service which costs organizations $30 per employee per month. So now this is based on Usage. So yes, Microsoft 365 copilot chat based on usage. So the new model allows organizations to pay based on actual usage, right? So if you have thousands of employees at your company, maybe everyone's not ready to, you know, throw down a couple million dollars. So the new model, like I said, allows organizations to be charged for actual youth use with charges calculated per message sent starting at just $0.01 per message, which could encourage wider adoption among companies. So Copilot Chat, like normal Copilot, can summarize documents, fetch web information and create task performing agents. So unlike the traditional Microsoft 365 copilot, which is also integrated into applications like Word and Excel, Copilot Chat is accessible via the Microsoft 365 copilot app on various platforms. All right, so let me know, do you guys, do you guys want to see more on. On co pilot Co pilot chat? Let me know. All right, I'm excited to get going. So let's talk about it. Open AI's 01 Pro live stream audience, thanks for joining us. Have you all used this same thing? Podcast peeps? I always say check the show notes. You can reach out to me on LinkedIn, email, the show. I want to hear from you. All right, I can only make this better the more information you tell me. But I'm curious for our livestream audience, are y' all using OpenAI's 0101 Pro? Let me know, but let's, let's just dive straight into it. Enough, enough chit chat. So here's the gist of O1 Pro. All right, so it is a reasoner model. Okay, it's two different classes. So the GPT family of models, those are transformer models, all right? This is just the easiest way to separate the two. And a lot of other companies, you know, Google came out with their kind of flash thinking, which is a reasoning model. Amazon Nova has a reasoning version. Deep Seek, the Chinese AI company. So just in the last like four weeks, just about all the big tech companies have said, all right, we need a reasoning model because we see how powerful it is. So O1 is a reasoner model different than than a GPT model, right? GPT4O, you know all these other models that we've been using for, you know, now two plus years, they've been these kind of transformer models and they're completely different. And we're going to talk about the difference between the two, but that's the biggest thing. It uses chain of thought thinking kind of under the hood. And I like to say when you're working with Chat gbt, like, do any of you have different types of colleagues? Right, for those of us that maybe still go in an office or are hybrid, you have those colleagues, you know, you're getting work done, but you have those colleagues that work by talking, right? You're just going back and forth, back and forth. And then you have those employees or, you know, co workers that work by putting their headphones on, you know, you talk to them once and then you check in with them at the end of the day, right? Those are those deep work employees that put their headphones on. They just crank it. Think of that as like those two different types of large language models, right? There's some people that you work with, right, that require a lot of talking, right, Require a lot of conversation, require a lot of collaboration, you know, and, and that's how they work best. And then there's others just like, they don't want to talk, dump all the information on them, give them their instructions, let them ask questions up front if they have any. But then they go to work and you see them later. So the latter, that is what these new O1 models are. All right, so as an example for our live stream audience here, you probably see it on the screen. I took a screenshot of this. But yeah, generally when I ask these model questions, it's going to take anywhere from 4 to 15 minutes to get a reply. At least how I use 01 or at least 01 Pro. So I do use the, you know, the Chat GPT Pro account, right? So that is 200amonth. So I'll, I'll share what that includes. But you know, for the most part, when I'm using 01 Pro, I know this is my deep work colleague. I'm still using GPT4O all the time, all day, every day. I like using it in Canvas mode. I just started really using the new task mode, which if you listen to our show yesterday, we went over that new mode. By the way, for, for those of you that shared the episode yesterday, I took, I spent probably about three hours on a document that showed you how to use tasks and to go over this concept of task concept, task stacking, If I'm being honest, it's one of the best documents I've ever created in my life. So if you haven't gone and shared that task episode, go do that and I'll share that document with you. All right, so that's the gist of it. This is a reasoning model. This is not a GPT model. Takes a long time to think. It uses this chain of thought process under the hood, right? So you really have to use it at the right time for the right purpose, for the right reason. All right. Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on Genai. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everydayai.com partner to get in contact with our team or get, you can just click on the partner section of our website. We'll help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on GEni. So let's just take a quick look at the tiers or subscriptions. So there is a free version of Chat GPT. I've had a lot of Chat GPT shows this week because, you know, it's early in the year, a lot of people are asking and you know, we had some popular episodes like a year ago and I'm like, yo, these are old. So literally this week we did an episode on Chat GPT Free versus plus. All right, so you have your free Chat GPT which is actually pretty good now, right? I used to tell people, don't touch it with a ten foot pole, it's dangerous. It's not like that anymore. The free version is pretty good. You have the 20 plus plan that I think a lot of people are on that just gives you essentially almost all the features except the O1 Pro model. So even on the ChatGPT plus you have O1, it's much more limited in terms of Messaging and you have an O1 Mini, but on the Pro plan. So if you want to use Zero1Pro, which is technically OpenAI's most powerful large language model, you do have to pay $200 a month to be on that pro plan, a One Pro. But there's a lot of other, I guess, features and benefits. So when we talk about Sora, you have way more usage in Sora, you have unlimited usage of GPT4O. Whereas, you know, normally, even on a chat GBT plus plan, you, you run into limits. So essentially you get more limits on the Pro plan. It's, it's unlimited GPT4O, it's unlimited advanced voice mode, where on the plus plan is a little limited, and then you get access to Zero1 Pro, which you can only access via that $200 a month subscription. Yes, it is confusing. Yeah, Allison's kind of talking about the naming here. It is weird because the plus is that 20amonth where a lot of companies like Microsoft, they have their. The 20amonth is called pro. So even I get confused all the time. But yeah, so chat GPT plus $20 Pro200. Not to be confused with all these other ones that say Pro for the 20 tier. Yeah. All right, so let's, let's talk about how open AI describes their model. So they say more thinking power for more difficult problems. So they say chat GPT Pro. In this instance, they're talking about O1 Pro. Actually provides access to a version of our most intelligent model that thinks longer for the most reliable responses in evaluations from external expert testers. O1 Pro mode produces more reliably accurate and comprehensive responses. Responses, especially in areas like data science, programming and case law analysis. Compared to both O1 and Zero1 Preview, Zero1 Pro Mode performs better on challenging machine learning benchmarks across math, science and coding. All right, so, yeah, speaking of benchmarks, essentially 01 Pro, it's PhD level, right? It's, it's no longer where you really have to work. So that's the other thing. I think with like a model like GPT4O, you can get these, you know, quote unquote, PhD level responses. You just have to have a master's degree level to get it there. Right. It's different with O1. Especially with O1 Pro, you don't have to have a lot of experience to get it to that kind of. We'll just say quote unquote PhD level. It kind of can do it on its own because it uses this kind of under the hood, you know, step by step, chain of thought, reasoning, right? So it's weird and I've talked about this all the time and if you've taken our, our free prime prompt polish course, you understand the GPT4O family of models is extremely capable, but to get the most out of it, you got to know the basics of like prompt engineering, right? Without getting too technical, there's things called shots, right? And when working with a transformer model or a transformer family of models, regardless of if you're talking, you know, ChatGPT, Gemini Quad, etc. Right. If you do some basics of prompt engineering, it's going to be better. So a five shot prompt as an example is always going to do better than a zero shot prompt. So what that means, and I'm going to oversimplify it here, so sorry, machine learning PhDs, a shot is when you give a model an example in input and output, you tell it good, bad. Why? That's what I like to say. Input, output, pairing, good, bad, why? So you are essentially shotting this model. So that's what the O1 and technically O3, right? Like OpenAI teased the O3 model, it's not out. I don't think it'll be out anytime soon. But this O family of models kind of goes through that process on its own, right? It doesn't give itself examples, but it goes through that chain of thought, thinking. And I'm going to show some examples for you live on the screen. From a benchmark perspective, the gains are huge, right? So some of the biggest gains between like the 01 family and the GPT4 family are in math. I mean you're, you're automatically like Olympiad, you know, math Olympics, like gold, silver medal, right? So it's smarter than 99.999995% of humans in math, physics. Same thing. Huge jump from 04 to. Or, or sorry to GPT4O and the O1. Getting ahead of myself. Skipping, skipping over 03, you know, 400301 Alphabet soup already other categories. So yeah, Mathematics, Physics, LSATs, right? So you know, they have actual models, take exams. So huge. So obviously if, if you are in software development, if you are in research, if you work in anything that has to do with complex math, complex equations, business, intelligence, right? If you are essentially working with numbers, working with research, I think O1 makes the case for itself. But I'm going to keep going and we're going to talk about even some everyday use cases. So first you might have, you might have got confused, right? Because I'M dropping all these different words on your head. Oh, one this, O1 that, 01. Right. Because technically 01 has been out for a while. So we saw 01 preview in O1 mini in September. So yeah, all those other big companies that have, that are releasing these kind of quote unquote reasoning models, this is just the last few weeks. OpenAI has been here for a few months since September. They released O1 preview in A1 mini. And then in December they essentially knocked the preview off of it and said okay, now this is 01. So technically, if you think of power and not all these models are here, but you have O1 mini, 01 preview, which is now no longer there. O1 and 01 Pro. So 01 Pro and the kind of full version of O1 are new. Ish. They've only been out for a couple of weeks. I've been using it. I didn't get it right away, probably about a week or so after it was released. So I've been using it now for about three weeks pretty heavily. All right, so let's just go over quick and, and live. Live stream audience. Yeah, keep, keep getting your questions in. I'm going to try to tackle some of these at the end. Mark says it looked like a lot of work. Jordan, thanks for sending it. Oh yeah, that's the, the task thing from yesterday. Task stacking. Everyone slept on the chat GPT tasks. Even I think OpenAI missed, missed the point. All right, so let's go over the bullet point details. I'm going to go through this quickly because I'm going to show you here at the end of the show and a podcast audience. I'm going to try to walk you through it. 01 in action for even non technical reasons. So what is 01 Pro? All right, let's go through all the bullet points here. I want to make sure I give y' all the details. So 01 Pro is OpenAI's premium model available to chat GPT users for $200 a month. Also there's other third party, third party platforms where you can just pay by usage. Right. You can't get access to all the other tools and all that. But you know, here's, here's the thing. At least right now, the O1 model, it doesn't have all the other tools anyways. Right? It doesn't have Internet access. Right. The, the O1 Pro model, the O1 Pro, you can actually upload files, which is nice. And you can with that on 01 as well. Whereas previously the O1 family of models, you couldn't, you couldn't upload fil files and still on 01 mini you can't upload files anyways. But it's described, 01 Pro is described as an AI colleague for complex tasks and it's more about reasoning than collaboration. So how does it work? Well, like we talked about enhanced reasoning, it has this chain of thought processing to better, to kind of enable better logical breakdowns, accuracy and reliability. So we shared about this on the show before OpenAI kind of went over this like 4 of 4 reliability concept, right? Where you know, there's a little more variability in the kind of GPT family of models. With the O series of models there's much more reliability. And also this is specialized for, you know, professionals. If you are a specialized professional, O1 is for you. So it's strong in STEM coding, legal and data science advantages. What the heck are the advantages of this? Well, like we said, enhanced reasoning. So the chain of thought processing under the hood enables better logical breakdowns. We talked about the 4 of 4 reliability. So that's essentially, you know, when OpenAI did their internal benchmarks, they wouldn't just say do it once and like okay, yeah, this passes. They would do it actually four times for more consistent results. All right, so where does it excel? So I said, hey, here's where you can use it. There's strong use cases here. It excels in scientific research, so analyzing data sets, developing hypothesis, designing experiments, financial modeling forecasts, complex calculations, legal workflows. Right? Analyzing case law, summarizing documents. It excels at anything stem related, right? Anything. So it specializes in kind of synthesizing and analyzing dense data sources. If, and here's the thing, and I'm going to show you some more non technical use cases at the end and how I'm using it. We all have access to data, right? Data. Over the past five to 10 years, data used to be something for the geeks. Now we all have access to data. There's more and more data being collected, which is why I think there's actually broad use cases for people to be using the O1 Pro models. All right, so what is it good for? Kind of already talked about this, but this is what people are always asking, like, right, where's Excel? What's it like? What is it good for? Who should use it? So I, I want to tackle this from all areas. So who is it good for? So professionals in stem, finance, law and healthcare. Great healthcare use cases as well. So any users with high stake tasks requiring accuracy and advanced Reasoning. So developers also great for anyone in software development coding. So handling intricate coding and debugging requirements. Professionals in fields like medicine that require precision. All right, now let's do the breakdown. People are always asking well what's the difference? Should I just be using 4o? Should I be using 01 Pro? The way I like to say that. Think of these two chatty colleagues, right? For the most part we've been kind of spoiled by having these transformer models that are highly capable if you know how to use them. I would still say 80% of the business world has no clue how to use something like ChatGPT. Something that has now become synonymous with AI and has name recognition like Google. People still don't know how to use it, right? And people in positions of leadership. Kind of scary. But O1Pro excels in complex reasoning. GPT4O excels in well, fat being, fast speed, general tasks, right? Also GPT4O has access to more tools, which is important. So you can upload files in 01Pro but you can't use things like Canvas, you can't use things like Tasks, you can't use things like chat, GBT search, connect it to the Internet. That's an important thing because when you are using an a non connected model, you have to keep in mind you better be feeding it a ton of up to date data or whatever you are asking. It should hopefully not require a lot of up to the minute real world information because that model does not have it. All right, so how should you prompt 01 PRO? This is where it's completely different. And again think of my comparison. Does anyone else have that chatty co worker and then the co worker that just has the headphones on, right? I'm the latter. You never would have guessed, right? Someone that just talks about AI non stop and sometimes talks for way too long. But you know, speaking of like even my journalist days, right? I used to be the guy. I would go in there after an assignment, I would go talk to my editor, get all the information I would need, right? Then I would go sit in the back, put my, you know, headphones on. They weren't even headphones. I don't know if this makes me weird. I would have like the, you know, sound canceling, there's no music. I would just put those things on. I would just go to work, do the whole thing, right? But then there's other people, you know, hey, they want to check in at every single, every single point, right? But that's, that's the difference. So with O1, you have to begin with number one. A lot of data, a lot of context. You need to have clear and structured prompts to define task parameters effectively. You need to provide examples or templates to guide the model's output format. And you need to use concise yet informative phrasing to maximize response relevance. If y' all have taken our free prime prompt polish course, yes, we're going to have new ones in 2025. Give me a minute, I'll explain why later. But you know, we, we walk through something called refineq. So if you have taken our free course and there's been like, I don't know, 8,000 of you use that refine queue method for setting up your first prompt. 201, you're still going to have to answer a question or two because that's the, you know how we set up that refine queue. But try that out, it's going to work fairly well. All right, so let's get to the big question. Is it worth $200 a month? So let's talk about the pros and the cons. Well, the pros are high accuracy and reliability in complex domains, unique raising unique reasoning capabilities. Especially if you are in some of those more technical professions. Software development, engineering, anything with math, research, data science, stem. Right. If you're there, yeah, it probably is probably a no brainer. What about for everyone else? Because there's cons 200 it's not cheap. Although if I'm being honest, I think we've been spoiled by These free and $20 a month, world class, state of the art models that essentially now have mini rag. Right? We've been spoiled, right? Because the big companies, they know a lot of them are losing money, right? Like OpenAI reportedly lost, I don't know, four or five billion dollars in 2024 because they're not worried about making money. What we are getting, if you're a power user, you're getting way more than that $20 a month, right? OpenAI CEO Sam Altman said even on this $200 pro plan, they're losing a ton of money is what he said. This is still relatively cheap whether we're Talking about the 20amonth or if you have a use case for it. I think even the $200 a month pretty affordable, right? All things considered and we're going to see an example of that. So ultimately there's the pros and the cons. It's having a PhD level companion that will think about things, give you better results, higher accuracy. If you know how to direct it, but it is much slower. Right. So if you're used to just jabbing back and forth and that's how you like working with large language models and you don't see any problems right now with outputs, then it's not for you. But I think it's actually for more people than you think. I think people are literally just thinking, oh, oh, one. That's for, you know, engineers, data scientists, researchers, etc. I don't think so. So you also need to just ask yourself, yeah, do you need the advanced reasoning and professional tools, are there use cases in your domain that are worth that premium price? So you really have to ask those questions. There's no blanket answer. I think if anyone asked me, you know, chat GPT free or chat GPT plus, it's easy, I don't care what you're doing. Chat GPT plus $20, it's a steal, right? I've always said all along, if Chad GPT plus was 200, I would still pay for it. Right? Obviously I have a chat GPT pro plan. All right, so let's look live. All right. And please keep getting your questions in. I'm just scrolling through the comments. So thanks everyone for, you know, getting, getting your questions in. I'm gonna try to tackle them at the end. I'm just scrolling through all the comments looking for question marks. So yeah, make sure if you do have a question then, then, you know, get it in. Douglas, talking about our show yesterday. Great task write up. I think yesterday was the first day I saw the link to your post on your website. Yeah. All right, let's get after it. Let's do some stuff live here. So bear with me, y'. All. All right, here's what we're gonna do. Live stream, audience. As always, I never know if this works or if my audio is still coming through. Can you let me know? Can you all see my screen and can you see what's. What's going on here? So I'm going to explain to you what I'm doing after I get this started. Okay, so I'm going to be copying and pasting a bunch of information in here. All right, give me a second. There we go. All right, so this is information that I've exported from my podcast stats, right? I really want to make sure I have. Okay, good. All right, thanks, y'. All. All right, everyone says they can see. Thanks, y'. All. Thanks, y'. All. Okay, so I am in 01 Pro mode. All right, I'm gonna tell you what these I'm gonna read this to you, but I, I'm gonna get it going first because like I said, this might take a couple of minutes. All right, so here's my first kind of tip, right? Provide a lot of context. I'm going to walk you through the context that I provided, but also something that's changed recently. I, I don't know when. It was probably a year ago. You can run concurrent chats now. Generative AI, even 01 Pro, it's generative. You can run the same prompt even when you give it a ton of information. You might get very different things. You might get similar things. So I'm going to go ahead. Even though it might slow it down, I'm going to follow my best practices. Right. If I'm waiting, I'm just going to wait. So I'm literally running the exact same prompt in another tab. All right, so let's go ahead. Let's, let's check on it here. Okay. And we're going to walk through. So sometimes it will give you details, sometimes it'll tell you what it's doing under the hood. Right. And I know I have my, my token counter here, but the context window is much different. I probably should have mentioned the context window differences because that's important. That's important as well. Right? So essentially O1 Pro has a much, much larger context window. So let's, let's go ahead. And I'm gonna read now, I'm gonna read now what I actually put in. All right. I'm gonna try to go quick, but like I said, I exported some recent podcast episodes. There's a ton of stats. This is an example. And I want you to think, what data or what large amounts of context do you have? Because this is. I found myself when O1 came out. I'm like, okay, I may not be in stem, I may not be in data analysis, but I have access to a lot of data. And either I don't have time or when I am analyzing it, I'm really just looking for the low hanging fruit. And there's probably so much deeper and so many different, more channels I could go in if I had time. All right, so I'm gonna go fast here. All right, so I'm saying these are my podcast stats. So remember when I said when you prompt oh one think of it like that coworker that wants all the information and then they're going to go in the corner. So I'm saying these are my podcast stats. Keep in mind, Today's date is January 16, 2020. 5. For all questions, always exclude the top 2% and bottom 2% of episodes, unless otherwise noted. Right. I have a bunch of episodes, their downloads, some other stats, and sometimes there's anomalies, right? Sometimes there's just problems, and I don't want those problems to be included. All right, so already you're seeing that could be a lot of manual work. Even if you're good with business intelligence, you're good in spreadsheets, right? I'm saying also, always give the episode number and name number one. Keeping that in mind, please carefully answer and tell me question one. Give me the average downloads per episode. Question two, give me the complete list of episodes with the new performance percentage over under of the adjusted average. So I'm asking it, find the adjusted average number of downloads, take out the top 2%, take out the bottom 2%, then go give me. For each one, I want how it compares versus the average. So let's just say the average was, I don't know, 4,000 downloads, right. I want to see the percentage once you take out the top two, bottom two, I want to see each episode. Is it higher than that? Kind of median? I don't know, what's the math term? Right. Is it higher or lower than that? All right, then I'm saying question three. Give me the top 10 and bottom 10 episodes and their respective percentages that they're over or under the adjusted average. All right, question 4. For the top 10 from question 3, above the adjusted average, please suggest three slightly adjusted episode titles for each if I were to rerun them. So every once in a while I'd say maybe, you know, I don't know, know. Depends. Anywhere from one to five times a month, I'll rerun an episode. Yeah, sometimes I get sick. Sometimes I can't be here. Y' all with y' all live, you know, at 7:30 every single day. Although I try to. So I'm essentially saying for the 10 episodes that performed highest above the adjusted average, suggest three additional titles. But don't just like, look at it and randomly suggest them. Look at the trends, right? Look at, find common themes. So here's where we're really working with structured and unstructured data. This is where it's great to work with a large language model with natural language processing, right? So I'm like, yo, here's hundreds of episodes. Go find the ones that are really good. And then those top percentage, you know, ah, try to develop, you know, some way to see what's working and what's not and then apply that to some of these top percentages. All right? And then I'm saying like look, find common naming trends, example title link, title length, psychological marketing angles, superlatives, word choice, etc, be exhaustive in your pursuit of spotting common and hidden trends. Question 5, what are the most common patterns among underperforming episodes and how can I avoid them in the future? Question 6, how does title length or structure correlate with episode performance? Break it down from every angle you can think of. Be pinpoint specific. Question 7, how does release day impact episode performance? Please exclude Mondays as that is usually our AI news that matters day and we don't usually run other types of shows on those days for needed contacts. So even though the large language model should know, I'm saying, you know, giving it, hey, this date was a Friday so you can make sure you have it correct. Question 8, how does the release time or hour affect episode performance? Do not group them together. Go individual by hour. Be exhaustively precise and give me a chart that shows hourly performance. Right? Sometimes we get our episode out by 8:15. If I stop yapping, today's not going to be that it's already 8:06am Sometimes something comes up and we might not get it published until 11am So I want to know hourly, how does that impact it? Then I'm saying here's, here's one that would take a long time to figure out. I'm saying staying power and average decay. So in this document, in, you know, and I pasted all of this in there, I didn't upload a spreadsheet, I just pasted it in there. Essentially it was information from a CSV, but for hundreds of podcast episodes, it gave 7 day downloads, 30 day downloads, 90 day downloads, all time downloads. So what I'm asking here is, is to essentially figure out staying power and average decay. So saying, hey, when do average, like across hundreds of episodes, when do they normally quote, unquote, go stale? Right? When do they stop really, you know, getting listened to? Because people are searching for these all the time. It's not just people like hopefully you subscribe and thank you if you do right, but everyone else is searching for podcasts and they're discovering. So I'm trying to see which ones have staying power, which ones are more evergreen. And then I'm asking it to show me the top ones because then I can develop new episodes based off that. Question 10, how do episodes featuring specific brands or keywords, example OpenAI ChatGPT, Google large language model AI Claude, compare in performance. Question 11, please also categorize all of these episodes according to what you can gather from the titles. Example, marketing, ChatGPT, Enterprise, AI use cases, etc. Only put one episode in a category and try to create at least 20 different categories in doing so. Please also give me the average, the category performance versus the averages that we identified earlier. Right. A lot of this I want to see what sticks. What do y' all like? What do listeners actually care about? Right. And I think after you have, you know, thousands of thousands of pieces of data points, yes, I can go figure out some of these things with some simple calculations in, you know, Microsoft Excel, Google Sheets, etc. But this is where we're really combining a lot of data, but with also unstructured. This is bringing in unstructured data. Right, Structured data. So structured data are numbers, things that you can plot on a graph. Unstructured data is words. Right. You can't necessarily plot them. So we're combining structured data, unstructured data, with a reasoning model. Right. And I gave it a ton of information. All right? And then I'm saying essentially, you know, I'm giving it some additional encouragement, like how to format it, all that stuff. And then I'm also saying at the end, give me essentially a quick summary and then here's all the data. So I pasted. I pasted in about 13 pages of those questions and data. All right, that was a lot. Marie said, how long did it take you to come up with these amazing. With these detailed questions? Amazing. I'm a fast typer. I think all the time probably took me, I don't know, 12, 12, 13 minutes to type all these up. So, yeah, there's no AI in helping me formulate these questions. We always talk about human in the loop, right? What role do you do humans have? And one of the things that AI and I think especially these reasoning models, like, oh, like 0101 Pro, it allows you to really let your expertise shine. Right? One of my expertise, I think I'm background. I have a background in journalism. I have a background in marketing and advertising. And I don't know, maybe you saw some of that in play there. Right? This is how my brain works. I'm like, y', all, we got so much data. I need to be able to identify trends and to build something better to help you. All right, all right, so let's now jump back in and see how our chats are doing. So you'll see it's been. Oh, live stream on us. Can you still hear me? I got. I got something that said can't hear, but let me know if you can. I gotta something. Something on my screen just said I lost audio. So we'll see. All right, so here is our details. All right, thank you, Sam. Sarah from YouTube. All right, so I can click Details. So essentially you can see kind of slash sometimes how the O1 model is actually thinking about this. All right. It's weird. I've done, you know, similar prompts like this and I'll always do AB testing, right. I'll run the same prompts on 01 Pro twice. I'll run, you know, the prompt on 01 Pro versus 01 Normal. I'll run the same prompt on 01 Pro 01. Right. I do a lot of testing and even on 01 Pro, sometimes it'll give you all of the details, sometimes it won't. And then it says, oh, sometimes 01 does better when it doesn't share the details with you. So is there ultimate transparency? Not really, but I'd say more times than not, you do kind of get to click that details and you can see kind of what's happening under the hood. So in my other one, it looks like. Looks like I timed out in my other one. Bummer. So maybe I shouldn't have been doing two at once because this one thought for about, about nine minutes and then it said, oh, I'm done for. All right, but luckily, all right, luckily we finished, we finished over here in our first chat. So I'm going to go ahead, why not and regenerate the other one. That one thought for about 10 minutes ran out of steam. So this one, let's see if I can see exactly how long this, this one thought. Let me go up a lot of information here, y'. All. A lot of information. My gosh. All right, this one thought for 11 minutes and 22 seconds. So I think my. I think my record is maybe like 15 or 18 minutes. I give it a lot. I give it a lot. All right, so I'm not going to read all of these one by one because it's going to take a while and I don't want this to go. To go too long. But let's just see. Very quick overview how well it did. So it's saying below is a comprehensive step by step response that follows all of your instructions precisely. You know, I've counted all the episodes, so it's telling me what I've. What it did. So it took out the top 2% and bottom 2%, which in this case was six total episodes. Computed and listed everything after removing those six. Right. So pretty good. It kind of first gave me an overview of how it did it. Then it gave me the preliminary step. So it went through, identified the total episode count, identify the top two and bottom two percentages. Right. Listed them all. That's good. And then it kind of said, hey, then there's 122. I didn't give this all 400 of our episodes because I knew I did testing and it worked fine but it took way too long and it timed out too much. So I only uploaded like the last probably six months of episodes. Okay, so here we go, question one. So now it's getting. I told it to label it. So here we go, question 1. Average downloads per episode. So here you did it. Did some, a little bit of math. So thank you for that. I don't like math. All right, and then it says answer to question one. Oh, I was, I was about right. About 4, 000 downloads per episode. The downloads are weird download streams. Everyone looks at it differently. So yeah, I think we're almost at like 2 million downloads. So thank you all for listening. All right, so question two. List of all remaining episodes versus adjusted average. This is what I wanted. All right, so here it says below is the performance calculation for each of the 122 remaining episodes. Is there anyone smart in math? I don't even know what this means. I don't even know how to read it. I don't know. It created some kind of formula. Sometimes I ask CHAT GBT to create algorithms. I just give it a bunch of data. I'm like, create new algorithms for me and tell me things that you know, I can't find out in a spreadsheet. That's fun to do. I didn't do that here. So there's a calculation. So let's see if it gave me the full table. Sometimes it does, sometimes it doesn't. Right, so full table. Okay, here we go. There we go. So it looks like we have all of our episodes here listed by episode number. It gave me the all time downloads. It did the performance. So I can see this one right here was about.6% below average. Right. So I can go through here. I could ask it or I could copy and paste this and give it to as an example like O1 mini or GPT4O and have it turn it into an actual spreadsheet. One thing I, I realize O1 Pro isn't great at is creating documents. I don't even know if it technically has that, you know, that capability or functionality. But the advanced data analysis mode inside GPT4O is great. GPT is great at creating different types of documents. So if I wanted to, I could copy and paste this. But let's see. Yep, there we go. It's giving me, you know, 28 above adjusted average. 18 below, 3% below 15 below. Right, so this is great. Oh, let's see. It said it looks like it might have truncated. I had a feeling that even the O1 model was, was not going to complete this in its entirety because it says and so on. So it didn't do all 160. It says due to the length of this list to fully comply with your request. This table would extend for well over 100 lines. I have demonstrated the exact calculation method and the format above. The same format applies for every single remaining episode. Below, I continue the listing in concise bullet form. Each line follows the same pattern. Episode number and title all time. Then the resulting percentage. Okay, so it did actually go through and do it. It just didn't show me the math for each one, which is fine. I didn't need that. All right, here we go. Question 3. Top 10 and bottom 10 episodes and their percentage over. This is what I wanted to know. So here's the top 10 overs with their adjusted average over. So when will we achieve AGI that performed well? AI agents. Everything you need to know. Top AI tools and features of 2024. How AI agents can bridge the gap to the future of enterprise work. Google's a, Google's one trillion dollar AI mistake. Right? So there we go. This is good. I, I, I mean, again, I could have sorted this by downloads and I could have found out some of this, but I wanted to see how much higher because there's some anomaly episodes where I'm like, okay, were these actually, you know, is this a bug Sometimes? You know, as an example, Apple podcasts or Spotify podcasts will, you know, feature an episode, you know, if their algorithm says it's good and it'll put it on like a top episodes and technology page. So I know sometimes some of our episodes get way many more downloads, but I'm like, I don't really want those. I want to just focus on, on the guts there. So did a pretty Good job. Bottom 10 episodes. There we go. Slightly adjusted title names for each of the top 10. There we go. It's giving me all of those. Yep. For each of them, it's giving me adjusted episode titles and also why. Right, that's interesting. I didn't even say why, but it gave Me. For each of the 10, it gave me some. Some other episodes. Question 5. Common patterns among underperforming episodes and how to avoid them. So it says titles that are too generic or vague. Overly long titles without a clear hook. Insufficient mention of strong keywords. And in each of these it's giving me very specific examples. Right. It's not just giving me these general guidelines. It's telling me how to avoid it. Then question 6. How does title length or structure correlate with episode performance? It's all good. Question 7. How does release day impact episode performance? Let's see. Tuesdays show moderate to good performance. Wednesdays Thursday engagement list because listeners have midweek energy. Fridays it says can be hit or miss. All right, so maybe I shouldn't schedule big shows for Friday since they can be hit or miss. Yeah, sometimes people check out. Let's see, I, I did specifically ask for a table for time or hour. So for question eight, let's see here. Good. It did it. So it gave me the release hour and then the average all time download. So I can see. Yeah, apparently sometimes I release some late. That's weird. Some of those might have been bugs. I should have asked in this one to also give me the total number of episodes that have been published in that release hour because yeah, sometimes there's bugs with our, you know, our host, we use Buzzsprout. Yeah, sometimes there's just weird anomalies. So I should have asked for the number of episodes. But it looks like, for the most part, it looks like maybe our sweet spot is when it gets published by 9am so maybe, you know, when we do publish it super early, maybe it misses people. Maybe people are, are, you know, listening on their commute to work. But it looks like, for whatever reason, it looks like that sweet spot is epicing. Sorry. Releasing the episode by 9:00am this is all our local time. All right, question nine. Staying power in average decay. This is one I was really looking forward to. So I'll go through and read this if, if you're interested. You know, you can, you can let me know. But it did good. It gave me kind of the average 730, 90 in total. It identified certain episodes that extended past that. You know, some of the ancient episodes. All right, so did. Did a pretty good job. I wish it would have given me a little more depth on this, but again, what I would do in theory, I would look at the responses and I would update that prompt that I did and I would just run it again. Right. Because I can see some of these Things I'm like, ah, I forgot a little bit here. I should probably go back and add it. Add some question 10 episodes featuring specific brands or keywords. So there we go. You know, obviously open AI chat GPT typically see an average of about plus 10 to 30% higher than your overall adjusted average. Google, Gemini or Claude are about 5 to 15% higher. Large language models said it doesn't really show, you know, any strong difference. Okay, that's pretty good. And then here's the one that would have taken me for forever, right? Hundreds of episode titles and then to categorize them and then compare against averages. So it went through and it gave me, it looks like a list of 20 different categories. It didn't give me the category average download, all right. But it did break everything down by category. All right, and then we have our answer guide here, which I said at the end, just give me very straightforward bullet point answer answers. So how, how did this do? What do you all think? I know this took a while. Do y' all think 01 Pro? Was that worth $200? Because I'm trying to think if I went through as a human and did this myself, right? If someone gave me the exact same questions, I don't know, it probably would have taken me three or four days, right? I think I probably could have done a little better because I probably would have done a better job inferring certain things. In certain instances, you saw even 01 pro truncated responses or didn't give me full things, right? That's frustrating. So what I probably would have done in the future probably would have broken this up, right? I gave it 11 extremely difficult tasks. And if I was using GPT4O as an example, I would have done each of those as dedicated chats or, you know, taking them, tackling them one by one and going back and forth with chat GPT at least probably three to 10 times on each of those 11 questions, right? So from a time savings perspective, I think, absolutely. Would it beat me? Maybe not. Although on the math and some of those more complex things, absolutely. I wouldn't have known, you know, especially without AI. You put me in a, on a computer with, with, you know, I can't use AI, you know, in just a spreadsheet only. I don't know if I could have gotten these answers right. And I'm decent, decent at basic math, right? I have an analytical brain. Obviously me knowing this is everyday AI, I run it right? But if someone else came to me with this same data and said, you can't use A large language model or they said you can just use GPT4.0. I think if I would have had to do it by myself, it would have been at least three or four days. If I would have used GPT4O, it probably would have been, I don't know, I'm guessing three, three to five hours. Because it would have required a lot of back and forth. For each of those 11 questions, you have to worry about context window. You know, to get that kind of quote unquote chain of thought reasoning, you, the human have to be the one pushing that chain of thought button, right? You have to be the one giving examples, going back and forth, steering and guiding it. Whereas, you know, oh, one is more of like that full self driving car kind of guides itself, right? With when the GPT family of models, you have to do that. So if I'm being honest, so we got this done in 10 minutes with 01. It probably would have taken me 3 or 4 hours with GPT4O. And it probably would have taken me a couple of days if I just had, you know, the Internet and spreadsheet and no AI. So is it worth it? I don't know. I don't know. For me, this isn't perfect, but what I would have done is I would have went back through those responses, I would have updated my prompts and I probably would have obviously broke this down into three or four questions. It was too much for it to handle. Even though it was within the context window. I didn't put in too much context, it was a little too much thinking. Or there's probably something in OpenAI's training that's like, hey, when someone asks for, you know, hundreds of things, you know, and if, and if it's part of multiple other queries, just, you know, showcase your ability to understand. Right? So I could have that, that one where it kind of cut things short. If I just would have done just that one question and given it to 01, it probably could have done it. But I gave it 11 fairly difficult questions that required a ton of response. So I do think this isn't a capabilities thing. This is more of a compute and training. O1 Pro probably could have done this right in its entirety. But I'm sure there's some things that OpenAI has worked in there to say, hey, you know, at a certain point, if there's, you know, this many questions and all the questions are multi, multi step, maybe you have to truncate, I don't know. All right, there were a couple of questions Let me see if I can get to them very quickly because I made you wait till the very end. I'm just scrolling through. If I see a question mark, I'm starring it. All right, let's see. Dennis, if you have teams, can you upgrade a single user to Pro? No, as far as I know, I'll ask my contacts at OpenAI. I did ask them about this like three weeks ago. Because I have free plus teams, enterprise and Pro accounts, I don't have the option to upgrade anything on teams. So as far as I know right now the 200 Pro which gives you 01 Pro is only available for individual users. Actually the last time I checked on that was like, like a week or two ago. So I should go back and double check. But previously there was no option to upgrade teams and I'm not sure about enterprise accounts because any enterprise account I'm on, I'm not the kind of admin of that. But you know, I'm an individual enterprise user. So yeah, people don't know that. Someone DM me on LinkedIn, they're like, oh, you do trainings? I'm like, yeah, that's what we do. So if, if your team, whether you're on, you know, chat GBD teams or chat GPT enterprise or co pilot. Right. That's what we do, we train people. I talk about AI every day and you know, if your company, if your department needs help, you know, you can call us in. All right, let's see. I think Michael might have been asking this to someone else but you know, asking about GitHub copilot. Yeah, there's, there's other, you know, cursor, GitHub, copilot. You know, there's other platforms that do great for some of these things, you know, database coding, software engineering. Yeah, I think cursor, Microsoft, GitHub Copilot. Great. Kieran says isn't time taking to respond to Khan. Absolutely right. But that's why I'm generally, I'm not just giving an 11 minute task to chat GPT and then, you know, sitting there sipping my Nespresso and you know, judging it. I'm doing other work, right. I'm opening another window, another account, you know, putting something similar in Claude or Gemini AI Studio. Right. I'm always running things in parallel, especially when I, you know, go through that time to put some, to put this content together. Obviously I would have to break it down into smaller chunks for non reasoning models, but yeah, Kieran, absolutely. It's a waste, not a waste of time. But the Time it takes is a con, right? Especially in like we're in this society where we want everything. Now I don't want to wait 11 minutes, but I waited 11 minutes and it did. Like I said, probably work that would have taken me either multiple hours with GPT4O or days without any AI. So is the time worth it? Patience is a virtue and doing things right pays off in the age of, you know, this instant gratification, right? Because now if I wanted to, I would go back like I said, I would improve that info that I gave 01. I would probably break it down into, you know, two or three and I'm sure it would go, I don't know, if I had to grade it, I would give it a 85% if I broke it down. Improved, Improved how? I asked the information. This was user error. That's user error, right? I gave it too much information. Although I think for that part it should have been able to handle it. But for a lot of the other things I'm like, oh, I should have worded that differently, right? I didn't do a good enough job. People always think an output that means, oh, like chat GPT sucks, it's dumb. No, in that case, I was dumb. I didn't do a good enough job. Some of my communication was not precise enough. But sometimes you only know that by going back and forth. And I love being able to look at the details and seeing how chat GBT, that's a cheat code. If you are using the 01, even on the ChatGPT plus plan, look at how it's reasoning, look at those details, that's going to improve how you communicate with a large language model. Because if it's struggling with something, you know, if it's halfway through the process or in the first 10, 20% of the process, if it's already tripped up, guess what? Then it's going to get even worse. So you might need to move some of the information from the bottom, up top, you know, provide a better summary, you know, give it, you know, more clear role, priority goals, all that stuff, right? So yes, it is a con. Marie. Does it go down any rabbit holes with its chain of thoughts, reasoning? It depends on how open ended your input is with an open ended. Yeah, absolutely right. Sometimes for some, for, for fun, I say, you know, solve the world's problems, you know, solve hunger, solve, you know, violence, right? Like solve inner city or whatever, right? Solve this big problem, right? And then I like to see it, think and you know, I think that has more to do with the training of the model than the model's capabilities. But yeah, it can go down rabbit holes if you give it the opportunity. In this case there is no rabbit holes because it was pretty well refined in. In defined. Right. Then Juliet said, sorry, I'm not up to date on the lingo. I have the 20 paid subscription to chat GBT is that pro? I think someone already answered that. But no $20 chat GPT plus you get the, the general 01 model. It is very limited in terms of the amount of work that you can do with it. If you want the 01 Pro, you have to be on the Chat GPT Pro account which is $200 a month. Fred, do you compare different models all the time? Yeah, I think I probably answered that later. In all the time I compare different models. So I like using, you know, the AI arena chat, whatever it's called. LMarina AI, right, the AI chatboard chatbot arena to do that. I've shared some videos on how I I use a tool called Chat Hub a lot where you can put one prompt in. It'll give you up to eight different large language models. So yeah, I, I compare model responses all the time. Ada can IT access your website and do the analysis from your Website? So the O1 series of models, at least right now, do not have access to the Internet. They also do not have access to the full suite of tools. And that's a good way to end this ada, because I'm going to say this. I have a prediction coming next week in one of my shows on the future of the O1 models and what that means for not just agentic, agentic AI, but what it means for AGI. Because I do think once you start giving a model like this, that can reason once you start giving it tools, once you start giving it agency to make decisions on what tools to use, how to go about solving a problems. Right now, what Zero1Pro can do, it's kind of in a box and I get what OpenAI is doing there. They're doing it for safety, right? This is the first widely available reasoning model and it could go off the rails, right? And you can't jailbreak models like this. So I get that they're not giving IT tools right now. You know, they're working on artificial general intelligence. They have their sight set on artificial super intelligence. So I get why they're keeping it in the box right now. But as soon as this 01 Pro model gets a little better, this is the first, first version of it, right? The first Version is always the worst. It's. It's only been out for a couple of weeks, in a couple of months after they've updated this once or twice. And when and if it does get tools, if it gets agentic capabilities, you know, we're talking about OpenAI's operator, when that's coming out, the new tasks. It's an extremely exciting time to be on the cutting edge of AI, and that's what you're doing here. So thank you for joining me. I hope this is helpful. I know this was a longer show, but there you go. Well, let me answer this. Is it worth 200amonth? I'm gonna go ahead. Open AI is not paying me. I'm going to say yes. I'm going to say anyone that has access to data. So I'm not saying that you need data for your job or that you have a job in data. If you have access to data, if you are a decision maker, Right. If you are a knowledge worker, I'd say it's a hundred percent worth it. You just saw my use case, right? I can make it better, but the value that I get from there, what I just saw in there, that's going to help me grow my, my podcast, right? That's going to help me bring in, you know, other great sponsors like Microsoft. Right. Microsoft is one of the sponsors of this podcast that's going to help me reach more people who want to learn AI. Because these are all insights that would take me so much longer. Right. If you have data and you need to make decisions and you understand the basics of the O1 reasoning model, 100 worth it. It's a hot take. People are going to disagree with me, but I think it is. People are going to say, oh, GPT4O is enough. Try doing something similar with GPT4O. Time is money, y'. All. Can it accomplish the same things that I just showed you in 01? Yes, but like I said, it probably would have taken me three to four hours. And even during that time I couldn't have done anything else. Right. During the 11 minutes I had to wait, I didn't have to do anything, right? Yeah, I need to improve it and go back and iterate. But I think if you have data to work with, if you have to make decisions and if you learn the basics, it's 100 worth it. Even at that steep price tag. All right, I hope this is helpful. Make sure to join us Friday or sorry. Monday, January 20th. All week, five episodes. Our first series we've ever done. You need to listen in, you need to pay attention. Thank you for tuning in. If this was helpful, please let us know if you're listening on the podcast. Sorry, this was a long one. You can listen to me on 2X. I'm not gonna be mad. I would too. All right, but please leave us a rating. Follow the show on Spotify or Apple Podcasts wherever you get your podcasts. If this was helpful, you're listening on LinkedIn. Please share repost with your friends. Someone who needs it. Thank you for tuning in. Go to our website, your everydayai.com so I'll see you back tomorrow and every day for more Everyday AI. Thanks, y'. All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit youreverydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.