Claude 3.5 Sonnet: Is OpenAI Falling Behind?

I need this for my email as soon as possible.

Yeah, I'm like imagining this workflow where, like, you're listening to your emails with some extra commentary.

There's something new being born there that never existed before.

Being a know it all used to be considered a bad thing, but in business, it's everything. Because right now most businesses only use 20% of their data. Unless you have HubSpot, where data that's buried in emails, call logs and meeting notes become insights that help you grow your business. Because when you know more, you grow more. Visit HubSpot.com to learn more.

Hey, welcome to the Next Wave podcast. I'm Matt Wolf. I'm here with Nathan Lands. And today we're going to discuss a lot of the latest stuff to come out of the AI world. Things from Anthropic and OpenAI and HubSpot and all of these big players that are building stuff in the AI space right now. But we're not just going to talk about the news, we're going to talk about what the heck do we actually do with this stuff. You know, there's a lot of discussion going on in the AI world around. That's a really, really cool thing. But how would I actually use that in my life? Well, we're going to talk about that, and some really cool stuff has come out from Anthropic and OpenAI lately, and we're going to give you our thoughts on where this might be going and how you can actually integrate this and implement it in your business.

So this week, I think the most exciting thing that I saw that's actually useful is the new Claude 3.5 sonnet. You know, I've been a big believer in OpenAI, and even though people were saying, hey, Claude's better at writing, I'm like, ah. But still, you know, with OpenAI and ChatGPT, I have my custom instructions and it knows more about me, it has more context about me. Well, now with the new cloth, not only is it better than ChatGPT, but also they've added this new thing called Projects. And so projects is basically like custom instructions, but better. For example, like with my newsletter, I created a project called the Lore newsletter and then I gave it all these notes. So I said, like, here's kind of my writing style. Here are like two of my favorite newsletters and the kind of writing style that I've learned from. Here are my previous issues of the newsletter and also the kind of general tier template I use to lay out my newsletter every week. And that has made The Claude such a better editor than before. Like, before it had no context. Like, who's Nathan? Or. Or like, what is he like? Or what's the newsletter? And now you don't have to do any of that. Like, it just knows all of it. And so it seems like, you know, yeah, for a newsletter, it's a great use case, but it feels like this is going to be where companies can really start to find value with AI. Because my understanding too is you can actually share the projects, right? So you can make this for any kind of project and feed it all the context that's relevant for that project. And now everybody who's involved in that project, when they go into the AI, it's already got everything set up so it knows, like, what you're trying to accomplish.

So honestly, I haven't played with the project's feature yet. You know, at the time of this recording, I've actually been traveling and I just got home and I haven't played with that yet. But they also rolled out a feature called Artifacts, where it puts this, like, sidebar inside of Claude. So on the left of your screen, you type your prompts and it looks just like Claude's always looked, but you. But then on the right, whatever the sort of output you're looking for will show up on the right. So one of the very first things that I tested with that was I wanted to see if I can have it write a game for me and actually test the game straight in Claude without ever leaving the website. And it worked. I asked it, I made a very, very simple game. I said, hey, make me a tic tac toe game and that I can play against the computer. And with a single prompt, it generated that game. And. And then it worked. First try. I mean, the computer actually sucked at tic tac toe. Like, the logic of where it should place the thing wasn't there yet. But I would put my X's down and it would put an O down and I'd put an X down and it put O down and I beat it every single time. Because the AI that it generated wasn't very smart for the game, but it was all playable right inside of Claude. I didn't even need to copy the code, paste it into a text document, open it as like a HTML file. Didn't need to do any of that. It just loaded up in this new Artifact sidebar and I was able to test it out. And I just thought that was like, so cool. You can test the code without ever leaving the website. And then if something doesn't work with it, you just go to the left side of your screen and say, hey, this didn't work. Fix it and it will rework the code and show it to you again in the right sidebar. And that was really, really cool. But I haven't actually played with the projects yet. I just know that, you know, I have some custom GPTs that I made over in, over in ChatGPT, where I mostly use them for like, duplicatable processes that I find myself doing over and over again. Kind of like you mentioned for your newsletter. And one of the ways that I was using the custom GPT was I was taking all of the links from one of my YouTube videos, right? I usually have like 20 tabs open and I'm like, here's this thing and then here's this thing, and then here's this thing, and then I take all of the URLs from those tabs. I would plug them over into ChatGPT and a custom GPT that I made and I said, you know, create a resource list for me and it would give every single URL a sort of title, a URL, title, URL. And it used to work really, really good. When GPT4O came out, it actually broke that custom GPT. It stopped working for me. So I'm like. When you mentioned this just now that Claude has this projects and it's kind of similar to like a system prompt or almost like a custom GPT. I'm like, well, that sort of eliminated one of the use cases I used to go to chat GPT for.

Yeah, for sure. I mean, this is like the first time I feel like I can actually switch over. But you're talking about the artifacts thing. So another thing with projects is apparently you can share the artifacts. I'm imagining like two teenagers who are like coding on a project together. Like, how cool that's going to be now, where they literally can go into the AI and say, hey, we want to make this little game together, or this little website or whatever, and you can talk to the AI and you can actually see the output. You can see the output of the code and, and go back and forth on that together. There's something new being born there that never existed before. Like in terms of collaboration on projects.

I wonder if you can make like a multiplayer game, right? Like, I make a game, like just for the most basic possible example, the tic tac toe game I made, I log in and I'm playing the game against you, you know, over in Japan.

Right.

Like that would be kind of crazy. I don't know if it's there yet. Right. You're probably just seeing like the front facing code. You're probably not seeing my input. But would be cool if we could figure out how to like make multiplayer games in there where you log in, you see like the move that I made and then you make your move and then I see the move you made and we can go back and forth and almost like create a multiplayer game inside of the artifacts, but with the sort of team group feature that's there.

Yeah, I mean, I think right now it's mainly JavaScript they're outputting. I could be wrong, but I think a lot of the stuff I've seen at least is JavaScript. So you'd probably want to have some kind of backend. But like, you know, you know, whether it's Microsoft or one of these guys or you know, Anthropic, I think they're funded partially by Amazon, so in theory they could just partner with aws. Right. And you could have something on the back end where it rolls up a server, usually with games. You know, netcode is one of like the kind of challenging things engineering wise, but my understanding is there's a few best practices for that now. So I don't see why AI couldn't learn how netcode works and then just roll up a server for a game and then yeah, now you do have a multiplayer game. So I don't think that's currently possible. But that feels like something that's probably six months away, maybe 12 months. Yeah.

So yeah, Anthropic's backed by Google too. So they have AWS and Google Cloud, you know, potentially at their disposal.

Right.

So yeah, I mean, I think that'll be really interesting. I need to play with that projects feature a little more. As far as the project features go, I mean, what are, what are some of the like super exciting use cases that we can think of for using that project's feature? Because you know, again, like we mentioned at the beginning of the show, like this is really, really cool. All right. Now, how do I use this in my business? How do I use this in my daily life for productivity? How do I actually like integrate this new technology? I'm curious your thoughts there.

One on the, on a personal side, you know, I recently saw a tweet from Greg Eisenberg talking about micro weddings and told me how that's like a new thing where people don't want to spend hundreds of thousands of dollars on a wedding. They Want to have like 10 to 20 friends and, and. But he talks about how that's a huge trend and it's so hard to like manage putting together a wedding. And obviously I just recently got married. We, we had a relatively small wedding, very expensive though. And in the meetings with the staff were so long, I was sitting there thinking, God, now that there's this client, you know, with the projects, I'm like, can made a project for the wedding. Told it like, here's the kind of wedding we're wanting to have. Help us organize everything and here's like some reference images and stuff. And yeah, sure, he couldn't have done everything, but I think it could have helped. Basically been like the wedding organizer for sure.

I wonder how close it is to like, you know, Anthropic's got like visual capabilities as well, right? You can upload a photo and then it can see what's in the photo and you can ask questions about that photo. I wonder, can Claude. And this is something, I don't know, maybe you do, but if you don't, that's okay. Can Claude actually like see what it built over in that right side, the artifact sidebar, Right. Because if it can, how close is Claude to being like what we saw from Devin, Right? Write me this game. And then it writes the game. The game pops up in the right sidebar. It tests the game itself to see if the game is functioning the way that specific game should. And if it's not, send that the image or the response or the error report or whatever back to the code on the left sidebar, fix it, iterate it, and then present it again on the right sidebar. Like, it feels to me like this is one step closer to that sort of agentic thing that everybody was looking for, especially for code. I mean, right now with Anthropic, the thing that everybody is sort of blown away by is how good it is at writing code. It's gotten like so much better than what GPT4 can do at writing code in a single prompt usually. So I'm just wondering that to me feels like where they're going with all of this is like that sort of agentic. It can see what the output was, determine if the output was the output we were looking for, if it is cool, if not, throw it back into the chat and try again, you know?

Yeah, I mean they definitely should go there. My understanding is currently it doesn't do that. I could be wrong. But you know, I think right now, you know, artifacts is basically like a UX Experiment or UI experiment for them, basically. You know, it's actually one of the first things they've invented, as far as I know, that Chat GPT didn't have first, because every other feature they've created was basically a copy of Chat GPT's features so far. Right. And artifacts is the first thing I was like, oh, that's new. That's. That's a. That's a really cool thing. You just see the output. Yeah, why not?

Yeah. Especially with JavaScript, HTML, CSS, there's. There should be no barrier to being able to just present that on the same screen.

Yeah. But I think that is where it's going, like, whether it's Claude doing it or someone else. And I think, you know, ChatGPT is coming. They're not gonna. They're not just gonna lose this fight. They're gonna roll out some amazing stuff too. But, yeah, that's. That's something I noticed a lot of engineers are talking about online is like, people are saying, like, in terms of, like, refactoring was a big thing. I heard they're like, oh, my God, like, you can go in, like, all your code that's messy. You can just copy and paste in all of it, and it just, like, makes it all, like, tidy and removes all the unnecessary stuff or anything that's redundant or inefficient. And it just. And it gives you code that's, like, way better, runs better, cleaner, easier to read, and nothing breaks. It's like before, apparently. Like, when you would do that. Yeah, it would, like, make it cleaner, but sometimes that would break things. And I'm hearing a lot less people saying, oh, it breaks things now. It seems like it's way better. Like, it understands what it's doing. And, yeah, it can clean up your code and it doesn't break anything. That's huge. Like, when you're coding, like, you can. You can. That's one of the hardest things about engineering is. Is, you know, the projects I've worked on is, like, when you go. When you go into something and especially if there's other people involved, to go and look at somebody else's code and understand what's going on.

Right.

It's so hard because, like, they've been sitting there for, like, you know, you know, sometimes even, like 24 hours coding on something, and. And it just, you know, it makes sense in their head and. But then when somebody else looks at it, it's like, how is all this stuff interconnected? And what the hell is this? And what's that, and, and so, yeah, I think that's a big thing that is going to actually be really useful for companies now is that like, yeah, your code will be cleaner, run better.

I think one of the things that I'm sort of feeling lately is that OpenAI is starting to fall behind a little bit. And again, we've talked about this plenty of times on the podcast. We have no idea what OpenAI is doing behind the scenes. Right. They're fairly secretive. Most of the stuff that they've announced and shown off, people didn't see that coming because they don't like just show it off until it's ready to show off. But lately it's starting to feel like companies like Anthropic are passing them in a lot of areas, you know, and other examples outside of the large language model is the video stuff. We actually kind of talked about this on a, on a different episode. But you've got Sora, which they announced seemingly to overshadow, overshadow Google. Right? Like when SORA was announced, I think it was the same week that Google announced Gemini 1.5 or whatever. It was a really big leap for Google with like a million token context window. Well, that week OpenAI went and showed off Sora and nobody was talking about Gemini 1.5. Once they saw Sora, Sora was all anybody in the AI world was talking about or looking at or caring about. Right. Well, here we are several months later. We still don't have SORA access, but now We've got Luma AI video available. We've got Runway gen 3, which is about to be available, most likely before Sora. We don't have it yet, but most likely before Sora. And then you've got the demo that they did for the sort of Advanced Audio Assistant, I think is what they call it, right? They did that demo once again. I feel like they timed that demo to sort of slap Google. Right? Like Google had their Google AI IO event the same week, like Google I O started on Tuesday. Well, what did OpenAI do Monday? We're announcing an event where we're gonna make a big announcement, right? They made this big amount announcement on Monday, which kind of took some of the flair away from the Google IO event. Here we are a month later, we still don't have it. And the. They just made an announcement that, oh, it's getting delayed, we're pushing it out, it's not going to be rolled out. I mean, some people seem to be getting it, but for the most part they're telling us we're not going to get it till fall now. And it's like they're making all of these big announcements with the intention of sort of overshadowing Google, but then they're not actually shipping anymore. And I don't think it's a good look for OpenAI, in my opinion. And it also kind of feels like it's giving a lot of these other companies an opportunity to catch up and pass them. Like, Claude 3.5 is beating OpenAI and all of the LLM benchmarks right now. Like, companies are starting to pass them up.

I agree that it's not a good look, but I'm still convinced that behind the scenes, they're pretty far ahead. And, like, and there's starting to be some evidence of that. It's like the, you know, the stuff with Sora, even though. Yeah, they did a demo and it's not out yet. But the video that came out, I think it was yesterday. The Toys R Us commercial, have you seen that yet?

I haven't showed that.

I saw the news that's generated with Sora.

Yeah, I heard that.

So, so, so Sora, there are deals going on behind the scenes where the technology is being used.

Right.

I think with Sora, it's probably just so expensive. They're like, yeah, we can't just roll this out.

Yeah.

Everyone is like, like to pay, like, hundreds of dollars to use it. Like, people are, you know, they're not going to do that.

The funny thing is I didn't even know that Toys R Us was, like, back.

It's like, yeah, and they're back with Sora.

Yeah. Like, this was their introduction to go like, hey, look, we're here again.

Yeah. Yeah. And so, you know, I, I thought that commercial was pretty cool. Like, it, you know, is it perfect? No, but, like, that's the worst it's ever going to get. And you can see that they're able to accomplish things in that commercial that probably would have, you know, traditionally cost a lot, lot more, probably to the point where they never would have made that kind of commercial. It's just like you're almost making like a short film at that point. I think that behind the scenes, that they're still doing quite well. I agree. They've kind of slipped up in terms of how they presented things. I don't, I don't think it's good that, like, I highly prefer how Claude's doing it where, like, they talk about something and then it's just out and, hey, try it. Like, you know, like the old Steve Jobs style. Like, I think that's way better, personally. But I still think, you know, you know, GPT5 is coming and it's. And it's probably way better than Claude. So right now, Claude's better for the moment, but, yeah, I guess we'll see.

Yeah, we'll see. We'll see. The Toys R Us video is interesting to me because I'm wondering what sort of reception it's going to get, like, if it starts. I don't know if that's airing on TV already right now or if it's, you know, just kind of circulating on the Internet. But almost every scenario where AI was used in this kind of thing, there was massive negative backlash. Right?

Like, oh, there's. There. There is line right now. There is, yeah. For this. It's like, really? Yeah, it's really. Yeah. People are quite hateful. Like, you know, screw. You know, Toys R Us. Screw, Stay Dead and all this kind of stuff. It's like. Like horrible stuff. I don't get it.

I don't quite understand it. You know, I guess the argument is, like, well, why didn't you just hire real people, real visual effects artists to go make that? You know, you would have employed more people if you did. But, I mean, they're a company that went bankrupt, you know, so, like, they're probably looking to cut some costs in some places.

Yeah, yeah.

But, yeah, I just remember when the. There was that Disney plus show, the Secret. What was it called? Secret Agent, Secret Invasion, and parts of the intro were done with AI and, like, people were boycotting watching that show because the intro was made with AI Right? And, like, there was, like, a magazine post where there was, like, a clock. I don't remember exactly what the ad was for, but there was, like, an ad where, like, the clock in the background, people are like, wait a second, the numbers on the clock look weird. That image is A.I. you know, boycott this company. They use A.I. for their images. I don't understand that. Like. Like, you're throwing the baby out with the bathwater. You're like, okay, because they use this one ad. The product that they're offering is. No. Like, I don't know. It's a weird take to me that people are, like, that passionate about. Like, this company is saving costs by using AI Therefore, screw that company. But that seems to be the sentiment whenever this kind of stuff comes out. I don't know if that's, like, the vocal minority or if that's how most people feel. I kind of get the impression there's, like, this vocal minority of AI haters, most of the world is either like, AI is cool, or I really don't care one way or the other. But then there's this vocal minority of people that tend to speak the loudest on the Internet.

Yeah, I think, I think it's the vocal minority. But it will be interesting to see long term. Like, it's definitely going to be like one of the, you know, the biggest transitions in human history, at least in our lives, right, where things are going to radically change. And a lot of it's going to be good. And there's going to be a lot of. There's going to be some bad and complicated stuff too, like we talked about before, right? It's, you know, I think we should accelerate. But also at the same time, it's like, yeah, but also, like, care about people because it's like some people are going to lose their jobs, you know, so we'll see how that works long term. I think there will be some backlash, but at some point, you know, like, you get these, like the new show from Disney, I already forgot the name of it, the new Star wars show. But it's horrible. I, like, I saw the commercial and they're like chanting the power of one and all. It looks like me and my wife have actually watched them all.

My wife and I have actually watched them all. And I know we're starting off on a tangent here. No, I think the show's horrible, but it's almost like I'm watching it now to watch this train wreck in slow motion.

There was an interview with the musician, Will. I am the guy who's being the black eyed peasant. And he said that, you know, the AI video stuff he's seen, you know, he feels like it is going to take power away from Hollywood and it's going to give it back to individual creators for them to create new things. And he thinks that's going to be like a more beautiful world where, like, people can bring it on, create the kind of stuff they want to see, you know. And so I am hoping, like, technology gets way better and then Hollywood's like, has to like, produce better films with humans in it and stuff that humans actually want to see. And then there'll be a whole other category, like you said, probably a niche of where people can use AI to make stuff that's personalized for them, what they want to interact with.

So, yeah, it's interesting. I definitely see the arguments, right? Like, I see the arguments of like, okay, this is taking away the need to hire Visual effects artists and things like that. But I also think that the visual effects artists that learn to, like, leverage a lot of these AI tools are going to be much better at their craft, you know, using all the tools that are at their disposal. Right. Why. Why let yourself get let be left behind because you're going to boycott a certain technology? Well, your competitors probably aren't going to boycott that same technology. Like, it's ridiculous. But I do understand it's. It's change. People are scared of change. But I think those that leverage it as another tool in their toolbox will only get better at their craft.

Yeah. One of the other interesting things that came out this week was 11 Labs, the AI voice company. They rolled out this. This reader app. So basically you can. You can feed it any document and then you can use an AI voice to read it to you. So, like, if there was a book that didn't have an audio version of the book, you could have it read it to you. Or if there's a present, I guess in theory a presentation or any kind of PDF or anything like that, you could feed it to it. Feed it to it and have it read it to you. So, like, it feels like, like, that's gonna be really powerful for people. Like, it feels like it moves us closer and closer to, like, not having to be on your computer all the time, right? Where you'll be able to go off and live your life and do work at the same time. Like, you can be going for a walk and staying healthy while hearing the presentation that your boss just sent to you or whatever. Like, yeah, I don't have time to, like, like, look at that this moment. But, like, hey, let's just send that to 11 labs and have it read it to me while I'm, like, going off and enjoying my life for a little bit.

Yeah, yeah, no, that. That to me, will be really interesting because it kind of makes it so, like, everything can be an audiobook, right? Like, articles you read online, they can be an audiobook. Like you mentioned. Like the memos from a meeting that you don't really feel like reading. Turn it into an audiobook. Go for a walk and listen. Listen to the summary of the meeting, right? Like, anything can be an audiobook or a podcast that you can listen to and multitask. You know, I need to run to the store real quick. I'll listen to the audio of this meeting that was generated while I'm there. Or I'll listen to this blog post that I came across on the way to the Store.

Real quick, I told Amara, who's working over there, I was like, I need this for my email as soon as possible.

Yeah.

And then when it gets to the point where then you're able to actually, like, talk back to it and tell it how to, like, craft response back, I mean, we're probably like six months away from all this, which is awesome.

Do they have an API for that feature? I think 11 Labs has a pretty robust API, so if they don't, I bet they will soon.

Yeah.

And that sort of gives me ideas of, like, you know, using something like zapier or make.com that you can start tying these tools together. And every time an email comes in, automatically send it over to the 11 Labs API, make an audio version of it, and then queued up in my podcast player. Right. And then when you go anywhere, you just open up your podcast player and listen to all the emails that have come in in the last couple hours, you know? Well, and what you could do, too. And I'm just like, I'm sort of totally nerding out right now, but, like, I like to make workflows with zapier and make.com and I'm like, imagining this workflow where, like, an email comes in, it automatically sends it to the Anthropic API, has Anthropic add its own commentary to the email for you, and then sends that over to 11 labs, makes a podcast for you, and now it's like you're listening to your emails with some extra commentary. Like, you can tell Anthropic, make me some jokes about this email. Make me some jokes about this contract. So as you're listening to the contract, it adds in a little bit of, like, entertainment factors. You're more, like, interested in and reading it. You know, like, every once in a while, laugh at something that's said in this contract. You could be listening to it and they're like, laughing at the contract as they're reading it to you. Like, that kind of stuff I could actually see myself doing just to, like, help me get through more of the content I'm trying to get through.

Yeah, I mean, so a lot of stuff we're talking about with the email, though, you know, a lot of that's, you know, maybe some of it can be doable with Zapier and other things. Right now I'm not sure what to go play with it and see, but if not, it's going to be available soon. But one thing you can actually use, like, right now, which I've been finding really Useful is, you know, and this is, you know, I'm not saying it because Dharmesh is, you know, the co founder of HubSpot, but, but Dharmesh, the co founder of HubSpot, the CTO launched this thing called Agent AI. And my understanding is he's building different agents for different business use cases. And I'm not sure that's going to get rolled up into HubSpot products or what. But what he has right now that you can use without even signing up is this thing where you can forward an email to agentagent AI and then it basically connects it to an LLM and you can ask it things like you can say, you can ask it a question about the email or you can ask it. The thing I've been finding very useful is summarize this email. Yeah, right. And I don't have to go anywhere else or anything. I literally just, you know, forward it to agent at Agent AI and say, hey, this email that's like six paragraphs, paragraphs, Summarize it into like three bullet points.

And you're just writing that in the message that you send. So you're like forwarding it and then in the message that above the forwarded message asking a question or whatever, that.

That'S all, that's all you do. And you get a response back in about 30 seconds. Was like 20 to 30 seconds every time I tried it. And it's great. It's like, you know, it's. I'm not sure what, you know, they're using behind the scenes, maybe they're using Claude or, you know, whatever. But responses are good. Like the summaries are pretty similar to what you'd get from putting it in the cloud. Of course you could go copy and paste and do all that, but you know, for a lot of people who don't know how to do all that, or you know, who are not as fast at doing that as I might be, like, it's, it's super useful and you can use it. Like right now you just like forward the email. So anyone listening, I said if you haven't tried it, you've got emails that are like super long. You know, I suggest like, just try it out. Like forward, forward an email to Agent at Agent AI and just say, hey, summarize this email. It's, it's pretty magical.

Yeah, yeah. And so, yeah, there's a lot there to think about. You know, we, we talked about a lot of the new tools that are coming out. We talked a lot about the use cases that you might be able to use them for. Even talked a little bit about the future implications of where some of these tools might go. I think this has been a really fun, fascinating discussion, and if you enjoyed this discussion, make sure you like this. If you're watching it on YouTube and subscribe wherever you're listening, slash watching this. If you're watching it on YouTube, subscribe to us there. If you're listening on a podcast platform, subscribe to us there as well. We really appreciate it. It helps get this show more reach in front of more people. We can't thank you enough for tuning in and nerding out around AI with us. We really appreciate you. We'll see you in the next one.