Early Uses for Anthropic's Claude 3.5 and Artifacts

Today on the AI Daily Brief, Anthropic releases its latest model. Before that in the headlines, is AI becoming a presidential issue? The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show. Notes. Welcome back to the AI Daily Brief Headlines edition. All the AI Daily news you need in around five minutes. We kick off today with some comments from former President Donald Trump, who is of course also running for president right now. He recently appeared on the all in podcast, part of a courting of the technology industry that he's doing in his campaign right now, and spent a bit of time talking about artificial intelligence as it relates to energy. Let's quickly listen to this one minute clip.

We have a phenomena coming up right now and I was talking about it the other day to David and that's a little things, simple, two little simple letters, but it's big. And I realized the other day more than anything when we were at David's house and talking to a lot of geniuses from Silicon Valley and other places, they need electricity at levels that nobody's ever experienced before. To have to be successful, to be a leader in AI, the amount of electricity that needs like double what we have right now and even triple what we have right now. They are, they are. It's incredible how much they need to be the leader and we're going to have to be able to do that. And a windmill turning with its blade knocking out the birds and everything else is not going to be able to make us competitive. You'll have China.

What about, what about nuclear, Mr. President? Yeah, so let me just give you a statistic on this building. So what's interesting to me about this is not only that like them or loathe them, when Trump speaks, it does set the tone of the conversation to come, but more that we're not just seeing a superficial surface level conversation around AI competitiveness and AI as it relates to a global geostrategic battle with China, but this specific awareness of the relationship between AI and energy. This is something that Sam Altman has talked about a lot before as well when he's been asked about whether he's concerned about the environmental impact of artificial intelligence. He's basically said that we're going to have to innovate new energy solutions for the sake of AI because we simply can't get enough with what we have. I'll be watching closely to see if AI starts appearing in this type of political discourse more. But for now, speaking of the government and AI. Also, in a recent interview, OpenAI's Mira Muradi said that they give the government early access to new AI models.

How do you minimize risk? Right, and providing people the tools to do that? And in the case of government, for example, it's very important to bring them along and give them early access to things, educate them.

As you'll hear more in the main episode today, as part of its announcement of its new model, Claude 3.5 Sonnet Anthropic shared more details about how they gave access to the UK AI Safety Institute. So again, maybe a bit of a shifting of the tide in terms of how these frontier labs think about their relationship with the government, even in advance of any sort of comprehensive legislation. Next up today we've got a set of different funding announcements. Heygen, the video avatar and visual storytelling company, has announced a $60 million Series A. They write that in just over a year they've grown from 1 million in ARR to over 35 million and have been profitable since quarter two of last year. What's more, they say more than 40,000 companies are currently using the tools. These numbers are pretty impressive to me, especially from the standpoint that we're only just beginning to scratch the surface on avatar use cases in the workplace. I happen to think that this is going to be an incredibly rich area of experimentation where we're going to see everything from company onboarding to change sales processes. And 60 million more in the war chest will definitely give hey Gen the ability to compete at the front of that pack. Another French company has also raised a big round. Poolside is, according to TechCrunch raising $400 million at a $2 billion valuation. This is not yet an announced deal. TechCrunch writes that Bain Capital Ventures and DST are in talks to co lead the round with BCV being a previous investor and DST being a new investor. You might remember this company from when they raised a $126 million seed round last year. Poolside is focused on using AI to accelerate and augment human software developers. TechCrunch points out that not only is France pumping out some very high profile AI companies, not one but two others have also raised nine figure seed rounds. Mistral raised 113 million and H raised 220 million. TechCrunch writes the city of Light might need to be renamed the City of AI at this rate. AI language Tutor Speak announced that it had reached a half billion dollar valuation, raising $20 million in a Series B3 financing. The company writes Speak's AI language tutor builds a path towards spoken fluency without reliance on human conversation. Users gain fluency by learning speaking patterns and practicing repetition in crafted lessons rather than memorizing vocabulary and grammar. An impressive statistic from this press release. Learners speak a thousand times on average in their first week week finally today, on the other end of the financing spectrum, the Information reports that chip company Cerebras has quietly filed for an ipo. The information writes the IPO plans show the eight year old company wants to ride a wave of investor enthusiasm over AI hardware sales that have made Nvidia the world's most valuable company and boosted scores of other stocks. The startup's financial results couldn't be learned. It said in a blog post in December that it had recently reached cash flow breakeven without elaborating. Still, the information writes that a new share authorization suggests Cerebrse is valuing itself at around $2.5 billion. So friends, that is the news from here and that's going to do it for today's headlines. Next up, the main episode. A quick note before we get back to the show. Today's episode is brought to you by Super Intelligent super is of course the platform that we built and released a couple months ago to help people learn how to use AI. It's built around fun, fast tutorials that get you actually using AI in minutes, not hours and certainly not days. The learning all happens in the context of an engaged community and we've got a bunch of exciting features rolling out in the weeks to come. One of those is a new teams version of the platform which includes a custom curated playlist as well as a showcase where people on your team can share their use cases and projects in AI across the organization. If you are interested in being a part of the super for Teams beta, go to BeSuper AI partner so you can learn about the program. Welcome back to the AI Daily Brief. Today we have one of my favorite types of stories which is when a very cool new tool gets released and people get all excited to go try it out. Of course, this tool coming from one of the biggest labs in the space Anthropic and it beating out GPT4O on a number of different metrics means even more excitement. First let's talk about what was actually released. It is called Claude 3.5 Sonnet. Anthropic writes, we're launching Claude 3.5 Sonnet, our first release in the forthcoming Claude 3.5 model family. So basically this is an update to their mid tier model Sonnet, which was previously behind their top tier model opus. Claude 3.5 sonnet is more intelligent based on benchmark scores, but also still costs less in terms of price per million tokens. The 200k token context window has remained the same, but the benchmarks appear very impressive, a highlight for example, that they call out in an internal agentic coding evaluation. Claude 3.5 Sonnet solves 64% of problems outperforming Claude 3 Opus, which solved 38%. On graduate level reasoning, 3.5 Sonnet scored a 59.4% compared to GPT4O's 53.6% on the MMLU. Its five shot score of 88.7% was the same as GPT4O's zero shot score. The Claude zero shot score was just a little bit behind at 88.3% and so on and so forth. You get the idea. Basically on most of these tests, with the exception of one particular math benchmark, Claude 3.5 Sonnet was outperforming not only Claude 3 Opus but GPT 4.0 as well. Anthropic also calls out Claude for having state of the art vision, they write. Claude 3.5 Sonnet is our strongest vision model yet, surpassing Claude 3 Opus on standard vision benchmarks. These step change improvements are most notable for tasks that require visual reasoning like interpreting charts and graphs. Claude 3.5 Sonnet can also accurately transcribe text from imperfect images, a core capability for retail, logistics and financial services. Another interesting note on the safety side of things is they say that they provided Claude 3.5 Sonnet to the UK's Artificial Intelligence Safety Institute for pre deployment safety evaluation. So basically the model testing wasn't just internal. Okay, so basically part one of this is that we have a model upgrade, the Verge writes. Claude 3.5 Sonnet is apparently anthropic's smartest, fastest and most personable model yet. The Verge sums up for now, the model is the big news and the pace of improvement here is wild to watch. Anthropic launched Claude 3 Opus in March, proudly saying it was as good as GPT4 and Gemini 1.0 before OpenAI and Google released better versions of their models. Now Anthropic has made its next move and it surely won't be long before its competition does so too. Claude doesn't get talked about as much as Gemini or ChatGPT, but it's very much in the race. Another reason for it to remain in the race is that this model update also comes with a pretty meaningful Interface Update Mike Krieger, the company's chief product Officer, who was previously the co founder of Instagram, wrote, We're also launching a preview of artifacts on Claude AI. You can ask Claude to generate docs, code, mermaid diagrams, vector graphics, or even simple games. Artifacts appear next to your chat, letting you see, iterate and build on your creations in real time. I've used it to work on writing, coding and diagram projects. Yesterday for Superintelligent, I did a tutorial about four early use cases for Claude and Artifacts. The first was writing an SOP document for a remote tech company. If you're watching this, you can see that what an artifact represents is basically a right side of the chatbot that's all about previewing the output of your instructions which are going on on the left side. This is in many ways primarily a user experience update. By separating the section where you are providing instructions and interacting with the chatbot from the output is arguably just a better way to set up the interface. For example, after I got that SOP document, I asked it to add a tongue in cheek section about taking advantage of the fun parts of work from home, including dress code and family time, which it did. As a second document preview that I could see alongside the first version, a couple other use cases I shared. I did a screenshot of growth statistics from our super intelligent YouTube channel that I got from Social Blade and asked Claude to turn it into a line chart, which it dutifully did. I had it write the copy and the code for an AI consultancy to help small businesses. One interesting note here is after I got the first version, I asked it to update the name to Mighty AI Consulting and explained that I wanted the name to reference the idea of these businesses as small but mighty, which it then took as a prompt to update the tagline, not just the name as well. If you've spent any time on Twitter or X since Claude 3.5 came out, you might have seen people coding up complete games. Ali Miller, for example, provided a screenshot of the instructions for the classic game man kala and asked Claude 3.5 to read the instructions, code the game, and then preview it so she could test and play it. Was able to successfully do this in a matter of seconds. There are literally dozens of examples as well of this type of game popping up. As an example of what Claude 3.5 Sonic can do outside of the capacities, people took note of the speed. Perplexity, writes Claude 3.5 Sonnet is now available on Perplexity with 2x faster speed than Opus. Claude 3.5 Sonnet unlocks new possibilities for complex AI applications across Reasoning knowledge and coding tasks. Dan Schipper, who just put out a new product called Spiral, writes, the really fun thing about building Spiral is the product just got way better and cheaper to run and we did nothing. Anthropic just dropped a new model that's smarter and lower cost. All we have to do is change one line of code to take advantage. Pretty crazy world. While a lot of the focus was on the artifact's interface, some like Maratka and Koilen, also tested it for its reasoning capabilities and found it much improved as well. So what about any critiques or skepticisms? One thing I did see a little bit from the safety crowd was summed up by Mikhail Saimin, who wrote As a reminder, Dario, the CEO of Anthropic, told multiple people Anthropic won't release models that push the frontier of AI capabilities. He then shared a couple of screenshots seemingly referencing that point. Given that Claude 3.5 Sonnet does seem to be ahead of GPT4O, is this a betrayal of that? The take that I've seen most often in response to this is that while 3.5 sonnet might be slightly better across a number of different dimensions, it's not some phase shift revolutionary jump up. And so the spirit of what Dario is saying is not necessarily undermined by releasing a slightly more advanced model than the state of the art. Also, the information about him telling people that they won't release models that push the frontier is all secondhand, so it may not even be really true. Wired continued their pattern of getting more and more negative about AI with a piece titled We're Still Waiting for the next big leap in AI. Anthropic's latest Claude AI model pulls ahead of rivals from OpenAI and Google, but advances in machine learning have lately been more incremental than revolutionary. This is functionally true. It's exactly the argument that I was just giving about Dario saying that they wouldn't push the state of the art. At the same time, I think this sort of critique from media rings a little hollow to me. One because as I said, I think Wired is now in a category of publications that is just looking for things to not like about artificial intelligence. And second, I think that it inherently fails to recognize how much additional value in terms of productivity, time saved, new opportunities, even quote unquote incremental advances really represent. One of my bully pulpits here on this show is the idea that functionally the way that AI is impacting the world right now is not that we wake up and entire categories of jobs are gone, but that people are winning back their lives 20 minutes at a time. Ultimately, I think the big advance here, even though the updated model is great, is really the artifacts interface. Professor Ethan Malik writes, the thing Anthropic is nailing is making their systems fun to use. ChatGPT has a lot of key features Claude is missing web access, full code interpreter, voice, GPTs but but it requires some trial and error to figure out since it isn't obvious. Even Gemini feels more complicated. For us, the consumers, it's basically nothing but upside. So if you haven't tried it out yet, go check out the latest claude. It's available right now at Claude AI and if you create anything really cool, please tag me on Twitter X so we can see what you did. Like I said, we're dropping a new tutorial about this on Superintelligent today, so if you are a member you will get access to that. Alright guys, thanks as always for listening or watching the show. And until next time, peace.