Karpathy Leaves OpenAI as OpenAI Launches Memory for ChatGPT

Today on the AI Breakdown, Andrej Karpathy has left OpenAI as ChatGPT gets memory before that on the Brief, Nvidia has jumped in the on device Chatbot game. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube, our newsletter and our discord. Welcome Back to the AI Breakdown Brief. All the AI headline news you need in around five minutes. One of the big trends in LLMs right now is to try to put them in new contexts rather than just being a cloud based bot. Specifically, people are looking into things like personalization, how they allow individuals and businesses to have LLMs that reference their own data to. Many companies and projects are also looking at how to get LLMs running on mobile phones or PCs in either a fully offline or hybrid cloud on device sort of way. This is of course where Apple's efforts are. So Samsung and Google and the other handset makers are thinking in similar ways and now it appears so too is Nvidia. They've recently announced Nvidia Chat with rtx which they call your personalized AI chatbot. They write Chat with RTX is a demo app that lets you personalize a GPT LLM connected to your own content, docs, notes, videos and other data leveraging rag. You can query a custom chatbot to quickly get contextually relevant answers and because it all runs locally on your Windows, rtx, PC or workstation, you'll get fast and secure results. So like I said, there are a lot of projects that are working on something similar right now, but obviously a company at Nvidia size and scale doing so shows what a big deal this is. Brian Romley writes Boom. Open source Chat with RTX is here and I am testing it. Now you can feed it YouTube videos and your own documents to create summaries and get relevant answers based on your data. This is a massive challenge to OpenAI. Now for those of you who are interested in this space, one project that I am watching is openinterpreter. Openinterpreter frames themselves as a new way to use computers. Let LLMs run code on your computer to complete tasks. You can check it out@openinterpreter.com now staying on the theme of Nvidia and AI chips for a minute, we recently got comments from Jensen Huang saying that he didn't think that it was going to take the 7 trillion that Sam Altman has apparently been talking about to meet the world's compute needs. Part of the reason for that is that he said he expected prices to come down. And now the CEO of Databricks has said something very similar for those unfamiliar Databricks is a company that helps big enterprises and software use AI. It's competitive with, for example, Amazon Bedrock. The they're very invested in the idea that there won't be one winner take all sort of LLM, but lots and lots of different solutions that run the gamut from open source to closed source that can meet any different company's particular needs. The CEO of that company, Ali Godse, said recently that just as Internet bandwidth constraints evaporated in the 2000s, quote, the same thing will happen with GPUs. Now part of where this came from was a comment in an interview where he said the challenging thing is a lot of these startups started on a weird funding or GPU commitment model that I think is going to pose an extra challenge to them going forward. We're going to see a lot of turbulence in the next 12 months because of that. And going further, he said, people were buying these GPUs and doing Bitcoin mining on them. Then Ethereum decided to switch the model and they didn't need GPUs anymore. But then it turned out that OpenAI is training gigantic models and we do need GPUs after all. Then there was this one vendor, Nvidia, and everybody rushed to get these GPUs. There's been such crazy scarcity, but in the 2000s there was a similar thing around bandwidth. It turned out that actually capitalism, supply and demand, takes care of that problem and the price of bandwidth just plummeted and bandwidth was abundant everywhere. The same thing will happen with GPUs. So then what happens to the startups that do GPU money laundering, which is when they get huge funds from big strategic investors. The valuation of those companies were high because they had to raise hundreds of millions of dollars just for the GPUs. So what happens when the price of those comes down? Then there's people that are sitting with three year commitments on these GPUs. That's going to pose a big challenge for anyone who's over committed on the GPU spend side. The whole interview with the information is pretty fascinating and I highly recommend it. And even if Alia is right, Wall street is still very focused on that bottleneck that exists here and now. In the weeks since UK chip designer ARM holdings announced its latest financial results, their stock prices nearly doubled up 98%. Unrivaled investing on Twitter wrote. My broader conclusion is that there's a huge mismatch between demand and supply for AI stocks and this is translating into absurd valuations where the companies can still grow a lot, but due to lofty prices the shareholders likely lose. The reality is that the winners of the AI revolution will be far spread and and not concentrated in just a few players. The challenge for these investors is that a lot of these initiatives and companies are still very early days. Now moving over to the new product side of things for a moment, a couple really interesting things. Cohere has introduced aya, which they call a state of the art model and data set pushing the boundaries of multilingual AI for 101 languages. The project involved 3,000 independent researchers, 56 language ambassadors and people from 119 countries and is meant to try to advance multilingual AI. Given that much of the state of the art in LLMs is of course developed in English Right now, Stability introduced a new open source text to image model called Stable Cascade that some are arguing is the best open source text to image model yet, they write. During our evaluations we found Stable Cascade performs best in both prompt alignment and aesthetic quality in almost all model comparisons. As with most Stability releases, Stable Cascade is now available on a non commercial license Quick update on the legal side. Earlier this week a US district judge dismissed four of the claims in the Sarah Silverman lawsuit against OpenAI. The dismissed allegations include vicarious copyright infringement, violations of the dmca, negligence and unjust enrichment. But the judge did allow Silverman and her co plaintiff's core claim of direct copyright infringement to move forward. The plaintiffs now have a month to amend their class action suit in response to this ruling. So friends, never a dull moment in AI. If you want to hear more about the Latest moves at OpenAI, including new features for ChatGPT and a majorly high profile researcher leaving the company. Come back soon for the main AI breakdown. Welcome back to the AI breakdown. OpenAI is once again dominating the news today for both good and not so good reasons, depending on your perspective. The first story to cover is that OpenAI researcher Andrej Karpathy, who is arguably one of the best known people at the company, has decided to leave the Information writes that Andre was working closely on an AI Assistant project in collaboration with the company's head of research, Bob McGrew. We talked a couple weeks ago about how OpenAI is moving and increasingly trying to pull competition in the AI space to this agentic future rather than just leaving it in the world of LLMs. Now Andrej Karpathy has had an interesting on again, off again relationship with OpenAI. He was one of the founding members, but then left and spent five years at Tesla. At Tesla, he was leading the development of the company's Autopilot semi automated driving software. About one year ago he posted on X that he was coming back to OpenAI and people were incredibly excited, said OpenAI spokesperson Kayla Wood. Andre is departing to pursue personal projects. We are deeply grateful for his contributions and wish him the best. His responsibilities have transitioned to a senior researcher who worked closely alongside Andre. Now, after this got a ton of buzz, Karpathy himself took to Twitter where he wrote hi everyone. Yes, I left OpenAI yesterday. First of all, nothing happened and it's not a result of any particular event, issue or drama. But please keep the conspiracy theories coming as they are highly entertaining. Actually, being at OpenAI over the last year has been really great. The the team is really strong, the people are wonderful and the roadmap is very exciting and I think we all have a lot to look forward to. My immediate plan is to work on my personal projects and see what happens. Those of you who've followed me for a while have a sense of what that might look like now. One thing I had noted before this came out is that Karpathy has been tweeting a lot more of late. For example, he wrote a whole screed on the shortification of learning, basically saying that education and entertainment aren't the same thing and that learning should be hard in some ways. And on Monday he wrote the Internet used to be fun. I remember visiting my friends websites. They were ugly and quirky and it was awesome. You'd wondered who'd stop by yours. They were a labor of love and a medium of self expression. Not your LinkedIn. We can fight this. Do people have opinions for the easiest way to host a static website today, not just the hosting but custom domain SSL deploy with git push. The point being here that the Occam's Razor explanation of a very smart guy just wanting to work on something different seems like in this case it might actually be true now. Still, in terms of your experience and my experience with OpenAI's products, the bigger news is the announcement of memory. Joanne Jiang, who does product@OpenAI we just launched a small experiment for memory on ChatGPT. How it works it's quite similar to custom instructions, except ChatGPT is the one driving it. Basically we taught ChatGPT to keep a notepad for itself every time you share information that might be useful for future reference, it'll hopefully add to the notepad. 2. You'll be in control There are five different ways to manage memory. You can 1. Just tell ChatGPT to remember or forget something in chat conversationally 2. Use temporary chat for one off convos involving topics you don't want ChatGPT to pick up on 3. Delete individual memory snippets 4. Clear all memories 5. Disable memory altogether finally, she writes, it's a small experiment. I'm sorry if you're not in the experiment and want to be. We're taking a bit more time than usual with this feature, given how memory almost by definition will take a bit longer and more interactions to build. We can't wait to learn from this experiment and iterate on the feedback so that personalization features like this can be truly useful to everyone now in the example screenshot they share of managing memory, they have things like Has a two year old daughter named Lena Daughter Lena loves jellyfish Prefers meeting summaries to have headlines with bullets and action items summarized at the end Prefers assistance with writing blog posts to be more concise, straightforward and less emotive Loves to travel Is interested in traveling to Mexico for April vacation Basically, these are all things that ChatGPT has extracted from your conversations and uses to make future results of conversations with Chat ChatGPT even better. They write as you chat with ChatGPT, you can ask it to remember something specific or let it pick up details itself. ChatGPT's memory will get better the more you use it, and you'll start to notice the improvements over time. Of course, the feature that they referenced was Custom Instructions, which was a really simple feature that came out last year that showed the direction that ChatGPT was headed. The Custom Instructions field simply says what would you like ChatGPT to know about you to provide better responses, and how would you like ChatGPT to respond? So, for example, the one of the things that I put in there was that I do a lot of content and education around AI, and that the way I liked it to respond tended to be at an undergraduate comprehension level for people who have some technical understanding but who aren't developers or technical themselves. Those are things that I often put in individual prompts that I was able to embed in ChatGPT's responses in general. In many ways it feels like memory is just a next step on that pathway that expands that capacity and just nudges chatgpt towards even more personally useful. Now, given how it appears that OpenAI or at least Sam Altman's goal with ChatGPT is to have it become the most powerful AI assistant. It makes sense that it could learn more about you over time and customize and tailor its responses to your queries based on that knowledge that it has now. By and large, the feedback has been really good. People are very excited about this feature. SWIX is among them, but also has a little bit of advice writing ChatGPT finally getting persistent memory and temporary chat I believe most chats should default temporary with opt in saved to history after the fact, but I wonder if OpenAI will ever recognize that in its pursuit of data. As for memory, it looks like this is almost a MEM GPT implementation. The flaw here is that you have to manually delete memories instead of having the AI figure out when to delete it. Net result is it is much easier to add memories than delete and there is no demonstration of edit in place. Probably more work to do here to make it good, but this is a nice start. Now speaking of starts, another start that is somehow related to OpenAI, although not of OpenAI, is that OpenAI's chairman of the board, Brett Taylor, formerly co CEO at Salesforce, has officially announced a new company called Sierra that is designed to build AI agents for customer service at corporations. The company has raised $110 million in that pursuit. TechCrunch writes, at its heart, the new company is a customer service bot. That's not actually all that earth shattering, but the company claims that it's much more than that, with its software going beyond an extension of an FAQ page and actually taking actions on behalf of the customer. In a blog post they wrote Sierra agents can do so much more than just answer questions. They take action using your systems. From upgrading a subscription in your customer database to managing the complexities of a furniture delivery in your order management system, agents can reason, problem solve and make decisions. Now, agents are of course one of the big themes in AI development right now, and it's not surprising that someone with Brett Taylor's background, especially a six year spend at Salesforce, has investors excited about the possibilities. I think honing in on a particular discipline like customer service is the way we're going to see AI agents come to market, and so this will definitely be one to watch. Now of course some people did ask about conflicts of interest with OpenAI, given that they also seem to be driving towards an AI agent future, to which Taylor said that he'll simply recuse himself. Quote whenever there is a potential for overl, we'll see just how competitive things get over time and whether that will actually work. Lastly, we got a little bit More information from OpenAI CEO Sam Altman when he spoke at the World Government Summit. When he had this to say about GPT5, I was sort of laughing a.

Little bit because this is going to sound like an annoying answer, but I think it is the important part. It's going to be smarter. There are all of these other things, you know, we can talk about. It'll be better at these kind of tasks, it'll be multimodal, it'll be faster, who knows what. But the thing that I think really matters is it's going to be smarter. And this is a bigger deal than it sounds, right? Because what makes these models so magical is that they're general. And so if it's a little bit better, if it's a little bit smarter, that means it's a little bit better at everything. And the thing that I think is most exciting is it's not like this model is going to get a little better at this task and not really better at these. Or, you know, it's not that.

It's.

It's because we're going to make the model smarter. It's going to be better at everything across the board.

So, friends, lots happening in the world of AI in general, but certainly OpenAI in specific. However, that is going to do it for today's AI breakdown. Until next time, peace.