The Next Phase of Generative AI

Today on the AI Breakdown, we're talking about how the next phase of generative AI burst into existence over the last week and a half or so. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube, our Discord and our newsletter. Hello AI friends. A couple of days ago I tweeted out the next phase of Gen AI burst into existence over the last five days and people's expectations have been totally redefined. The three legs of the stool Gemini 1.5, Sora and Grok now I will get into what I meant by that in just a little bit, but to kick us off, given that this is a long reads episode of the AI Breakdown, we're going to go look at another piece from Professor Ethan Moloch, which actually came out on the same day that I sent that tweet. It's called Strategies for an Accelerating Future and it won't take you long to see how it relates to what I just read. Ethan writes, I didn't expect to have to update my views on the state of the art in AI so soon after writing about Google's Gemini Advanced, the first real competitor to GPT4. But there have been two leaps in LLMs this week with real practical implications. The first has to do with memory. There is a new version of Google's Gemini that has a context window of over a million tokens. The context window is the information that the AI can have in memory at one time, and most chatbots have been frustratingly limited holding holding a couple dozen pages at most. This is why it is very hard to use ChatGPT to write long programs or documents. It starts to forget the start of the project as its context window fills up. But now Gemini 1.5 can hold something like 750,000 words in memory with near perfect recall. I fed it all my published academic work prior to 2022. Over a thousand pages of PDF spread across 20 papers and books, and Gemini was able to summarize the themes in my work and quote accurately from among the papers. There were no major hallucinations, only minor errors where it attributed a correct quote to the wrong PDF file or mixed up the order of two phrases in a document. You can see how the advent of massive context windows gives AI superhuman recall and even new use cases. If I asked a researcher to read through all my papers and summarize major themes, including illustrative quotes, it would take days. The AI did it in less than a minute, and Google has announced that Context Windows will soon reach 10 million tokens, or nearly 17,000 pages. The second big advance is speed. You may have been frustrated by the relatively slow speed of ChatGPT, but one AI company, Grok, no relation to Elon Musk's Grok, has developed hardware that gives almost instantaneous responses from GPT 3.5 class models, bridging the gap between questions and answer in the blink of an eye. This shows that AI needs not always involve waiting for replies. Speed and memory are both vital to making AIs more usable and powerful in the real world. Imagine feeding AI hundreds of pages of instructions on how to do something and then having it quickly do exactly that. In an experiment, I gave the AI a 352 page rulebook for an obscure game and it was able to make sense of the scattered documentation and actually figure out how to correctly play. Plus, Google demonstrated exactly this capability in their Gemini 1.5 documentation. Researchers gave the 500 or so available pages of reference material on a language with 200 speakers, and so with no real online presence to Gemini and to a human translator, they found that the AI was able to learn the language about as well as the human could from the same documentation, despite the fact that the AI itself was only about as smart as GPT4. Together, rapid answers and massive context windows suggest that even without smarter AIs, and those are coming soon, we will see large leaps in AI capabilities continue for the near future. Hello AI friends, Quick note before we get back into the show, we have just opened up registration for the March edition of the AI Education Beta program. The whole philosophy of this program is to get you learning by doing so. We have short tutorials, think 3 minutes, 5 minutes, 7 minutes around specific features and use cases in AI, followed by challenges that are step by step instructions that get you actually using the most interesting and relevant tools. We have now built out a library of more than a hundred of these lessons and step by step companion instructions and we'll be dropping more each week for the first time. We'll also be moving beta users this month to a new dedicated platform where you can access that library of content, build lists of lessons you want to learn from later, and other features that we hope will help make this the single best AI learning experience available. If you want to check it out, go to Bit ly aibeta. That's Bit Ly aibeta. Registration is only open this week until next Monday, so go check it out. Out to the edge of the possible with this rapidly approaching future in mind, I taught the students in my class this semester how to build prompts and distribute them as GPTs. Most of my undergraduate and MBA students had no programming experience, but that was not a problem. Building a GPT is more like providing instructions to a person than coding a machine. After those lessons, I gave my students an assignment titled An Impossible Thing Build a GPT that will get you a job by showing a potential employer that you are a prompt engineer. It should automate a task in a job you want to do. One of the things I emphasize to them is that no one really knows what current LLMs are capable of, since most of the tests conducted by AI companies focus mostly on coding and testing benchmarks, not real world applications. Since my students come from many industries and countries, they had a tremendous diversity of potential use cases. By the time they started building GPTs for their specific needs, whether private equity deal memos or suggesting a perfect wedding ring to their fiance, they became the world experts in using AI for their specific field. Ethan then goes on to talk about some of the GPTs that his students made, including one that helps engineers and managers develop a common language around performance reviews, one that automated the process of creating social media posts driven from company thought leadership reports, et cetera, et cetera. Ethan continues, I also asked students for a reflection on what they learned as a result of this process. An almost universal belief among the students, whether they were in aviation or consulting or banking or nonprofits, was that AI was going to have a big impact on the future of their industry very soon. Even though many saw the limitations of today's tools, they also got a sense of where the future was heading. Despite this conviction, they also tended to think that the leaders and executives of the organizations they were joining did not yet see the full significance of AI and what it would mean for their industry. Four questions to ask about your organization how can leaders start to think about the rapidly advancing nature of AI? The first thing they should do is use it. No amount of reading and research can substitute for spending 10 hours or so with a frontier model learning what it can do. After getting familiar, companies should think about the following four 1. What useful thing you do is no longer valuable? AI doesn't do everything well, but it does some things very well. For many organizations, AI is fully capable of automating a task that used to be an important part of your organizational identity or strategy. AI comes up with more creative ideas than most people, so your company's special brainstorming techniques may no longer be a big benefit. AI can provide great user journeys and Personas. So your old product management approach is no longer a differentiator. Getting a sense of what AI can do now and where it is heading will allow you to have a realistic view of what might soon be delegated to an LLM. 2. What impossible thing can you do now? The flip side of the first question is that now you can do things that were impossible before. What does having an infinite number of interns for every employee get you? How does giving everyone a data analyst, marketer and advisor change what is possible? Number three what can you move to a wider market or democratize? Prior to AI, companies were often advised to put their effort into servicing their most profitable customers. But AI has greatly changed the equation. Services and approaches that were once expensive to customize have become cheap. Prior to AI strategy consulting firms would only work for giant clients for large fees. But now they may be able to offer effective advising to a wider range of businesses at lower costs. Custom tutoring and mentoring, once available only to the rich, may be widely democratized. What can you move upmarket or personalize? At the same time your organization's capabilities have increased? If you were once a small marketing firm, you can use AI to punch above your weight and and offer services to elite clients that were once only available for much larger firms. With giant context windows and fast answers, every customer may be able to have a personal AI agent who knows their preferences in previous interactions with the company and communicates with them according to their preferences. Figure out the most exciting thing you can do and see if you can make it happen. Misguided companies will see any increase in performance from AI as an excuse to lay off staff, keeping their output the same. More forward thinking firms will take advantage of these new capabilities to both improve the lives of their employees and expand their own capabilities. This is an area where leaders have agency over the future of AI and work. A lot depends on getting it right because it is possible. We are just getting started. Many skeptics about the impact of AI are focused on the flaws that LLMs have hallucinations, short context windows, slow answers, and so on. These are legitimate concerns and if AI advancement were to stop, they might prove to be huge issues in the utility of AI. But AI is advancing rapidly and some of these concerns may soon vanish even if others like hallucination are not completely solved. What that means is that it is fine to be focused on today building working AI applications and prompts that take into account the limits of present AIs, but there is also a lot of value in building ambitious applications that go past what LLMs can do. Now you want to build some applications that almost but not quite work. I suspect better LLM brains are coming soon in the form of GPT5 and Gemini 2.0 and many others. When they do, you can swap them into the almost but not quite working applications for a fast start. This is similar to the philosophy of the big AI labs which build ambitious solutions which will benefit when the next version of their core LLMs are released. So don't just build for what is possible today, but what is possible in six months. At this point, I think things are unlikely to slow down in the near future, and focusing on where things are headed rather than where they are prepares you for a world of continuing change. All right, back to NLW here. Another great essay from Ethan. So like I said, the connection point to what I had originally tweeted is that I really do think people's sense of the possible got shifted over the last week and a half. As you heard in my tweet, I think Sora was another component of this, just from sheer power and impressiveness. But in terms of this question that Ethan ends on exploring of how companies and organizations can think about how to use these tools, this is something that I notice a lot and I think about a lot. There's almost a natural progression of the way that companies think about how AI can impact them. The first is a cost savings mentality, let's call it, where they're looking for one to one replacements for things they already do. In other words, automating away tasks that used to take them a long time. There's nothing wrong with that. There's a lot of areas where that's incredibly valuable right here and right now. But it is a very simple and limited way of looking at this. Another level which some companies get to, is thinking about the new opportunities that AI opens up. How they can, to use a phrase from this piece, punch above their weight class in their particular industry, how they can service new types of markets, how they can expand service that used to be for a few to the many. Basically they have not a cost savings mindset, but an opportunity expansion mindset. It's an abundance view that instead of saying can we do the same with less? Says can we do more with the same. But then there is a third level which almost no one has gotten to yet, which is to zoom out even farther and ask not just can we do more with the same, can we do things that were literally not even possible before, that people don't even know that they want but they will when they realize that it's possible. This is obviously the most nebulous section, the hardest to know before we get there, but it seems highly likely to me that the vast majority of impact and change that AI will have on business and work in the economy will will not come from either one to one replacement of existing functions or even supercharging individual employees, but from fundamental restructurings that totally reshape our sense of the possible. I don't think it will be those things that get companies to actually start institutionalizing these tools, but it is where I believe their biggest impact will be felt. Anyway, one more big thank you to Ethan Molik for another great essay. Go check out his blog. It's one useful thing.org, he has a book coming out in just about a month, and until next time, peace. Sat.