Zapier’s Mike Knoop launches ARC Prize to Jumpstart New Ideas for AGI | Training Data

Right now I think what I see happening is there's sort of this mythical story of a very bad outcome once we get to like super intelligence, right? It's a very theoretical driven story. It's not grounded in sort of empirical evidence. It's basically based on sort of reasoning our way to like this outcome. And I think the only way that we can really, really effectively and truly set good policy is by you have to look at what the systems can and can't do and then regulate or decide, make decisions at that point about what, what it can or can't do. I think anything else is sort of like, you know, you're cutting off potential really, really good futures way too early.

Hi and welcome to Training Data. We have with us today Mike Knoop, co founder of Zapier. Mike has recently stepped up to co found and sponsor the ARC Prize, which is one of the most unique benchmarks in AI that measures a machine's ability to truly learn things intelligently versus just to parrot patterns in the training data. We're excited to ask Mike for an update on how things are going with the ARC Prize two weeks in and to hear his views on why we need radically different approaches and benchmarks to achieve true general intelligence.

Mike, thanks for being here today. So we're excited to talk about the ARC AGI initiative. Before we get into that, I'd love to spend a few minutes on your background at Zapier, because I think Zapier has emerged as probably one of the best examples of what an existing application company can do with the power of AI. And the way that you guys were sort of early to that and the way that it's now kind of interwoven in the product has been really interesting to watch. So maybe can you just say a few words on what is Zapier and what has your approach to AI been at Zapier?

Yeah, Zapier is a workflow automation platform. We support 6,000 different integrations, things from Salesforce to Gmail to, you know, any basically SaaS software that you can imagine we connect with. And I think the unique thing about Zapier is it's intended to be very easy to use for non technical users. So the majority of Zapier customers, users would not self identify with being a programmer or being technical, being an engineer, something like that. Even though I also, I kind of think of myself as an engineer and I find lots of interesting use cases for Zapier but the large majority of our users don't and I think that's what is quite special about Zapier and why people tend to like fall in love with it is the sort of feeling of power and leverage that you get as a non technical user being able to have software do work for you basically. And I think that's in an interesting way the exact same promise of AI, right? Like that's what people want of AI. They want software that's just going to do more work for them. And so in many ways I think the mission of Zapier and sort of the mission and purpose of AI intersect and I've been sort of, I guess, call me AI curious I've since all the way back to college and I gave like a whole all hands at Zapier, can't remember what year, but like when the GPT3 paper came out and showed that to a little company. And so I'd been sort of tracking and following along the progress, but it really wasn't until I think January of 2022, when I think this was the chain of thought paper when that one came out that I saw that and really surprised me because I thought I had priced in everything that sort of AI language models could sort of do up to that point. And this idea of like, you know, let's think step by step, this chain of thought technique, breaking down these language models as just tools for reasoning instead of, you know, just a one shot sort of completion or chat engine felt very special and something that I think most people didn't expect that they could do even though the technology had been out there for over a year at that point. And so that actually that kind of moment caused me to give up my exec team role. I was running all of our product engineering at the time and I went back to basically be an individual contributor at the company as an AI researcher alongside my co founder Brian, and happy to talk more about sort of that journey. But yeah, I think that's what caused Zapier to be relatively early in terms of AI.

What are some of the things that you've put into the product that you're most proud of at Zapier in terms of AI features?

You know, at this point I think Zapier is pretty. I'd say there's like probably two main places where we've gotten a lot of value from AI. The first is that over half the company now, individual level, uses AI on a daily basis. And I know this because we're actually measuring Zapier's own platform usage of our own company. So this is like we have over half the company is like Building zaps, building automations that use an AI step, like a ChatGPT step in the middle, you know, either to do so content generation or data extraction over unstructured text. All sorts of really interesting use cases that we can sort of talk about. In fact, one of the top internal use cases is probably getting us about I think like 100x labor enhancement rate. Wow. Which, which has been phenomenal.

What is that?

Yeah, you want to talk about it? Yeah, Yeah.

I mean 100x improvement.

Yeah, we got to talk about that. I think it's our, it's our, it's our personal like high watermark for what we've been able to achieve using AI internally for like an operational perspective. And so Zapier has these things on our website called Zap templates. They're effectively recipes that help users figure out what can Zapier do and help them get started. And these templates are in order to make them, they've all been historically handmade because they require a bit of right brain and a bit of left brain. They're very creative. You have to inspire the end user, the customer, the would be user of what could Zapier do for you? What's the outcome, what's the ROI you might get? And then there's also a very like technical way they have to be crafted and built as well. You have to map JSON fields from one integration app to another integration app to make sure it actually works. And together that's what actually helps users get started and activates them into the product. And we had a whole of maybe a million of these things that we knew we wanted to build that we hadn't built yet because they're just, they take so much effort. The sort of rate of production for our contractors was about 10 a day up until last summer. And we had a person, a member of our, I think it was our partner marketing team, actually a background by the way in freelance writing who built a zap, a system of actually several zaps using OpenAI as some middle steps here and built an end in system that whenever a new integration got launched on Zapier it would automatically try to like figure out what are the most interesting Zap templates that could be built. Write the inspirational use case behind it because we've tons, we have millions of these things already today. There's lots of training data and then also do the exacting field mapping as well. And we moved the human in that workflow from the do loop to the review loop. So instead of having those contractors now actually Generating them, thinking really hard about what the use cases should be in building the sort of field mappings. Now they're basically reviewing output from, from the system in a spreadsheet and saying, yes, this one is good, this one's bad, this one's good, this one's bad. And the funny thing is, because the cost of production of generating these things is solo, we don't even try to fix the bad ones. We just throw them away and say, well, just generate another stack and throw them on top. And so that rate of production is about a thousand a day now. So we've gone from, you know, 10 a day to a thousand a day per sort of contractor. And we've been chipping away steadily at that hole of a million and keeping up with sort of the launch of like new integrations on Zapier. I think one of the main things that that showed me was, you know, probably the space you want to look for in businesses, if you're thinking about how to deploy AI is like really up, like top of funnel or bottom of funnel, tend to be like, you want to get something that's really close to like an important conversion rate for your business. And then, you know, I think if you can identify any manual work that your organization does that has high volume where human is doing the work, I think those are sort of opportunities to like introspect and say, okay, is there an opportunity to get that human out of a do loop and sort of craft a system that can do the work, but then put the human in the review loop, which is still quite needed at the sort of maturity level of the technology today, but still phenomenal from an ROI perspective.

Are there any metrics you can share on the impact Zapier AI has had on the overall Zapier business?

The, the biggest one today is we're just about to hit 10 million AI tasks per month now. I think we're on a run rate for about 10 million AI tasks per month. And I think what's. I think I would love to be shown wrong or if corrected, if, you know, examples. But I think at this point, Zapier might be the biggest automated AI platform in the world. In the sense there's a lot of researchers, entrepreneurs, builders who are trying to build these agentic AI systems where the AI is working without human in the loop. And yeah, I think at this point, with 10 million AI tasks a month, Zapier may be the biggest example of that in the world right now.

Really cool. Can we talk about ARC AGI? Let's do it maybe start with a recap of what is ARC AGI. Why did you and Francois set out to establish this prize?

Yeah, this was so a follow on to my like, AI curiosity. So, you know, the reason I gave up my exec team role back in 2022 was I kind of wanted to know for myself, like, are we on path for AGI or not? I felt like very important to know for Zapper's mission. But also just like as a human, I was very curious and wanted to know like, is this going to happen? Like, you know, there's definitely some interesting scaling that's happening. Is that sufficient to like get to. I think, you know, what naively I had in my head is like this, you know, super intelligence AGI. And you know, surprisingly what I learned is the answer is no. I revisited actually got to first know Francois Chollet, who is my co founder on ArcPrize. I first heard about him and got exposed to his research back during COVID Actually it was during 2020 he did, I think another podcast where he was explaining his paper, 2019 paper on Measure of intelligence, where he tried to formalize a definition of what AGI actually is. And I thought it was interesting at the time, but you know, I kind of parked it with lots of other stuff going on. It's a year and doing our own AI product building. But as Zapier got more into sort of building with AI, I built my intuition of what language models could do. But the apparent limits were I started getting more into AI evals and trying to like understand where sort of the limits were. What could we expect from a product on a product building perspective? Like, where are, where are our products going to, like tap out? Where should we invest our engineering and research effort versus just wait for the technology to keep scaling and mature. And you know, the thing I found was that most AI evals were saturating up to human level performance and it was accelerating. And when I went back to look at the ARC eval back from 2019, expected to see a similar trend. And instead what I found was basically the opposite. Not only had it not reached human performance yet, but it was actually decelerating over time. And this was like really supremely sort of surprising to me. And maybe it's worth defining, you know, we use these terms of AGI, but like, what's the actual correct definition for it? Right. I think there's a kind of popular definition in the world today. Actually there are probably two schools of thought. I think one school of thought that I see is AGI. Is undefinable and we shouldn't even try. This is a quite popular perspective. And I think the other school of thought is, you know, that AGI is the system that can do the majority of economically useful work humans do. This was popularized by OpenAI and the Microsoft deal. This is actually like in their, their deal together, like once this is achieved, like OpenAI retains all the future IP is very interesting. I think the node coselect, you might get credit for coining that definition. But nonetheless, I think because of OpenAI's success, that definition has sort of become accepted by a lot of people and as a target and goal we should shoot for. The challenge is I think it's a fine goal by the way, and I think current model architecture may be within spitting distance of it. I think it says way more probably about what the majority of humans do for work if it's a true goal than what AGI actually is though. And Francois defines AGI as the efficiency of acquiring new skill. That's it. And here's a quick thought experiment I think you can use to grok this is. We've had AI systems now for many years, five plus years that can beat humans at games like Go chess, poker, diplomacy even. And the fact remains that you cannot take any one of those systems that was built to beat one of those games and simply retrain it with new data, new experience to beat humans at another game. Instead, what researchers and builders and engineers have to do is they have to like go back to zero. They have to tear it all down, rethink of new algorithms, new architectures, new ideas, of course, new training data as well, often new amounts of scale in order to beat that next game. And yet this is in complete contrast to how you two both learn, right? I could like sit you both down here, teach you a new card game in probably about an hour. I could probably show you a new board game and get you up to proficiency within a couple hours. And that fact is what makes, I think it's highly representative of what makes you generally intelligent. It's your ability to very quickly and efficiently sort of gain skill in order to accomplish some open ended or novel task that you've like never encountered before in your life. And that's what's special about Arkansas. So RKGI is an eval that tries to take that definition and actually measure it. And it was designed specifically to resist the ability to memorize the benchmark, which is very different from most other AI evals that are out there. Every task is completely novel and there's a private test set that no one's seen outside of a handful of people that have taken it, to verify that all of the puzzles are solvable. And that degree of novelty and that degree of not having ever been seen before is what makes ARC a really, really strong benchmark for trying to distinguish between, you know, this more narrow AI that can be beaten largely through memorization techniques and egi, which is, you know, a system that can sort of very, very rapidly and efficiently acquire this skill at test time.

What is the definite, what is the definition of efficiency? I imagine there's a compute component, a data component. What's the definition of efficiently and efficiently acquire new skill?

Yeah, Francois, I'll probably do a bad job trying to like summarize his research, if you want to read more. By the way, his On Measure of Intelligence paper is like the source of truth for all of this stuff. I think it's really, really good. And I think before I get to the answer, I think one other important thing to sort of see is that ARC has been unbeaten since 2019. And I think its endurance to date is probably the strongest set of empirical evidence that the underlying concepts of the definition are correct. Which is why I think it's worth paying attention to and why it's such a special eval and special set of research. So I think Francois would sort of describe efficiency as the ability for a system to sort of translate from core knowledge priors to being able to like attack, sort of like the sort of search base or task space around it. And so a very weak generalizable system is only going to be able to take on sort of very near term adjacent tasks to sort of the core data, the core knowledge that it was, that system was trained on. Whereas, like a highly generalizable system is going to be able to have a much larger field of sort of tasks and novelty that it's able to attack and be able to be able to effectively do with a small set of training data. And that's what we hope to see with the eventual solution for ARC AGI as well, is that if someone's able to beat it, the goal is to get to 85% on the eval. Today's state of the art is I think, 39% as we record this. And I think what's special is if someone can actually beat ARC at the 85%, that would mean that you've created a computer program that can be trained on this very small set of core knowledge priors, Things like goal directedness, objectness, symmetry, rotation, These are, these are sort of things that even that emerge very early in childhood development and be able to use those core knowledge priors and recombine them and synthesize them into new programs in order to solve tasks with exacting accuracy that that system has never seen before, never been exposed to in its training data. That would be a really, really important thing, particularly at the application layer where like the number one problem today is like hallucination accuracy and consistency. And that results in this low user trust which limits deployment of like, really AI.

Right now you have some peculiar rules for competing for the prize. I think there's a limit on how much compute you can use. You can't use the Internet. I don't know if you can use GPT4 and closed models. Like, why set those? Why put those limits in place?

Yeah, so the two big ones are. You're right. So the competition hosts on Kaggle and Kaggle enforces no Internet and you have limited compute. So specifically you get one P100 for 12 hours. And no Internet means you can't use frontier closed models that are available through APIs like Claude Sonnet or Gemini or GPT 4.0 or 4.0. Maybe a little take them in order. I think the compute one is maybe more interesting. The reason for the compute limit is to target efficiency first and foremost, because if there wasn't any compute limit at all, then you could simply define AGI as saying it's just a system that can acquire skills with no degree of efficiency attached to it. And if that was true, that would mean that this system could brute force basically every possible program, think through every possible future outcome here, generate every possible single archetype of puzzle, and use that in order to sort of win the challenge. And we know that's not actually what happens in human general intelligence. You can read more in sort of Francois paper about why. But the way that I think about it is, you know, you can think about it, you can introspect even yourself why you're taking the ARC puzzles and see that when you're trying to solve one of these, that you're not brute forcing every possible transformation from the pattern, trying to recognize the pattern and apply it to the test. Instead, you're using your intuition, you're using your prior experience to try and identify maybe 3, 4, 5 possible possibilities of what the pattern is, and then you check them right in your head. And I think this shows that the sort of efficiency humans have is not brute forcing every possible solution and checking. It's actually there's degree of efficiency. So the compute limit, it sort of forces researchers to reckon with that definition. Now I think it is worth important acknowledging like we don't know exactly how much compute is necessary to beat our kit and we're going to like keep upping the compute bar over time is what I expect. For example, we already upped it. We over 2x'd it from prior versions of the competition. So I think it prior usually got somewhere between like two and five hours to run on the GPU and we bumped that up to 12. Interestingly, all of the state of the art techniques right now are actually maxing out that 12 hour runtime as well. So I do expect we'll continue to increase it over time. But I think it is an important tool in order to force the generality out of the sort of full solution that we're looking for. And the new Internet is a little more of a practical reason. We're trying to reduce cheating, reduce contamination, reduce overfitting, not be able to leak the private test set and largely just increased confidence that when we reach the 85% grand prize mark that someone has actually be narc and be able to sort of say that with some sense of sort of authority and confidence. That's a true statement. One of Francois and my goals for ArcPrize is to establish a public benchmark of progress towards, or maybe towards AGI, or maybe the lack of progress towards AGI and have it be sort of a trusted public tool that, you know, policymakers, students, entrepreneurs, venture capitalists, you know, employees, everyone can look at to get a sense of, you know, how close or far are we away from this sort of important technology existing and then using that insight in order to help try to drive more AI researchers to work again on exploring new ideas, which is something that's unfortunately kind of fallen out of favor in the last several years as LLMs have taken off.

What, what have you seen or maybe what do you expect to be true about the efforts that are successful or more successful toward ARC AGI that makes them different from what we're seeing out of the frontier models and the big research labs.

Yeah, so it gets into like the details of like how does an LLM work? Because that's kind of the bet most frontier AI researchers lives have been taking the last several years is we're going to scale up language models and that's going to more scale, more data is going to get us to AGI. And even though that's the dominant story, I actually don't think it's what Most of the AI labs actually believe internally most of them are working on new ideas. So I think there's like an interesting story there. But it is definitely in their interest to sort of promote a very strong narrative of like, scale is all you need. Don't compete with us. Yeah. You know, we're just going to steamroll you.

Nothing to see here.

Yeah. I think there's true competitive dynamics that have emerged in the market that have that, that are unfortunately, I think shaping a lot of attention, investment, effort away from exploring new ideas. And if it is true that new ideas are needed, which I believe it is, and I think ARC AGI and ARC Prize show that like at least some new idea is needed, then due to the sort of competitive dynamics that emerged in the market over the last couple years, we're kind of like headed in the wrong direction. Right. There's like, you know, the, all the frontier research has basically gone closed source. The GPT4 paper had no technical details shared in it. The GPD, the Gemini paper had no technical deal shared on its longer context. Innovation, things like that. And yet this is like in direct contrast to the history of how we even got here today. Right. The sort of innovation set that led the sort of chain of research that led from, you know, Ilia sequence to Sequence paper at Google, out to Jacobs University, back to Google, then to Alec Radford and back to Ilia OpenAI like there's like a six or seven year chain of research that only happened because of open sharing, open progress and open science. And I think that's a bit unfortunate that we don't, we don't really have that right now. Again, somewhat just due to the market dynamics and commercial success of language models, kind of forcing a lot of that closed frontier research up. So yeah, one of the goals of archprises is to help counterbalance a lot of those things. You were asking about the difference between. Yeah.

What might look like what you said resonates because it seems like a lot of the foundation model companies are going down very similar, somewhat clearly defined paths. And I'm sure that internally there's all sorts of work being done to find the next breakthrough in architecture. But in terms of what's working today, they're all fairly similar paths.

Yep. And I imagine, yes.

And I imagine that what works for the sake of ARC AGI is going to be a little bit of a different shape.

Yep.

And I'm wondering if you're starting to see what shape that may take.

Got it.

And have a sense for what may be different about this more general architecture than what we're seeing out of the foundation models.

Great. So I think, you know, a useful shortcut on how to think about language models is that they are effectively doing very high dimensional memorization. Right. They're able to train and memorize tons and tons of examples and apply them in slightly adjacent contexts. I don't want to under. I don't want to dismiss language models too much because I think that they are very special, something very magical and something that has lots of economic utility. Zapier is an existence proof of that fact alone. So, you know, I don't want to like throw it under the bus too much. I think there's some like really good things that it has like unlocked as the technology goes. But there are limits to it and the sort of limits are not being able to effectively leverage its training data to compose it or combine it at test time to go attack and accomplish novel tasks that had never seen before in its training data. That's what ARC sort of shows, right, Is that this is a skill that these language models don't sort of possess. I think it's kind of maybe useful to look at the history of the high score so far and maybe where we expect it to go. So from when the eval was first introduced in 2019, 2020, the first there was a small kaggle competition that ran to kind of get a baseline on it was 20% and from 20% to 30% the techniques that worked were effectively researchers crafted a like handcrafted domain specific language by looking at the puzzles that were in part of the public test set. There's two test sets, there's a public set and then the private one. That's the, say the rsmeasure on. And they looked at the public test set and they tried to infer and write down programs and Python code or C or whatever. What are the individual transformations that you do in your head to go from one puzzle to the next. And so they called this a DSL and then they wrote a brute force search to try and search through all possible permutations and combinations of those subprograms in order to find the general pattern and then apply it at real time. And, and that got to about 30%. What's gotten from 30% up to close to 40% now is a slightly different technique. This is Jack Cole and his approach is effectively using a code based open source language model and doing test time fine tuning. So he has some pre training done on the code gen model and then at test time taking the puzzles they get, the novel puzzles that's never been seen before and permutating variations of it and then training this like codegen based model in order to write that program then and find a program that fits the pattern and then apply it at test time. And that's gotten to 40%. I suspect that we probably have, I bet we have the ideas in the air already to get to like the 50% mark may maybe even a little beyond the 50% mark without a lot of new innovation. I bet just ensembling or combining these sort of existing idea sets that have already worked toward dark probably gets you about halfway, I think to get to the 85% mark or beyond. I think the ultimate solution probably looks more like the shape of at least to solve arc something that looks like a deep learning guided DSL generator where you have some sort of. Instead of hand coding and handcrafting the DSL ahead of time by trying to infer from the public test set what those sub programs would be, you need somewhere to generate that DSL dynamically by looking at the puzzles in real time and being able to learn from past puzzles and apply that towards future puzzles. This is also another important thing humans do when they're going through the architect. Sometimes the first or second puzzle are actually a little trickier because you're orienting yourself around what am I doing, what task am I doing, what does the possible solution space look like. And then as you get further into the task set they tend to get a little easier because you've. They start, you know, some of the sort of space of possible transformations is just finite. So you start kind of recognizing patterns there and then combining that with some sort of deep learning based program synthesis engineering something that can not brute force all possible programs of how to combine those DSLs together, but something that has some sort of deep learning approach to shape. Which program traces do you try to generate or test and try against the pattern. And then it kind of goes back to this human introspection of how we take the puzzles, which is we're not brute forcing all possible programs in our head. Instead we're trying to identify just a handful of likely candidates and then testing those deterministically in our head in of find the one that works.

It's really interesting that co generation and program synthesis kind of underlies all of the methods you just talked about. And there's something very special there. Like program synthesis is very general and allows you to actually get closer to that definition of generalized intelligence that you mentioned at the beginning.

It's very exacting. And I think this is one of the reasons why the solution to ARC AGI is going to be useful very quickly. So there's a, you were talking this before. There's a history of like toy AI benchmarks over the last 10, 15 years that you know, kind of looked like Ark. There were games, there were puzzles and really never amounted to much in terms of being beaten. They, they all got handily beaten as sort of scale emerged. And you know, they, they really didn't add to our, our understanding of how to build useful AI systems. And so what's one of the common questions I've been fielding the last couple weeks is like, what's different about ARC is that, you know, isn't that likely to just happen here again? And I think the reason why we're likely to see something much more useful, assuming we get like a really good solution to ARC from the sort of first, you know, grand prize win is that the number one problem at the application layer. And this, we see this with Zapier too with our new AI bots that we launched a couple months ago. Been surprising to me in how that has gotten adopted actually by our users. You know, there's like, how I kind of describe, there's a lot of concentric rings of use cases that you can use AI, AI automation for. And what we're seeing is people are sort of restricting the use cases for the AI bots where they're sort of fully automated, totally hands off to the use cases where there's sort of a very low need for user trust or where the sort of, Let me say that a different way. If, if it goes wrong, it's not catastrophically bad. So they deploy it for use cases like personal productivity or team based, like workflow, automation, things where, you know, if it's wrong or it's right only 9 out of 10 times, or it takes me, you know, maybe a couple days to like really work with the system to like do the prompt engineering to steer it towards getting maybe 95, 99% reliability that's acceptable because the risk of being wrong is just, you know, quite low. In order to get much higher up and expand the number of concentric rings to, you know, moderate risk to high risk deployment scenarios where we want these systems working autonomously, we're going to need. The main thing that is missing is user confidence in the exacting nature of what it can do and what it can't do. This is what Zapier, like Core Classic gives us, right? Is like it's A deterministic engine to execute automation. So you know, once you build it and set up, it's going to do the exact same thing every single time. But that's also what makes it fragile and hard to use. And on contrast, you know, these like AI core based LM systems that are totally autonomous have the opposite set of trade offs, right? They're much easier to use. You can steer them, guide them and fix them entirely through natural language. But because the accuracy is still inexact, confidence is low. And I think that's what ARC gets us. A solution to arc at 85 or 100% means that you've written a, again you've written a computer program that can generalize from like very simple core knowledge priors to solve with exacting accuracy, 100% reliability. These like sight unseen puzzles. And I think that that tool as a, that will be a new tool in like the programmer's toolkit basically in terms of building products and building systems that can achieve that same thing.

We're two weeks in I think to. When you launched ARC AGI Prize, what have you learned so far? What types of people are working and competing on this? Is it like the pedigreed researchers or the big labs? Is it scrappy hustler types like who's competing? How many teams are submitting solutions?

Yeah, let's see. So the response by the way after launch was phenomenal. It was much bigger than we expected. I think we were like trending on Twitter twice during the launch week. The number one Kaggle competition in the world, over a million social views, I think over all the launch channels. So just very phenomenal. I'm really thankful for everyone who helped sort of promote and help share ark. Hopefully we actually like can get a solution here in some, some short time. I think the like the most interesting thing about the folks that are working towards arc, it's probably a historical glance here and then what I've seen over the last two weeks. So the historical answer is most of the people that have worked on ARK are outsiders to the field. They like this is not actually the first year that there's ever been a contest about it. There was a past competition called arcathon. It was much smaller, it was hosted out of this Lab 42 AI lab in Switzerland. And so last year There was actually 300 teams that worked on trying to beat Arc and again no one had sort of beat it. And almost to my knowledge all of those teams were effectively individuals or effective, you know, outsiders in some way. They're not, you Know, know people at like big AI labs, they're folks with backgrounds in, you know, engineering, mechanical engineering or video game programming or physics, folks that just kind of like got curious and interested in the problem at hand. And I actually think that's more likely than not. Where the breakthrough for ARC is going to come from is I think it's going to come from an outsider, somebody who like somebody who just thinks a little bit differently or has a different set of life experience where they're able to like, you know, cross pollinate a couple like really important ideas across fields. That's one of the reasons why I put as much money as we did at arcprize. I felt like the progress was idea rate limited actually. And one of the best ways to sort of increase the amount of ideas is to try and blow up awareness, which is what the launch kind of did. Over the last two weeks I've kind of seen like two probably like camps of people, at least on Twitter, emerge. I think there's one camp of people who are sort of the, you know, they're in it for the mission. They agree with the underlying concept. They think that like we do need some new ideas. They're excited to like try and figure out what those are. And then there's like a second group of people that are sort of like, I'm going to prove you wrong. LLMs are definitely enough. Scale is definitely what we need. And I'm going to do my best to go like beat this benchmark just using like existing off the shelf technology and sort of prove you wrong. So I'm actually quite happy for both those games to exist.

One of those approaches is currently up in the leaderboards. Right.

So yeah, we can break some sort of news here. So this week, this Thursday, we're launching or I guess when this comes out, it'll have launched just a couple days in the past, a brand new public task leaderboard. So we talked about how ARC doesn't allow Internet access and there's compute limits. I know personally how unsatisfying that is to not be able to use frontier models though. I also want to know how good can GPT4O, how good can Cloudsonnet like do against this benchmark? And also because you know, no compute owner, it kind of also is a bit of a barrier to entry. Right. You have to use open source models. You have to do just like there's quite a bit of engineering work you have to do before you can even start just testing and experimenting. So we're going to be, we're we're launching a new public task leaderboard. It's going to be a secondary leaderboard. We're going to be committing about $150,000 for a reproducible fund towards this secondary leaderboard. It won't be officially part of the competition this year. You know, we want to, we want to like maintain that aspect of like assurance on you know, cheating and contamination overfitting with the private test set. And that's also the test set that has sort of the most empirical evidence against it over the last four years. But the secondary leaderboard is going to allow folks to basically submit scores towards it against the 400 public task set. And we'll verify and reproduce the scores locally to sort of ensure good fitting with the approach and we'll publish that. And you're right, I think the top score or one of the top scores on that is this guy Ryan Greenblatt. He came out a couple days after the competition launched with a pretty interesting novel approach actually towards beating it. And he's using GPT4. Oh and, but not, not just GPT4O. I think the interesting thing is he's, he created a, like an outer loop around 4.0 where he is using 4.0 to generate programs or sample from GPT4O. These like program these reasoning traces to beat the tasks or identify the patterns, then testing these patterns against attends against the demonstration set and then finding the one that works and applying it. And that approach seems to be getting in the low 40s, maybe 40%, 41% somewhere in that range. And it's pretty interesting because I think someone might look at that. I think and it sort of at first blush say well isn't that evidence that scale is all you need? And I do think there is something interesting there. Right. It's like it's showing that hey, the more training data these things have, the more sort of programs that they can spit out. That might be kind of right but also shows that I think that new ideas are needed still. Like this outer loop is novel. Like that might actually be frontier LLM reasoning that Ryan published. And we're going to make all the approach whenever we put similar to ARC Prize, we're going to like open source all the code for all the reproducible solutions so folks can, can take these and apply them and try to reproduce them and sort of using private closed or open. Close open open source models for the sort of the closed private data set. But yeah, I do think it's pretty interesting how much innovation you get when you, how Much information we've gotten over the last few weeks is just a result of putting even just the awareness against the public test.

Are the folks at the big research labs like, why are they not working towards this benchmark? Because it almost like when you explain the benchmark, it seems so clear that obviously this is the thing you want to solve. You don't want to solve the memorizing the textbook use case. Why do you think the folks at big research labs aren't trying to solve this benchmark? Or are they?

So I am aware of a handful of big AI labs that have tried in the past several years ago. So like, you know, this was perhaps at smaller scale with weaker models and things like that. I would. One of the things I would hope is that actually more do in the future actually love to see if there was. If we could make ARC AGI an actual measure on some of the model cards that get reported against future models. I think that would be like a really cool thing. We're willing to do it. So if anyone is listening to this and wants to reach out and make that happen, more than happy to work with them and find some way to do that. If I had to guess, well, let me say, let me not guess. And let's say what I have more sort of confidence in. I've been surveying. Once I got exposed to Francois work again and was sort of for deep, deep thinking deeply about ARC AGI, I started surveying a lot of my friends and researchers in SF in the Bay Area about had they heard of Francois and had they heard of arc. And Francois has pretty good name recognition because he's really, really big on Twitter. Been big on Twitter for many years. Probably nine out of 10 people I talked to like, knew who Francois chollet was. Maybe 1 in 10, 2 in 10 had heard about the ARC AGI eval, and probably half those were confused because there's like five other AI evals called arc and I had to like, sort of do some. So I had to like, you know, disambiguate with them. So I had really low awareness. This was one of the first things I asked Francois about when I, when I met him for the first time in person this year. I asked him, why do you think that is? Why do you think you have such high awareness, but ARC has such low awareness. And his answer effectively was that it's hard. You know, the way that benchmarks get, like, gain popularity and notoriety is we make progress towards them, right? Researchers are working against it. Somebody has an idea, they have a breakthrough, they publish that in a paper, that paper gets picked up and cited by others, that generates awareness and attention. Other researchers say, ooh, interesting. Okay, something might be possible now on this, like, really hard benchmark. And so you get the snowball effect, right, of like, attention. And because ARC has sort of endured with very low rates of progress, in fact, decelerating progress over the last four years, I think anyone kind of in the lab looking at that would just say, well, maybe the time's not right for you. Maybe we don't have the idea set in the world. Maybe we don't have the scale we need yet in order to sort of beat this thing. And it looks like a toy and it doesn't like, you know, I don't fully understand why I don't get the necessary importance, how it's qualitatively different. I haven't just spent that much time. We got a million other benchmarks I could use. And, you know, I think that's some of the somewhat of the dynamic that has existed in the past and, and is one of the, again, reasons why we launched ArcPrize. Right? I think there, there are some. There's lots of like, market tools you can use to like, shape markets and shape innovation. And I think prizes do have a narrow. There's a narrow spot where prizes can be, like, outrageously effective. And it's where, like, the idea is small and it's. And it's like idea rate limited. One person or a small team can make that breakthrough and it's very quickly and easily inspectable, reproducible and like built. And you can build on top of it very rapidly. And so all those sort of boxes got checked. And yeah, one of the reasons why I decided to go to Archmise and.

One, and you've mentioned curiosity around AI or AGI dating back to college that was sparked a few years ago in the context of Zapier and has kind of been nurtured ever since. Beyond the curiosity, I'm curious why this is important. Meaning if you could paint the picture of what life looks like for the world post AGI, where we've defined it as the ability to efficiently acquire new skills, what do you think that version.

Of the future looks like?

Why is this an important thing to solve?

The thing that I feel like I have a unique insight into at this point, having spent a lot of time thinking about ARC and this AGI definition is, I suspect the advent of AGI is going to look very differently than most people expect, especially of the group who are in the camp that like AGI is undefinable because it's so mythical and you know, scary or big or awesome that like we can't even hope to ever define. It's just going to be this like magical, special thing. And you know, it turns out like, you know, or something that I believe quite deeply is. The definitions are really important because definitions allow us to create benchmarks and benchmarks allow us as a society to measure progress and set goals towards things that we care about and want to happen. And this idea of efficiently acquiring skill, one of the, you know, we've talked about it a handful of times today, but one of the direct near term things that you get from that is you get systems that can do exacting accuracy, generalization from a small set of core priors and apply it towards novel solutions. That is again the number one problem that rate limits AI adoption for more real world use cases today. And so that's what you're going to see. You're going to see basically like the application layer of AI get like amazingly good at accuracy, consistency, low hallucination rates, which is going to allow us to use it in a much more unfettered way, in a much more trusted way because of the underlying way that in which it's built. So I think that's like the reason I think that's important, I think that's the reason why I think that's important is, you know, I think there's a lot of, we don't know what that set of capabilities is going to build on top of into the future. Right? There's lots of unknowns, I think, of how AI AGI evolves beyond the actual inception moment of a system that can efficiently acquire skill. But I think it's going to be a much more gradual and incremental rollout where there's a lot of contact with reality as we build and engineer these systems, which is going to give us as like a society a lot of time to update based on what those capabilities, what it can do, what it can't do, and make decisions at that point about how do we want to deploy this technology. Where might we as a society say we don't want to deploy for this set of use cases. I think that's one of the reasons why I've been such a proponent, I think of open source AGI progress with ArcPrize is like right now I think what I see happening is there's sort of this mythical story of a very bad outcome once we get to like super intelligence. Right? It's a very theoretical, driven story. It's not grounded in sort of empirical evidence. It's basically based on sort of reasoning our way to like this outcome. And I think the only way that we can really, really effectively and truly set good policy is by you have to look at what the systems can and can't do and then regulate or decide and make decisions at that point about what, what it can or can't do. I think anything else is sort of like, you know, you're cutting off potential really, really good futures way too early. And that's sort of what's happening I think with a lot of this early AI regulation where, you know, I'll try to, you know, be, be the paint the good side of this picture. It's like, hey, I care, you know, maybe the risk of this bad outcome is so high in the future that we should like pause here. I think the, the risk of that is you've, you've like trimmed off every possible possible good path of the future way too early. And the reason it's way too early is because we still need new ideas. We need new ideas from researchers, we need new ideas from students, we do new ideas from young people, we need new ideas from labs. Otherwise we're actually, there's a chance that we've never actually reached the high degree of useful AGI that we actually want. And so that's kind of my nuanced take, I think, on probably what the advent of AGI looks like. I think it's much less likely to be a moment in time and much more likely to be a stair step of technology that we build on on top of past technology. And that creates a lot of moments to sort of update beliefs based on what it can and can't do.

Do you have any predictions on when we'll cross 85% on ARC price?

You know, before the competition started, my, there was a, the first data scientist we hired at Zapier gave me this idea a long time ago and stuck with me. He, he said the longer it goes, the longer it goes. And so it's this idea that like the longer something takes, the more you should update your prior advantage. It's going to take longer to compete. So coming into this year my expectation was like hey, at least three or four years probably before we get to the grand prize mark, based on sort of like past, based on the past track record. Having seen what we've seen over the last like two or three weeks though, I think it is quite likely we get to 50% during this competition period. I would be very Surprised? I'd be surprised in a good way if we actually get to the 85% grand prize in this competition period. But I think it is not unlikely that we crest the 50% mark before the end of or middle of November, which is when the contest period ends for 2024.

And is there a good why now? Because people have been trying at this for five years now and you're galvanizing interest around it. And now a lot more researchers around the world are interested in AI and solving hard problems. But is there a good why now in terms of enabling techniques, technologies, et cetera? That's different now than than five years ago when Francois first kind of defined the benchmark.

If it is true that deep learning is an important part of the solution, right? A deep learning guided program synthesis engine or a DSL that is generated on the fly through deep learning technique. If that's true, the world has a lot of experience now on building and engineering and scaling such systems over the last three or four years. And there's a lot more compute online, which brings the cost down into a territory where some of these things may have just been out of practical cost before. For example, actually, Ryan Greenblatt's solution right now is maxing out our cost limits we're going to have against the public leaderboard costs $10,000 to generate the 8,000 reasoning sample traces from GPT4O that he then deterministically checks. And so that would have been a technique that would not have been possible, you know, three or four years ago in any way. It's guard. So I think if it is true that there's amount of like a minimum amount of scale that's necessary to be dark, I think, hey, we've gotten more of that in the last three or four years than we had when the first competition ran. And then I think the other why now? Is just can largely due to awareness. If it truly is. Actually, let me answer the opposite. Like, I think the risk that it is, the reason we launched ArcPrize is that it is actually not why now, right? It's like actually it's not going to happen is the problem. It's not a why now? It's not like, oh, the ideas are in the world. We just need to like get people to work on it. The risk is that it isn't. It's actually not why now? And not why now is, I think a much more interesting story right around this, like LM driven, you know, focused attention on sort of LM solutions. Only the closed research do the competitive dynamics, all of these things have like shifted and shaped attention away from the new ideas and towards scale, towards LLMs, towards like you know, know application layer AI. And I think that we think we need some shaping reshaping back towards the new idea set. So hopefully the why now is because ARC has now lots of attention to be seen.

Do you think LLMs will be part of the solution? I'm curious what you think of. It seems like in the big research labs right now, all of the frontier research is around. Let's merge kind of LLMs with the insights that you get from the inference, time, compute and the Q AlphaGo stuff. I'm curious what you think of that kind of direction of research.

There's some pretty interesting research I've come across with transformers. That transformers are capable as an architecture of representing very deep deductive reading change with 100% accuracy. I think this is interesting and the challenge with them is actually we just don't have the learning algorithm. Backpropagation is an ineffective learning algorithm in order to teach a transform architecture a set of network weights that can do deductive reasoning with 100% accuracy. And so I think it's possible that the systems that we kind of, or at least the core concepts that underlie language models have sufficient capability in order to do this type of reasoning. And we have not yet discovered the like algorithm that can train the model in the right way. We haven't quite discovered the right outer loop around the transformer that is going to do the program synthesis engine or the DSL generator. I feel more confident in saying that deep learning is almost certainly going to be a part of the grand prize in some way. I bet it won't. I'm pretty confident it wouldn't be just like a pure deterministic program is going to be how it gets solved. I think transformers are effectively the technology that has the highest degree of awareness in research like literature. There's a lot of hardware now that's going towards accelerating transformer based models. I think actually just earlier there was like an ASIC that got announced recently that's trying to accelerate like the transform architecture. So to the degree that actually like some degree of like scale or compute is necessary to be dark. I think those are like things that I would say I'm bullish on sort of the transform architecture though I would point out that the search base of alternative architectures is quite rich. Right. We've had maybe like nine or ten now mainline architectures from transformers to LSTMs, CNNs, RNN's XLSTM state space models. This would suggest that the search base of those architectures is quite rich actually and they all have like slightly different varying properties. So like I think it's certainly possible that someone comes up with some innovation there. I'm less confident or like bullish that LLMs in their exact form are going to be part of the 85% solution though I would think it probably like a sub component of the architecture instead of the entire application system itself.

When somebody does ultimately hit the 85% threshold, what do you hope they do with the solution? Like what would you like to see out of that person.

Other than submit it to the leaderboard? So this is one thing we didn't talk about a ton. But you know, one of our prize's goals is to accelerate open progress towards AGI. So we are going to be requiring that in order to claim the prize money you do have to publicly share and reproduce publicly share reproducible code and put it into the public domain. And this is, this goes for both the public leaderboard and the official competition leaderboard as well. You know, this is with the spirit of trying to re accelerate open progress again so that we have research in small ways out in public that other researchers can build on top of and hopefully stairstepper would actually, actually building AGI and not getting stuck in sort of the plateau that we're in today. So I think that's probably my first pickup. I've actually seen a handful of people online that have said, hey, if you've got a solution in rkgi, I'll give you this like million dollar offer. My company. Yeah, exactly. Which on one hand I'm like, okay, that's kind of interesting. But on the other hand I'm kind of like, I think that's great awareness and it shows actually the importance of solving this. I think more people are starting to become aware of the lack of progress, lack of frontier progress. I think ARC is kind of becoming a lightning rod for folks that want an actual measure towards this. I think growing sentiment in the field today.

Should we close out with some rapid fire questions?

Yeah, let's do it.

Okay. Who do you admire most in AI?

I mean, Francois Chalet is a bit of a copy answer. I wouldn't have co founded arcprize if I didn't admire and respect his work over the last four years. I mean I think the two people that I have learned the most from like directly and have inspired my own belief and work, Rich Sutton and Chile, both of them published papers in 2019. Right. Rich Sutton published the Bitter Lesson, which I think is fairly well known now in the industry at this point. I think his idea set is quite right there with maybe the one asterisk that, you know, the one aspect that has not yet had scale applied to it is architecture search itself. You know, we certainly have unbiased search and learning applied on the inference side and the training side, but every architecture still has a very human handcrafted story and journey to it, which is an important insight, I think about that. And then on Measure of Intelligence from Chollei in 2019 and I think both of these papers are in like, or I guess maybe Sentence was more of a blog post. But both of these pieces of writing I think are very important because history has proven them right as time has gone on. Language models Transformer Scale has sort of shown Sutton's ideas to be even more true than they were in 2019. And I think the Endurance of ARC has shown Francois definition of AGI to be more and more true as time has gone on.

What is your most contrarian point of view in AI?

I feel like everything we've talked about today, most people don't agree with me on, so all right, we'll count it. Scale is not all you need. New ideas are needed.

What's your favorite AI app other than Zapier?

Let me look and see. What do I have? I have a handful, I think. I'm not going to like surprise you with anything. So I've got ChatGPT, Perplexity and Claude and I'm a pain user of all three of those services. You know, one interesting thing actually, I'll add, I have gotten way more value out of language model based tooling over the last like call it six months than I ever did in the first aspect when I was starting to start working on it at Zapier. And it's because one of the things they're really, really good at, the thing they're perhaps like best at, is summarizing tons of unstructured text and helping be like an educational tutorial for you. So it's significantly like ramped up my learning rate on actually building with AI, learning these fundamental different architectures, starting to actually do model training. That's something zapierilizing done. But I've started to do myself over the last six months to get a sense of that type of work. And yeah, language models and AI tooling has definitely accelerated my learning process. Awesome.

All right, last question. Let's do something optimistic, something that we can all dream about. What change in the world are you most excited to see over the next five or ten years as a result of AI?

I think the. I've always wanted to like, live in the future. I think that's maybe something that has always driven me towards like, working on like frontier tech. I've always, you know, bought the latest gadget, always tried the latest app. I think it's led me to work on Zapier and AI and it's one of the reasons I'm working on AGI right now is because I think it's like the biggest thing that you can. I can potentially have an influence on trying to pull forward, pull forward that future. You know, I personally get, I think like one of the things that feels very limited from AI right now is that with the narrow form of AI that we have, if we never get to AGI, what that will mean is that we will always be rate limited on developing things by the human that's in the loop. And that means we'll never have AI systems that can invent and discover and sort of innovate alongside humans and really help pull forward and push forward the frontier in I think, a lot of really interesting ways, like, you know, understand more about the universe, invent, discover new pharmaceutical things, you know, and discover new physics, discover how to build AI. You know, I think we're always going to be rate limited by the human today. And I think if you just sort of care about living in the future and you want to pull forward the good aspects of the future, some form of AGI is necessary to do that. Awesome.

Thank you, Mike.

Thank you both for having me. It.