An unfiltered conversation with Alex Atallah, CEO of OpenRouter

And this was the first local model I ever used that I was like, wow, clearly we can actually get within shooting distance of the CLOZ models. And there is a way for indie developers to look like a company spending $100 million training a model, but with way less money and with just one computer.

You know, you're not really showing what the best model is, but more so like what model has the most traction. How representative do you think some of those numbers are?

We've experimented with a couple different complicated ways of ranking models, but we figured To start with, we would just do raw tokens in and tokens out. And that has a big con.

Alex, do you want to take us back? And I'm curious what the inspiration was to start OpenRouter, sort of what the conviction was, what was happening at the moment when you decided to create the project?

Yeah, so it started back in February or March when I was thinking about, uh, Llama had just come out and this was the first time we were kind of getting exposure to local models. This was Llama 1 and it was surprising how like useful it was, though it wasn't really like a great chatbot and it wasn't reliable as knowledge for knowledge retrieval, but it, it, there was something there that was worth exploring and I couldn't, I didn't quite know what it was. So, um, first project was that I started was a, uh, Chrome extension, very much following the pattern of like my current paradigm and, and seeing like, what about AI? Um, could benefit from it. My current paradigm was crypto because I was wrapping up my time at OpenSea. I was the co-founder and CTO, still on the board, but I figured I wanted to look for something zero-to-one again and try to explore the AI space and figure out what sort of interesting intersections would appear. So I worked on a project similar to a Web3 wallet, but for bringing language models to the browser. And it was a standard for frontend developers to just build API and model agnostic apps. And that was the experiment. And what became clear is that there weren't that many closed models at the time. It was really just like GPT-3, GPT-4 had just come out and Cohere. And, uh, And there were a few open source models that were really tough to use. And then Llama came out. And after Llama, a group, a research group took Llama and generated a bunch of synthetic data and then fine-tuned Llama on that data and spent about $600 in the whole process. This was a group at Stanford and they created Alpaca. And this was the first local model I ever used that I was like, wow, this is, this is huge. This is something I like. Clearly we can actually get within shooting distance of the closed models. And there is a way for like indie developers to look like a company spending $100 million training a model, but with way less money and with just one computer. And I thought that was huge. It meant that maybe we'd have a world with thousands or tens of thousands or hundreds of thousands of models. And, uh, another takeaway from this is that the data becomes more of the moat and data becomes like a more critical part of what makes a new model unique and a new model useful where a closed or centralized model wouldn't be as good. You know, even if a new model is just not as smart, even if we have like, you know, take a person who's not as smart as somebody else, but that one person just knows a whole bunch of stuff. They just have all of these secrets locked in them, um, or they've been exposed to, like, they know how something works. The, the smart person's not gonna be able to just figure it out. There's just knowledge they're missing. And that is something that, like, you know, empowers the like, or allows people to take data that's— there's no way of selling this data and find a way to monetize it. So it creates a new economy. And I thought, well, you know, we might need a marketplace then for models. And the closest thing at the time was Hugging Face, but you couldn't really use the models. Inference was tricky and often left up to the developer. And it was very hard to find the models to like figure out which ones would be good at different things. So that's why I started OpenRouter. Actually, the, the, one of the guys who was building like the, one of the top browser extension frameworks called Plasmo, which I used to build WindowAI, the Chrome extension for local models. He ended up joining as a co-founder and we started OpenRouter together.

I love that story. And how, for folks who haven't used OpenRouter, Um, like what's the basic pitch of like why as a, as a developer, do I want to use OpenRouter? Um, and also like who is the, does it tend to be skew more indie developer persona today? Or is it like, you know, you also have large Fortune 500 companies or whatever, like larger companies, um, who are building on, on OpenRouter?

So today it, it skews indie developer for sure. Um, and there are, there are a couple of companies that use OpenRouter. To do benchmarking where they run a whole bunch of tests against new models that just emerge. And OpenRouter is the easiest API for doing that. The primary developer use cases that we see are people building B2C apps. The top ones are either games or role-playing apps or novel writing assistants. They involve Probably the very top one involves mostly generating code and that code gets rendered into top experiences. And then we have some that just help users generate code. They're more programming assistants. So it's really a big mix of lots of stuff. And we try to, developers can opt into sharing their data with us and we give them a discount. If they do. And that data we just use to classify prompts coming in and figure out which models are good at what. And we'll do, you know, we're working on a router and some ideas for creating a router to help people find the best model. And we're sort of shipping new experiments all the time. But the primary use that people have for OpenRouter is they know what model they want to use, but they don't know where to get it from. Or they, you know, they want to explore models and they want to figure out which model is really good at finance or really good at roleplay or really good at programming or really good at machine translation. And we just help them find the models that people are really using a lot for those cases. It's not really an eval. It's more of a, you know, App Annie style, like engagement metric that we try to show for every model.

I have so many random follow-up questions to this, but I'll just, I'll just ask one of them right now, which is how, um, based on that metric of, you know, you're not really showing what the best model is, but more so like what model has the most traction across a specific domain. Similar to that, like, how representative— and I was looking through some of the leaderboards, um, before we jumped on this call just to see where some of the models were— how representative do you think some of those numbers are, like, with actual production usage, or maybe, maybe more broadly, like developer mindshare? Like, and the part of— for folks who haven't seen the leaderboards, they should go look because it's interesting to see what's showing up at the top. And right now showing up at the top are some of the Claude models. Um, and I'm curious if that's, you know, obviously we've seen more people using Claude models, uh, because of the recent releases, but do you think it's like actually representative of mindshare or some, some other metrics or proxies?

Yeah, it's a good question. We've experimented with a couple different complicated ways of ranking models, and we're zeroing in on one that'll come in the future. But we figured to start with, we would just do raw tokens in and tokens out. And that has a big con. The big con, uh, the disadvantage is that one developer who's just going bananas can skew the data for some model. And they do have to pay a lot to skew that data. So it's not like— I'm not aware of anyone trying to game a model or trying to like game the rankings. But it's not ideal. I think like a closer ideal would be something like retention, where of all the people who like test a model, how often are they coming back with that same kind of prompt? That, I mean, that's kind of what we look for in just analyzing websites. You don't look at raw traffic. You do sometimes. It's the simple metric. But what really matters is like, are people sticking and coming back? So we're going to experiment with some of those other ways of ranking models soon.

Alex, I guess kind of hitting on the creation of OpenRouter, curious to hear kind of from a technical from a technical perspective, what issues or roadblocks you faced when, you know, creating the platform itself, and then, you know, how you're able to kind of combat some of those.

There have been a couple. One was routing speed. We wanted to build something that ideally would have almost zero latency impact on the process of routing through OpenRouter to the language model that you want to use. And to do that, we, we ended up moving more and more of our logic and infrastructure to the edge. And Cloudflare is just huge here. I, I don't think people realize how many things Cloudflare offers. And I'm not, you know, really familiar with Cloudflare's competitors, but we, we ended up like leveraging a lot of Cloudflare's newer features to reduce the latency of our routing process down as small as we could. And really sort of like advanced types of caching that we put in place. And we like, I'll give an example. Cloudflare has a cool feature called Hyperdrive that lets you execute SQL that is cached in the edge region. Of the user executing the SQL. And you can kind of connect Hyperdrive to your database, and ideally you locate your database as close to the edge center as possible. But we found Hyperdrive to be really, really effective in reducing routing latency, and just any sort of database work that we have to do is like almost zero, zero latency, um, because of, of things like Hyperdrive in combination with a couple other components. So that was one. And we've been spending, I mean, it's a continuous project. It's never going to be fully over. There's all kinds of caching that we're moving closer and closer to users. But I haven't heard anybody complain about routing latency in many, many, many months. Another technical challenge has been making analytics that scale. And we've, like, the first thing we did wasn't terrible, but it definitely is not going to scale. And that was building our own sort of in-Postgres analytics tooling to do all of our analytics in Postgres and just move as much into Postgres as possible. The Postgres ecosystem, similar to Cloudflare, seems to be a bit underrated. There are a lot of amazing Postgres extensions and having a lot of our very stateful logic in the Postgres ecosystem has been pretty interesting and allows us to feel like there's just a lot of integrity in our data. Also allows us to leverage triggers, which have scaled very well for us, actually. So when things happen in OpenRouter, Postgres triggers, like, update a lot of the analytics tables that we have, or update a lot of our analytics logic. And we still have, we currently have cron jobs doing analytics that are going to go away eventually. We're looking at Timescale. DB as one, like, as like the next gen of all of our analytics, which will unlock tons of other cool ways of ranking models and is also just a lot more scalable for managing a really massive dataset like we have now. So those have been exciting. And in the more like AI sector of technical challenges, we integrate a lot of different APIs and we host a few models ourselves. Every time a new LLM provider comes up, its API is never the same as all the others. There's always some weird edge case that we see pop up at some point. I think one thing that's worked to our advantage a bit is that we have this community of power users. They're not developers. They're not companies. They're just normal users who love LLMs. And they, they like have an OpenRouter account that they connect to apps that let you bring your own key. There's also a way of using OpenRouter to sign in with OpenRouter, like to do an OAuth into some apps. I'm not familiar with like another way of doing OAuth with LLMs. So this community of power users has been really good. For just teaching us things. When new models pop up or when we start, when we host a new model or when we start integrating a new API, they'll discover these very niche Finnish reasons that come out or strange errors that will emerge that we get immediate alerts for and then we can fix. And then I think a last answer that I would have that just reminded me of. Since we're aggregating so many models and so many APIs, we decided early on to get really crazy about type safety. So we have extremely strict type checking across the whole codebase. And we sort of believe that as like a foundational engineering principle, which is new. We, you know, I haven't done that at another company before, and I haven't seen it really thought through as an engineering principle. But by making it really serious for us, we just catch a lot of errors before they ever happen. And it's like, for us, it's almost an imperative because there are just so many different schemas that we're working with all the time. And there's so many APIs and so many different formats for models. And Getting like really, really good alerts and really good error reporting kind of necessitates us knowing exactly the date, the shape of the data going in and out of every pipe in the machine.

Alex, I love that. A couple of, a couple of quick reactions. One, the finish reason PTSD thing. I was just having flashbacks of all the, of all the horrible finish reasons that are possible that are oftentimes not fully documented. So a lot of empathy for that. I also think your comment about solving— I, I think that it's a very underappreciated problem space, this like analytics problem for LLMs because of the volume of queries that you're often working with and like the size of the data. It's like a, another order of magnitude where like trying to deal with these problems at OpenAI and it's just really hard. And like most of the off-the-shelf analytics tools, like don't scale up to the, the order of magnitude that you might actually see with with some of these analytics things. I'm curious though, like your comment about this standardization or lack of standardization, it does seem that there's some amount of momentum around people standardizing on the way that OpenAI has built their API. And you actually see this like even more so with the SDKs where like Together AI and a bunch of other providers don't even make their own SDKs. They just wrap them directly as part of OpenAI's SDK. I'm curious to get your reaction to this. Where you think that's going. Shout out to the folks at Stainless who make the OpenAI SDK, and obviously in conjunction with the team at OpenAI, and have done an incredible job. They also do Anthropic's SDK and actually the Cloudflare SDK as well. So they're sort of helping some of this. But it's been interesting to see how— and maybe just as one other comment to get your perspective on, like, the how potentially conversely things happened in, um, in like the NFT blockchain ecosystem where it seems like, like all the different chains have like completely independent toolchains and like everything is, um, hard to do. It's hard to deploy projects to multiple chains. Like none of that really works super well. I'm curious.

Yeah.

Why people have been so willing to standardize around OpenAI, which seems like an interesting decision.

At a high level, I think there's, there's a healthy duality in both spaces. There's a healthy duality between standards and standard breakers, or, you know, you might call it like innovation generously and standard breakers less generously. Standards in either space, they can be good for consumers. Because they make it easier for new entrants to get adoption and they lower the barrier of entry for them. So it makes it easier for developers to, to switch their code over to a new language model or new API. And that increases competition, which increases quality and, and reduces costs for consumers. Standards can also be bad for the consumer too. If nobody tries to like improve upon the standard and everyone just like locks in to the way things are, it becomes really hard to like, for a company to get any kind of traction, uh, from like deviating or making a breaking change. OpenAI had the first attempt that I'm aware of at standardizing something about the language model communication process with ChatML, which for anyone listening is a way of structuring a multi-turn conversation with an AI where a user says something, the AI responds back, the user responds with something else, and maybe there's a, you know, overall context for the whole conversation called a system, the system prompts. And it's really a simple standard. And it's very extensible. And it was the start of something easy to set up. And I mean, correct me if I'm wrong, I'm not aware of an earlier attempt at standardizing something about the prompt. So ChatML was really interesting to us and it was part of the reason we decided that our API was going to look very similar to OpenAI's and just be a superset. And I think, you know, some of that mentality probably meant a lot of other companies thought the same thing and started implementing things that looked like OpenAI's standard. And this is like, I think, a really healthy standard because if you want to deviate, you can still just send a raw prompt. There's like basically an easy opt-out for people who are trying to innovate on their prompt format. And there's tons of people who do. There's basically like a really clear pathway for a developer to make a model with like a really unique, interesting prompt format. An example is OpenChat, which I believe at some point, I think they were training on, this was a research model and they were training on synthetic data, and they found that the model performed better, uh, if these, if the assistant's name was like GPT-4 response. If, like, if the assistant thought it was GPT-4, this researcher found that the performance improved. So basically, like, doing these little tweaks to the prompt format was one way the model developer community has chosen to innovate and deviate from the standard. And they're not hosed. They can still plug in to a lot of the existing tools. And Hugging Face now has a really flexible standard for tokenizer in the tokenizer config for the chat format that lets you use Jinja, which is a templating language for really, really complex chatML formats. So back to the overall picture, I think like crypto, crypto is just very different, in part because it's financial fundamentally. Within like the Ethereum ecosystem and all the Layer 2s on top of it, conforming to the same RPC standard as all the other miners just allows you to participate and earn mining rewards. So there's like direct financial incentive to adhere to the standard. And there's direct, there's direct financial disincentive to deviate, which is really tough. If you want to, like, deviate from the standard in crypto at that level, you kind of have to create a new blockchain. And that's why every new blockchain, not everyone, but a lot of them have very different APIs for interacting with them as, as clients, or as miners. And I think that's just why things work differently in that space.

Yeah, just a quick reaction, Nolan, and then I'll kick it to you for a question, which is when ChatML was coming out, Greg Brockman was incredibly gung-ho about ChatML and was like, you know, we're going to build this standard. It's, I think, the original blog post when we released the Chat Completions API was accompanied by us saying like, you know, I actually think the original iteration of that blog post was more of like, we're building this open standard, yada yada yada. And I think it got sort of tampered down to like, you know, we're putting out this product and we're also releasing the spec of ChatML so that people can understand. And then sort of almost immediately, even we diverged from what the standard was. And like we sort of had this reference implementation that was available to the world with ChatML. And, you know, what the path that we were on was like not actually keeping that, you know, quote unquote standard up to date. So I think this is a good reminder. I'm sure Greg's not listening to the podcast, but Greg, someone Someone needs to ping Greg. Greg's coding somewhere and putting out tweets about fixing machine learning optimization problem code issues. Somebody's got to ping Greg and remind him that they need to— OpenAI should do some active work to actually keep that standard somewhat up to date because it hasn't been maintained and it was actually removed from a bunch of the stuff that they had put out. So it's a good reminder.

I'm curious, are there other parts of the API standard that drive more adoption of it in your eyes?

I think it's part of my mental model is it was less that like OpenAI sort of figured out the right level of abstraction, which they might have. And like, I don't know if I actually have the intuition to know if that's true or not, because I think people just became— it became so clear that OpenAI was so far ahead that like Essentially, if you wanted any of those users to be on your platform, you were required basically to, especially in the world where like everyone is experimenting always, like you were essentially required to use that spec and the spec specifically of the SDKs so that developers didn't have to switch over to something else. And it just, from my perspective, like I think it's obviously good for OpenAI, but it's interesting how it really was like the first like the first iteration. And it seems like to a certain extent, like the first iteration without any level of incremental innovation is like the thing that people are sticking with as like, oh no, you know, don't make a different thing. Like, let's stick with the one that OpenAI came up with, like literally on the first shot. And that just seems, um, yeah, it'll be interesting to see like how, how much that plays out over time.

Yeah, we, I mean, our view at OpenRouter has been like make every model work with the OpenAI API formats, but provide extensibility and advanced features that deviate from the standard but aren't breaking changes. For example, like our model, you know, normally the model field in your request is just one single string with the model that you want to request. OpenRouter, it can be an array of models. And we have different kinds of prompt transforms that you can configure. One of them we call middle-out, which allows you to, a little bit of a play on the Silicon Valley TV show joke, take your prompt. And there's research that LLMs mostly pay attention to the beginning and end of very, very large texts and pay less attention to the text in the middle. So you can have, if, oh, you know, if your prompt is too large for a model, OpenRouter can like squeeze it and remove parts from the middle in like a strategic way. So we add different things like that to our API, but, but try to like, our overall goal is to reduce switching costs between models. So it has to you know, we try to make it really, really easy to let developers like experiment with new ones.

Alex, my question kind of ties back to your statement around the power users that you have currently. I guess kind of curious as far as use cases, as far as what those power users are using this tool for, and then kind of, you know, higher level question too, when you're originally rolling out with OpenRouter, you know, your thesis as far as who would be the end users, did that stay true or was there kind of a shift in personas as you kind of gradually rolled it out and got more and more feedback? I'm kind of curious to see where that stands And again, kind of how it changed throughout the evolution of OpenRouter itself.

We have seen more of our usage in terms of tokens and dollar volume move towards developers over time. And it's a little bit hard to know this question for sure, but we still have a good chunk, probably more than half of our users are just prosumers or power users or people using OpenRouter directly. And yeah, we also have like a playground where you can, where people can experiment with models and test new ones out. And the playground is often used, or it's used by some people as just like a consumer experience for directly chatting with models and saving your history with all of them in one place, but locally and privately on your device. So we've been seeing the ratio, we think roughly not change for, you know, in terms of users versus developers over time, but developers have been getting more serious and more active and spending more over time.

Yeah, Alex Atallah, I have a follow-up on this. And I'm even more grateful that we're having this conversation because I think like this is to a large extent some of the, almost the exact situation that the product that I work on, you know, many hours a day, is sort of in the position of Google AI Studio. Like, there's a large consumer footprint for Google AI Studio because it's an easy way for folks to sort of get access to the models for free and try out what's latest. But like, the core market that we're going after is developers. And I'm curious, like, how you've sort of navigated the world of like, you can't do everything, um, and you ultimately have to make those trade-offs, and like, how you've made how you've thought about the trade-offs between like, we want this thing to be for developers potentially, or, um, but we also still like don't want to actively make a bad product for, for the sort of consumer-oriented, uh, user persona.

We struggle with this question all the time. Honestly, it is like, uh, uh, or I, I at least like think about it all the time. I think the, what the, the biggest source of the tension between building for users and building for developers, at least to me, is differentiation. Just being different than everything else out there or different in a way that really works for us and feels like the OpenRouter thing to do. The answer to this question for us right now is building a platform that works for developers and building a marketplace that allows people to discover and explore new models with ease. And, and the people doing that are not just developers, um, but we, we've been— we just, we launched like a new marketplace UI last week, um, that lets you, you know, see all the language models that we have, but figure out which ones are free, which ones are really popular on finance or legal topics, which ones support JSON output, which ones support tool calling, and, uh, and then sorting all of them by price or looking at which ones are new, um, sorting them all by token usage in the last week, giving people like really powerful tools for looking up models that they want to use, um, and for exploring like how models rank and compare with each other, is sort of like the big, not purely developer product that we spend a lot of time on. The Playground, which we're going to be, which we've been upgrading a lot recently, is like a really easy to use developer tool for models and it looks a little consumery. But I think like the developers for language models are like bringing a lot of value that just the, you know, the chat room or playground is not going to bring. And we want them to feel like they're welcome. So we've been, we've been sort of building a lot of developer platform and performance improvements, uh, recently.

This is kind of a high-level question too, but I'm curious, and again, kind of over the past, I mean, close to a year and a half, not 2 years now, again, we got kind of go back to all of this AI hype. And Alex Atallah, I'd love to get your feedback or perspective, you know, being at OpenSea when NFT and crypto was kind of on that bull run as well. You know, what similarities you see between the AI hype and kind of those NFT crypto hype? And, you know, kind of where do you see that space going as we continue to move further and further up that kind of chart?

And maybe as an additional add-on to this, like anything that you have thought about doing differently with OpenSea to like, or with OpenRouter this time to sort of make it resilient to like more resilient perhaps to like, you know, if we are in some crazy hype bull run with AI right now and it sort of flatlines in the future, like is there still a feasible way to make, you know, the business or the project work, however you want to frame it?

With OpenRadar, we haven't really been building that many hedges against like the AI market into the product. Like, you know, if the AI market flatlines or declines, I expect we will too. And I think that's just part of the bet. It is a bet that like when some part of the language model space is growing, a rising tide lifts all boats. And I expect other parts of the language model space, or the model space, to grow too. We just focus on language models today, but others will come in the future. So, and examples of this, and this is similar to this part, I'll, I'll draw similarity to crypto in a second. If you look at the rankings page on OpenRouter today, um, there are a couple moments where it just jumps up significantly. You know, a recent moment like that was the launch of Claude, or actually a better example is the launch of Claude 3, which happened in early March of this year. And you see a lot of people trying out Claude 3, just tons of tokens processed. And then you start to see open source models grow a lot as well. Like they follow it up. Yeah, it's kind of one of the mysteries of life why that happens. I think like one theory is that the like interest that Claude brought to the ecosystem from developers who just weren't getting something out of it before, or for developers who wanted to achieve something and now could achieve it in a low-cost way. It just made people look at the other things available. It just made people look around to see, wow, I thought I knew AI, but then this new thing came out. What if there's something else new that I don't know about that is also underrated or undiscovered? There's something about developers everywhere, not just in AI, where I think this just happens all the time. Like, if a new thing emerges, you start to wonder if you were overlooking the whole space and if there are other new things that are just hiding in corners and just need a light shown on them. And this happens in crypto too. Like, there's very— and we made a lot of A lot of inside jokes at OpenSea about rising tide lifts all boats because everything is sea-themed. And we would look at the analytics of the space and every time a new project emerged, maybe it was like a new blockchain like Polygon, you could just see a ton of other projects kind of similar to that project. Or other creators or apps start to just get more interest and get more traction and sort of follow the lead of the breakout project. Additionally, when crypto, like when prices are going up, just fungible tokens like Bitcoin and ETH, when the prices are going up, NFT values go up too, and like they lag a bit and they're not super correlated. But I think it's partly because, you know, people like have more crypto liquidity and are looking for things to do with them. And NFTs are one of the ways that you can use your crypto that's like easiest and most like visually and digitally exciting. So just like the whole rising tide lifts all boats phenomenon, I love seeing it happen and I just started to follow it around everywhere and think about what it means. And I guess that's one of the biggest commonalities between the two spaces too.

Yeah, I love that. I know we're almost coming up on the end of time, so I want to be respectful of your time, Alex. The two questions that we always close with, and I'll ask the first one and then I'll kick it to Nolan. Just curious what your personal AI tech stack looks like. And this could be from a tool perspective, like what are you using? What's getting you excited from an actual tool, consumer tool perspective?

Yeah, I've recently started to get more into SuperMaven as my tool for code generation. I've just found it to be so much faster than competitors. I've been kind of blown away. It's really hard to know, but it feels like it has more of my code in context when it generates code. And Creator seems really smart. And it's also a New York-based team. I'm in New York. And they just launched the ability to chat with models, with different kinds of models with your code. And I've started sort of like a new workflow where I'll write the code that I want and then I just ask OpenAI or Claude or Gemini to generate tests that just like show me which tests thinks I should write, which unit tests would be good for the code I just wrote. And I accept most of them. They've been getting better over time. And if enough code is in context, they improve exponentially. So, I write more tests now than I ever have in my life because I'm not actually writing them. And the tests, I read it and check over. And I mean, the end result is so much better. So that's been exciting. Another one that I've been using is, I mean, I'm an angel investor in Devin by Cognition and we've hooked Devin up to our codebase and Slack and it's improving and particularly good at research-driven tasks that really, really need to use explore things on the internet, Devin is like the most powerful scraper I've ever seen. And consumer-wise, I mean, I switch back and forth between, like for just knowledge retrieval, I sort of bounce around to everything partly because I enjoy bouncing around. I bounce between like OpenRouter's Playground and Claude and Perplexity and ChatGPT directly. As my primary like knowledge retrieval places. I think there's definitely in the future, like it feels like there's gonna be some more UI differentiation that every platform does. The UIs all started the same and have gradually diverged and tried to like keep users locked in with some unique thing. I'm guessing that's gonna continue.

And Alex, our final question we like to end it on is, uh, already 6 and a half months into 2024, we like to get the perspective of our folks we speak with as far as, you know, something that you're looking forward to or hope that happens within the next 6 to 7 months, maybe early into 2025. And then on the flip side, uh, you know, kind of given everything that's going on, something that you hope does not happen, whether it has a negative effect on personally, society, politics as a whole, etc. Kind of the two-sided questions, uh, as far as again what you're looking forward to and something that maybe you're not.

Starting with something that I'm not, or something that I'm worried about, uh, I'm really worried about Taiwan. It doesn't feel like the AI industry or even the computer industry has a great plan for removing the dependency on all kinds of industry going on there, but primarily TSMC. Like it is like that kind of, that kind of like critical component, basically not having an alternative or not having like a great plan for what happens if, if Taiwan, you know, if we can no longer depend on our supply chain in Taiwan, it would be so disastrous. And it's literally being verbally threatened all the time. I just haven't seen like something that's calmed me down about it. So that's one thing I think about and worry about. And in terms of like things I'm looking forward to, there's a lot of stuff. I'm, you know, I, I, first thing that pops into my mind is a new architecture for language models and, uh, particularly one that can do search. Um, maybe, maybe that's, yeah, reasoning would be maybe a subset or maybe I'm misunderstanding how some language models are, are trying to approach reason. But it seems kind of odd that the search process, just like, like doing inference and then revising your answer over and over again, is being handled by apps and agents today. Uh, you, you get like a huge performance boost from generating code with any of the language models and then saying, do it better. Or like, you know, prompting the model to like revise the code they wrote and improve it in some way. And you don't even have to explain how you want it improved, just think twice, not just once. Give me your best answer, not just the first answer that you thought of. And moving this sort of search process or multiple inference passes inside the model itself, I expect would be wild, wild improvement. So I'm excited for that to happen somehow, or some kind of new architecture that allows for the model to actually think a bit.

I love that. I completely agree on both of those fronts. Alex, this was a ton of fun. I wish that we had another hour to chat about stuff. Super excited about OpenRouter. It was awesome to hear your perspective. And yeah, looking forward to hopefully all the cool new improvements that y'all are working on.