In this interview, Arin (Speaker A) and Tsahi Levent-Levi discuss Meta’s recent blog post detailing how the company re‑engineered the AV1 video codec for massive mobile‑first deployments. They compare Meta’s deep customizations—such as a proprietary AV1 encoder that rivals H.264, asymmetric codec selection (AV1 inbound, H.264 outbound on lower‑end devices), battery‑aware switching, and a machine‑learning driven device‑capability detection system—to the earlier OpenAI WebRTC stack overhaul. Tsahi outlines the practical thresholds for building a custom WebRTC stack, emphasizing that only organizations with billions of monthly active users can justify the engineering and lifelong maintenance costs. The conversation then moves to the challenges of using AV1 in group calls, the role of software versus hardware decoders, and the broader industry adoption timeline. Finally, Tsahi offers actionable takeaways for smaller teams: adopt a multi‑codec strategy, start experimenting with AV1 where its strengths (low‑bitrate screen sharing, SVC) matter, and avoid postponing AV1 work until 2028 to prevent a massive backlog.
Chapters
All right, thanks for joining me today, Tsahi Levent-Levi. Last month on WebRTC Live, in our main interview, we learned from OpenAI how they re-architected their WebRTC stack by replacing SFUs with lightweight transceivers in order to better optimize their voice AI deployment at scale. Today, what you and I are going to talk about is how Meta has written a blog post about their adoption of the AV1 codec. Again, using some custom logic on encoding and decoding to better optimize their deployment at scale. Two big-scale deployments with a lot of custom logic in it. My opening question to you is, should we all be building our own custom WebRTC stacks now? What did you think of this Meta blog post?
The first thing is, it's interesting that you mentioned the OpenAI work because that's For me, it's totally different. What they've done wasn't to go into the innards of WebRTC, change it, and figure out what to do with it. It was working around WebRTC and saying, this is what we have, that's how the browser works. Now how do we make it work for us?
Yeah.
With our infrastructure scale and requirements. What Meta has done is something different. They said, well, everyone is coming, it's coming to WebRTC as well, which is nice. We use WebRTC. Which is nice, but we're mobile first, in a way mobile only in many, you know, in many aspects. So WhatsApp on my desktop still doesn't allow me to do phone calls, not audio, not video. You need to put their app, which means proprietary.
Right.
Messenger does allow that, doing a video call. So that's like browser and the rest is all native. So when they decide we're going to use AV1 and now what we're going to do is to look at the codec itself and do a bunch of optimizations or changes to it, this is first and foremost something that someone that doesn't use the browser or someone that relies on the browser for most of its calls cannot do, period. Yeah. Because you won't be able to go back to, you know, Google and tell them these are the changes of my code, please go upstream. You can try. Most of the time they won't like it, or you need to sign too many things. And that still needs to be in their main roadmap to make that new into the code. Now what they've done in many ways is to say we need AV1. We want that as the main codec for the coming years. At least that's how it looks like from the amount of work that they're doing. And it makes sense at their size. It's not only video conferencing. It's also live streaming and just file downloading, videos on Instagram and Facebook and wherever. So supporting AV1 means reducing bitrates, but now they need to do it in low CPU environments, so they need the CPU to be lower. And what they said, for example, is that they have their own implementation of AV1 encoder that is almost as good or even better than the H.264 software implementation. So that means that they did a lot of optimizations there to fit what they need. They probably decided not to use some of the tools in AV1, use others, use different heuristics together, which is fine, especially if it got them the better quality for lower bitrate.
Yeah.
Now, should you do that? How many people do you have in your organization dealing with ORTC today? If I say the number is below 4 experts, don't even try, right? Um, and then you need to maintain it for life.
Yeah, yeah, exactly.
If you have a billion plus subscribers, monthly active users, definitely go for it. Yeah, I wouldn't, I wouldn't think about it, I guess, not even in the 100 million active monthly users. So you wouldn't see a lot of the other vendors doing that even at that scale.
Right, right. But I think they're still interesting even though, you know, maybe we just have one viewer watching this who has billions of users on their platform, but for the rest of us and thousands, millions of users, right?
If you said that I have one encoder, a lot of decoders, then go invest in the encoder, not in the browser. Do it in the cloud. And that's done today. I mean, the first uses of AV1 were YouTube and such, or Netflix or whatever, where what you have is one encoder running in the cloud doing the actual CPU work in order to reduce bandwidth on the line so that they can send the same file to gazillions of people, usually not live, but you can do it live as well. And they can just decode AV1 and eat less bandwidth, so I need to pay less for that bandwidth. There's economic value there that you don't derive elsewhere. But that, for me, is not the WebRTC kind of— I'm going to do that in the browser. You do only the decoder, not the encoder. So it changes the whole story to something else.
Yeah, yeah. No, I think so. And for anyone who hasn't read the blog post, it's a, it's a long read and a long URL, but we've got it in the QR code up here. It's a really Good read. And I think there's some interesting things, even if you're not at billions of users, that we can learn from it for sure. One I wanted to call out is this asymmetric design that they did. So this is an image from the blog post here, and they built some advanced logic around the codec selection so that when a mid-range device is communicating with a high-end device, you can use different ones in different directions. So the mid-range device could receive AV1 from the high-end device but send H.264 as one of the things that they built in. And then they also had some similar type of logic around when the device battery is low, switching things to H.264 since AV1 consumes more battery. So I was curious, Tsahi, have you seen this kind of asymmetric innovation before?
Yes, it's not the first one that you see. We've seen it with VP9 earlier. Earlier on you had that as well. You see it with those that do HEVC. Okay, iOS devices can encode, some Chrome devices can only decode, so you're using HEVC one way. The whole thing of using AV1 with WebRTC means that you're going to go to dynamically be able to switch and adapt to the codecs that you're using because it's not— you can't do it in one fell swoop. Which you can do with H.264 and VP8 easily, VP9 also almost. But if you're using AV1, it's like, okay, can I use it for this scenario? And even if the answer is yes now, it might not be the same answer 2 minutes from now because of CPU issues, because someone else joined the same meeting and other dynamics that make changes that mean that you cannot use it anymore. So the first thing that you need to do with AV1 is say, well, okay, Let's pause for a moment. We're moving from a world of a single video codec at all times, doesn't matter which one you're using, you can switch from one to another, from one call to another call, to being able to run dynamically and switch codecs dynamically within the same session. Otherwise, don't even start using AV1. And then you see this additional optimization that says, well, I might be able to only decode and the other side might be able to encode. So let's go for that as well. So we're like squeezing the lemon of AV1 in each step there. And you see how Meta is thinking about that and what they're doing in each step and how they're trying to optimize. Many of the things there are like they're proprietary to them because you will not be able to do that. You don't have access to the actual encoder to be able to do that. But others like this one are things that make sense.
Yeah. Yeah, and as you said, these are kind of mobile-first optimizations where you have more control when your use case is mobile-first. And so, you know, if you're a browser-first application, some of these things are going to be harder for you to control. Another thing that Meta calls out in the blog post is they kind of refer to like group calls with AV1 as sort of the next frontier. To what degree can you use AV1 in group calls now? What are kind of the limitations around that?
Google Meet already does that. And in group calls, there's different mechanics because in group calls you've got a lot of small tiles. So a lot of, let's call it, small decoders and one big encoder. So if the encoder can do AV1, you're good. That's one thing. And on the decoder side, if you're passing the number 3, you can't use hardware decoder anyways. You're using software decoders. That means that anyone doing today multi-party calls and they're decoding H.264, VP8, VP9, they're doing it in software, all of the decoding side. So you might as well just do AV1. It might take you a bit more CPU, but for less pixels, depending on the layout that you're using.
Now, another interesting thing that we don't have time to dive into, but it's worth reading about in there, is how Meta knows what devices can support AV1 or not. And they have kind of a custom machine learning-driven algorithm that they use for that. But as a result of kind of the work that they've done, they say that AV1 now covers the majority of mobile devices and Messenger and WhatsApp calls. I'm curious, is that further along than you expected the industry to be at this point in terms of adoption?
It seems too optimistic if I look at my own home. Now it's on the total because it's not the same size that they have. And then you need to align it also with the fact that we have a kind of a— someone put the brakes on mobile devices in 2026, and this is going well into 2027. I wrote about that about 2 or 3 months ago, and since then it got a lot worse because you've got an increase in price points for all Apple devices, not including iPhones. All flagship devices either increase their price points or reduce the memory that you have in on the device. Yeah, so we've got, we've got an issue here. Would people switch to the new, stay with their current device, wait a year for that, wait for 2 years? Because most scenarios state that we have, we're going to have the same memory crunch into 2027. It's going to continue for several years now. So I'm not a fan of this, this approach of, well, we're going to have the best devices if we wait just a bit more longer, at least not in 2026.
Yeah, yeah, we may at least temporarily be hitting a wall on that. So, so most of these changes, you know, apply to the billion-user type of applications. But let's, let's sum up, uh, what's something that you think a a more typical WebRTC deployment can learn from Meta's blog post about this? I mean, you know, they're putting— they describe this specifically as a multi-year effort to do this AV1 optimization within their team, which most of us are not going to do that level. But yeah, what do you think? What's the takeaway for smaller teams?
I think it's time to start looking at multi-codec in WebRTC. Up until today, you know, we started 15 years ago, you and me, going into conferences, and the main discussions were, well, when is Internet Explorer going to support WebRTC? That was one, right? The other one was, is it going to be H.264 or VP8? Which one is going to be the mandatory-to-implement codec? Yeah, now we ended up with two, but that summed up in, you can pick whichever you want and use only one. And then you had all vendors pick either VP8 or H.264 depending on what they were trying to achieve, and that's about it. Yeah. VP9 came out, some used it, some haven't, and again, that's— it was okay to keep it that way. Now AV1 is coming, it is better to the extent that you actually really need to switch to that. Maybe not this year, probably next year. Definitely in 2 years' time, that's going to be— without that, you're going to lose market. So it's time to start going towards that path. And that means having parallel tracks. One is saying we're going to do more than a single codec. That's life. And the second is going to be how do we actually understand what AV1 is? Because AV1 means dealing with memory and CPU differently. It means thinking at which part of the scenario this is going to be really useful. Screen sharing with text on the screen, for example, and very low bitrates. These would be my main targets. Yeah, at least at the beginning. And then there's this small part of SVC, Scalable Video Coding, which AV1 comes with. If you haven't done VP9 yet, then you're not using that. And if you have used VP9, you might not have used SVC. So that's another, you know, another step in that ladder that you need to go through. And if you just wait until 2028, you're going to have all of that backlog to do in one fell swoop. Not, not a good idea.
All right. Well, as always, Tsahi, thank you for sharing your insights with us. To learn more about Tsahi's work, check him out at bloggeek.me. To learn more about future episodes of WebRTC Live, check us out at webrtc.ventures. Thank you, Tsahi.
