Tsahi Levent-Levi explains how WebRTC decides the video codec bitrate. WebRTC combines four constraint sources — the application's maximum bitrate setting, the network bandwidth estimation (congestion/packet loss/latency), the device's CPU capability, and the observed content complexity — and takes the minimum of these maximums as the encoder target bitrate. The encoder then tries to reach that target across frames, sometimes overshooting or undershooting due to variability.
How does WebRTC makes the decision of what bitrate to use for the video codec? It starts with the max bitrate of the application. The application makes a decision, a policy decision, of what should be the maximum bitrate, and that's going to be passed to WebRTC. The second value that WebRTC looks at is the bandwidth estimation. WebRTC sniffs the network check to see if there are any issues, if there's conjecture, congestion or corruption or issues, and packet losses and latencies in the network. Based on that, it is going to decide what is the estimated bit rate or bandwidth that is available for it. That's going to be another maximum target value that WebRTC is going to look at. Now, the same way that WebRTC is going to look at the network, it is going to look at the device itself. It's going to look at the CPU and see if there is enough resources and power there to encode the things that it wants to based on what it sees. For example, if the CPU is being overburdened and taxed already, or the CPU is not that new for the type of content that we're trying to encode, that the resolution and frame rate, it is going to decide to reduce the bandwidth or the bitrate that we want to encode. So the resources are going to be lowered so that the CPU can actually do what we asked it to do. The fourth thing is content. The type of content is going to affect what we're going to encode. If the image itself is static and there is nothing there, we're going to have a lower bit rate than what we want. Why? Because there is no need for that. Okay, so that's a dynamic decision that takes place there. Now, WebRTC would take all of these values. The minimum, the maximum bit rate for the application, the bandwidth estimation, what the CPU can do, and that type of content, and it's going to take the minimum value out of all of them. So we're going to take the minimum of all the maximum values. That value is going to be passed to the encoder as the target bitrate that the encoder needs to generate. The video encoder will try to hit that target, but it's not exact science. So it might overshoot or undershoot that value. If it undershoots in the next frame, it might try to overshoot because it has a higher budget. If it overshoots in one frame, it might go and give lower bitrate for the next one because it knows that it went above and beyond what it could. So that's how WebRTC handles the decisions of how much to put on the video encoder. If you want to learn more about WebRTC, check out my website at Blogic Me and follow me there. Thank you.
Meta rebuilt AV1 for mobile scale. Should smaller teams follow? | Chats with Arin
