In this presentation, Tsahi Levent-Levi, a veteran of WebRTC and video conferencing, explains that peer‑to‑peer (P2P) media routing is optimal only when media stays strictly between two endpoints. He identifies three concrete triggers that compel a shift to a media‑server architecture: (1) recording, which typically requires a server to mix and store streams; (2) server‑side AI processing such as transcription, translation, vision, or object detection, which needs the media in the cloud; and (3) the addition of a third participant—human or AI bot—which breaks the pure P2P model. He discusses how the presence of these triggers determines whether a media server, an SFU, or a direct server termination is appropriate, noting that many AI‑driven solutions now terminate WebRTC at the edge without an SFU. Levent‑Levi advocates a “start with P2P, promote to a server only when a trigger fires” strategy, illustrating it with a dual‑connection approach used by GTT: two parallel connections (one direct P2P, one to a server) that seamlessly switch when a third party joins. He also suggests sharding architectures to separate one‑on‑one calls (P2P) from group calls (SFU) when the majority of traffic is bilateral. Finally, he offers a consulting service, “Tsahi in Your Corner,” to help teams make these architectural decisions.
Chapters
Adding one AI agent can end your peer-to-peer architecture. Now here's the thing. Everyone today is adding AI agents to whatever it is that they're doing. And while we need to remember that most calls today are one-on-one, once you add an agent to this call, that is going to kill the peer-to-peer architecture that you have. I'm Tachi Levent-Levi. I've got 15 years. Of, you know, experience covering WebRTC and over 20 in the video conferencing space. And this is where I think you're going to learn something new. There's going to be 3 triggers for you to switch from a peer-to-peer architecture to a media server one, and I'm going to give you one rule on how to do that. Stay with me. Peer-to-peer wins in exactly one scenario, and that's when the media stays between 2 endpoints, always. There is no media server that you need to pay for. There is one hop, so the latency is the lowest available, at least theoretically. And there is nothing in the middle that we need to trust. Even the TURN servers, if they are there to relay the media, get all the media encrypted, so they don't understand what goes on there in any case. Now break any one of these And you end up without peer-to-peer. Here are the 3 reasons that peer-to-peer might not work for your use case. There are 3 triggers and each one of them ends peer-to-peer. The first one is recording. When you want to record calls, you're going to usually do that with a media server. The media server is going to get that media and then mix that and record it and place it and store that media somewhere on the cloud. Usually you can use MediaRecorder API to do that. That's a client-side API that allows me to record the media that I have locally and then send it and upload it to the server. For most use cases, this is not really usable. When can I use it? In podcasts, for example, when all of the people in the session are managed in a way I know and control them, and I can even tell them don't close your browser when the session is over. Because we still need to upload your media. Otherwise, a media server means recording means that you need a media server. Then there is server-side AI that needs the actual media. Say you want to do transcription or translation and you don't have the models locally to run, you are going to send that media to the cloud. When you do that, you might as well put a media server in there and just skip the peer-to-peer architecture altogether. That also goes for things that relate to video, like vision and the object detection, things like that. The other thing, and the most common one, is if a third participant is joining the call. When you have a third wheel in your call, you likely need a media server. You can do mesh or peer-to-peer. I wouldn't bet on that. Now, 3 participants can be 3 humans, or let's say 2 humans and a bot, because again, the bot is another participant that adds media in that call. Each of these 3 is going to push you and put you from peer-to-peer into a solution and an architecture that will require something in the middle, a media server. So we're moving from peer-to-peer to something that requires a media server. Every one of the triggers that we've seen puts us in that media server path. Whether that server is an SFU, that's a second question. And today, many people skip that, especially when it comes to AI. An SFU is a media server that routes the media between many participants. In most situations today, when we talk about voice AI or AI in general in meetings, What happens is that we've got one person and one AI. There is nothing to route. There are no multiple participants. So I've got this user here connecting to the cloud to handle AI. Now, most solutions up until recently used an SFU to bridge that gap. OpenAI today terminates WebRTC at the edge. Using a transceiver. They're not using an SFU anymore. They started with an SFU and they stopped. And the question here is, where do you terminate? That's going to decide if you're going to use an SFU or not. For 1-2-AI, I would not go with an SFU, for example. I would skip that because if I do that, then I buy or build the wrong thing for my architecture and for my solution. So how do I make that selection? How do I decide, especially if what I want to do is peer-to-peer because I don't pay for media on that one? So the obvious idea that you should think of or lean towards is to start peer-to-peer and then to promote to something bigger, to a media server, only when a trigger actually fires. GTT, for example, ship such a solution. They have peer-to-peer in for one-on-one calls. It's a configuration parameter. What happens there, they open 2 separate peer connections when you have 2 people in the call, one to the media server and one direct between the peers. So I've got almost 2 calls at the same time. Media is routed only on one of them. So if I'm on a peer-to-peer call, the media goes direct. If a 3rd person joins That connection breaks and the one that is already open to the SFU starts writing the media. This gives us the impression that there is no delay in that switching, and we don't even perceive that. Other solutions might start with 2. When a 3rd person joins, move, move and shift towards the server, you might have a hiccup of a second or not, depending on how the solution was built. The best thing about that Is that when you do that, you don't have the media server in the path for the one-on-one calls that don't need that, which means no recording, no AI. Okay, why do we want that? Because in most scenarios, for most of the use cases that I've seen, the calls are one-on-one. If you go for telehealth, for example, that's one patient, one doctor. So think about that. Now, almost nobody builds such a solution. The reason why? It's too complex. That's one. Doing that is going to take more time, and the solution is already good enough. So if we have an SFU, a media server, and media runs between the 2 clients through the media server, most often than not we'd say, well, you know, it's good enough, that's fine. Sometimes we'll try to optimize the quality there, Unsuccessful in a lot of cases, and we will make do with what we have. So pick the approach that you need, think about what you need to do and what architecture fits best to your actual use cases and scenarios. So we've seen peer-to-peer, we've seen media servers and SFUs and the different approaches. Here's how I think about that and how— what I would ask my clients in one pass in order to decide on the solution. The first question, will media ever leave the 2 endpoints? If the answer is no, it's not going to be in the cloud in any way, I don't think recording, there's no AI that needs that media, it's only 2 participants, just go with peer-to-peer. If the answer is yes, we're going to need in the future or now more than 2 participants, we're going to record or something similar to that, Then we're going to put a media server now within that path. If there are many participants, yes, we're going to put an SFU. That's likely. If it's one person and an AI agent, that's a totally different story. There's going to be a media server, but not an SFU. SFU is counterproductive here. Now, if many of the calls are one-on-one and they don't need recording, That's where I would start thinking. If it's 80%, 90% of the calls like that, I would try to shard the solution into 2 parts. The part that does one-on-one and put that in peer-to-peer, and the one that does group calling. If I can't do that in a way that is good enough, then I will do the dynamic movement or shift from peer-to-peer to a group call solution, something that does that switch. How dynamic, how fluid, that depends on the use case and scenario that you have. In any case, I'm Tsahi Levent-Levi. You can come to me. There is a service that I give called Tsahi in Your Corner, where what I do is I come and I become part of the decision loop that you have for your architecture in WebRTC. Check out more in the description below.
