HelixML Walkthrough: From WebRTC, to WebSockets, to JPEGs 🙀

In this 45‑minute walkthrough, Tsahi Levent‑Levi (owner of Blogic.me and WebRTC Courses) uses two HelixML blog posts as a case study to dissect real‑time remote‑desktop streaming for AI agents. He begins by describing his mentorship of high‑school groups building various communication projects (WebRTC‑like, TCP/UDP video, JPEG frame streams) and why the HelixML solution caught his attention. He then walks through a series of takeaways: (1) “If it works, don’t fix it” – acknowledging that HelixML’s service meets its own needs but may not be a universal pattern; (2) a deep technical comparison of WebRTC versus a custom WebSocket + WebCodecs pipeline, covering NAT traversal, TURN/STUN/ICE, jitter buffers, adaptive bitrate, packet‑loss recovery, latency, and debugging ergonomics; (3) an examination of HelixML’s implementation details – raw H.264 NAL units over TCP, audio‑video synchronization without a jitter buffer, and the consequences for latency and quality; (4) the realization that fixed bitrate, lack of adaptive mechanisms, and TCP head‑of‑line blocking cause failures on variable or enterprise networks; (5) the eventual shift to a JPEG‑snapshot fallback, dramatically reducing bandwidth (100‑500 kbps vs 20‑40 Mbps) and simplifying the stack, while still providing graceful degradation for poor connections; (6) broader lessons about knowing the target audience and edge cases, conducting a “this‑is‑like” feasibility game, up‑skilling before optimization, measuring before optimizing, and managing developer motivation. Throughout, Tsahi interleaves practical advice (e.g., use existing services like Zoom/Teams as reference, leverage rtcstats.com for WebRTC debugging) and personal reflections from mentoring. The session ends with a call to action for viewers to reach out for further guidance.
Chapters
Hi, my name is Tsahi Levent-Levi. I'm the owner of Blogic.me and WebRTC Courses, and I'm here to break down real-time communications so you don't have to. Now today I want to try something new, um, kind of a service walkthrough where what I'm going to do is look at the vendor and what he did and analyze that for you so you can learn from that as well. Before I start, I want to begin from why, where I got this, this idea from. Uh, this year I'm a mentor to 9 different groups, and these groups are kids, school kids in 12th grade, so they are 17 years old. Each group is 2 kids where they've learned in the last 4 years computer software, and they are developing with a lot of focus around communications and security. In the last year, which is this one, they need to actually write a project, and that project is something that they choose out of a menu of different projects. One group that I have does compilers, two of them do kind of a TorChat solution, so distributed chat with security and privacy, and the other six are doing a kind of a TeamViewer service, which means screen sharing between two machines, one is remote. Being someone that is bored quickly, 6 groups doing the same things means that it's not that fun, for me at least. So for each group, I decided to let them flow with what it is that they want and then guide them to specific areas that I think would best fit what they need. So I've got groups that are using something like WebRTC, not exactly WebRTC, they don't have enough time for that and they don't need to use browsers. They write these things in C#, C++. C++ or Python. Um, so applications that run full video encoding with H.264, VP8, or VP9, and then run it over TCP or UDP with or without a server to route that through that server. And I've got one or two groups doing JPEGs, so they're sending JPEGs as images one after the other, so you've got a video out of that. And each one has a different focus and different area that look at some look at the quality, some at the network, different areas. Okay. Um, so it's interesting to see. And when I saw these two posts coming from HelixML, I knew that there was something there that pissed me off. Uh, that's because there are certain things that need to be done in certain ways. That's how I grew up as a video conferencing person. I need to unlearn a lot of that in order to do WebRTC. and I think that they need to unlearn some things because of HelixML as well, and we'll go through that. Um, I sent these articles, these two articles to all of my kids, the ones doing the TeamViewer projects and had discussions with them. And that brought me the idea that, you know what, I should share that here as well. So here we are and we're going to go look at these two posts by Helix and then I'll go through that. Into my own learnings out of that and what I think you should take away from these posts and from this session. So let's start. Um, what I want to say is this, and this is going to be the first lesson immediately at the beginning: if it is working, don't fix it. HelixML does something related to real-time GPU streaming over low-latency Simpler, Minfra, whatever. So what they're doing is they are taking a remote machine and they send that machine so I can view that and interact with it from my own machine. Okay, so a TeamViewer kind of a service, which is interesting. Um, it's not new, and it seems like they have a service that works for them and works for them well enough for their scenario, which means that if it's working, don't fix it. That's okay. Okay, I'm not here to trash them or say that they've done things that are wrong because it works for them. I'm not sure that this is a path or route that others should take, and they mention that in their post as well. So let's begin after this first takeaway. So at Helix, we run AI coding agents in GPU-accelerated containers. Users watch these agents work through a live video stream. Think remote desktop. So this is a remote desktop service, but for AI. The standard solution for browser-based real-time video is WebRTC. Yes, it is. This is what you should use. After months of turn server hell, we threw it out and replaced it with plain WebSockets. Why? Okay, that's my first thought about that. Let's see, let's understand. So the problem— oh, and by the way, the result: lower latency, simpler infrastructure, and it works everywhere. Yay, great! Spoiler: this solution worked so well that we eventually threw it away too, but that's a story for next week. We'll review that one as well. It goes to show that the solution that they explain here didn't work quite well for them. Although they say that it did. Now for the problem with WebRTC. WebRTC is designed for peer-to-peer video calls. Okay, it handles NAT traversal, codec negotiation, bitrate, adaptive bitrate, packet loss recovery. It's an impressive piece of technology. Yes, but we don't need peer-to-peer. Yes, you do. Our architecture is strictly client-server. No, your architecture might be client-server from the point of view and perspective of the way you think about it. Me, the client, using my laptop to connect a server machine that runs the service for me, the workload for me. From a WebRTC perspective, there is one client in front of another. That's peer-to-peer. There is no media server needed in the middle. So that's like a misunderstanding of what the technology actually does and enables you to do. For this use case, WebRTC's complexity becomes pure liability. We'll see about that. TURN server hell. Okay, enterprise customers don't allow random UDP ports. Okay, WebRTC also uses TCP and you can decide not to use random ports for some of the services in some places. They have L7 load balancers that only speak HTTP/HTTPS on port 443. WebRTC requires UDP, TURN, TCP, and UDP media relay on different ports. No, it doesn't. This is what the protocol states. In real life and in the best practice, what you do is use UDP and TCP ports 443 and not the standard TURN port 3478. Why do you use port 443? Because of that. If you want to pass and traverse through enterprise networks, if you relay through port 443 in TURN, it will work better. It might not work all the time, but it will work more of the time. Okay, so that's the kind of a misunderstanding of what you can actually do with WebRTC once you deploy it. Getting this through a corporate firewall, good luck. Okay, so I say this out of where we're at at the moment. Um, let's continue for a few more seconds. Getting through a corporate firewall, good luck. It means that they didn't manage to do that. We spent weeks debugging TURN configurations. Coturn, Twilio, custom deployments, each had its own failure modes. The TCP fallback that TURN promises, in practice, it's unreliable and adds 50 to 200 milliseconds of latency. We had customer whose WebRTC connections worked 80% of the time, the other 20% Black screen, no error messages. WebRTC negotiations would silently fail after 30 seconds of connecting. Now, here is a challenge for you. These enterprise incorporate networks and firewalls. Do they use, let's say, Zoom or Google Meet or Microsoft Teams? Because all of these services rely on either WebRTC or something very similar to WebRTC in the way it works and behaves. Cisco Webex as well and other solutions. So I think the second takeaway is that first you need to conduct— this is just like game. Okay, what do I mean? They are building a solution that does peer-to-peer and they need to go through a client firewall and it does remote desktop sharing, whatever, something similar to remote desktop sharing. Now, if I use Google Meet for remote desktop sharing, would that work for me in enterprise networks? Yes. Better than their service? Probably. So then the question should be, okay, what are we doing wrong? Because our service is just like that one. Okay, we're doing video conferencing and video real-time sharing, whatever, in WebRTC, inside browsers, across Firewalls today in enterprise networks, in corporate environments, it is doable. Okay, you might need to configure things, but it is definitely doable. So second takeaway, before you start, think of another service that exists out there that is similar enough. If you see that that service uses WebRTC and uses it well, then you might want to use it as well. Okay, and we'll get to to the reasons why later on. So here's the first insight that they got to. Here's the thing, WebSockets work everywhere. Yes, it does. There is just HTTP upgrade. Okay, every blah blah, it works out of the box. And for real-time video, WebSockets might actually be faster than WebRTC in our architecture. Nope, can't do it. Here's the reasons why. No jitter buffer, we can render frames immediately. Yes, sure. Do you know what jitter buffers are there for? They are there for if you have delayed packets and you need to reorder them, and if there's packet loss and you need to wait for stuff and you need to make decisions. The jitter buffer in WebRTC is adaptive. It means that it will be smaller if the network is better, and it will be longer if there are issues with the network. With a WebSocket, you're running on top of TCP. TCP is reliable. It means that it is going to wait there and retransmit and do things that is going to add to the latency. The reason WebRTC uses UDP and not TCP is to reduce latency and to work faster, not slower. So WebSockets are slower than WebRTC, hands down, all the time. And if it's not, you're doing something wrong with WebRTC. No turn relay, direct connection through existing proxy. Okay. The hint is here. A proxy is an actual relay. So it's similar to a turn relay. So you can install, maintain, and put a turn relay in a different way that you would do with a proxy. In their service, it might not be possible because of the type of customers and architecture, but it is still possible and doable. The other thing is if they're doing peer-to-peer with WebRTC, there is no TURN relay. So that again depends on the type of sessions that they have and how they've exposed the machines in the cloud. We don't have the answer for that, of what they've done and why. It might just be that they mandate the use of TURN relay because of different reasons. It still doesn't mean that it's going to be slower. No ICE negotiation. Okay. So that would be faster in the time to connect, not in the actual session. Time to connect in WebRTC can take a few seconds, but you can speed that up using a TURN relay, for example, the connection time itself. So there are ways to mitigate that, and there are today work being done to improve and speed up things in WebRTC as well a bit. Trade-off is TCP head-of-line blocking. But in modern networks with low packet loss, barely matters. Not true. And you don't have that. Okay, we'll get there again. That's an assumption they're making here and then they're breaking later on. The implementation, and then they stream from Moonlight, NVIDIA. We're not going to do that. This is, let's say, the technical part that is uninteresting to us. Over the WebSocket, they've built their own binary protocol. To send these frames and have the clicks back, and that's— which is cool, it's neat, that's great. It shows that they know how to develop and what to develop. Now let's continue. Video frames are raw H.264 null units, no RTP packetization, audio in ISOPUS frames. Input goes the other direction. So they're doing null units in H.264 and sending that over WebSocket over TCP. This means that the network is going to segment these things in the way that the network wants. So if they have a null unit that is longer than, say, 1,400 bytes or 1,200 bytes, this is going to be split into several packets. So they might not be aligned in the right way. And a null unit is a kind of packetization in a way. It's not a full frame. WebCodecs for decoding. So they're using WebCodecs API to decode, which is fine. Um, let's continue. They're doing hardware-accelerated H.264 decoding straight to canvas, no media source buffering, no jitter buffer. Eh, great. Okay, audio sync. So they are synchronizing audio with the video. Okay, so here's what happens. Audio uses the same approach: audio decoder, audio context, schedule playback based on presentation timestamps. So whenever they present something, they play the audio for that. This means that they cannot display every frame the moment that it appears because the audio needs to be aligned with the video. But if I've got video frames that are created and in the other side I just decode and play them back immediately, I might have different gaps and distances because between one frame to another, and then some frames are going to have a large latency between two of them and others a small one. And if I play audio this way, it's going to be either too fast or too slow at different areas of the video that I'm playing back, which is one of the reasons that we have a jitter buffer and lip synchronization. And if you remove the jitter buffer, you now need to add it back after you add audio sync. They probably haven't noticed that because most of the workloads don't need that audio coming from the remote machine. The moment they will need that, or if they will need that, then they will need to do back the jitter buffer and the lip synchronization, not just audio synchronization. Okay, and that's going to crap out what they are doing, which is slightly said. So first, first, first audio frame establishes the baseline. Subsequent frames are scheduled relative to it. If frames arrive too late, above 100 milliseconds, we drop each rather than accumulate latency. That's great. Do they do it only for audio or for video? I don't know. From the way it's written, it's probably for the audio, which again means they miss something on the video side. Input forwarding. Input goes the other direction. That's fine. We don't care about that. The thing is, you can do that with the data channel in WebRTC, or you can keep it as a WebSocket. Both approaches are fine, especially if you want to do things that they do in the second post. What we lost. Now this is interesting. Nothing is free. Here's what WebRTC gave us that we had to handle ourselves. Adaptive bitrate. Well, WebRTC monitors network conditions and adjusts bitrates automatically. That's true. We don't. Bad for you. Our bitrate is fixed at connection time. Okay. For enterprise deployments on stable networks, this is fine. No, it isn't. For variable mobile connections, it might be a problem. No, it is a problem. It's a problem everywhere. I have a customer last year, they had an issue. Telehealth in hospitals, network conditions change variably during the day. They couldn't push and pump through the network the video in the rates that they wanted, and we needed to balance the system down so that they don't overuse the network because, well, what would you say, stable networks? No, they're not. Now, you might have that with the first, second, third customers. At some point, this is going to break. So that assumption that you don't need adaptive bitrate for a live video service of remote whatever is, doesn't hold water. Okay. This is something that they will need to fix. Packet loss recovery. WebRTC uses NACK, PLI to request retransmission lost packets. With TCP, we get reliable delivery, but head-of-line blocking. Guess what? Lost packets stall the stream until retransmitted. And this is shitty. That's the shittiest experience possible. And this is why WebRTC is hands down always faster. In practice, on data center quality networks, packet loss is rare. No, it isn't. When it happens, TCP recovers fast enough that users don't notice. Oh, they do. They don't notice because you're running at low frame rate or running a service that if you run it on a low frame rate, it wouldn't matter for the users. That's a different story. With a different type of a solution. So you need to understand not only your users, also the use case, the technology that is using, and the requirements that your users have. Then browser fallback. Okay, WebCodecs, da da da da da, works on everything. We're not going to do things for other browsers. What we gain? Works everywhere. No, it doesn't. It doesn't work on a lot of different types of networks. It works on different machines, maybe with different configurations. but not on network conditions. Simpler infrastructure. True. It is simpler. Is it better? I doubt it. Now, current servers or Twilio. Yes, you need that. STUN server. That's like TURN server. TURN server doubles as a STUN server. So you don't need a separate STUN server. ICE configuration management. That's part of TURN. I wouldn't put that as a separate bullet. The certificate management. That's That's a no-brainer, if I might say. It's not that hard to deal with. And then UDP port ranges, again, I'm not sure that's a problem with the infrastructure. There are other things, the fact that you need a TURN server, and you need slightly different architecture and need to think about signaling. But I think that at the end of the day, WebRTC versus WebSocket plus WebCodecs, WebRTC has a simpler infrastructure or simpler solution at the end of the day for this specific use case. Low latency without a jitter buffer, which you need. Turn relay, which doesn't really hurt. We measured 20 to 30 milliseconds lower end-to-end latency. Great for you, but you didn't measure what you lost. WebRTC Adaptive Jitter Buffer bitrate sometimes causes quality drops that look took seconds to recover. Our fix, big trait is fixed, and I'm sure that customers were really happy about that, which is why you switch to another solution later. Debugability. WebRTC failures are famously opaque. No, they're not. ICE connection fails tells you nothing. WebSocket failures, you get HTTP status codes, error messages, blah blah blah blah. Okay, if you have issues with WebRTC debugability, there is rtcstats.com. Go check it out. I'm building that with Philip Penke and Olivier Arnault. The best solution ever for debuggability. And then WebRTC is not opaque at all. Go check it out. Should you do this? Probably not. Great. Yeah, that's true. You got to that conclusion and that conclusion is the right one. Um, and then they put the reasons why you should. I don't think that that matters that much. We'll skip that and move on to the next post. And before that, We'll take the third takeaway, and this time, I think that we started it here and see, we'll see it in full force in the next post. And that is, you need to know your target audience and use case, and you need to know it by, by heart. You need to know it only for the main set of users, like we run on corporate networks that are great, but rather on the edge cases as well, because the edge cases is what's going to kill you in WebRTC. And if you're moving away from that, you need to understand, are you taking care of the edge cases as well? And WebRTC does that well if you plan and architect it properly. So that's my third takeaway. Know your target audience and use case. Let's go to the next one. Now the next one is nice. We mass deployed 15-year-old screen sharing technology and it's actually better. It's older than 15 years. 15 years ago, WebRTC started. 20 years ago, I was working in TradVision doing video conferencing with H.264 video codec. We didn't do moving JPEGs at that time. So the year is 2025, now it's 2026, and we're sending JPEGs. Okay, first question is why? Let me tell you about the time we spent 3 months building a gorgeous hardware-accelerated web codec powered 60 frames per second H.264 streaming pipeline over WebSockets. Did they tell you that you need to know the target audience and use case? Who in Earth— on Earth sends 60 frames per second over H.264 from a remote desktop to a current— to this desktop, probably at 4K? Okay, it makes no sense unless what we're doing is cloud gaming. This is an AI-powered solution to run something on a remote desktop hardware, stream it locally, and look at the results. Unless what they are doing is rendering video in real time on a remote machine, they should probably lower the frame rate and bitrate considerably, and the resolution. Doing that at 60 frames per second makes no sense. Even if I've got the network to do that, I wouldn't do that. Okay. Act 1: Hubris, also known as enterprise networks exist. Okay, so they built Helix, AI platform, autonomous coding agents work in cloud sandboxes. So this is for coding. Why do I need 60 frames per second? Think screen share, but for things being shared, uh, is a robot writing code. I don't need 60 frames per second to watch a robot writing code. 10 frames per second, 5 frames per second work good enough. You want to go overboard, go to 15 or 30, don't go to 60. So this is pure waste, and that waste means unbalanced network use. You don't want that. Last week we explained how we replaced WebRTC with custom WebSocket streaming pipeline. Okay. This week, why that wasn't enough. Last week it worked, now it doesn't anymore. The constraint that ruined everything. It has to work on enterprise networks, which is why they moved from WebRTC to WebSockets. Not sure why. And they explain here again, UDP blocked, so it deprioritized, dropped security risk. WebRTC requires TURN servers, which requires UDP, which is blocked. WebRTC can run on TURN servers with TCP. It can also run on ISE TCP Direct without a TURN server, and it can run on TURN TLS as well. But, you know, who wants that? Custom ports firewall says no. You can run it on port 443 using TURN or ISE TCP. Then, Stan, ISE NAT traversal in my corporate network? Absolutely not. Okay. Literally anything is fine denied by policy that's in enterprise networks. And yes, that's true. But all of these services, all of these networks run either Zoom, Google Meet, Webex, or Microsoft Teams. And all of these use custom ports and firewalls get configured for these things if needed. We tried WebRTC first, worked great in development, worked great in the cloud, deployed to enterprise customers, the video doesn't connect. True, you need to make sure that you know WebRTC well enough. Okay. Uh, we could fight this, set up TURN servers, uh, configure enterprise proxies, work with IT departments. Or we could accept reality. Fighting this means making this work for your customers. Accepting reality, everything must go through HTTPS on port 443. Not necessarily, that would depend on the customer, that's the first thing. And the second thing, if it needs to go through HTTPS port 443, WebRTC can still do that for you. So we built a pure WebSocket video pipeline. Yes, we know. The line here is 60 frames per second at 40 megabits per second with sub-100 milliseconds latency. And the question is why? Why over-architect a solution? That beats me. And here's, here's the money line. Someone, then someone tried to use it from a coffee shop, which means that you didn't map the target audience and your use case properly from the beginning. You didn't look at the edge cases of what customers might want to do. That's the sad part. So, act 2, denial. Video is frozen, Wi-Fi is bad, no video is definitely frozen and not Now my keyboard isn't working. That's what you get with 4 megabits per second running on a video stream on Wi-Fi in a coffee shop. I don't think that my home network can handle 40 megabits per second, and I've got, well, fiber to the home with 1 gigabyte. Real-time is tough when the network gets congested. Frames buffer up the TCP WebSocket layer. That's killing you. They arrive in order, but increasingly delayed. Well, it doesn't— the fact that it's in order is not helping you, it's hurting you. That's the thing. TCP is built somewhat on top of UDP, and it does things that you might not need at all. Video falls further and further behind real-time. Yes, it does. You're watching the AI type code from 45 seconds ago. Okay. The solutions that lower the bitrate, great idea. Now it's 10 megabits per second of blocky average that's still 35 seconds behind. Here's the thing. You don't lower the bitrate. You first balance the system on what you need. And again, 60 frames per second at 40 megabits per second. It's not what you need. Not for these customers. So start by balancing it. Bargaining. What if you only send keyframes? So you generate the whole H.264, 60 frames per second, but then send only the keyframe toward the network. Okay, so why generate 60? Why not generate 1 frame per second with H.264 and have each one as a keyframe? And if you go that route and you say, this is good enough, then stop and ask yourselves, what is it that our customers really need? Do they need 60 frames per second or do they need only 1 or 2 or 5 or 10 or 15 or 30? What is the number that they need? Because it ain't 60, I can tell you that. And then they had another technical issue, trying to fix that. Weren't able to. And they go to Act 4, which is why they did JPEGs. One late night while debugging with— while the stream was frozen again, open a screenshot. The screenshot loaded. It was great. So I decided let's press F5, mash the F5 for like a degenerate 5 frames per second of perfect screenshots. So they ended up saying, well, you know what, WebSockets is great, WebCodecs is great, H.264 is great, but JPEGs are enough for us. We don't need 60 frames per second, 5 is just fine. Great. Why did you start with 60? And then they go and implement the solution, which is simply putting JPEGs. And they do a table here, the bandwidth. H.264 stream, 20 to 40 megabits per second. It can be also 500 kilobits per second, by the way. That's up to you, up to the network to make a decision of the bandwidth. JPEG, that's 100 to 500 kilobits per second, okay. Stateful H.264, yes. Corrupt means dead, yes and no. That depends again. However, TC plays with both sides if you do web codecs that's different. Stateless. Okay. For the JPEG part. Latency sensitivity, very high. Yes, but that's what you want. JPEGs don't care about sensitivity, but your users care. That's what you said. So am I wrong? What, what, again, what is the requirement of the audience and what is the use case? Do you need low latency or you don't care about low latency? It's a question and you need to answer that. And then recovery, wait for keyframe, seconds, no, it's a lot less if you implement that properly. Next frame, 100 milliseconds. Okay, not sure, that depends. We talked about 5 frames per second, this is 10 frames per second at 100 milliseconds, but fine. And if you've got congestion, that can be a second or two because of the head-of-line blocking of TCP, and we don't go there. JPEG screenshot self-contained, okay, whatever. Um, and they're sending 1080p, not 4K, which is okay, nice, at least that. They can do the same with H.264. I'm not sure why it takes them more, but no, it is what it is. And then they understand that they have a problem. They can't use JPEGs. Because it's not good enough for the enterprise environment where the network is good, and they can't use the WebCodecs solution for the ones that run in coffee shop. So what they do is they do a hybrid solution. They do a pipeline of H.264 and one of JPEGs on a good connection when the round trip time is below 150 milliseconds, whatever that means. Full 60 FPS H.264 hardware decoded, battery smooth. Again, why 60 FPS? I have no clue. Bad connection detected, pause video, switch to screenshots polling. So they move from 60 FPS to 1 FPS or to 5 FPS on JPEGs. Connection recovers by the user clicking the retry video, and they run the WebSocket for the key— keyboard inputs. Which is why I said with WebRTC you can do it with the data channel, but if you want something like a JPEG fallback, which you probably shouldn't, then doing the keystrokes on a WebSocket and not on WebRTC might make sense. And then with added controls, try to do it automatically, did oscillation, we're not going to go in there, but there are a lot of issues and technical problems that they needed to solve. Let's go to their takeaways. And I'll add the two of mine that are left. This was their final architecture, by the way. Frame took more than 500 milliseconds to receive, drop quality, you know, lots of things that they are doing here for JPEGs and, and I don't know. Lessons learned: simple shell solutions often beat complex ones. That's true, but I think that their solution is too complicated. Not the JPEG one, but the fact that they moved through all of these and ended up with two proprietary solutions that are subpar to what you can actually achieve with something simpler. Graceful degradation is a feature. Users don't care about your codec, true, they care about seeing their screen and typing yes. Graceful degradation is not what you have moving from 60 FPS to 5 or 10, from H.264 streaming at high quality directly to the lowest level possible. That's not, not it. WebSockets for— are for input, not necessarily video. True, but that's true for TCP in, in general, which means JPEGs as well if you're looking at video. Ubuntu: I don't care, measure before optimizing. Okay, that brings me to my two last takeaways. Number 4: you need to upskill before you optimize. And develop. Okay, yes, you need to measure before you optimize, but before you optimize, you also need to understand the actual technologies that you're using. Video, live video is, is challenging. It's not that easy, and you can see that from the fact that they started with WebRTC, moved to WebSockets, and then moved to JPEGs and trying to do things all over the place. Now, WebRTC could have solved the problem from beginning to end Maybe, maybe with a fallback to this JPEG thing. In 2026, I wouldn't do that, but maybe they would have needed that. I'm not certain. I'm almost certain that without it, it would have worked better and simpler. If they would have known how to use WebRTC and not get stuck on the TURN servers at the beginning and think, thinking that 60 FPS is what they need, they would have solved their problems a lot earlier. With a better solution that is more polished, that has graceful degradation. And that means understanding the technology that you use before you design, before you develop, and definitely before you optimize. Okay, that's for me. And yes, you need to measure before you optimize as well, but there you need to understand the technology in order to optimize. Now the fourth one— the fifth one, sorry— is an interesting one, and this is Developers want to develop. And for me, there are like, there are two extremes in developers that I've managed to work with in the past as a manager. There is a type that say, well, yes to everything that you ask of them to do. And they would go and they are clueless and they will not ask questions. They would not succeed in what they're trying to do. And they would come back and you ask them, well, have you made progress? Yes. and you'll never know what's happening until you just fire them. This isn't the case. This is the case of the great engineers that sometimes see a wall and they would go and hit that brick wall with their heads until that wall breaks and they get to the other side. They solve the problem by just breaking that wall. Sometimes that's great because they almost always succeed. But sometimes just saying, hey, look, there is a door next to that wall. You can just go and go through that door, solves that problem. This is something that happens a lot with very good developers because developers like to develop and they like to play and tinker with new stuff, a new shiny toy. I'll go back to the kids that I'm mentoring. Almost with all of the groups. There are two things that you need to do at the beginning. The first one was to say, well, look, we've got only 4 months, 5, 6 sprints of 3 weeks each. It's the two of you. You've got school, you've got exams, a lot of other things in your life. You're not going to have so much time to do that. So let's dilute and reduce the actual feature set to the bare minimum, the basic, the most important things that you need, because you're not going to be able to implement all of WebRTC. In this solution, including encryption, Diffie-Hellman, and all of the other stuff that is there. It's impossible to achieve. I wouldn't go and do that if I were you. And it takes them time to get there with me, more, usually more than one session. They need to fail it at least once to see that, because they are great developers, they are enthusiastic, and they're young. The second is to, to go and focus on what matters. They have a very short time span for the project. Sometimes, again, they need to show a demo at the end that actually works. So we go and say, let's first have something that we can present, something presentable. What is it with video? Well, having video running from one machine to the other. Let's do it locally, not in the cloud. Let's do it between two laptops sitting one next to the other. We might do that other thing that you want, but let's call it fairy dust. We do it at the end. I go to a session with them and then they say, well, we are starting to do the keyboard controller, remote controlling the machine. Okay, but is that really the most important thing now for your project, or is that fairy dust on the top? Okay, let's leave the fairy dust towards the end. Developers want to develop, and sometimes you need to focus them back to what's important. Uh, the Helix team did a hell of a job, a great job at a solution that actually works for them and their customers. They're solving the problems of their customers in a lot of different ways. So remember the takeaways. We started with the first one. If it is not working, don't fix it. So if this is working for them, that's great. They don't need to fix it. The second thing, know your target audience and use case. And you need to know it really, really well. You need to know the common use case for the majority, but also the edge cases that you are going to need to solve. You need to think about them as a whole when you start architecting and designing the product for these customers, especially when you're doing something with WebRTC in real time. Third takeaway, conduct a 'this is like' game, especially if you want to understand if this is doable or not. I had a client once come in and say, well, I want to implement something that works in the Himalaya in poor networks, and it should be better than Skype. I almost gagged on that. Oh, and he's going to use WebRTC because he knows that it's good and it needs to be better than Skype and Google Meet. And I asked him, look, Seriously, do you think that you can do a better job with WebRTC inside the browser where Google Meet uses WebRTC in the browser and they own the technology in Chrome for that? And that seemed like a surprise to him. Okay, so you need to conduct this is just like game in order to understand if this is doable or not, at least to some extent, to understand how big of a problem you're in. Fourth one, upskill before you optimize and develop. That's important. If you're going to develop something, at least understand what you're developing and what are these tools, what they are capable of doing. And if you need help, okay. And then developers want to develop. We need to give them something to develop that is interesting enough, especially if you are a manager. In many cases, when someone comes and asks me, well, what should we use as a media server, an open source media server, there are these solutions. I simply tell them, look, ask your developers if they are, if they are leaning toward the specifics one, just go with them because it won't matter if they don't like it. They first need to like it because they're going to show you that it works there. Okay, so you need to understand that and work with that. With the developers that you have. I'd like to thank you for joining this session. It's the first time that I'm doing such a thing. It's a bit long, I know. So sorry for that. If you like more of these, let me know. If you want me to go and do a walkthrough of your own service and give you this kind of advice, let me know. I'll be happy to do that publicly if you want, or even privately. And see you next time. Until then. If you want to 10x your work and the efforts what you're doing in making sure that you get faster towards your target in live real-time communications, reach out to me. I can help you with that. And don't forget to follow this YouTube channel and my blog at bloggeek.me. Bye!
More from Tsahi

Meta rebuilt AV1 for mobile scale. Should smaller teams follow? | Chats with Arin









