We’re a little bit late to the party, many of the other WebRTC vendors and consultants have already wished everyone the best, but better late than never. We also list a few of the goodness you should expect from CoSMo this year.Continue reading
by invited guest blogger: Sergio Garcia Murillo
There has been quite some buzz (and fuzz) in the webrtc world recently due to a serie of great posts from Natalie Silvanovich, Project Zero in which she demonstrated the power of end-to-end fuzzing to find vulnerabilities and bugs in webrtc stacks and applications, including browsers, FaceTime, WhatsApp….
Inspired by the above wonderful but admittedly complicated work, we decided to investigate how we could provide a
much easier to use fuzzer to stress test all webrtc apps with. Combined with the automation and interoperability testing capacities of KITE, it would become the perfect stress tool for the entire WebRTC community.
QUIC has been the source of all the heat during the internet meetings for the past two years at least, after being kind of restricted to a smaller group since its existence became known in 2013. There is a reason for that, QUIC as a transport protocol takes the best of TCP and the best of UDP, add encryption for security, multiplexing for speed, and bring additional improvements to make sure deployment will be fast on top of existing equipment, and updates (in user land) will be faster than before (in the kernel). For those interested in a first overview (possibly slightly biased as indicated in the respective sections) wikipedia is always a good start. For the hardcore Fans, the corresponding IETF working group and mailing lists will satisfy all your desire for detailed information. The latest drafts (#16), are hot out of the press as IETF 103 is ongoing in Bangkok this week.
Recent discussions with several parties, make me realise that the first steps to master webrtc are not yet documented enough. Once, as a challenge, I asked a master student doing his graduation project with us to try to recompile and example provided with libwebrtc separately from the compilation of libwebrtc itself. Basically to make a project with the same example source code but that would link against the pre-compiled library. 5 months later, it was still not successful. Thanks to our internal tool that I shared then, we eventually did it in two weeks. This post is about the usual ordeal people have to go through to understand the state of affair, and of course how CoSMo can shield you from that.
How to make sure the quality of a [webrtc] video call, or video streaming is good? One can take all possible metrics from the statistic API, is still be nowhere closer to the answer. The reasons are simple. First most of the statistic reported are about network, and not about video quality. Then, it is known, and people who have try also know, that while those influence the perceived quality of the call, they do not correlate directly, which means you cannot guess or compute the video quality based on those metrics. Finally, the quality of a call is a very subjective matter, and those are difficult for computers to compute directly.
In a controlled environment, e.g. in the lab, or while doing unit testing, people can use reference metrics for video quality assessment, i.e. you tag a frame with an ID on sender side, you capture the frames on the receiving side, match the ID (to compensate for jitter, delay or other network induced problems) and you measure some kind of difference between the two images. Google has “full stack tests” that do just that for many variations of codecs and network impairments, to be run as part of their unit tests suite.
But how to do it in production and in real-time?
For most of the WebRTC PaaS use cases (Use Case A), the reference frame is not available (it would be illegal for the service provider to access the customer content in any way). Granted, the user of the service could record the stream on the sender side and on the receiving side, and compute a Quality Score offline. However, this would not allow to act on or react to sudden drops in quality. It would only help for post-mortem analysis. How to do it in such a way that quality drops can be detected and act on in real-time without extra recording, upload, download, ….?
Which webrtc PaaS provides the best Video Quality in my case, or in some specific case? For most, this is a question that can’t be answered. How can I achieve this 4×4 comparison, or this zoom versus webrtc, in real time, automatically, while instrumenting the network?
CoSMo R&D came up with a new AI-based Video Assessment Tool to achieve such a feat in conjunction with its KITE testing engine, and corresponding Network Instrumentation Module. The rest of this blog post is unusually sciency so reader beware.
We are really laser focussed on delivering our projects and preparing for the upcoming standard committee meeting. With all of that going on, it took several message from linkedin to congratulate me for my work anniversary to realise the truth: CoSMo was 2 years old. I’d like to take a moment to look back at what we have achieved.
While many rejoice after Tokbox acquisition today, they do so in the same way France rejoice after the world cup: we won, but we’re not really proud about the way we won. Yes, it’s good to have a high-level acquisition in the WebRTC field, it had been while. But 35 Millions for Tokbox, something is terribly wrong there, and there is nothing to rejoice about. Unfortunately, this was somehow predictable. Let’s look at the details in this post.
VP8 and H.264 codecs are mandatory to implement to be webrtc compliant. Simulcast is a way to use multiple encoders at a time to provide different resolutions of the same media to chose from as a way to adapt to bandwidth fluctuations (and other good things). Unfortunately, while some patches were proposed some two years ago by some including HighFive, libwebrtc did not implement support for simulcast with the H.264 codec. H.264 was then a de-facto secondary codec, and Safari which only supported H.264, could not achieve the same level of adaptation (or quality) than VP8 and some other browsers could. This blog gives more details about the epic journey to get that done, the design of the implementation, and the impact for WebRTC products.
Such is the title of one of the latest blog post by wowza. While it is a very interesting question, I believe the blog post is conveying the cliche the streaming ecosystem as been carrying about WebRTC and that are not longer true. I do not believe wowza to be knowingly deceiving people, i see their point, I just believe that recent advances in the webrtc protocols make most of their statements inaccurate. This post is an attempt to document the statements that can be proven wrong today. Fact-checking in some way.
Since our “Real-time communication testing evolution with WebRTC 1.0” article was accepted for publication by IEEE, not many new scientific papers have been published on WebRTC testing apart from  and  ( is also available, but without indication wether it has been peer-reviewed and accepted for publication or not), but a lot has happen with KITE.
Beyond being the official testing tool for webrtc.org, and adding tests for multistream and simulcast to better cover the last pieces of the WebRTC 1.0 spec, support for mobile browsers and mobile app testing, as well as for Electron apps testing have been added. KITE can now support up to 20 clients configurations, making it the most complete and most versatile #webrtc testing tool known to date.
Beyond the Interoperability Mode, a Load Testing mode has been added to KITE, specifically to stress test webrtc servers and infrastructures, with a capacity of 1 millions users. It has been tested in production by several customers in a one-to-many streaming environment.
Of course, you can have CoSMo run KITE for you (hosted and managed), or you can run it yourself, on premises. More details in this Post.