WebRTC has been a magical word for the best part of the 2010 decade, starting, if we had to put a date on it, with 2011 Google IO presentation. From conversations as early as 2018, and many small signs (dropping support for official mobile release in m80 release notes), it was clear that, for Google at least, the WebRTC star itself was already the past. Still, more people depend on webRTC, or want to adopt it, today than ever. What are the options out there? How should one prepare for WebRTC in 2020?Continue reading
W3C TPAC @ fukuoka Japan, October 6 to 20, 2019
Greetings from Fukuoka, Japan – or, your essential update to all things WebRTC…
As ever year, the technical plenary meeting of the World Wide Web consortium took place in the fall. After Lyon, the French “culinary capital”, in 2018, it was hosted this year in Fukuoka, Japan, during the week before the first Rugby World Cup hosted by Japan.
If you only have a few seconds or have an attention deficit 🙂 here’s a very quick wrap of what happened at Fukuoka; however, I DO encourage you to read the the remainder of the blog to get a full perspective of the SIGNIFICANT DECISIONS that were taken, and the background perspective.
As far as the standard and corresponding specifications of WebRTC are concerned, the job is done. I know, we’ve all heard this before. However, with the IETF RTCWEB group closed last year, and the W3C WebRTC group closing in April, 2020, there will be no official space, nor resource, to make any more WebRTC standard work. Thanks God!
With Unified Plan now implemented by almost all browser vendors, Simulcast and the Statistics are the last two big items whose implementation needed more work. We should see the browser vendors work more on these items now on.
It is clear for some time that the focus of the browser vendors has moved from WebRTC proper to the next big thing.
WEBRTC NV, including QUIC, then WASM and AV1 codecs were the big thing last year, and even if NV is now being called WebCodec and WebTransport now, it feels like those four items are still the focus.
Looking at the still active or newly active standard drafts being proposed, one can read between the lines as well, and IETF 106 (November 18-22) should tell us more about what has been, or is, brewing.
I believe that WebTransport is going to be the main group to follow, but some drafts like RIPP are also introducing interesting techs.
Let’s get things in context (short history reminder)
Last year, beside the usual maintenance items, the two main working items of the WebRTC working group were Simulcast, to finish WebRTC 1.0, and WebRTC NV use cases. BUT not a single test from the WebPlatformTest suite was passing on all 4 main browsers, that said, it was not an excessive worry at the time.
WebRTC NV had many small use cases, but the overall idea was open up the WebRTC API even further to allow people to bring their own codec, and bring their own transport. In conjunction with web assembly (WASM), being made available through another group, this could lead to support for a lot of use cases, and give much more flexibility to applications.
The only part of WebRTC that would NOT be opened was the DTLS-(S)RTP part, as it was not only mandatory for WebRTC 1.0, but also needed to be handled by the Browser itself to protect users.
Both Apple and Mozilla were pushing back on WebRTC NV, for different reasons. Apple wanted to focus on finishing WebRTC 1.0 and did not want to see WebRTC NV draining away already limited resource.
Mozilla, heavily involved at IETF with Cisco in the PERC standard, would oppose anything that would allow overlapping features with PERC. That eventually lead to two Mozilla employees, who were Area Directors at IETF, to send a formal liaison letter from the IETF to the W3C claiming that the W3C working group was trying to alter the security foundation of WebRTC by removing DTLS-SRTP.
We, at cosmo, do not remember such removal to have been proposed, but it did not matter, the WebRTC working group had to drop the corresponding item
Meanwhile, at IETF, the PERC standard, chaired by Cisco and Mozilla, was declared “a rough consensus” by the chairs, and accepted as a standard. This was, after multiple problems had been raised both on the mailing list and during the meetings, as well as no less than 12 negative comments raised during Last Call.
Simulcast was generating a unanimous concern. With Unified Plan now out of the way, the Simulcast holy mountain could finally be cleaned, but nobody knew really how much work was left to be done. More worrying maybe, no testing tool existed which would allow the browser vendors to test their implementation. As Bernard Aboba from Microsoft was also pointing out, testing Simulcast meant testing against an SFU, and in this absence of a reference SFU, that meant testing against enough SFUs to make sure it worked. The feature was deemed at risk, and marked as such by the W3C itself.
CoSMo then committed to allocate resources to provide a reference SFU, as a variation of Medooze, and corresponding KITE tests to all browser vendors to help address simulcast. Rendez-Vous was taken to each IETF Hackathon (which most browser vendors attend, and which take place three times a year instead of only one time for the W3C TPAC) and to make sprints and discuss progress.
And now to Fukuoka..
Fast forward to Fukuoka this year. CoSMo had implemented a reference SFU, with a lean-a-and-mean compliance mode that would only accept the latest specs. Apple had, for example, used it during their QA to accelerate their development. The WebRTC working group meeting was more relaxed, at least while it came to Simulcast, which was taken out of the features-in-danger list.
The KITE simulcast tests and infrastructure (Medooze) have been made publicly available. The extension of this test bed to the WebRTC-SVC specification, and/or to the alliance for open-media’s AV1 RTP payload specification is under discussion.
The coverage of the WebRTC specifications has seen a great surge, going from 0 to almost 50% of tests that would pass on all 4 browsers (see below). If we take into account all tests that pass on at least 3 browsers, that goes up to almost 70%, and with 2 browsers, which is the criteria for W3C, that’s 85%.
INTEL who had announced their intention to open-source their full WebRTC implementation at TPAC last year had since came through with their promise in April. They are planning to join the Google / CoSMo lead Hackathon at IETF 106 in Singapore on 16th and 17th November (see here) to integrate better their tests in KITE, to participate in the SFU Compliance testing session, and to collaborate with Apple to bring H.265 Hardware Acceleration to libwebrtc.
The WebRTC working group was chartered only until April 2020.
The questions was now, what to do next? It was pretty clear that the number of active members in the group had reduced, and following the problems arising during the WebRTC NV discussion, a lot of interest to keep working this way, with that group of people, had declined (more below). Once the decision ad been taken that we would NOT reach out to extend the working group life, there remained the question of what to do with pending documents. Dom, as representative of W3C for the group, presented simple conditions: for a draft to be kept alive and get support it needed to have two things:
– One editor that was taking responsibility to finish the document.
– At least one browser vendors which would commit to implement it.
The INTEL device group stood up and took responsibility of the Depth Image support, as they had already provided an implementation in Chrome. Microsoft confirmed support on their side. Same for some other contributions to the Device support (PTZ cameras, ….). One by one, the remaining specifications were distributed, or dismissed.
Following the “PERC Drama”, and because of the little remaining time left in the WebRTC working group charter to do things, most of WebRTC NV items had been rebranded into WebTransport and WebCodec, and dispatched into different working groups.
With the imminent closure of the WebRTC working group, the new Media Group got extra attention. Traditionally, at W3C media meant the Video element, streaming of non-ICE, non-real-time media (Media Source Extension), DRM (Encrypted Media Extension). Just like in the rest of the industry, we can now see the two groups (real-time and VOD) coming together and working together on a set of converging technologies. Logically, the Media Group will be Hosting the WebCodec effort.
Until next time ……
There are many that happened this week, both during IETF sessions and in the ecosystem, that troubled me enough to write a dedicated blog post. People from the streaming industry, and from the webrtc industry alike, are approaching OTT media in general, and webrtc specifically, the wrong way. It works, but it does not work well enough. It connects, it streams in perfect conditions, but it does not stream with good quality, at scale and in real-time. What is missing?Continue reading
For those who are following me on twitter @agouaillard, or are following cosmo @rtc-cosmo this is old news: CoSMo Software has been selected to be part of the INTEL Innovator program. Let’s dig into what it means really.Continue reading
Libwebrtc does not include support for H264 by default. While the code could support it, there are legal obligation when it comes to compile and distribute H.264 in your product: License and royalties! CoSMo just contributed a patch to make libwebrtc use OpenH264 (free as freedom) for encoding, and to dynamically load the Cisco provided library to make it free (free like beer).Continue reading
Now that we understand the basis of libwebrtc code management, we can start answering otherwise problematic questions. This week I was at CommConUK, and was discussing the number of contributors to libwebrtc, pointing my interlocutor to the AUTHORS file to start with. “Less than 100” was the other party position, and to be honest, I had never checked. So, who would risk a guess as to how many contributors to the webrtc stack there were in the past three years, and more importantly, how to check?Continue reading
I first started writing about libwebrtc source management, build and test systems almost 5 years ago. While the posts are still here, and mostly accurate, people forget, and/or the system as changed just enough that we need to update what is the de-facto reference for libwebrtc. As we are writing a book, with examples and illustration to be used in classrooms to teach the underlying principles, and in companies for e.g. on boarding Engineers, we though we should put some extract here.Continue reading
After the release of the Codec Spec in march 2018 (“frozen bitstream” and reference decoder), the next step qs to show that AV1 could be used in production. The decoder had to be made fast enough on commodity hardware, hardware vendor had to integrate AV1 support in their chips, for the base profile. Then advanced profiles (SVC, …) and modes (lossless, Real-Time) would deliver. For the work on real-time and SVC modes, a specific subgroup was created to continued the work beyond just the codec Spec. Integration with the Real-Time-Protocol (i.e. writing an AV1 RTP Payload specification), and usage of SVC in conjunction with Media Servers (RTP Header extensions, …) needed to happen. On Halloween 2018, CoSMo demo’ed live the first AV1 RTP integration in WebRTC. It did not support SVC and was not Real-Time. On June 26 2019, Cisco demonstrated live from New York the first Real-Time AV1 RTP Integration in WebRTC, through a modified version of their flagship product: webex. It denotes a new step in the evolution of AV1, one that happens 12 months earlier than anybody thought it would.Continue reading
Let me be honest, I really dislike marketing in general. Public researcher by training, I’m looking forward to the truth, to reproducible results, and claims that are backed up by data and processes I can access by myself to reproduce the results and the conclusions of the analysis. Marketing is very often the opposite of researching of the truth: it aims at making the market buy your product(s) and service(s). Very often, the end justify the mean, and FUD, deceiving and/or unsubstantiated self-serving claims become the norm.
Let’s be clear, if you don’t have a killer feature, you might as well pretend that either a/ you have it, b/ it is not something important, c/ what you have is so much better. Eventually, as all lawyers know, if you can’t argue the facts, go after the credibility of those bringing them to light. Microsoft have made this approach very famous and even gave it a name: Fear, Uncertainty and Doubt: FUD.
When I read some Wowza marketing piece I cannot help but noticing a strange correlation with those technics, and it’s frankly upsetting. We have just made a “Fact checking” session about WebRTC at Live Streaming West in New York. I think it’s time to do it again here, so at least people can have access to enough verifiable information to make their own mind.
Satyagraha, (Sanskrit and Hindi: “holding onto truth”), was the philosophy proned by gandhi when addressing the British Imperialism: non-violence, self-scrunity and research of truth. It lead to citations like “First they ignore you, then they laugh at you, then they fight you, then you win”, and more simple saying like “believe what they do, not what they say”, which should be applied any marketing material, really.
For the past two years, all CDNs and most streaming server vendors, have been trying to downplay the disruptive impact that WebRTC would have on their business. It wouldn’t scale. There would be no market for it. It would not be needed. It has now changed.Continue reading