Extending Janus #webrtc bandwidth Management


Main Coders and Architects: Sergio Murillo, Lorenzo Miniero,
Facilitator, Motivator, and Secretary: Dr Alex Gouaillard.

Janus Bandwidth management has been incrementally updated to support the latest technologies available in a joint effort between CoSMo and Meetecho, a.k.a. The WebRTC A-Team. This article describes the original design of Janus and its VideoRoom plugin with respect to bandwidth management, and the incremental changes that were needed to bring it to automatic bandwidth estimation and adaptation on the sender side, and availability of simulcast for bandwidth management on the receiver side. A concrete example about how to leverage simulcast with the Janus VideoRoom Plugin is provided for illustration and testing purposes.

— old time Janus —

A Janus media server with the VideoRoom plugin (hereafter “Janus server”) acts as an SFU, by design providing client software feedback at the application level (as in “slow link” events) on bandwidth related problems. This solution works for both sender and receiver side. When you receive one such event from a Janus server, it means that either it is having trouble receiving your packets (publisher) or Janus noticed many incoming nacks/loss notifications (viewers). You can differentiate between both scenarii thanks to a flag in the event, but since when using the VideoRoom plugin all Peer Connections are unidirectional, it should be obvious anyway.

You can then send an event back to slow publishers down, ie., send a request to force a lower REMB. How you compute the new requested bandwidth value is up to you (eg., force half the current bitrate, or cut it by 1/3). Usually you are more aggressive on the ramping down than on the ramping up, as the consequences of overshooting are equally unbalanced: if you put the requested bandwidth usage too high, you will starve the connection and get errors (huge impact on UX), while if you decrease requested bandwidth usage too much, you just get lower quality / resolution stream but without glitches, or freezes.

Ideally you’ll want to ramp it back up again slowly once everything has been fine (lack of slow link event) for a while (stable). Again, finding the right balance between when and how fast you ramp down, and when and how fast  you ramp up is more art than science. If unbalanced or unstable temporary issues will cripple the bitrate forever. That is, among a myriad of other reasons, why the original design of Janus server was delegating the responsibility of bandwidth management to the application, which more often than not knows better about its environment to make the good choice.

Unfortunately, in many other cases, app developers prefer to delegate this to the browser (on sender side), and/or would like to have more options on the receiver side.

— Receiving-side semi-automatic bandwidth management through simulcast —

Simulcast is another bandwidth management option possible today with webrtc mainly targeting receiver side. It’s about getting several video tracks from the same source, at different resolutions. It increases bandwidth and CPU usage on the sender side, but gives flexibility to the Janus server to choose which one to relay through to the receiver, and thus, how much bandwidth will be used on the receiver side.

The Media Server needs to know how they are related and the bandwidth logic. E.G. relation between tracks: if track A is HD, track B is HD/4 (each dimension / 2), and track C is HD / 16, the bandwidth logic could be:

  • supposing bandwidth at start can accommodate HD, start by sending A to the remote peer, and drop B and C;
  • if at any given time the available bandwidth goes down, switch and send B,
  • and so on and so forth.

The relation between the streams needs to be sent (“announced”) by the sender to the media server through signalling first. The SFU can then check on the wire specific identifiers associated with each video track to know the relationship between them (i.e. which one is the high resolution, and which ones are the lower resolution versions of the same original track). The track choosing and relaying logic needs to be implemented in the media server. The switching between tracks to adapt to available receiver side bandwidth can then be automated in the SFU itself, or can be triggered by the receiving side through an API.

Around mid 2017, the Meetecho team added early simulcast support to the Janus server. Despite the fact that Janus typically doesn’t support multiple streams of the same kind in the same PeerConnection (that is, no support for either Plan B nor Unified Plan), with simulcast multiple video resolutions are sent as a video tracks in the same peer connection, thus partially overcoming the original single stream design of Janus. The relationship through the tracks is signalled differently for Chrome and Firefox, although the tracks are in both cases identified through their SSRC. When simulcast was first added, the SFU bandwidth management logic was semi-automatic, and triggered from the receiver side: if the stream  the receiver wants is available, Janus forwards it. Otherwise (e.g. when publisher has low bandwidth and so is only sending some of the streams, not all), Janus switches to the next available stream of lower quality.

For what concerns Simulcast’s tracks bandwidth management on the sender side, Janus server only supported RTCP for the base stream (the “first” SSRC), while it would ignore the other streams that may be there. This was due to the single stream design of Janus, and prevented the implementation of automatic sender-side bandwidth estimation and management in conjunction with simulcast. This was then fixed, and support for RTCP feedback for the additional simulcast tracks was implemented shortly after that. That said, it did not matter, sender-side automated bandwidth estimation and management was not implemented as part of that effort, which meant more work was needed to accomodate that too.

— Sender-side automatic bandwidth estimation and management (a.k.a. Congestion Control) —

There are multiple way to manage the bandwidth on the sender-side. Most of the time the adaptation itself is automated, only the estimation of the bandwidth differs. You can find references to this as congestion control sometimes. Three options co-exists today among browsers: REMB, TMMBR/TMMBN and transport-wide-cc. Long story short, transport-wide-cc is the latest and best available today across most browsers. It uses RTP extensions and feedback packets (RTCP) to pass information that allows the browsers to compute bandwidth estimation. The beauty of it is that it is fully automatic and you have nothing to do to enjoy it. That is, apart for implementing support for the corresponding RTP extensions, RTCP packets and their content.

Following the enabling contribution from Lorenzo, Sergio Murillio then contributed in January the missing pieces to support transport-wide-cc.

Janus has a modular design, and can be used in a multitude of configurations. Not all of them would benefit from having automatic sender bandwidth estimation and management. Moreover, it is always bad for backward compatibility to enable by default an option that modifies the behavior of a server. As a result, it was decided that transport-cc would be negotiated (as all other features are), and that each plugin would have a say in deciding whether to enable it or not, while keeping it off by default. In order to keep things simple and still have a realistic testbed for the new feature, this feature was only added to the VideoRoom plugin, in order to evaluate its effectiveness in scenarios involving multiple publishers and subscribers.

— Play Time is NEVER OVER —

To be able to switch between streams on the receiver side, you need to first enable simulcast on the sender side (publishers).  You just need to send a simulcast SDP for that. Then on the receiver side, you send a configure request to the janus server indicating your layer of interest (note that the term layer is usually for SVC codecs, and not for simulcast, and we conscientiously abuse it here).

You can use the default VideoRoom demo as a reference implementation. Add “?simulcast=true” as a query string to the demo URI to enable simulcast. When simulcast streams are detected, the JS code has buttons to change layer programmatically. In the videoroomtest.js file, on Line #649, one can check the “configure” request. There are two different payloads: temporal and substream which maps to temporal and spatial scalability. They are not mutually exclusive and can both appear in a single request:

{ request: “configure”, sub stream: 1, temporal: 1 }

By default the demo, and google implementation in chrome uses 3 spatial layers and 2 temporal layer.

The Janus VideoRoom plugin will automatically drops to a lower resolution if the requested layer is not available.

In the demo, the layer change is purely manual, but one could (and probably should) connect it to a network event. Basically the callback for the “slow network” event might be configured to trigger a layer change, modulo a certain duration to prevent changing too often. The logic about when you should send an event from the receiver  to ramp up bandwidth usage again is left to the application.                                                                                               

About CoSMo

CoSMo was funded in 2015 by WebRTC experts with a goal to develop tools to make WebRTC easier to use, and help Businesses adopting it. Contributing to WebRTC code since the early days of 2012, leader of the webrtc-in-webkit project, Invited expert to the WebRTC working groups, co-chair of the IMTC WebRTC interoperability group, speakers and chairs of several RTC conferences, the CoSMo team is part of Real-Time Communications DNA, from standard to implementation. They provide a wide range of technical expertise on WebRTC, with a focus on system level design, technical due diligences, and Customisation/Integration of the WebRTC stack. Today, CoSMo is a team of almost 30 individuals, working across four countries and two continents. More information at www.cosmosoftware.io

About Meetecho

Meetecho was born in 2009 as an official academic spin-off of the University of Napoli Federico II. Since Day One they’ve been working hard on real-time multimedia applications over the Internet, ranging from VoIP to more advanced applications based on top of the emerging WebRTC technology. Today’s Meetecho team is composed of world level experts in Real-Time Communication, proud authors of the Janus® WebRTC server! They provide design and implementation consulting services of WebRTC products on top of Janus®, ad-hoc solutions for streaming of live events to the world with remote participation, as well as Ready-to-use web based conferencing and collaboration services. Their website is listing the major companies already trusting them.

About the WebRTC A-Team

10 years ago, a crack student unit was sent to PhD by a European court for WebRTC, a technology they didn’t write. These men promptly graduated from maximum security universities and vanished into the Napoli and Singapore Undergrounds. Today, still wanted by their government-paid past teachers, they survive as WebRTC consultants. If you have a WebRTC problem, if no one else can help, and if you can find them, maybe you can hire… the WebRTC A-Team.


Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.