This post is also available in: Français (French)

An updated version of this post has been posted here.

What is WebRTC again?

WebRTC is a set of browser APIs and protocols being worked on by the W3C and IETF standardization bodies. With WebRTC, developers can quickly add real-time peer-2-peer audio, video and data capabilities to their web applications through a set of standardised JavaScript APIs.

WebKit today lacks support for this exciting new standard. The intention of the webrtcinwebkit.org project is to add WebRTC support to WebKit, starting with the WebKit GTK+ port (Linux), by means of the OpenWebRTC implementation. Much of the WebRTC support will be implemented in the core of WebKit and therefore shared among all WebKit ports. This will also enable integration of other WebRTC backends such as webrtc.org. The intent was also to make it easier for Apple, if they decided to, to add support for webrtc in safari, and for apps on iOS through the web view framework.

This morning I tweeted about apple finally adding support to safari in webkit, and I have been flooded by questions ever since, so I thought I’d write a post.

Past and current efforts

As you know, Stefan H from Ericson R&D, and Chair of the webrtc working group @ w3c and myself have been instrumental in starting the webrtc in webkit initiative. Adam B, the openwebrtc team (stefan A, robert S, …), igalia and centricular (Sebastian D) were the main contributors to the effort as far of coding was concerned, with small contributions from two french interns I used to supervise, Yin Liu from INSA Lyon (Data Channel), and Adam Tiamou from Grenoble University (using webrtc.org instead of OWR for the platform layer).

Recently, the project had managed to push most of the foundation for Media Streams and GetUserMedia in the common part of webkit (webcore), as explained in this july post.

At this time, the final note was giving hints about what was to come:

“Some things still need more work, like the Constraints mechanism and the functionality to enumerate local devices, but the foundation is now up to date and testable.”

Browsers and Ports

webkit is a browser engine. Each browser user webkit is called in the webkit community a “port”. webrtcinwebkit uses the linux port (webkitGTK+) to test the implementations of all the layers, even though in theory everything in webcore is reusable by all ports. Since July, you can test getUserMedia in the linux browser.

Safari on mac is another port, and so is safari on iOS. One does not have access to the specific browser layer code unless one works at Apple. However, you can recompile webkit and tell safari to use that DLL instead for testing.

Following webrtcinwebkit’s Ericsson’s adam’s push on july of all the generic plumbing for MediaStream, apple has been hard at work to bring the apple specific parts to the source code. Three things in the list of commits from apple are telling (see the extracted list of commits here until september 28):

• Changes are specific to mac and iOS
  • /platform/mediastream/mac
  • /platform/mac-mavericks/
  • /platform/mac-yosemite/
• Changes use AVFoundation API (AVVideoCaptureSource.h)
• Changes uses apple only build projects (WebCore.xcodeproj/project.pbxproj)

So what was implemented by apple exactly?

For background information on difference of webrtc implementations between browsers and stack, you can go look at these slides.

Basically, apple just implemented all the specific plumbing for GetUserMedia and MediaStream supports on mac and iOS. Support for audio/video devices enumeration and selection, support for media stream as a source for <audio> and <video> DOM elements, support for capturers (through AVFoundation), support for snapshots of MediaStreams, …

Of course, this needed a solid implementation that respect the layering of the browsers. The capturing and media handling needs to be done at the browser level to allow for sharing streams between tabs and for global echo cancellation. The rendering needs to happen within the tab, sandboxed from other tabs, so special attention was given to the layering and message/data passing between each layers.

In short, with this code, all features of GetUserMedia and MediaStreams seem to be implemented in the mac ports of webkits. It is almost all the Device API specifications that indeed are more advanced than the webrtc (peer connection) specifications as they are in “Last call” status.

I hope it finds its way soon in safari, or at least in apple provided web view for iOS.

Next step would be the implementation of peer connection and data channels, parts which belongs to the main webrtc specs, but are still in flux.

This was first published in WebRTC by Dr Alex.

Webrtc in Safari?

This work by Dr. Alexandre Gouaillard is licensed under a Creative Commons Attribution 4.0 International License.

This blog is not about any commercial product or company, even if some might be mentioned or be the object of a post in the context of their usage of the technology. Most of the opinions expressed here are those of the author, and not of any corporate or organizational affiliation.