I.Introduction

libwebrtc is the base of many products. It is of course the base of the webrtc JS API implementation in chrome, firefox and opera, but it is also the base of a lot of mobile SDKs and other products out there. Keeping an updated version, based on a stable code base, tested, has been notoriously difficult if you’re not a google employee. This post will be the first one in a serie of post explaining step-by-step how to set up a system that can automatically compile test and package libwebrtc for you, for several platforms. It follows a previous post that explained how the google build system works (depot tools), and will focus more on the CMake part of the automation.

II. There are already (a few) very good resources out there

There is not a “best way” to compile libraries. As long as the library are built, tested, and integrated automatically, the job is done. Specifically for mobile the pristine scripts (see below) are doing a good job at this. I m biased toward CMake since I worked with it for many years, however, there are reasons why i kept working with it for so long. Not having to deal with compiler differences is a joy, not having to deal with OS differences is also a joy. Being able to add test on the fly with simple command, which in turn will also enable sending result to a dashboard is an overjoy, especially when support for valgrind and coverage comes out of the box, is AWESOME, and finally, installing and packaging is equally made easy (yes, cross platform). Following those blog post, you will be able in a few days to set up a state of the art compilation, testing, and packaging system for libwebrtc, where it would take (from my experience) much longer. Of course, if you have an existing testing infrastructure, and CI solution, that might not play along well, but it’s often best to separate building the webRTC libs and building whatever you build around it, be it a webrtc plugin, or a cordova wrapper (see below) or a mobile SDK.

further reading

• http://tech.pristine.io/build-ios-apprtc/
• https://github.com/pristineio/webrtc-build-scripts
• http://tech.pristine.io/automated-webrtc-building/
• https://github.com/eface2face/cordova-plugin-iosrtc

III. How to best use CMake

CMake is a cross-platform and cross compiler configuration manager, sometimes called a meta-build system. Its main purpose is to find libraries and program for you on the host computer, set up targets (executable, libraries, tests, ..), whatever the operating system and the compiler you use.

At the bare minimum, you can use it like any other script language and try to encapsulate the webrtc build system in system calls. the “execute_process” command would be use for that purpose. It has the disadvantage of doing everything at configuration time, and linearly. A second call to the script would re-run everything, every time. It is preferable to set up targets and dependencies at configuration time (the only time you would directly run cmake) and then to use the native compiler to handle the rest, in a resumable fashion.

A full usage of cmake would be to use it to replace entirely webrtc build system, and redefine all libraries and executable, the corresponding source files, compilation options, etc. While this might be tractable for single executable, or libraries, doing it for all of webrtc would mean migrating all the .gyp[i] files and keeping them synchrone at every update. Anybody that look at the mess the 6,000 lines src/build/common.gypi is, and still be sane, would shy away from this.

In this case we are going to take the “super build” approach, and let cmake drive the google build system, import and run the tests, add coverage support, prepare packages and so on and so forth. It allows to use the google build system untouched for the basics, and to use CMake where it’s best.

IV. Let’s do it

1. A new CMake module to detect depot tools

CMake has a collection of modules installed alongside to make detecting and using your usual libraries and packages easier, cross platform. Most of those modules are made of CMake code in a file named FindXXX.cmake. From your code you can use it by invoking the “find_package( XXX )“. If the package is mandatory, you can tell find_package to fail right away by adding the REQUIRED argument. For example, CMake comes with a FindGit.cmake pre-installed. In our code we can simply write this:

1. find_package(Git REQUIRED)

Now, depot tools is not yet recognize by vanilla CMake. However CMake has a mechanism for projects to extend its capacity. You can write your own FindXXXXX.cmake modules and let CMake know about them by setting the CMAKE_MODULE_PATH variable in your code. You should deb careful to extend and not overwrite the variable, in case your code is called from another CMake code (super build style). In our case, we put the scripts in a “Cmake” directory right under the root directory.

1. set( CMAKE_MODULE_PATH
2.   ${CMAKE_MODULE_PATH} # for integration in superbuilds
3.   ${CMAKE_CURRENT_SOURCE_DIR}/Cmake
4.   )

You can see the corresponding full code here.

Now we still need to code the FindDepotTools.cmake. CMake provides all the primitives to do this. In our case, we just find client and we’re good. The rest is handled by CMake for us, including OS and Paths differences.

1. find_program(DEPOTTOOLS_GCLIENT_EXECUTABLE
2.   NAMES gclient gclient.bat # Hints about the name of the exe. client.bat is the windows version.
3. ) 
4. # below this line is standard CMake stuff for all packages to handle QUIET, REQUIRED, ….
5. include(${CMAKE_ROOT}/Modules/FindPackageHandleStandardArgs.cmake)
6. find_package_handle_standard_args(DepotTools
7.   REQUIRED_VARS DEPOTTOOLS_GCLIENT_EXECUTABLE
8.   FAIL_MESSAGE “Could not find the gclient executable.”
9.   )

Now our main script core is relatively easy to write.

1. project(webrtc)
2. set(CMAKE_MODULE_PATH
3.   ${CMAKE_MODULE_PATH} # for integration in superbuilds
4.   ${CMAKE_CURRENT_SOURCE_DIR}/Cmake
5.   )
6. find_package(DepotTools REQUIRED)
7. find_package(Git               REQUIRED)

You can see the corresponding full code here.

2. Implement “gclient config” as a CMake custom command and custom target.

Now, we want to run the “client config” command line from CMake. This command line will create a .gclient file which should not exist beforehand. If the file exist, the command should be a noop. Moreover, I want the build system to try every time it is run. I want it work both across Operating systems.

let’s first define what the command should be, in a cross-platform manner.

1. set( gclient_config
2.   ${DEPOTTOOLS_GCLIENT_EXECUTABLE} config # gclient has been found before automatically
3.   –name src
4.   https://chromium.googlesource.com/external/webrtc.git
5.   )
6.   if(WIN32)
7.     set(gclient_config cmd /c ${gclient_config}) # Windows command prompt syntax
8.   endif()

The first constraints are met with CMake’s add_custom_command(). It allows to create a CMake command that will generate a file as output and will not be triggered if the file already exist.

1. add_custom_command(
2.      OUTPUT   ${CMAKE_SOURCE_DIR}/.gclient
3.      COMMAND  ${gclient_config}
4.    )

However, nothing tells you in the script when to run this custom command. That’s where the add_custom_target() comes in the picture. It creates a (potentially empty) target for the chosen build system, to which commands can attach themselves. Also, it helps defining dependency with other targets to define build/execution order. In the case of client config, there is no previous step, so there is no dependency. By making the target depend on the file generated by the command you automatically link them. The ALL parameter always include this target in the build.

1. add_custom_target(
2.      webrtc_configuration ALL
3.      DEPENDS ${CMAKE_SOURCE_DIR}/.gclient
4.    )

You can see the corresponding full code here.

3. Implement “gclient sync” and add dependency

Implementing “gclient sync” is more or less the same: define the command, add a custom command, then add a custom target. This time though, we want this target to only execute AFTER “client config”. So we need to add a dependency. The code is actually self explanatory. Note that we force sync NOT to run the hooks (-n option).

1. set(gclient_sync ${DEPOTTOOLS_GCLIENT_EXECUTABLE} sync -n -D -r 88a4298)
2.    if(WIN32)
3.      set(gclient_sync cmd /c ${gclient_sync})
4.    endif()
5.   add_custom_command(
6.      OUTPUT  ${CMAKE_SOURCE_DIR}/src/all.gyp 
7.      COMMAND ${gclient_sync}
8.    )
9.   add_custom_target(
10.      webrtc_synchronization ALL
11.      DEPENDS ${CMAKE_SOURCE_DIR}/src/all.gyp
12.    )
13.    add_dependencies( webrtc_synchronization webrtc_configuration )

4. Implement “gclient runhooks”

Gclient runhooks will generate the build files, wo we need to be sure that the most obvious parameters are set beforehand. Ninja handles Release and Debug at build time, so no need to take care of that now, but if you had chosen another type of compiler, you might need to do it now. the switch between 32 and 64 bits architectures (or arm flavors for mobile) are handled through environment variables. CMake has a proxy function to access those:

1. if(DEFINED ENV{GYP_DEFINES})
2.   message( WARNING “GYP_DEFINES is already set to ENV{GYP_DEFINES}”.
3. else()
4.   if(APPLE)
5.     set(ENV{GYP_DEFINES} “target_arch=x64”)
6.   else()
7.     set(ENV{GYP_DEFINES} “target_arch=ia32”)
8.   endif()
9. endif()

Then the rest of the code is pretty much the same as before:

1.  set(gclient_runhooks ${DEPOTTOOLS_GCLIENT_EXECUTABLE} runhooks)
2.    if(WIN32)
3.      set(gclient_runhooks cmd /c ${gclient_runhooks})
4.    endif()
5.    add_custom_command(
6.      OUTPUT  ${CMAKE_SOURCE_DIR}/src/out # it’s arbitrary.
7.      COMMAND ${gclient_runhooks}
8.    )
9.    add_custom_target(
10.      webrtc_runhooks ALL
11.      DEPENDS ${CMAKE_SOURCE_DIR}/src/out
12.    )
13.    add_dependencies(webrtc_runhooks  webrtc_synchronization)

Full code for sync and unhook can be seen here.

5. Prepare the Build with Ninja target

As mentioned previously, we have to deal with Debug / Release modes here. We follow the CMake syntax, which luckily fits ninja syntax.

1. if(NOT CMAKE_BUILD_TYPE)      # allow to set it in a superscript and honor it here
2.   set(CMAKE_BUILD_TYPE Debug) # Debug by default
3. endif()

We can then run the usual.

1.    set(webrtc_build ninja -v -C ${CMAKE_SOURCE_DIR}/src/out/${CMAKE_BUILD_TYPE})
2.    add_custom_command(
3.      OUTPUT  ${CMAKE_SOURCE_DIR}/src/out/${CMAKE_BUILD_TYPE}/libwebrtc.a # it’s arbitrary.
4.      COMMAND ${webrtc_build}
5.    )
6.    add_custom_target(
7.      webrtc_build ALL
8.      DEPENDS ${CMAKE_SOURCE_DIR}/src/out/${CMAKE_BUILD_TYPE}/libwebrtc.a
9.    )
10.    add_dependencies(webrtc_build webrtc_runhooks)

6. Finally building!

you can then run cmake once and then launch the build with your native compiler. This code has been tested on mac (use “make”) and windows (open webrtc.sln in MSVC, and build the ALL target, or hit F7).

This work by Dr. Alexandre Gouaillard is licensed under a Creative Commons Attribution 4.0 International License.

This blog is not about any commercial product or company, even if some might be mentioned or be the object of a post in the context of their usage of the technology. Most of the opinions expressed here are those of the author, and not of any corporate or organizational affiliation.