I first started writing about libwebrtc source management, build and test systems almost 5 years ago. While the posts are still here, and mostly accurate, people forget, and/or the system as changed just enough that we need to update what is the de-facto reference for libwebrtc. As we are writing a book, with examples and illustration to be used in classrooms to teach the underlying principles, and in companies for e.g. on boarding Engineers, we though we should put some extract here.

Intro and history

The chrome/libwebrtc build system is complex and fully integrated. It has been designed, and optimised time after time as the code base grew, the team grew, the number of target architecture grew, and the number of downstream products grew. If you are a single developer in your basement, working on a single project, with a single programming language, there are a lot of “optimisations” that come with the system that becomes overhead for you. Also, some parts of the system (patch and code review in gerrit, distributed build with GOMA, test isolation and distributed testing with LUCI, …. ) that most outside google do not have access to. Do not let that upset you, don’t throw your toy and start a new competing webrtc implementation, there is a lot you can do without access to those tools. Today, first things first, the source code management.

To understand the source code management part of the google build systems and tools (depot_tools), one has to go back in history to a time where git was not ubiquitous. Long time ago, in a far far away land, people had a lot of different Source code management solutions, including but not limited to SVN, CVS, Mercurial, …. Moreover Git submodules did not exists. The temptation to have one tool to rule them all (see the cartoon above) was high.

gclient, the google client for their infrastructure and the corresponding data structures were created to implement the conceptual equivalent of git submodules and to implement an overlay above the source code management software to abstract away the differences. You can mix GIT and SVN repositories for examples, and have a command that will act conceptually as a git command across all the submodules.

Configure

First thing first, the main repository needs to be set up by “gclient configure”, which is hidden behind the “fetch command”. We prefer to document here the configure command, because it allows more flexibility, as explained later. The result of the configure command is a .gclient file on disk with the following content by default:

solutions = [
  { "name"        : "src",
    "url"         : "https://chromium.googlesource.com/external/webrtc.git",
    "deps_file"   : "DEPS",
    "managed"     : False,
    "custom_deps" : {
    },
    "custom_vars": {},
  },
]

For liwebrtc it is mandatory to use “src” as the name of the directory in which the code will be cloned, otherwise one of the script run by gclient will detect it and fail. If you have your own fork of the main libwebrtc code, which use the same submodules that the main code, you can simply replace the url here with yours at this stage, and you re done. This is the clean way to make a clone of the webrtc project, which allow easy rebasing in the future. We will come back to the “custom_xxxx” fields later, once we will have described the DEPS mechanism.

Note that the only difference for iOS and android is to add an additional line which request that the target OS be iOS or android. by default, the target OS is set to the host OS.

DEPS mechanism and the sync command

The git submodule mechanism is a nice way to make sure you get all the dependencies you need for your project, with the right revision for each. The google equivalent is the DEPS mechanism. The “DEPS” file name can varies and can be passed through the configuration “deps_file” field. It allows more flexibility: a single repository can list several DEPS file, and you can switch between them at configuration time.

a DEPS file contains three things: the vars, the deps, and the hooks. the vars are just aliases that allow the file to be more concise. The deps are very similar to submodules, they contains a local directory where the repository should be cloned, and the repository to clone. Note that it can handle both git and svn repos, … . The parsing of the vars and the deps must happen after the main repository has been cloned (so DEPS exists). It generates a flatten list of dependencies in the .gclient_entries file on disk. The cloning of those entries happens at “sync” time. The hooks, which you are going to learn to hate, are arbitrary pieces of code that will be run at sync time as well. It is usually download of pre-compiled binaries, the toolchain, the files needed as input for the tests, and so on and so forth. Interestingly, the DEPS file is a little bit smarted than you usual configuration file, as you can make some of the dependencies conditional to, e.g., a specific OS.

To be thorough, there are more entries in a DEPS file than just those three, you can find recursedeps, include_rules, and some other less used entry, but the above three will occupy most of your time.

solutions = [
  { "name"        : "src",
    "url"         : "https://chromium.googlesource.com/external/webrtc.git",
    "deps_file"   : "DEPS",
    "managed"     : False,
    "custom_deps" : {
    },
    "custom_vars": {},
  },
]

Let’s revisit now the file generated by the configure command. “custom_deps” and “custom_vars” allow you to provide and insert your own deps and vars that are either going to add to the list, or more likely overwrite the existing ones. This way you can chose a different revision number, and/or a different repository for any given dependency. Let’s say you have your own fantastic implementation of libsrtp for encryption, or libsctp for datachannel, and you would like to see the impact on libwebrtc, just go ahead and replace the corresponding entry to point to your repo.

Note that the above works with upstream project as well. If you have a modified version of liwebrtc that you want to use in Chrome or in electron, you can do exactly the same thing. You can use our implementation of PERC_lite in webrtc 61, 65, 69 (here) in chromium to enable End-To-End Media encryption by modifying the custom_deps entry of the .gclient file.

If your dependencies modification are heavier, with multiple lines, etc, you might just want to keep those modifications in the DEPS file of a clone main directory.

Conclusion / wrap-up

Today we have learned how to properly configure a libwebrtc project. We have learn how to use different dependencies than the default one, including our own repository in lieu and place of external one. Adding a new codec, replacing encryption, ICE, <name it> by your own can be very easily achieved thanks to the DEPS mechanism.

The goal of next blog post will be to make you discover the bases of GN: the build configuration mechanism. We will write about the notion of build args, external vs internal args, gn args, chrome args, rtc specific args, toolchains management, targets types, targets configuration. At the end of the post, one should be able to quickly and easily find the source code for a given target, the includes, the compilation flags, the defines, …. as well as quickly navigate through the BUILD.gn files in the repository. Special builds/toolchain/configuration will be quickly pointed out (coverage, sanitisers, fuzzer, ….) but not thoroughly explained.