Remote Build Execution Setup

Getting started with Remote Build Execution (RBE) is less daunting than it may seem. We've put together a guide that not only helps you get started with BuildBuddy RBE, but also helps you understand what is going on under the hood.

The basics

The very simplest Bazel command needed to enable RBE is the following:

bazel build //... --remote_executor=grpcs://cloud.buildbuddy.io

This points Bazel at BuildBuddy Cloud as a remote executor. A simple repo that has C/C++/CGO or Java dependencies will build just fine like this. Most interesting repos have some dependencies on C/C++/CGO or Java - so we'll need to tell our remote executors where to find tools like gcc or the JRE. We do this with platforms and toolchains.

Configuring your workspace

There are several options for configuring your platforms and toolchains, the most fully features of which being bazel-toolchains. It comes with an rbe_autoconfig rule that works nicely with BuildBuddy.

Unfortunately, bazel-toolchains has a dependency on Docker and can take quite some time to start up in a clean workspace, so we provide a simple and easy-to-use BuildBuddy toolchain that enables you to get up and running quickly, and works for most use cases.

To get started with the BuildBuddy Toolchain, add the following lines to your WORKSPACE file:

http_archive(
    name = "io_buildbuddy_toolchain",
    strip_prefix = "toolchain-master",
    urls = ["https://github.com/buildbuddy-io/toolchain/archive/master.tar.gz"],
)

load("@io_buildbuddy_toolchain//:rules.bzl", "register_buildbuddy_toolchain")

register_buildbuddy_toolchain(name = "bb", llvm = False)

You'll likely want to pin this to a specific commit/version and add a SHA for reproducible builds.

Platforms

The first thing you'll want to do is tell BuildBuddy RBE in what environment you'll want to run your build actions. This is tools can be found in different locations on different platforms. This is done with the --host_platform flag.

BuildBuddy's default platform is Ubuntu 16.04 with Java 8 installed. We can specify this platform with the --host_platform flag:

--host_platform=@bb//:platform

If you want to use a different environment, you can specify a custom Docker container image to use. More information on how to do this can be found in our platforms documentation.

Toolchains

Toolchains sound complicated (and they can be) - but the concept is simple. We're telling our remote executors where to find tools that are needed to build our code.

C toolchain

The first toolchain you'll likely run into the need for is a C/C++ compiler. Even if your code isn't written in one of these languages, it's likely that one of your dependencies is - or calls some C code with something like cgo.

You'll know you need a C toolchain when you see an error for a missing gcc or clang that looks like:

exec: "/usr/bin/gcc": stat /usr/bin/gcc: no such file or directory

To use BuildBuddy's default C toolchain, we can use the --crosstool_top flag:

--crosstool_top=@bb//:toolchain

Java toolchain

If your project depends on Java code, you'll need 4 more flags to tell the executors where to look for Java tools.

Using BuildBuddy's default Java 8 config:

--javabase=@bb//:javabase_jdk8
--host_javabase=@bb//:javabase_jdk8
--java_toolchain=@bb//:toolchain_jdk8
--host_java_toolchain=@bb//:toolchain_jdk8

If you need a different version of Java, we recommend using bazel-toolchains for now.

Putting it all together

Now that we've got our platform and toolchains setup - we can do a remote build that includes C & Java dependencies with a command like:

bazel build //... --remote_executor=grpcs://cloud.buildbuddy.io --crosstool_top=@bb//:toolchain --javabase=@bb//:javabase_jdk8 --host_javabase=@bb//:javabase_jdk8 --java_toolchain=@bb//:toolchain_jdk8 --host_java_toolchain=@bb//:toolchain_jdk8

This can be a lot of flags to tack onto each bazel build, so instead you can move these to your .bazelrc file:

build:remote --host_platform=@bb//:platform
build:remote --crosstool_top=@bb//:toolchain
build:remote --javabase=@bb//:javabase_jdk8
build:remote --host_javabase=@bb//:javabase_jdk8
build:remote --java_toolchain=@bb//:toolchain_jdk8
build:remote --host_java_toolchain=@bb//:toolchain_jdk8

And running:

bazel build //... --config=remote

Authentication

You'll want to authenticate your RBE builds with either API key or certificate based auth. For more info on how to set this up, see our authentication guide.

Optimizations

By default, bazel will download intermediate results of remote executions - so in case an artifact isn't found in the remote cache, it can be re-uploaded. This can slow down builds in networks constrained environments.

This can be turned off with the flag:

--remote_download_minimal

While this flag can speed up your build, it makes them more sensitive to caching issues - and likely shouldn't be used in production yet.

Need more help?
Get in touch