When using Bazel within a large monorepo, there comes a time when memory starts becoming a problem, and it is important to be aware of the available tools that can help with diagnosis. Fortunately, Bazel offers Starlark memory profiling, which we can utilize to see how much memory is being swallowed by the analysis phase.

Setting up

Surprisingly, it is a little tedious to set this up, but in the end, it is not that difficult either.

First, we need to download the allocation instrumenter JAR from Maven Central. Once that’s on disk, we can start Bazel in memory tracking mode like so:

bazel --host_jvm_args=-javaagent:/path/to/java-allocation-instrumenter-3.3.4.jar \
      --host_jvm_args=-DRULE_MEMORY_TRACKER=1 \
      build --nobuild //path/to/target

Here, we start Bazel with memory tracking enabled and pass --nobuild so that only analysis is performed, which is what we want to measure.

After analysis completes, make sure not to shut down the Bazel server because you’ll lose the gathered data.

Now, to produce a memory profile, we’ll use Bazel’s dump command:

bazel --host_jvm_args=-javaagent:/path/to/java-allocation-instrumenter-3.3.4.jar \
      --host_jvm_args=-DRULE_MEMORY_TRACKER=1 \
      dump --skylark_memory=memory_profile.gz

Notice how we repeat the startup flags:

--host_jvm_args=-javaagent:/path/to/java-allocation-instrumenter-3.3.4.jar
--host_jvm_args=-DRULE_MEMORY_TRACKER=1

Failing to do so will result in a Bazel server restart, and you’ll have to repeat the whole process from the beginning.

Dealing with gathered data

Now that you have memory_profile.gz, you need to make sense of it. The Bazel docs recommend using pprof to inspect the profile.

There doesn’t appear to be a pprof formula in the main Homebrew formula index, but installing it is straightforward if you already have Go:

go install github.com/google/pprof@latest

The binary is installed into $GOPATH/bin, which is $HOME/go/bin by default.

Once installed, a useful starting point is a flame graph:

pprof -flame memory_profile.gz

or a text report annotated with source lines:

pprof -text -lines memory_profile.gz

Conclusion

Most Bazel users won’t ever need to gather a memory profile, but I believe it is good to be aware that this exists so you can reach for it when needed. Rule authors should be especially aware of it, as it is not that difficult to make a coding mistake in a rule that results in huge memory usage.

You can find more information about optimizing performance in the official Bazel docs.