I always find BazelCon to be an exceptional experience, and this year was no exception. From the things I learned by randomly talking to people, to meeting familiar faces and learning from the talks and BOF sessions, it is simply an incredible experience. Today I want to go over some things I observed and what I am looking forward to in Bazel 10.

AI is everywhere

Nowadays, when we are all using AI agents every day, I find that using AI is becoming just like using any other tool. There was no talk about doomsday scenarios, but rather encouragement to use AI even more. On more than one occasion, people would say “point your agent at the documentation” instead of reading the documentation yourself. Also, EngFlow’s report on how much people are using AI was eye-opening. It’s not like I didn’t expect it, but seeing the numbers really puts things into perspective.

IDEs as we know them are going away

It seems that nobody is focusing on IDEs as much as we did before. For example, we’ve seen the use of rules_xcodeproj going down in some large organizations in favor of more lightweight alternatives like VS Code. I feel like the next opportunity for IDEs is a more integrated code review experience and better code navigation. Investing heavily in better autocomplete seems wasteful since fewer people are relying on it as their primary way of writing code. Sure, it is always good to have it in some shape or form, but I don’t expect it to be the main focus of IDE improvements in the coming years.

Bazel 10

Bazel 10 continues the effort of moving language-specific functionality out of the core and into Starlark extensions. There are also a lot of welcome performance improvements, some of which are being accelerated by the use of AI.

Project Skymetal

This is an effort to replace parts of Bazel’s Java implementation with native code, currently focusing on Rust. It is going to take some time, as the Bazel team won’t simply replace all Java with Rust overnight. Instead, the plan is to migrate component by component, using JNI to bridge Java and Rust. Early results are promising, but I expect we’ll get a better picture of how it plays out once more of the codebase has been migrated.

Skycache

Skycache is basically an attempt to remotely cache Bazel’s analysis state, which is becoming more expensive to compute over time, especially in larger monorepos. This effort started last year, and by the looks of it, it is gaining more traction. In fact, Google is already experimenting with an early implementation internally, though their results do not automatically translate to analysis caching for everybody, as Google relies on infrastructure such as its cloud-based filesystem.

Stripping native flags

Rulesets like those for Apple platforms have historically relied on Bazel’s native flags for CPU selection and various other things. The effort to remove these language-specific flags from Bazel core is continuing, with equivalents being provided through Starlark-based configuration, including the apple_support module.

Talks and other relevant information

Snapshotting Bazel server state

This is an interesting approach to snapshotting the entire Bazel server state so that we don’t have to pay the analysis cost every time. Of course, this can be fragile, but I believe it can be useful if done carefully.

Hermetic LLVM

Hermetic LLVM got a lot of attention during this BazelCon, and I think it is well deserved. David and Corentin did an amazing job of providing a fully hermetic C/C++ toolchain, which has always been a challenge in Bazel. Hopefully, this finally puts the issue to rest.

rules_rs

Similar to Hermetic LLVM, rules_rs is getting a lot of well-deserved attention due to its fast Cargo dependency resolution and strong emphasis on hermeticity.

Sandbox on macOS

Of course, it is well known that Bazel sandboxing on macOS is not as performant as on Linux, so folks over at Aspect have been working on providing a way to plug in a custom sandboxing binary. Hopefully, someone finds a way to finally make sandboxing on macOS as fast as it is on Linux.

Any to Bazel

This is a quick note on any2bazel, a tool developed by EngFlow that aims to help humans and AI agents migrate projects to Bazel. It uses a deterministic approach to compare what the original build system actually compiles and links against what Bazel produces. This helps guide agents toward a correct migration, because simply getting a green build is not enough.

Apple Birds of a Feather

The last time we had an Apple BOF was in Mountain View, California. This time, there was a lot of discussion about excessive disk usage in tests when statically linking dependencies. The quick fix is to use dynamic frameworks, but that comes at the cost of slower startup times and potentially other issues. Another popular topic was Linux-to-macOS remote build execution (RBE), where Linux is used as the CI runner and macOS machines as remote executors. This effectively allows us to initiate builds from Linux while executing Apple-specific build actions on macOS. There was also a discussion about building Apple targets directly on Linux, but the potential licensing issues make this approach unappealing to large organizations. Of course, the Apple BOF would not be complete without mentioning Swift explicit modules support in rules_swift. Thank you, Keith, for doing the work!

Conclusion

There you have it. Yet another incredible BazelCon behind us, and I am already looking forward to the next one. P.S. I know I haven’t covered a lot of ground here, but these are just my quick notes and observations.

Ever since Swift gained the ability to control warnings via diagnostic groups in SE-0443, we’ve been able to promote specific diagnostic groups to errors.

However, passing these flags via copts on swift_library targets isn’t always convenient. To apply them globally, we’d either need to modify every swift_library target, pass them via the command line, configure them in .bazelrc, or use Bazel macros to apply them automatically. All of these approaches have their downsides, such as accidentally applying the flags to external repositories, introducing additional complexity through macros, or making it difficult to opt out for specific targets.

In the past, I’ve written about Bazel’s features mechanism, as well as how to suppress warnings in external repositories.

Bazel feature-based approach

In rules_swift 4.2.0, we added a way to control these warnings using Bazel’s features mechanism in #1960.

Simply build a target while passing --features, like so:

bazel build //... --features=swift.werror.DeprecatedDeclaration

And all warnings belonging to the DeprecatedDeclaration diagnostic group will be reported as compiler errors.

This approach is extremely flexible, as it allows per-target control via the features attribute, repository-wide configuration via REPO.bazel, which only applies to the current Bazel repository, and, of course, configuration via the command line or .bazelrc.

Furthermore, we can take advantage of Bazel’s feature negation mechanism by prefixing feature names with -, allowing us to opt out of specific features when needed.

For example, if we’ve enabled swift.werror.DeprecatedDeclaration globally but want to opt out for a specific target, we can simply negate the feature in its features attribute:

swift_library(
    name = "legacy_library",
    srcs = ["Legacy.swift"],
    features = ["-swift.werror.DeprecatedDeclaration"],
)

This way, deprecated declaration warnings will remain warnings for legacy_library, while still being treated as errors for the rest of the project.

Conclusion

There we have it: a simple and clean way to control Swift warnings with Bazel.

P.S. I wish Swift would give us a way to suppress warnings per diagnostic group. If that ever comes to fruition, I’ll be the first to add a feature-based approach to rules_swift.

When writing a Bazel module extension, we sometimes need a way to let the user pass some values to configure things, download binaries from a specified URL, etc. A Bazel module extension itself does not support attributes in the same way repository rules and normal rules do. Instead, it offers another API called tag_class, which lets us define attributes for tags exposed by a module extension.

A single module extension can have multiple tag classes. During extension evaluation, we can iterate over the modules that use our extension and inspect the tags of the class we are interested in.

Downloading a binary

For the purposes of this simplistic article, we will define a very crude repository rule to download any sort of binary. Then we will wire it up through a module extension, which will allow the user to specify the URL and SHA-256 value.

Our repository rule will look like this:

def _binary_download_impl(ctx):
    ctx.download(
        url = ctx.attr.url,
        sha256 = ctx.attr.sha256,
        output = ctx.attr.file_name,
    )

    ctx.file(
        "BUILD.bazel",
        'exports_files(["%s"], visibility = ["//visibility:public"])' % ctx.attr.file_name,
    )

binary_download = repository_rule(
    implementation = _binary_download_impl,
    attrs = {
        "url": attr.string(mandatory = True),
        "sha256": attr.string(mandatory = True),
        "file_name": attr.string(mandatory = True),
    },
)

Then we need to make a module extension with a tag class:

binary = tag_class(
    attrs = {
        "name": attr.string(mandatory = True),
        "url": attr.string(mandatory = True),
        "sha256": attr.string(mandatory = True),
    },
)

def _impl(module_ctx):
    for module in module_ctx.modules:
        for binary_tag in module.tags.binary:
            binary_download(
                name = binary_tag.name,
                url = binary_tag.url,
                sha256 = binary_tag.sha256,
                file_name = binary_tag.name,
            )

binary_download_extension = module_extension(
    implementation = _impl,
    tag_classes = {
        "binary": binary,
    },
)

There are two loops here that are worth pointing out.

module_ctx.modules contains the modules participating in our extension. Each module can then have zero or more binary tags, which we access through module.tags.binary.

For every binary tag we find, we call our repository rule and forward the values from the tag to it.

Finally, the user can configure the extension from their MODULE.bazel:

binary_download = use_extension(
    "//:extensions.bzl",
    "binary_download_extension",
)

binary_download.binary(
    name = "some_binary",
    url = "https://example.com/some_binary",
    sha256 = "...",
)

use_repo(binary_download, "some_binary")

And that’s it.

binary_download.binary(...) creates a tag containing the values we defined in our tag_class. When Bazel evaluates the module extension, _impl receives those tags through module_ctx, and our extension can use them to create repositories.

In this case, the flow is essentially:

MODULE.bazel
    |
    | binary(name, url, sha256)
    v
tag_class
    |
    v
module extension
    |
    | binary_download(...)
    v
repository rule
    |
    v
downloaded binary

Conclusion

This stuff is reasonably well covered in Bazel’s official docs, but I felt the need to try and simplify it further. It is also true that I haven’t come across anything more interesting in the past week. So, looking forward to the next one.

When running a binary through Bazel with bazel run, we might find ourselves in a situation where it is desirable to wrap it in a “parent” binary or script. This could be to profile the running binary, fiddle with its environment, run it through a debugger, or any sort of thing really.

To achieve this, we could build the binary and then manually wrap it with whatever we want, but that is a bit tedious and perhaps prone to errors.

Using –run_under

Bazel has a flag specifically for this use case: --run_under. Simply put, it prepends a command to the executable being run.

It is a rather old Bazel feature — --run_under was already around in 2015, during the early days of the public Bazel project.

For example, if we wanted to measure the CPU time of a rules_xcodeproj executable target, we could do something like this:

bazel run //:xcodeproj --run_under="time"

This effectively results in Bazel running something along the lines of:

time <path-to-xcodeproj>

and we get the timing summary once the executable finishes.

The wrapper does not have to be a single command either. --run_under accepts a command prefix, so arguments can be passed to the wrapper as well. For example:

bazel run //:xcodeproj --run_under="time -p"

It is worth noting that --run_under is not specific to bazel run — it can also be used with bazel test.

Going further

Fortunately, this flag is not only able to wrap the binary with something available on the host OS. It can also refer to another executable Bazel target.

This means we can have a *_binary target in our repo and, by referring to it using normal Bazel target label syntax, Bazel will build both targets and use one to wrap the other:

bazel run //:xcodeproj --run_under="//tools:my_wrapper"

Conceptually, this becomes something like:

my_wrapper xcodeproj

This is particularly useful when the wrapper itself is part of the repository and we want Bazel to take care of building it rather than relying on some separately installed host tool.

Conclusion

In my opinion, this is yet another somewhat obscure Bazel feature that can come in handy in certain situations, as it did for me quite recently.

Typically, when writing Bazel rules, we output files as the final step, and that is what we need most of the time. However, today I want to make a case for outputting directories—or, as Bazel calls them, “tree artifacts.”

There are many reasons to reach for a tree artifact instead of, say, a ZIP file. For one, zipping can be expensive and time-consuming, depending on the size and contents of the output. Every downstream action that needs the directory then has to unzip it again, adding more CPU work, disk I/O, and temporary files.

Archives are also opaque blobs. By contrast, Bazel represents a tree artifact as a directory whose individual files are stored in the content-addressable store. The tree is still treated as a single declared output for dependency and action-cache purposes, but remote caching and remote execution can deduplicate the files it contains instead of repeatedly storing and transferring one large archive.

In the context of Apple-platform apps, we typically need .app bundles during intermediate build steps—not ZIP or .ipa archives. Signing, validation, installation, and other tools naturally operate on the bundle directory. Keeping the .app as a tree artifact means that we spend much less time zipping and unzipping it.

This is especially noticeable when developing locally through rules_xcodeproj, where unnecessary archive and extraction steps slow down the edit-build-run cycle.

There are other advantages too, particularly for remote execution, but it is hard to enumerate them all. The general idea is simple: keep structured outputs as directories for as long as possible, and create an archive only at the boundary where something actually requires one—for example, when producing the final .ipa.

Creating a tree artifact

It is quite easy, and perhaps not worthy of an entire blog post—but here we are.

In the rule implementation function, we simply invoke:

output = ctx.actions.declare_directory(
    ctx.label.name + ".app",
)

The returned value is a File, just like the value returned by ctx.actions.declare_file(), and it can be passed to actions and downstream rules in much the same way:

ctx.actions.run(
    executable = ctx.executable.tool,
    arguments = ["--output", output.path],
    outputs = [output],
)

The action producing the tree artifact must create the declared directory and place all of its contents inside it.

The main thing to remember is that Bazel does not know the tree artifact’s contents during the analysis phase. You cannot inspect the directory and turn arbitrary files inside it into regular declared outputs. Its contents are normally available only at execution time.

One newer option is map_directory, which lets Bazel expand actions over the contents of directories. It is still fairly new, however, and is not yet widely used.

Conclusion

My advice is to always evaluate whether an output really needs to be an archive. If it does not, it is usually better to declare it as a tree artifact.

In the world of Bazel, we prefer to be explicit as much as possible and register actions ahead of time. In fact, this is not just a preference but a strongly enforced rule that enables all the good stuff Bazel gives us.

However, there is a world where a bit of dynamism is useful—or, in some cases, unavoidable.

Today I am exploring the little-known map_directory API that was introduced in Bazel 9.

Registering actions dynamically

Say we write a rule that generates a couple of files and then want to register a separate Bazel action for each of those files.

The Bazel API did not facilitate this use case before version 9. Sure, we could have resorted to all sorts of tricks and hacks, but there was no nice native way to register actions based on the contents of a generated directory.

This is where map_directory comes in. It allows part of action registration to be deferred until execution time, when the contents of an input directory are known.

map_directory in action

The example below demonstrates generating a couple of files and then copying them with separate actions:

def _copy_files(template_ctx, *, input_directories, output_directories, tools, **_kwargs):
    for src in input_directories["src"].children:
        out = template_ctx.declare_file(src.tree_relative_path, directory = output_directories["out"])
        # Here we register one action per file; Bazel executes the command later.
        # Each action depends only on its input file and the copy tool.
        template_ctx.run(
            executable = tools["copy"],
            inputs = [src],
            outputs = [out],
            arguments = [src.path, out.path],
        )

def _demo_impl(ctx):
    src = ctx.actions.declare_directory(ctx.label.name + "_input")
    out = ctx.actions.declare_directory(ctx.label.name + "_output")

    ctx.actions.run_shell(
        outputs = [src],
        arguments = [src.path],
        command = """
mkdir -p "$1/nested"
echo hello > "$1/a.txt"
echo world > "$1/nested/b.txt"
""",
    )

    copy = ctx.actions.declare_file(ctx.label.name + "_copy.sh")
    ctx.actions.write(copy, '#!/bin/sh\nexec /bin/cp "$@"\n', is_executable = True)

    ctx.actions.map_directory(
        input_directories = {"src": src},
        output_directories = {"out": out},
        tools = {"copy": copy},
        mnemonic = "CopyFile",
        implementation = _copy_files,
    )

    return [DefaultInfo(files = depset([out]))]

map_directory_demo = rule(implementation = _demo_impl)

Creating a target from this rule and building it produces an unremarkable result, but it demonstrates what is now possible with this API at our disposal.

The important part is that _copy_files does not run during the regular analysis phase. It runs later, once Bazel knows the contents of the src tree artifact. At that point, it can inspect its children and register an individual copy action for each file.

Conclusion

While still greatly limited, map_directory allows us to introduce some controlled dynamism into our Bazel builds and makes certain things possible that previously required considerably more creativity.

I am pretty sure there are far more interesting examples of this feature being used, so feel free to look around on Github.

Finally, as always, I suggest consulting the official docs, at least as a reference, as well as the GitHub discussion around dynamic dependencies.

I was recently writing about the problem of disk space usage when building with Bazel due to its many caches. That led me to pay more attention to the work being done in this area, and I discovered that Bazel 9.3.0 is expected to make better use of copy-on-write cloning on macOS #30776 to avoid unnecessarily duplicating files between the disk cache and the output base.

On APFS, this essentially means copy-on-write: the cloned files initially share the same underlying data blocks, so they don’t immediately take up twice the physical disk space. If either copy is modified, the filesystem only needs to allocate storage for the changed blocks.

Using APFS clones from Swift

Naturally, I got interested in learning how this feature works, and it turns out to be pretty simple. There is a low-level API function, clonefile(...), which does exactly what you would expect: it creates a copy-on-write clone of a file.

If I were to wrap that function in Swift, here is roughly how I would do it. In production code, I would also provide better error handling:

func clone(from source: String, to destination: String) -> Bool {
    let source = source.cString(using: .utf8)
    let destination = destination.cString(using: .utf8)
    let result = clonefile(source, destination, 0)
    return result == 0
}

I recommend taking a look at its man page to learn more about it.

Conclusion

I wanted to share this because I feel like APFS cloning is not talked about enough. It’s a simple API backed by a pretty powerful filesystem feature, and I hope you find it handy someday.

Ever since the advent of coding agents, a lot of engineers have started utilizing Git worktrees and, by extension, multiple Bazel output bases. We all know the story: Bazel caches tend to take up a lot of disk space when working with large projects. Manual deletion and garbage collection can only take you so far.

Enter bb-clientd

bb-clientd is a daemon that runs on your machine and can act as a local remote cache and proxy for Bazel.

More importantly, it also implements Bazel’s Output Service protocol, introduced in Bazel 7.2. This allows bb_clientd to manage Bazel’s output tree through a virtual filesystem—FUSE on Linux and NFSv4 on macOS—and lazily materialize files when they are actually accessed.

Combined with its content-addressed local cache, this means that multiple Bazel output bases can reuse the same cached content instead of each storing their own copies of identical files. This is particularly useful when working with multiple Git worktrees.

Because bb-clientd already has a nice README, there is no point in me explaining the setup in detail. But here is an example .bazelrc configuration that I keep in my global .bazelrc so I can enable it when needed:

common:bb_clientd --disk_cache=
common:bb_clientd --remote_cache=unix:///Users/<user>/Library/Caches/bb_clientd/grpc
common:bb_clientd --remote_instance_name=local/projects
common:bb_clientd --remote_upload_local_results
common:bb_clientd --experimental_remote_output_service=unix:///Users/<user>/Library/Caches/bb_clientd/grpc
common:bb_clientd --experimental_remote_output_service_output_path_prefix=/Users/<user>/bb_clientd/outputs

This lets me use just the local cache with:

bazel build --config=bb_clientd //...

NOTE: If you’re using rules_swift, make sure to check out the swift.module_map_home_is_cwd feature if you’re not using the Xcode toolchain. It makes module-map generation and compilation assume that header paths are relative to the workspace root, which can be important when using a virtualized output tree.

Conclusion

This is one of those things that I don’t see discussed enough in the Bazel community, yet it can save gigabytes of disk space—especially if you’re regularly working across multiple worktrees.

Swift’s permissive nature when it comes to module dependencies has always annoyed me and made my life a bit harder than I expected. Of course, I’m talking about transitive module imports.

You know the situation: there is a ModuleA which is a dependency of ModuleB, and if you declare ModuleB as a dependency of ModuleC, ModuleC can import ModuleA even though it never declared ModuleA as its own direct dependency.

There are a lot of reasons why you might not want that, from making dependencies harder to visualize to enforcing a certain module structure, and so on.

Unfortunately, Swift itself does not provide a mechanism to enforce direct module dependencies—or at least I don’t know of one.

Layering check via rules_swift

One of the reasons I like working with Bazel is the flexibility it provides. One example of that is a feature in rules_swift called layering check, which prevents importing modules that aren’t declared as direct dependencies.

It was introduced a couple of months ago in #1780.

So, for example:

swift_library(
    name = "Bottom",
    srcs = ["Bottom.swift"],
    module_name = "Bottom",
)

swift_library(
    name = "Middle",
    srcs = ["Middle.swift"],
    module_name = "Middle",
    deps = [":Bottom"],
)

swift_library(
    name = "Top",
    srcs = ["Top.swift"],
    module_name = "Top",
    deps = [":Middle"],
)

Now let’s say Top.swift tries to do this:

import Middle
import Bottom

Even though Bottom is available transitively through Middle, Top never declared it as a direct dependency.

With the Swift layering check enabled, building it:

bazel build //:Top --features=swift.layering_check_swift

should fail with an error resembling:

Layering violation in //path/to/package:Top
  The following modules were imported, but they are not direct dependencies:

      Bottom

  Please add the correct 'deps' ...

And that’s exactly what I want.

To fix it, simply declare Bottom as a direct Bazel dependency:

swift_library(
    name = "Top",
    srcs = ["Top.swift"],
    module_name = "Top",
    deps = [
        ":Bottom",
        ":Middle",
    ],
)

Conclusion

This is one of those features I wish more people knew about and used. It gives you much tighter control over your dependency graph and makes it harder for accidental dependencies to get imported.

When using Bazel within a large monorepo, there comes a time when memory starts becoming a problem, and it is important to be aware of the available tools that can help with diagnosis. Fortunately, Bazel offers Starlark memory profiling, which we can utilize to see how much memory is being swallowed by the analysis phase.

Setting up

Surprisingly, it is a little tedious to set this up, but in the end, it is not that difficult either.

First, we need to download the allocation instrumenter JAR from Maven Central. Once that’s on disk, we can start Bazel in memory tracking mode like so:

bazel --host_jvm_args=-javaagent:/path/to/java-allocation-instrumenter-3.3.4.jar \
      --host_jvm_args=-DRULE_MEMORY_TRACKER=1 \
      build --nobuild //path/to/target

Here, we start Bazel with memory tracking enabled and pass --nobuild so that only analysis is performed, which is what we want to measure.

After analysis completes, make sure not to shut down the Bazel server because you’ll lose the gathered data.

Now, to produce a memory profile, we’ll use Bazel’s dump command:

bazel --host_jvm_args=-javaagent:/path/to/java-allocation-instrumenter-3.3.4.jar \
      --host_jvm_args=-DRULE_MEMORY_TRACKER=1 \
      dump --skylark_memory=memory_profile.gz

Notice how we repeat the startup flags:

--host_jvm_args=-javaagent:/path/to/java-allocation-instrumenter-3.3.4.jar
--host_jvm_args=-DRULE_MEMORY_TRACKER=1

Failing to do so will result in a Bazel server restart, and you’ll have to repeat the whole process from the beginning.

Dealing with gathered data

Now that you have memory_profile.gz, you need to make sense of it. The Bazel docs recommend using pprof to inspect the profile.

There doesn’t appear to be a pprof formula in the main Homebrew formula index, but installing it is straightforward if you already have Go:

go install github.com/google/pprof@latest

The binary is installed into $GOPATH/bin, which is $HOME/go/bin by default.

Once installed, a useful starting point is a flame graph:

pprof -flame memory_profile.gz

or a text report annotated with source lines:

pprof -text -lines memory_profile.gz

Conclusion

Most Bazel users won’t ever need to gather a memory profile, but I believe it is good to be aware that this exists so you can reach for it when needed. Rule authors should be especially aware of it, as it is not that difficult to make a coding mistake in a rule that results in huge memory usage.

You can find more information about optimizing performance in the official Bazel docs.