Pruning Unused Action Inputs in Bazel

Bazel has grown significantly since its initial open-source release, and it is hard to keep up with every release. One useful feature that may have slipped past you is unused_inputs_list, a parameter of ctx.actions.run(...) that lets an action report which of its declared inputs it did not actually use.

After the action executes successfully, Bazel can prune those files from the action’s effective input set. On subsequent incremental builds, changing only one of the reported unused files will not cause the action to run again.

This is post-execution dependency pruning, not pre-execution input discovery. The action still starts with all of its declared inputs. It reports unused inputs only after it has run. If the action later needs to execute again, Bazel restores the original input set first, because a previously unused input may be needed during the new execution.

A natural use case would be a compiler-like rule that begins with a conservative list of possible headers or source files, but can determine which ones it actually read.

A simple example

To demonstrate the unused_inputs_list parameter on ctx.actions.run(...), I came up with a deliberately contrived example that:

  1. Receives choice.txt through a selector attribute.
  2. Reads the contents of choice.txt.
  3. Selects one file from srcs.
  4. Copies the selected file to the output.
  5. Reports every unselected source file as unused.

It is an extremely unrealistic rule, but I believe it demonstrates the idea clearly.

The select_one rule

Given the following select_one rule:

def _select_one_impl(ctx):
    output = ctx.actions.declare_file(ctx.label.name + ".out")
    unused_inputs = ctx.actions.declare_file(
        ctx.label.name + ".unused_inputs",
    )

    args = ctx.actions.args()
    args.add(ctx.file.selector)
    args.add(output)
    args.add(unused_inputs)
    args.add_all(ctx.files.srcs)

    ctx.actions.run(
        executable = ctx.executable._tool,
        arguments = [args],
        inputs = [ctx.file.selector] + ctx.files.srcs,
        outputs = [
            output,
            unused_inputs,
        ],
        unused_inputs_list = unused_inputs,
        mnemonic = "SelectOne",
    )

    return [
        DefaultInfo(files = depset([output])),
    ]

select_one = rule(
    implementation = _select_one_impl,
    attrs = {
        "selector": attr.label(
            mandatory = True,
            allow_single_file = True,
        ),
        "srcs": attr.label_list(
            mandatory = True,
            allow_files = True,
        ),
        "_tool": attr.label(
            default = Label("//:selector_tool"),
            executable = True,
            cfg = "exec",
        ),
    },
)

The unused_inputs file is still an ordinary declared output: it must appear in the action’s outputs list, and the tool must create it.

Passing that same File through unused_inputs_list gives the output additional meaning. It tells Bazel to read the file after execution and interpret its contents as a list of inputs that can be pruned.

The tool itself

Here is the small Swift program that reads the selector, copies the selected file, and reports the remaining candidates as unused:

import Foundation

func run() throws {
    let arguments = Array(CommandLine.arguments.dropFirst())
    let selectorPath = arguments[0]
    let outputPath = arguments[1]
    let unusedInputsListPath = arguments[2]
    let candidates = Array(arguments.dropFirst(3))

    let selectedName = try String(contentsOfFile: selectorPath, encoding: .utf8)
        .trimmingCharacters(in: .whitespacesAndNewlines)
    let selectedPath = candidates.first(where: {
        URL(fileURLWithPath: $0).lastPathComponent == selectedName
    })!

    let selectedContents = try Data(
        contentsOf: URL(fileURLWithPath: selectedPath)
    )
    try selectedContents.write(
        to: URL(fileURLWithPath: outputPath)
    )

    let unusedPaths = candidates.filter { $0 != selectedPath }
    let unusedContents =
        unusedPaths.isEmpty
        ? ""
        : unusedPaths.joined(separator: "\n") + "\n"

    try Data(unusedContents.utf8).write(
        to: URL(fileURLWithPath: unusedInputsListPath)
    )
}

try run()

The unused-input file contains one input path per line. These need to be the action’s execution paths, not merely arbitrary workspace-relative names. Bazel reads each line and matches it against the mapped execution paths of the action’s inputs.

The example avoids having to reconstruct those paths by writing the exact candidate strings that Bazel passed to the tool.

The selected file must not appear in the unused-input list because its contents were copied to the output. The selector must not appear there either because changing choice.txt may change which source file is selected.

Putting it all together

Here is the final BUILD.bazel file:

load("@rules_swift//swift:swift_binary.bzl", "swift_binary")
load(":select_one.bzl", "select_one")

swift_binary(
    name = "selector_tool",
    srcs = ["main.swift"],
)

select_one(
    name = "demo",
    selector = "choice.txt",
    srcs = [
        "a.txt",
        "b.txt",
    ],
)

Suppose the files contain:

# choice.txt
a.txt
# a.txt
Contents of A
# b.txt
Contents of B

On the first build, the action receives all three declared inputs:

bazel build //:demo

The tool selects a.txt, writes its contents to demo.out, and writes the path of b.txt to demo.unused_inputs.

At that point, Bazel knows that b.txt did not contribute to this execution. Modifying only b.txt will therefore not cause the SelectOne action to execute again during a subsequent incremental build:

Changing a.txt, on the other hand, must rerun the action because a.txt produced the output.

Changing choice.txt must also rerun it. When that happens, Bazel restores the complete original input set before executing the action. The tool can then select b.txt, even though b.txt was reported as unused during the previous execution.

Conclusion

I have not seen many rulesets use unused_inputs_list directly, but it is worth knowing that it exists. It is most useful for actions that must declare a conservative set of possible inputs but can reliably determine which subset mattered after execution.

I enjoy exploring the more obscure corners of Bazel from time to time, since I tend to find features I did not know I needed.

A Note on Bazel’s config.exec()

When writing a rule that executes in the exec configuration, we need to communicate that to Bazel so it doesn’t attempt to build it for the target configuration:

generator = rule(
    implementation = _generator_impl,
    ...,
    cfg = "exec",
)

While there is nothing wrong with cfg = "exec", my opinion is that we should use the newer transition object API.

This means that instead of:

cfg = "exec"

we use:

cfg = config.exec()

Why?

Apart from legitimate use cases such as transition composition and passing an exec group, I think using the object form is better for readability and discoverability.

"exec" is a special string. You need to already know what it means and where it is supported.

config.exec(), on the other hand, looks like an API. It is easier to discover, easier to search for, and makes it clearer that we are applying an execution transition.

Passing the "exec" string is not deprecated, and I am not suggesting that it is incorrect.

Conclusion

This is simply my take on config.exec().

Migrating an entire codebase from cfg = "exec" to cfg = config.exec() will not result in any build-time improvement. It will, however, make the API usage slightly more explicit and leave room for features that the string form cannot express.

Making Bazel Module Extensions Work Together with override_repo

Recently, I have noticed more rulesets adopting Bzlmod-specific features. With Bazel 6 no longer supported and Bzlmod adoption continuing across the ecosystem, rulesets can increasingly rely on newer module APIs.

One feature that caught my attention while I was setting up a hermetic Android toolchain is the ability to override a repository generated by a module extension.

override_repo requires Bazel 7.4.0 or newer and can only be used by the root module.

Overriding a repository

There are several reasons to override a repository generated by a module extension.

It can simplify migrations by allowing existing call sites to keep using the same repository name. It can also avoid introducing multiple similarly named repositories. Most importantly, it gives module authors a way to provide a smoother developer experience when integrating with other rulesets.

A good example is Keith’s hermetic Android toolchain. Setting it up requires only a small amount of configuration:

bazel_dep(
    name = "hermetic_android_toolchains",
    version = "0.3.0",
)
bazel_dep(
    name = "rules_android",
    version = "0.7.3",
)

android = use_extension(
    "@hermetic_android_toolchains//:extensions.bzl",
    "android",
)
android.sdk(
    build_tools_version = "37.0.0",
    version = "37.0",
)
use_repo(android, "androidsdk")

# Make @rules_android's @androidsdk labels resolve to the hermetic SDK.
rules_android_sdk = use_extension(
    "@rules_android//rules/android_sdk_repository:rule.bzl",
    "android_sdk_repository_extension",
)
override_repo(rules_android_sdk, "androidsdk")

register_toolchains("@androidsdk//:all")

The important line is:

override_repo(rules_android_sdk, "androidsdk")

The rules_android_sdk extension normally generates its own repository named androidsdk. The positional form of override_repo tells Bazel to replace it with the repository of the same name that is already visible to the root module—in this case, the androidsdk repository generated by hermetic_android_toolchains.

As a result, references to @androidsdk from the rules_android extension resolve to the hermetically downloaded SDK rather than a separately configured Android SDK repository.

The keyword form can be used when the two repositories have different names:

override_repo(
    some_extension,
    generated_repo_name = "replacement_repo_name",
)

Here, generated_repo_name is the repository produced by the extension, while replacement_repo_name is a repository visible to the root module.

Conclusion

That is all there is to it. override_repo is a small Bzlmod feature, but it can make integrations and migrations considerably cleaner. It is worth keeping in mind whenever two module extensions need to agree on the repository behind a well-known name.

Stamping iOS Builds with Bazel

Stamping is the act of embedding build metadata into the product that we ship to customers. It can help with issue diagnosis, analytics, and so on. Conveniently, Bazel offers us a first-class solution, and it is very easy to take advantage of it in the context of iOS apps.

Workspace status script

The first step in enabling stamping is to create a workspace status script. For example, we can create a script that emits the current Git commit hash:

#!/usr/bin/env bash

set -eu -o pipefail

echo "STABLE_GIT_COMMIT $(git rev-parse HEAD)"

Now we need to tell Bazel to execute the script:

--workspace_status_command=./tools/workspace_status.sh

Reading the value at build time

For iOS apps, or really any Apple platform app, it is usually best to embed this data in a plist file so we can read it at runtime. First, we use a genrule to read the workspace status data and materialize a plist file:

genrule(
    name = "commit_plist",
    outs = ["Commit.plist"],
    cmd = """
commit="$$(sed -n 's/^STABLE_GIT_COMMIT //p' bazel-out/stable-status.txt)"
plutil -convert xml1 -o "$@" - <<EOF
{
    "GIT_COMMIT": "$${commit}"
}
EOF
""",
    stamp = True,
)

From there, it is just a matter of adding this target to the infoplists attribute of any Apple platform application target, like ios_application. Because rules_apple performs plist merging, this value will end up in the final Info.plist file that we ship in the app bundle.

A word on the stamp attribute

Notice the stamp = True attribute on the genrule? That is what allows the genrule action to access bazel-out/stable-status.txt and bazel-out/volatile-status.txt.

Without it, the action should not rely on those files being present. In this example, the important part is stamping the genrule that materializes the plist.

This is separate from the stamp attribute you may see on Apple rules like ios_application, where stamping controls whether build information is encoded into the binary. For this plist-based approach, we do not need to rely on link stamping at the application target level.

Reading the value at runtime

Because the value ends up in Info.plist, we can read it at runtime through the Bundle / NSBundle API.

Conclusion

There you have it: an easy way to stamp iOS builds. I hope it helps you discover and fix bugs in production more easily.

Making Developer Tools Available Through Bazel

Traditionally, when setting up a developer machine, instructions include something like “install the following tools using Homebrew”. What if we could always have tools available without asking developers to install anything but Bazel?

This is easily achievable with Bazel since it gives us a way to download and execute binaries. Before diving into the implementation, let’s first explore the downsides of asking developers to install tools on their own.

Problems with Homebrew for developer tools

brew install …

When developing on macOS, the “default” package manager is Homebrew, so we install tools like linters and formatters using it. However, it is not great for versioning in this use case. By default, we usually end up installing whatever version Homebrew currently resolves, unless we specifically do extra work to avoid that.

This is the first problem: we can’t expect people to ensure that they have exactly the same version of a tool as everybody else, especially if the organization is large.

Distributing tools

Just like we can’t easily enforce versions, we also can’t easily enforce tool replacement. Say that we built an internal linter and we want everybody to use it. What is the distribution mechanism? Perhaps an internal Brew formula? It works, but it is tedious to set up and maintain.

Using Bazel and rules_multitool

There have been a couple of community posts about rules_multitool, and I want to give my take on it.

How it works

This ruleset provides a convenient way to tell Bazel to download a binary and expose it as a target under the @multitool//tools/{TOOL_NAME} label. It takes in multitool.lock.json files and uses information from the lockfile to invoke Bazel repository rules, download the specified binary, and expose it as a runnable tool.

Setting it up

  1. Declare the dependency in MODULE.bazel as described here.

  2. Call its module extension from MODULE.bazel:

multitool = use_extension("@rules_multitool//multitool:extension.bzl", "multitool")
multitool.hub(lockfile = "//tools:multitool.lock.json")
use_repo(multitool, "multitool")

Of course, you can put your multitool.lock.json wherever you like. I tend to keep it under the tools/ package.

  1. Add multitool.lock.json and give it some tools to work with, e.g.:
{
    "$schema": "https://raw.githubusercontent.com/theoremlp/rules_multitool/main/lockfile.schema.json",
    "bb": {
        "binaries": [
            {
                "kind": "file",
                "url": "https://github.com/buildbuddy-io/bazel/releases/download/5.0.350/bazel-5.0.350-linux-x86_64",
                "sha256": "d14e6a240dc5e8bc3ebb625ff7c139ba8e380f1440f9e2f60e9c1d7850d012c9",
                "os": "linux",
                "cpu": "x86_64"
            },
            {
                "kind": "file",
                "url": "https://github.com/buildbuddy-io/bazel/releases/download/5.0.350/bazel-5.0.350-darwin-arm64",
                "sha256": "f16cc54449eb62ee65ac7ec3b45d7bce7922225a3c004925cbb25982faa8a9cd",
                "os": "macos",
                "cpu": "arm64"
            }
        ]
    },
    "yq": {
        "binaries": [
            {
                "kind": "file",
                "url": "https://github.com/mikefarah/yq/releases/download/v4.52.4/yq_linux_arm64",
                "sha256": "4c2cc022a129be5cc1187959bb4b09bebc7fb543c5837b93001c68f97ce39a5d",
                "os": "linux",
                "cpu": "arm64"
            },
            {
                "kind": "file",
                "url": "https://github.com/mikefarah/yq/releases/download/v4.52.4/yq_linux_amd64",
                "sha256": "0c4d965ea944b64b8fddaf7f27779ee3034e5693263786506ccd1c120f184e8c",
                "os": "linux",
                "cpu": "x86_64"
            },
            {
                "kind": "file",
                "url": "https://github.com/mikefarah/yq/releases/download/v4.52.4/yq_darwin_arm64",
                "sha256": "6bfa43a439936644d63c70308832390c8838290d064970eaada216219c218a13",
                "os": "macos",
                "cpu": "arm64"
            }
        ]
    }
}

This is it. You can now execute bazel run @multitool//tools/yq or bazel run @multitool//tools/bb, and Bazel will download and execute them just fine.

Making it more convenient

Typing out the label from above will quickly get tedious. What if we could instead run the yq tool as easily as ./tools/yq?

To achieve that, we could do the following trick:

  1. Create a script at tools/_run_tool.sh with the following content:
#!/usr/bin/env bash

target="@multitool//tools/$(basename "$0")"

bazel run \
    --run_in_cwd \
    --noshow_progress \
    --show_result=0 \
    --ui_event_filters=-info \
    "$target" -- "$@"
  1. Create a symlink for every tool with its name, e.g. a yq symlink that links to tools/_run_tool.sh.

That will expand $(basename "$0") in tools/_run_tool.sh to the name of the symlink and allow you to execute ./tools/yq. Or, if you put the symlink at the root, it gets even more convenient: ./yq.

Conclusion

I think the trick with symlinks is quite powerful, and you don’t really have to use rules_multitool if you don’t want to. The same idea can work with any Bazel target that exposes a runnable tool.

Overall, I like this setup very much because it requires almost nothing from developers. They can just use the tools provided to them while not even thinking about versions.

Avoiding .DS_Store Cache Misses in Bazel

It is well known that macOS Finder .DS_Store files should never be checked in to a repo, or leave the single machine for that matter.

Fairly recently, I noticed that a lot of my iOS resource processing actions were missing the cache for seemingly no reason. That is, until I looked at the Bazel action inputs. There, I noticed that every action that missed the cache had an extra input. Of course, it was the .DS_Store file.

The problem

The problem popped up because of the act of balancing developer convenience and build correctness. Given the following glob pattern:

resources = glob(["Assets.xcassets/**"]),

we allow engineers to freely add or remove files in an iOS asset catalog without needing to constantly modify the list in the BUILD.bazel file.

This, of course, means that .DS_Store files can get picked up if the engineer ever opened a Finder window at the given path. One might say that rules_apple should take care of this. However, that’s easier said than done, since asset catalogs can host many different resource types. Plus, Apple might extend the list of accepted resources at any time, which would require a rules_apple release just to add a file extension to some list.

The solution

The solution to this problem is quite simple: just make a macro for the glob() function:

def safe_glob(include, **kwargs):
    exclude_pattern = kwargs.pop("exclude", []) + ["**/.DS_Store"]
    return native.glob(
        include = include,
        exclude = exclude_pattern,
        **kwargs
    )

Now just load this symbol and use it instead of plain glob(...).

Conclusion

A neat trick to get around the fact that neither .bazelignore nor REPO.bazel solve this problem. I bet similar annoying files exist on Linux as well as Windows.

External Repo File Checks In Bazel 9

In a quest to speed up Bazel builds we tend to pick every available low-hanging fruit once somebody discovers it. One of those used to be telling Bazel not to check external repos for file changes, since that can take a while in a dependency-heavy repo.

Prior Art

Historically we used --noexperimental_check_external_repository_files to skip checks for files in external repositories. That flag still exists in Bazel (source), and bazelrc-preset.bzl still sets it (source).

Bazel 9 gained the repo contents cache via --repo_contents_cache. Cacheable external repos can now be served out of that cache.

That matters because Bazel does not treat repo-contents-cache-backed files as the old EXTERNAL_REPO case. In the source they are tracked as EXTERNAL_OTHER instead. Bazel 9 also added --experimental_check_external_other_files to control checks for those paths..

Conclusion

If you have repo contents cache enabled and your goal is the old “don’t spend time stat’ing external repos on no-op builds” behavior, you likely want both:

  • --noexperimental_check_external_repository_files
  • --experimental_check_external_other_files=false

The old flag still matters for repos that are not served out of the repo contents cache. The new flag matters for cache-backed repos.

If repo contents cache is disabled, --experimental_check_external_other_files=false can still help with those broader EXTERNAL_OTHER checks, but it does not replace the old external-repository flag.

Cleaning up old Bazel patterns

From time to time, it is worth cleaning up old Bazel stuff in your repositories. This is especially useful before a major Bazel upgrade, because it reduces the amount of migration noise you need to deal with. Most of these cleanups are not difficult, but they make the codebase a little easier to deal with.

The suggestions below are relevant if you are on Bazel 8.1.0 or newer.

Sets

Starting with Bazel 8.1, Starlark has native support for sets, which removes the need to use sets from bazel_skylib.

So instead of:

sets.make([1, 2, 3])

you can write:

set([1, 2, 3])

Native sets support the usual set algebra, such as union, intersection, difference, and symmetric difference, so this should cover most use cases where you previously reached for bazel_skylib.

Remove function_transition_allowlist when creating transitions

The conventional wisdom used to be that you needed to create a private _allowlist_function_transition attribute for Starlark transitions to work.

That is no longer necessary in modern Bazel versions, so you can remove:

"_allowlist_function_transition": attr.label(
    default = "@bazel_tools//tools/allowlists/function_transition_allowlist",
),

repo_name is usually no longer worth keeping

Historically, many repositories used reverse-DNS-style names for external dependencies because that was the common WORKSPACE convention. With Bzlmod, the module name is usually the better default.

For example:

bazel_dep(name = "rules_swift", version = "3.6.1", repo_name = "build_bazel_rules_swift")

can become:

bazel_dep(name = "rules_swift", version = "3.6.1")

This makes labels and load statements shorter and easier to write by hand.

Just make sure you update any remaining references to the old apparent repository name, such as @build_bazel_rules_swift, before removing repo_name.

Start using REPO.bazel

I already wrote about this in my article about dropping .bazelignore and in suppressing warnings in external Swift repositories, so I encourage reading those if you want more details.

The short version is that REPO.bazel gives you a better place to express repository-wide behavior. It marks a repository boundary and lets you set repository-level attributes in a way that fits better with modern Bazel.

compatibility_level is a no-op

If you are a rules author, do not spend time tuning compatibility_level. Starting with Bazel 8.6.0 and 9.1.0, both compatibility_level and max_compatibility_level are no-ops.

This makes me extremely happy because this thing often created more pain for users than it solved. If you introduce a breaking change, it is better to provide clear error messages and an actionable migration path instead of relying on Bazel module version selection to protect users.

A word about flags

In every major Bazel version, there are flags that get removed, become no-ops, or get flipped.

I do not recommend tracking all of that manually. There is a good chance you will miss something, or sometimes get it wrong. The better approach is to use bazelrc-preset.bzl, which applies version-appropriate flags for the Bazel version you are using.

Conclusion

There is probably more cleanup work that I am missing. However, these are low-hanging improvements that are usually easy to apply and easy to review.

Running Multiple Bazel Targets in a Single Invocation

There are many instances where it would be really convenient to run multiple targets at once. By default, Bazel will not execute all targets even if you pass multiple ones:

bazel run //:lint //:format

In this case, only one of them would be executed.

Enter rules_multirun

rules_multirun is a set of rules that helps with running multiple targets either sequentially or in parallel. It is developed and maintained by Keith Smiley.

Running multiple targets

It is extremely easy to get started. First, load the multirun rule and use it like this:

load("@rules_multirun//:defs.bzl", "multirun")

multirun(
    name = "xcodeproj",
    testonly = True,
    commands = ["//apps/app1:xcodeproj", "//apps/app2:xcodeproj"],
    jobs = 0,
)

Here, I used the multirun rule to create a single runnable target that generates Xcode projects for two of my apps:

bazel run //:xcodeproj

Execution modes

The jobs attribute specifies whether targets should run sequentially or in parallel. The default value is 1, which means that targets will run one after the other. In the example above, I explicitly set it to 0 to make sure both Xcode projects are generated in parallel.

This is something that needs to be decided on a case-by-case basis, since parallel execution might not be a good fit for tools that modify files.

Why testonly

This is typically not required, but given the specifics of rules_xcodeproj and my project setup, I need to pass it in this scenario.

That is because xcodeproj passes testonly = True as soon as you add test targets, and it does that to satisfy Bazel’s restriction that non-test targets cannot depend on test-only targets.

So, like I said, this is typically not needed, but you might run into it, so I figured it was worth explaining.

Other rules from rules_multirun

rules_multirun is a set of rules, not just the multirun rule. There is a rule for configuring individual targets and commands, rules for executing targets with transitions, and more.

It is best to consult the rules_multirun docs on GitHub for the full list of available options.

Conclusion

This is a ruleset that I find extremely convenient in my daily work, and I tend to strive to optimize that last mile of developer experience whenever I can.

One thing I intentionally changed: your intro said Bazel executes only the “first” target, but then the example said only format runs. I made it “only one of them” to avoid the contradiction.

Suppressing Warnings in External Swift Dependencies with Bazel

It’s very common to want to apply some Bazel feature only to your first-party repo while omitting external dependencies.

A common case in the Swift world is suppressing warnings for external dependencies brought in by rules_swift_package_manager, since we usually can’t do much about third-party code. There are countless other examples too, like treating warnings as errors for our own code while avoiding that for third-party deps.

REPO.bazel to the rescue

I wrote about REPO.bazel in an earlier article, where I explained how to replace .bazelignore with glob semantics.

For the use cases described in the intro of this article, REPO.bazel is extremely useful. It lets us apply Bazel features, which I’ve also written about before, only to our own repo.

Suppressing warnings in external Swift libraries

To achieve this, we need to do two things.

First, suppress warnings globally in .bazelrc:

# Suppress Swift warnings
common                --features=swift.suppress_warnings
common                --host_features=swift.suppress_warnings

# Suppress clang warnings
common                --features=suppress_warnings
common                --host_features=suppress_warnings

Then, disable those features for our first-party repo using the repo(...) function in REPO.bazel. The repo(...) function accepts the same arguments as package(...):

repo(
    features = [
        "-swift.suppress_warnings",
        "-suppress_warnings",
    ],
)

Conclusion

And that’s really it. A neat trick to have in your toolbox.

I also want to make it clear that I wasn’t aware of this trick until a fellow Apple rules maintainer, Aaron Sky, shared it in the Bazel Slack workspace.