Overlay System

Overlay System

Compatibility mechanism.

This page documents the legacy symlink-merge overlay machinery. The current architecture boundary for owner-specific deployment facts is Implementation Overlays. Do not use this page to justify repo-specific runner labels or repo-shaped ARC lanes.

This document describes the mechanics of build/overlay.bzl and build/extensions.bzl, which together implement the symlink-merge overlay that combines upstream and private files into a single Bazel repository.

Purpose

Institutional deployments need to add private configuration (tfvars, secrets references, environment-specific configs) and occasionally replace upstream defaults. The overlay system provides this without forking: the overlay repository contains only the delta, and the merge happens at build time.

That broad idea still matters, but the current product contract is stricter: private implementation values belong outside the core repo, while workflow labels remain shared capability classes.

How the Merge Works

The overlay is implemented as a Bazel repository rule. At a high level:

  1. The rule receives two directory paths: the upstream source tree (resolved via bazel_dep + local_path_override) and the overlay source tree (the overlay repository root).
  2. It walks both trees recursively, collecting every file path relative to the root.
  3. For each file, it creates a symlink in the output repository (@attic_merged):
    • If the file exists only in upstream, the symlink points to the upstream copy.
    • If the file exists only in the overlay, the symlink points to the overlay copy.
    • If the file exists in both trees, the symlink points to the overlay copy. This is the private-wins-on-conflict rule.
  4. The resulting @attic_merged repository looks like a single coherent source tree and can be built with normal Bazel commands.

Private-Wins-on-Conflict Semantics

The conflict resolution is intentionally simple: the overlay always wins. There is no file-level merging, no patch application, and no conditional logic. If an overlay provides tofu/stacks/attic/terraform.tfvars, that file completely replaces the upstream version of the same path.

This makes the system predictable. To understand what Bazel sees for any given file, check whether the overlay contains it. If yes, that version is used. If no, the upstream version is used.

File Tree Watching with ctx.watch_tree()

Bazel 7.1 introduced ctx.watch_tree(), which allows repository rules to declare file-system watches on directories. The overlay rule registers watches on both:

  • The upstream source tree
  • The overlay source tree

When any file in either tree is created, modified, or deleted, Bazel marks the @attic_merged repository as stale. The next cache-backed build command will re-execute the repository rule, regenerating all symlinks. This means developers working on either the upstream or overlay codebase see their changes reflected immediately without running any manual invalidation commands.

What Overlays Typically Add

Overlays are used for files that are specific to a deployment and should not be published upstream:

  • tfvars files — variable values for OpenTofu stacks (cluster_context, gitlab_token, environment-specific sizing)
  • Stack configurations — additional stacks that exist only in one deployment (e.g., {org}-runners)
  • Environment configs — organization.yaml or similar files that define the institutional identity
  • CI pipeline definitions — .gitlab-ci.yml tailored to the institution’s GitLab setup

For current GitHub ARC deployments, owner-specific GitHub App installation media and repo URL anchors, when unavoidable, belong in an implementation overlay. They should not be committed to the core GloriousFlywheel stack as active product structure.

Overlays can also replace upstream defaults. For example, an overlay might provide its own tofu/stacks/attic/main.tf if the upstream version does not support a required backend configuration.

Build Targets from the Merged Repository

Once @attic_merged exists, the overlay BUILD.bazel can define aliases and aggregation targets that reference it:

  • just cache-contract-strict, then just bazel-build-cached builds all overlay and upstream targets once the shared cache-backed substrate is available.
  • scripts/bazel-cache-backed.sh build //:deployment_bundle creates a pkg_tar of overlay configs combined with upstream artifacts.
  • scripts/bazel-cache-backed.sh build //:validate_modules validates all upstream OpenTofu modules via @attic-iac//tofu/modules:all_validate.
  • scripts/bazel-cache-backed.sh build //:app builds the SvelteKit application from upstream source.

If BAZEL_REMOTE_CACHE is unset or the strict cache-contract check fails, raw local Bazel remains a compatibility/debug path only and does not satisfy the intended heavy-work contract.

GloriousFlywheel