Current State
This is the canonical internal status document for GloriousFlywheel.
Use this page for the current internal operating contract, repo/admin reality,
and active route. The conservative future public package lives under
public-docs/; do not treat the whole docs/
tree as a future public surface.
Snapshot date: 2026-06-08
What GloriousFlywheel Is Now
GloriousFlywheel is an internal-first pooled build, cache, and control-plane platform built around:
- ARC on the
honeyon-prem cluster - Attic for Nix acceleration
- Bazel remote-cache acceleration
- a SvelteKit dashboard and related operator surfaces
- OpenTofu, Bazel, and Nix tooling to operate the platform
The intended product contract is one shared substrate for local development and CI. Shared capability classes are the normal path. Repo-specific runner lanes are debt, not the product direction.
Owner-specific GitHub App install media, tfvars, backend settings, and private registration anchors belong in implementation overlays. They may point at the same backend substrate when isolation is explicit, but they are not product runner taxonomy.
Current state: the owner implementation-overlay repositories exist, strict
preflight is green, and the six Jess personal-boundary compatibility releases
have been rehomed into the Jess overlay state. tinyland-inc/tinyland-infra is
the Tinyland Honey authority. Jesssullivan/jesssullivan-infra is the Jess
personal-boundary authority. Both overlays use read-only deploy keys
(GF_CORE_DEPLOY_KEY) for core checkout; GF_CORE_READ_TOKEN is kept only as
a compatibility fallback. Both overlays still keep their stable enrollment pins
at selected core 637b7167c400a842cdc7af0709b2251c0542a48a; the Jess ARC
deploy workflow separately pins the state-rehome authority commit
defff7fb7d1f3457c5270ce2e57ac6077e797b1c. The remaining owner/repo-shaped
work is compatibility-lane retirement or capacity policy, not unperformed
state rehome.
See Implementation Overlay Workstream for the active monitoring surface.
Current Repo And Admin Reality
- PR #768 exposed a product-contract regression: merge-blocking
ValidateandSecret Detectionstill ran onubuntu-latest, so a GitHub-hosted billing lock blocked a repo whose core product is pooled self-hosted runner infrastructure. That was not just an external outage; it was dogfood drift. PR #769 removed the hosted-runner escape hatch, PR #770 hardened the publication paths that regressed after that correction, PR #771 made the workflow self-trigger paths prove their own changes on pull requests, PR #772 refreshed the post-merge dogfood evidence, PR #773 made the GitHub Pages artifact-quota branch non-redlining without moving publication to a hosted runner, PR #776 hardenedjust dogfood-contract-auditso scalarruns-on, block-listruns-on, matrix/env hosted-runner literals, and configured exceptions all fail, and PR #777 refreshed artifact upload hygiene without creating a hosted-runner publication exception. Currentmainhas zero configured hosted-runner exceptions: first-party validation, secret scanning, Bzlmod/Bazel canaries, RBE chaos/status workflows, runner authority proofs, docs publication, FlakeHub publication, image mirrors, and release metadata all run on sharedtinyland-*capability-class lanes. Hosted runners are not control-plane publication exceptions for this repo. If runner availability, queue pressure, billing, or artifact quota blocks a first-party workflow, the fix is runner-substrate repair or best-effort artifact handling, not routing the path to GitHub-hosted runners. Public fork PR code stays skipped or moved to a trusted branch rather than executed inside the cluster-backed pool. - the latest audited default-branch checkpoint is
42b06019609c04ae25a34f48df284aeaec661a0bafter PR #828. Post-merge main provedValidate(26657097810),Secret Detection(26657097814),Platform Proof(26657097799),Source Bazel Proof(26657097846),Publish to FlakeHub(26657097788),Tranche Proof Status(26657097756),RustFS State Authority Canary(26657097748and later26657934610), andDeploy Docs(26657097770) on sharedtinyland-*lanes. PR #828 raised the W3.4 vendor-mode scratch preflight to match the observed full-scope proof footprint. This is runner/dogfood and external-input-canary hygiene, not RustFS repair, durable distdir authority, or CAS/action-cache authority promotion. - the macOS REAPI image/digest and Bzlmod-overlay lane is active in
GloriousFlywheel rather than TCFS. PR #870 added the immutable
gf-reapi-celldigest resolver, PR #871 repaired the consumer proof default branch frommastertomain, PR #872 staged Bzlmod external inputs for proof stability, and PR #873 made setup-flywheel and nix-job use Nixgithub.comaccess-token plumbing for first-party GitHub flake/source fetches. Nix flake source input authority is now tracked byjust nix-flake-source-input-manifest,just nix-flake-source-input-authority, anddocs/contracts/nix-flake-source-input-authority.json. The Nix flake source mirror package primitive isjust nix-flake-source-mirror-package,just nix-flake-source-mirror-verify, andjust nix-flake-source-mirror-restore; it packages already-materialized archives while keeping tarballarchive_sha256separate from NixnarHash. The next package-selection primitive isjust nix-flake-source-authority-package-gate, with an intentionally non-live template fromjust nix-flake-source-authority-package-template; it records the dedicated mirror/bucket/prefix and scopedGF_NIX_FLAKE_SOURCE_MIRROR_*credential names without committing secrets or promoting the authority contract. The current status is authenticated-live-GitHub-only, not durable source authority. Promotion still requires mirror placement, devshell restore proof, retention, provenance, auth boundary, and consumer exposure evidence. - PR #716 promoted the public
tinyland-inc/omux.xoxd.ai //:puppeteer_local_route_smokeproof class. PR #689 promoted the privatetinyland-inc/tinyland.dev //:app_typecheckRBE proof class. PR #690 made the enlargedgf-reapi-cellproof envelope scale to zero between proof runs so the sharedtinyland-nix-heavylane is not starved by a resident proof cell. PR #693 promoted the privatetinyland-inc/tinyland.dev //:app_buildRBE proof class after GF REAPI Cell proof run25978934708proved it from tinyland.devmain. PR #695 added the read-only queue-pressure diagnostic tojust arc-burst-capacity-auditafter the PR #694 heavy-lane contention window. This is runner capacity and queue visibility, not a new RBE target class. PR #698 promoted the privatetinyland-inc/tinyland.dev //packages/tinyland-activitypub:testclass. PR #699 promotedJesssullivan/MassageIthaca //:sveltekit_node_build. PR #700 promotedtinyland-inc/tinyland.dev //packages/tinyland-a11y-engine:typecheck. Run26001030662then proved the boundedtinyland-inc/tinyland.dev //:web_package_typecheck_fanouttarget after tinyland.dev PR #445 added the finite consumer target. PR #701 recorded the TIN-666 worker toolchain model. PR #702 recorded the next web RBE candidate queue, PR #703 made consumer proofs run from the checked-out consumer workspace, and PR #704 lowered the proof cell scheduling request. tinyland.dev PR #447 and repair PR #449 then enabled//:web_package_vitest_fanout; GF REAPI Cell proof run26002645581proved that bounded package Vitest fanout class from tinyland.devmain. TIN-668 stays active for the next target-class gate; TIN-1290 is now proved by main GF REAPI Cell proof run25989829826, TIN-1303 is now proved by main GF REAPI Cell proof run26005817853after omux PR #67 addedtinyland-inc/omux.xoxd.ai //:playwright_local_route_smoke, and TIN-1402 is now proved by GF REAPI Cell proof run26037732121after omux PR #69 addedtinyland-inc/omux.xoxd.ai //:puppeteer_local_route_smoke. TIN-1403 is now proved by GF REAPI Cell proof run26051698671after tinyland.dev PR #458 addedtinyland-inc/tinyland.dev //:puppeteer_local_route_smoke. - the latest merged owner-boundary truth package remains PR #626 at
12340a42f1a0b95f034f67d23554c7d3f6d8b61f. That package is older than the current RBE proof tranche but still records the owner-boundary scoreboard state. PR #597 through PR #602 added the pilot repo guide, exported pilot workflow template, workflow/guide sync guards, internal workflow-cache publication alignment, and cache/state reality-doc synchronization. PR #607 kept the Go example explicitly pure-Go, PR #608 promoted//examples/hello-go:hello_testas a narrow pure-Go REAPI test class, and PR #609/#610 hardened the dashboard Docker pnpm/Corepack bootstrap after the image workflow exposed a missing Corepack shim. PR #611 synchronized the May 10 RBE truth across the sprint plan and current-state docs. Forced REAPI run25638930305recorded the C++ worker-closure blocker, and run25648975728then proved the trivial//examples/hello-cc:hello_testclass with4 remoteprocesses; PR #624 promoted that proof on main. Run25649628233proved the separate trivial cgo-backed//examples/hello-go-cgo:cgo_testclass with11 remoteprocesses. Open repo issues remain focused: #413, #412, and #407. - the GitHub repo is public, but the whole tree is not yet treated as polished
public API.
docs/remains operator-facing internal truth; the safe consumer alpha route is the scrubbed fresh-history export produced byjust public-alpha-export. - the GitHub tracker is intentionally narrow rather than empty: PR #419, PR #420, and PR #422 are merged; #433 and #426 are closed after the April 26 Attic authority and tubebrain placement cleanup; PR #444 merged the Docker placement and ARC deploy-lane correction; PR #445 merged the bounded hygiene validation contract and repo-owned OpenTofu wrapper; #438 / TIN-681 is closed after the Docker-capable MassageIthaca shared-runner access proof; #421 / TIN-568 is closed after the Honey implementation-overlay authority rehome; #417 is closed after the downstream lab package-canary cache proof; #487 and #493 are closed after the KVM pickup/signal-9 closeout. Open issues #413, #412, and #407 track shared-lane reachability and compatibility-lane retirement follow-through
docs/is the internal source-of-truth surface for operator, product, and architecture materialpublic-docs/is the conservative future public package, with generated references and scrub gates already enforced in CI- the month-1 hardening lane is complete
- the square-one platform-definition lane is complete
- the public-surface MVP lane is complete
- pooled substrate, cache authority, and auth authority are complete enough to govern the current product contract
- the PR #400 through PR #404 parity and hardening lane is complete: current-state, roadmap, and Linear/admin surfaces distinguish the landed GitOps product slice, the proof-recording slice, and the heavy-lane signal-9 hardening slice
- the latest falsehood-retirement tranche has landed: repo-shaped runner taxonomy was removed from active ARC config, stale planning surfaces were fenced, and the source Bazel proof now passes a real remote-cache endpoint explicitly instead of relying on Bazel rc shell expansion
- the operator-gated ARC lane-retirement apply has completed: the live stack
destroyed the ten retired repo-derived scale sets, removed the stale
honeylabel fromtinyland-nix-gpu, and passed post-apply source/platform proofs - a later 2026-04-25 live ARC audit found
massageithaca-browser/massageithacadeployed as repo-shaped ARC residue; closed issue#409records the initial finding, closed issue #438 / TIN-681 records the separate Docker-capable access proof, and remaining compatibility-residue disposition belongs with the implementation-overlay and personal-lane follow-ups rather than new repo-shaped lanes - the implementation-overlay boundary repair moved the six selected Jess
personal-boundary ARC Helm releases out of core ARC state on 2026-04-29:
massageithaca-browser,massageithaca-dind,personal-package-nix-a,personal-package-nix-b,personal-docker, andpersonal-nix; themassageithaca-browserHelm release creates the live ARC scale set namedmassageithaca, so release names and scale-set names must not be conflated - the 2026-05-16 runner-capacity burst updated the live
massageithaca-dindcompatibility bridge: it is admitted tostingwithmaxRunners=3, the compute-expansion taint toleration, andlocal-path-sting-fast-ephemeralgeneric ephemeral PVCs for both runner workspace and Docker graph scratch. This is a bounded compatibility bridge, not a new product runner class and not durable HA. just arc-runner-residue-auditremains the read-only live classifier; after the state move it classifies the six selected compatibility releases as Jess-rehomed compatibility lanes, thejesssullivan-*lanes as Jess overlay-owned, andtubebrain-nixas a manual standalone compatibility lane forJesssullivan/yt-text;tubebrain-nixis bounded compatibility debt to rehome into an owner overlay or retire, not a product runner classconfig/arc-runner-residue-rehome.json,just arc-runner-residue-rehome-plan, andjust arc-runner-residue-state-checkencode and verify the completed six-release state transition without printing state values; the 2026-04-29 maintenance-window move pushed both remote state files and the remote post-move check reportedsummary: 0 blockers- the post-rehome apply gates are complete: Jess overlay apply reconciled the
six adopted releases onto
github-app-secret-jesssullivan, verifiedjesssullivan-nix=16,jesssullivan-docker=20, andjesssullivan-dind=12, and a core output-only apply completed with0 added, 0 changed, 0 destroyed; no destroy or Helm uninstall was performed - the TIN-548 dogfood reality audit and TIN-549 contract guard are complete;
just dogfood-contract-auditis now the repeatable guard against hosted-runner, stale-endpoint, and raw-Bazel drift - TIN-551 is complete: PR #419 added the scrubbed public-alpha export seed and
default-branch proof is green. The 2026-05-02 closeout decision is to use the
scrubbed fresh-history export/mirror for public alpha, not to flip this
private repository public. Direct full-repo visibility remains blocked by
just public-alpha-visibility-reportuntil a separate history-rewrite or exposure-acceptance decision exists. - a local 2026-04-27 gitleaks pass found no committed secrets in either the current checkout or the 592-commit history; direct visibility remains blocked by topology/planning surfaces, not by a known secret leak
- GitHub #417 is closed after downstream package-canary cache proof in
tinyland-inc/labPR #304; TIN-568, TIN-613, TIN-620, TIN-627, TIN-629, TIN-632, TIN-643, TIN-650, TIN-681, TIN-758, and TIN-851 are complete after the Honey implementation-overlay authority rehome, bumble placement guard decision, ARC network/session continuity closeout, shared-label capacity policy disposition, tubebrain listener placement, Attic public-read/key authority cleanup, Bazel external-fetch authority proof, developer-machine exposure policy guard, developer-machine shared-cache attachment proof, MassageIthaca Docker-capable shared-runner access proof, and owner-overlay queue-drift diagnostic correction just arc-shared-label-capacity-auditis the live read-only evidence surface for TIN-627. It groups owner-overlay ARC scale sets by workflow-facingtinyland-*labels and shows wheremaxRunnersis only a per-scale-set cap, not a global shared-label concurrency policy.just arc-diagnostic-self-testskeeps ARC network-continuity classification, runtime session/cap drift, queue-drift owner matching, and shared-label capacity warning fixtures covered without cluster access.just kubelet-imagefs-capacity-audit-self-testkeeps TIN-613 fixture coverage for healthy, warning, and critical kubeletrootfs,imagefs, andcontainerfsavailability boundaries without cluster access.just arc-burst-capacity-auditis the May 13 operator evidence surface for shared-label burst incidents. It joins node pod-slot headroom, workflow-label fanout, namespace quota, kubelet root/imagefs summary data, top node pod consumers, active ARC runner job repository/workflow attribution, and activetinyland-dind-compute-expansionfast-local PVC evidence before anyone proposes another ARC capacity mutation. Its JIT assignment trap section separates no-job cleanup candidates from assigned GitHub jobs at risk, so operators do not delete live assigned runners as a queue-drain shortcut.just runner-scale-contract-checknow guards the TIN-613 placement decision: ARC and GitLab runner selectors must not place default runner workloads onbumble; ARC baseline/controller placement stays onhoney, stateless Docker/listener placement stays onsting, and GitLab compatibility runner managers/jobs stay onsting.- the core Honey ARC scale-set cap target is now deliberately higher for the
primary GF-owned shared lanes:
tinyland-dockermax 20 onsting,tinyland-nixmax 16 onhoneyplus 8stingoverflow slots, andtinyland-dindmax 20 onhoneyplus 16stingoverflow slots. The additivetinyland-dind-compute-expansionlane is backed bylocal-path-sting-fast-ephemeralgeneric ephemeral PVCs for runner workspace and Docker graph scratch. Its container root ephemeral-storage requests are intentionally small so the scheduler reserves only rootfs overhead while the heavy recoverable build churn lands on the fast-local PVCs. Heavy, KVM, GPU, and compatibility-residue lanes remain separately bounded. - the May 10 DinD queue-drain work classified the bottleneck as an ARC admission/storage-envelope policy issue, not raw cluster exhaustion. Future burst relief should start from source-owned capacity changes, additive overflow lanes, storage-envelope review, and runtime audits. Do not patch active baseline ARC scale-set caps under load as the first response; ARC can churn listener/config reconciliation while existing runners drain.
- the May 12 post-merge runner burst added one more hygiene rule: completed and
failed runner-namespace utility pods must not consume honey’s finite pod
slots indefinitely. The ARC runner stack now enables the repo-managed
runner-cleanupCronJob inarc-runners; live manual pod deletion remains an operator action, not the normal product path. - the May 15 follow-up raises the source-owned DinD overflow envelope to
tinyland-dind=20on Honey plustinyland-dind-compute-expansion=16on Sting, with each overflow pod using48Giwork plus96GiDocker graph fast-local PVCs. This follows a live recurrence where both baseline and overflow DinD lanes were at cap, Honey had zero pod slots, and Sting still had pod, CPU, memory, and fast-local scratch headroom. The product issue was not lack of raw compute. It was that shared labels span multiple owner overlays and can saturate Honey pod slots before CPU/RAM, while Sting overflow health depends on explicit toleration and PVC-backed scratch. - the May 18-19 Nix runner recurrence keeps that boundary honest: Sting’s fast
local storage is real, but Nix compute-expansion needed its own storage model
rather than a blind cap bump. The source-owned TIN-1400 path gives the
tinyland-nix-compute-expansionoverflow lane per-podlocal-path-sting-fast-ephemeralPVCs for/nixand/home/runner/_work, with an init copy from the baked runner image so the installed Nix toolchain is not hidden by an empty mount. Until that is applied and observed live,Insufficient ephemeral-storageon a Sting Nix pod remains a missing fast-local Nix proof, not proof that the whole cluster is out of CPU, memory, namespace quota, or raw SSD/NVMe. - the May 24 GloriousFlywheel PR #768 dogfood run sharpened that again:
live first-party validation correctly ran on shared
tinyland-*runners, but atinyland-nix-compute-expansionpod still hit schedulerInsufficient ephemeral-storagebecausestingadvertised only roughly 71GB of kubelet root/nodefs ephemeral-storage while the node otherwise had CPU, memory, namespace quota, and fast-local storage intent. This is not a hosted-runner escape hatch and not evidence that the cluster lacks raw resources. It is storage integration debt: kubelet/local-path/root ephemeral accounting must be reconciled with the 2TB fast-local runner scratch model. - the May 25 TIN-1600 correction keeps that truth but right-sizes admission:
tinyland-nix-compute-expansionkeeps PVC-backed/nixand work storage onlocal-path-sting-fast-ephemeral, adds PVC-backed/home/runner/.cachefor Bazel/Bazelisk/package-manager cache churn, keeps the runner root ephemeral-storage request at1Gi, and raises only the root ephemeral-storage limit to16Gi. With the module’s existing1GiNix PVC init request, the eight-slot lane still reserves16Giof Sting node ephemeral-storage instead of40Gi, without treating fast-local scratch as durable HA storage. The correction still has to be applied and observed live before the Nix overflow cap can be counted as fully usable. - the May 27 dogfood window exposed a distinct memory envelope defect in the
same lane: a
tinyland-nix-compute-expansionrunner fortinyland-inc/labPublish to FlakeHubreached the old8Gicgroup limit and OOM-killed whilestingremained below node memory pressure. The source-owned response raises the shared lane to a4Girequest and16Gilimit, with the namespace request-memory quota raised to280Giso honey-heavy admission does not starve the sting overflow lane through a global quota artifact. That keeps the product dogfooding invariant intact: fix the shared capability envelope, do not route first-party proof work to hosted runners or repo-specific labels. - the May 14 RustFS backend-authority pass keeps TIN-1147 as an active
stop/go gate through
rustfs-trusted-publication-backend-gate.json. Trusted Attic publication is still quarantined until one of three paths is proved: non-restart RustFS repair/reindex, RustFS upgrade/topology fix, or backend replacement. Restart-only recovery, green canary-only coherence, source-only admin route existence, RBE proof evidence, and OpenTofu state-only HA proof do not unblock TIN-1046. - TIN-1152 is the concrete RustFS upgrade/topology candidate path under
TIN-1147. Its packet,
rustfs-upgrade-topology-candidate.json, treats upstream RustFS1.0.0-beta.4as a candidate to prove, not a fix claim. The release and beta.1…beta.4 comparison include ListBuckets CreationDate, filemeta/metacache, bucket metadata, list_object_v2/listing, HeadObject, scanner/rebalance, and S3 tracing changes, but a pinned Docker Hub candidate digest is only readiness input. Its preflight requires the normal main suite to be green while treating RustFS State Authority Canary as expected-red TIN-1147 evidence with an uploaded artifact that preservestofu-state,attic, NAR integrity, and HA candidate inventory output. Explicit operator approval, post-upgrade state/RCA evidence, and representativesmall-check/medium-checkpublication proof are still required before trusted Attic writes can return. rustfs-upgrade-topology-proof-plan.jsonis the source-owned proof plan for that candidate path. It is intentionally non-mutating: the currenthoney.tfvarsimage stays on beta.1, and the future maintenance-window change is narrowed torustfs_imageonly. The plan requiresjust tofu-plan-guard attic, protects live secret authorities, node selectors, service selectors, and the OpenEBS PVC, rejects Civo, and keeps TIN-1046 blocked until post-upgradetofu-state, bucket-index RCA, NAR integrity, and representative publication evidence clear the knownNoSuchBucket,curl 18, andsize_download=0failure classes. The managedDeploy Attic Stackworkflow has a manualplan_scope=rustfs_upgrade_topologymode for producing this guarded saved plan: normal plans and all applies keep strict RustFS state authority, while the proof-scope plan may continue past the expected-red TIN-1147 canary state only far enough to run the generic Attic plan guard plus the specific upgrade-topology guard.just rustfs-upgrade-topology-plan-guardis the specific saved-plan guard for that future maintenance window: it allows only the beta.1 -> upgrade-topology candidate RustFS image update on the live Deployment and, if the shared module input causes it, the drained legacy StatefulSet pod template. It rejects Secret, selector, PVC/storage, service, delete/create, and wrong-image drift before apply.rustfs-trusted-publication-decision.mdis the operator decision runbook for the same TIN-1147 gate. It keeps the three allowed lanes explicit: non-restart repair/reindex, the TIN-1152 upgrade/topology proof path, or backend replacement. It also records the current non-proofs so ARC runner dispatch, Bazel RBE proof, green plan-only checks, restart-only recovery, and OpenTofu state-only HA proof do not accidentally reopen trusted Attic publication or package-runner flips.- the same May 12 audit confirmed that shared-label overflow can hold ARC
listener recreation open after scale-set spec changes. The May 15 recurrence
added the missing productionization detail:
Deploy ARC Runnersmust not run plan/apply/verify on labels it quiesces, and quiescing must close listener admission by settingmaxRunners=0, not onlyminRunners=0. The workflow now usesARC_DEPLOY_RUNNER_LABELwithtinyland-nix-heavyas the bootstrap fallback, max-freezes the shared Nix/Docker/DinD baseline and compute-expansion scale sets in one guarded quiesce/apply step, snapshots caps for restore on failure, gives active shared jobs a bounded 20-minute drain window, generates and guards a fresh post-quiesce apply plan so manual cap freezes are reconciled by OpenTofu on success, restores caps from source tfvars targets on success before listener proof, keeps the pre-quiesce snapshot only as the failure rollback, keeps best-effort failure restore in the workflow trap, and keeps a longer strict post-apply listener proof.tinyland-nix-operatoris the source-owned control-plane lane to use after bootstrap. Warm pools remain allowed, but they cannot be treated as proof that the scale-set listener is healthy. - the 2026-05-12 owner-boundary refresh keeps #407, #413, and #412 open:
Jesssullivan/Dell-7810,Jesssullivan/XoxdWM,Jesssullivan/scheduling-kit, andJesssullivan/scheduling-bridgeeach report zero accessible repo-level self-hosted runners. The package repos already setPRIMARY_LINUX_RUNNER_LABELS_JSON=["tinyland-nix"], so #412 is not blocked on workflow label text. It remains blocked becausepersonal-package-nix-aandpersonal-package-nix-bstill exist as Jess-overlay-owned repo-registration compatibility lanes. The active sprint packet is May 12 Owner-Boundary Proof And Retirement Sprint.
Current Toolchain Coverage
Bazel-hermetic language toolchains today are narrower than the substrate narrative suggests. The truth as of 2026-05-18:
- TypeScript / JavaScript: Bazel-hermetic via
aspect_rules_js,aspect_rules_ts,rules_nodejswith pinned Node 22.13.1 and pnpm. REAPI proof targets include//app:build,//app:unit_tests,//docs-site:build,//docs-site:playwright_chromium_smoke, public omux//:unit_tests,//:build, Puppeteer, and Playwright smoke classes, publicjesssullivan.github.ioVitest, SvelteKit/Vite, Puppeteer, and Playwright classes, private MassageIthaca Vitest, svelte-check, TypeScript no-emit, Playwright TMD, and SvelteKit/Vite production-build classes, and privatetinyland.dev //:app_typecheck. These are explicit target classes, not broad/default web RBE. The privatetinyland.dev //:app_buildVite/SvelteKit production-build class is also proved separately by run25978934708, andtinyland.dev //packages/tinyland-activitypub:testis now proved separately by run25981546207.Jesssullivan/MassageIthaca //:sveltekit_node_buildis now proved separately by run25983800544.tinyland.dev //:playwright_local_route_smokeis now proved separately by run25989829826, andtinyland.dev //:puppeteer_local_route_smokeis now proved separately by run26051698671. - Python: Bazel-hermetic via
rules_python1.4.1 with pinned Python 3.11 - Shell: Bazel-hermetic via
rules_shell, with Nix-provided coreutils - Rust: Bazel-hermetic via
rules_rust0.70.0 from BCR. Proof targets//examples/hello-rust:helloand//examples/hello-rust:hello_testexercise the rules_rust prebuilt toolchain (not Nix-sourced —rules_nixpkgs_rustis not in BCR). Run25648670844proves the trivial Rust test class through GF REAPI Cell with5 remoteprocesses. - C++: BCR rule surface via
rules_cc0.2.18 with auto-resolved system/Nix cc toolchain. Proof targets are//examples/hello-cc:helloand//examples/hello-cc:hello_test. Forced REAPI run25638930305reached the remote compile action for//examples/hello-cc:hello_test, then failed because the worker lacked/nix/store/zx71vq7s1v840wqsrw2m2ckmxn413a2b-gcc-wrapper-13.3.0/bin/gcc; after the worker image carried the C/C++ wrapper closure, run25648975728proved the trivial C++ unit-test class with4 remoteprocesses. This does not prove all C++ tests or broad/default C++ RBE. - Go: Bazel-hermetic via
rules_go0.60.0 from BCR (prebuilt Go SDK). Proof targets//examples/hello-go:helloand//examples/hello-go:hello_test; the separate cgo proof target is//examples/hello-go-cgo:cgo_test.gf-reapi-cellcontinues to build via NixbuildGo124Module. Run25634296833proves the pure-Go//examples/hello-go:hello_testclass through GF REAPI Cell with11 remoteprocesses. Run25632300253still captures the earlier rules_goruntime/cgo/ no-ccblocker; run25649628233proves the separate trivial cgo-backed Go class with11 remoteprocesses. - Zig, Chapel: absent. No Bazel rules and no Nix devshell wiring today; these remain backlog
See Toolchain Coverage for the working
table and active proof targets. RBE eligibility for any toolchain is owned
separately by config/rbe-target-eligibility.json and the broad-RBE gate
(see Roadmap).
RBE Production-Gate Reality
The machine-readable RBE eligibility manifest now records 34 proved target classes. That is meaningful product evidence: GloriousFlywheel can already run target-scoped remote check, test, smoke, package, and build work across the source repo and several spoke-style web repositories. It is still not default-capable multi-tenant RBE.
The production-gate blocker is not target breadth alone. The next default-RBE
promotion requires the gates tracked by the RBE production-readiness project:
durable CAS/action-cache authority (E1), trusted-writer AC policy and audit
(E2), durable external-input authority through distdir/repository-cache (E3),
tenant enforcement through instance_name, IAM/OIDC, executor pools, and
quotas (E4), and operator/developer visibility through TTFCH, cache-hit,
fairness, and poison-signal panels (E5). E6 target-class breadth should stay
downstream of those gates.
Proof-of-concept versus product, in code terms (2026-05-29). The
gf-reapi-cell REAPI surface already implements all five REAPI v2 services
(Capabilities, ByteStream, CAS, ActionCache, Execution) for real, plus
instance_name scoping, opt-in OIDC/JWT authz (warn/enforce), AC writer
attestation, an AC write audit log, on-read and on-write digest verification,
and gf_reapi_* Prometheus counters. These are landed code, not design docs.
The distance to a production, adoptable product is four pillars, none of them
yet complete:
- Durable storage. A provider-neutral
BlobStoreseam landed 2026-05-29 with a dependency-free S3-compatible backend. It does not select MinIO or any other replacement provider. The live self-hosted object-store substrate is RustFS for existing cache/state paths; using RustFS for RBE CAS/AC still requires the TIN-1147 repair/proof gate and a dedicated CAS/action-cache namespace. Readiness now gates on backend reachability, age-based TTL garbage collection (E1 W1.3) is wired for the local backend, and the first W1.4/TIN-1461 local CAS size-bound primitive exists behindGF_REAPI_CAS_MAX_BYTES. That size evictor is lease-protected byGF_REAPI_MIN_CLIENT_CACHE_TTL, LRU-ordered, quota-reconciled, and observable throughgf_reapi_size_eviction_*plus thegf_reapi_evicted_while_referenced_totalpoison tripwire. The default backend is still node-local, and sharded/replicated topology is not built. - Multi-tenancy and identity. Routing, authz, AC attestation, per-tenant
quotas (E4 W4.4 — Execute concurrency + per-blob size; W4.6 — durable
CAS-byte and AC-entry limits surviving restart via a startup scan + post-GC
reconcile), first executor-pool admission (W4.3 —
Action.platformpool property checked before AC lookup/execution), an in-process scheduler/placement seam with queue metrics, bounded local worker-pool leases, static worker inventory/provenance, and an opt-in in-memory live worker heartbeat registry now exist in code. The first Bazel credential- helper slice also exists for projected-token and explicit-token callers, so Bazel can attachAuthorization: BearerJWTs without storing them in.bazelrc. Still open: token exchange, full IAM/OIDC tenant-claim rollout, remote worker dispatch, and durable worker-pool placement. - A distributed worker pool. Execution still runs locally on the
single-replica cell pod itself. The cell now records scheduler enqueue,
start, completion, queued, inflight, and histogrammed queue-time metrics by
instance_nameand pool, plus local worker-pool slot metrics whenGF_REAPI_WORKER_POOLSis configured. It can also lease from a static named worker inventory for provenance and prefer non-expired token-authenticated live heartbeat workers for lease provenance. There is not yet durable worker registration, heartbeat fanout, or remote executor dispatch. - Observability and fairness. Counters exist, the first W5.3 Grafana
fairness contract now exists at
docs/monitoring/gf-reapi-fairness-dashboard.json, and the first W5.4 TTFCH probe contract is wired through.github/workflows/ttfch-probe.yml,scripts/gf-runner-ttfch-probe.sh, anddocs/monitoring/gf-runner-ttfch-dashboard.json. The probe distinguishes cache-hit outcomes from clock-valid latency samples so stale cache metadata cannot be misread as zero-second TTFCH. Cache-hit, poison-signal, alerting, live multi-tenant TTFCH evidence, and runner-dashboard SvelteKit panels remain open E5 work.
Until those land, the honest claim is target-scoped RBE on a single-replica cell, not default-capable multi-tenant product RBE.
Live runner substrate on 2026-05-28 is not the scarce resource: honey,
bumble, and sting are Ready, all tinyland-* ARC scale sets are Running,
and tinyland-nix-heavy is committed at a 64Gi memory request, 160Gi
memory limit, 192Gi ephemeral-storage request, and 256Gi
ephemeral-storage limit. That request is based on the green W3.4
current-main canary, which proved the target class is scratch-heavy rather
than memory-heavy. The remaining RBE blocker is product authority and
enforcement, not a need to fall back to GitHub-hosted runners or invent
repo-specific labels.
Current Operating Contract
honeyis the only physical cluster target- GitHub is the primary forge surface
- GitLab remains a compatibility path, not the primary control plane
- the GitLab runner compatibility stack is bounded by per-manager
concurrent_jobsplus HPA manager replicas. It does not provide ARC’s queue-driven scale-to-zero semantics. - the committed
arc-runnersandgitlab-runnersstack changes now declare ResourceQuota plus LimitRange policy for the rollout. This is the aggregate in-namespace admission backstop for finite Honey/Sting capacity; it is not a global concurrency policy across every owner overlay, and it may intentionally be lower than the sum of every lane’s theoretical max if the quota is the machine-limit stop.just runner-capacity-model-checkkeeps the committed quotas tied to the declared runner burst envelopes and verifies at least the largest modeled pod fits. Treat this as rollout-ready IaC until a live plan/apply proof is recorded. - operator access is tailnet-first and private
- environment-owned S3-compatible state is the configured authority for
attic,arc-runners,gitlab-runners, andrunner-dashboard, but the current RustFS-backed API is not healthy enough to treat as deploy/state authority without passing the guard commands below. - that state authority is guarded interim infrastructure, not HA
infrastructure. On 2026-05-08,
just tofu-state-ha-readiness --expect-interimpassed as evidence capture:tofu-statewas visible through the S3 API and all required state objects were readable JSON state bodies. Strictjust tofu-state-ha-readinessstill fails because the live path is one RustFS pod, one service endpoint, one OpenEBS ZFS node, a bumble-scopedReadWriteOncePVC, and no detectedrc/rustfs-adminrepair surface. TIN-1012 remains In Progress until a dedicated HA state authority, equivalent managed HA backend, or real replicated RustFS repair and node-maintenance proof makes strict mode pass. - on 2026-05-19, the interim state path regressed again after the latest
controlled restart window.
just tofu-state-ha-readiness --context honey --expect-interimandjust rustfs-bucket-index-rca --context honey --bucket tofu-state --since 1hboth failed becausetofu-statewas absent from S3list-bucketswhile/data/tofu-stateand/data/.rustfs.sys/buckets/tofu-stateremained present. The same window produced GitHub canary failure26079424346and GloriousFlywheel PR #735 check failure26082188888inPlan ARC Runners; both stopped on the same state-authority guard. PR #735 fixes only the list-buckets parser so owner display names cannot masquerade as bucket names. It does not repair RustFS. Current conclusion: repeated bucket-ensure or restart-only recovery is not a completion path. TIN-1026 must provide the managed/appliance HA endpoint package and scopedTOFU_HA_STATE_*credentials, then TIN-1017 must prove scratch S3, disposable OpenTofu, lockfile, maintenance, and cleanup evidence before protected state migration. - as of 2026-05-21, RustFS State Authority Canary runs remain expected-red
backend-authority evidence, not normal-suite health gates. Latest observed
run
26213404755failed in the HA readiness, read-only Attic bucket-index, and Attic NAR integrity steps, then succeeded in HA candidate inventory, evidence summary, and artifact upload. Artifact7130361579(rustfs-state-authority-canary-26213404755) is the preflight evidence to preserve before any beta.4 upgrade/topology maintenance window. The material risk remains the recurrence pattern: the same RustFS singleton can move between coherent and incoherent S3 API views while disk markers persist. - on 2026-05-25, the RustFS bucket-index recurrence reproduced again after
bumble returned:
tofu-stateandatticdisk markers existed, while the S3 API omitted the buckets fromlist-buckets; Attic narinfo stayed readable, but the representative NAR body failed with curl 18 and zero bytes transferred. A controlled restart restored coherence temporarily. PR #801 then added and merged a managed, self-hosted Attic stack plan/apply workflow ontinyland-nix-operator. Managed apply run26410714188succeeded withallow_destroy=false, moved liveattic-rustfs-openebsto generation 35, setfsGroupChangePolicy=OnRootMismatch, and removed recursivechown -R /datafrom the init path. Independent post-apply state authority checks and Attic NAR integrity passed, and current-main canary run26410874298passed. That green window did not hold: after PR #802 merged at4068f8727d7e9e15df38cbf7424524a563fbcad9, post-merge canary runs26411985313and26412011228failed on the same authority shape. A second controlled restart moved live RustFS to generation 36 and temporarily restoredtofu-statereads plus representative Attic NAR streaming, butatticbucket-index evidence still failed becauselist-bucketsomittedatticwhile disk markers existed; minutes latertofu-statelist-buckets coherence failed again. This is recovery and restart-hygiene evidence, not a RustFS trust promotion: TIN-1147 remains open until sustained canary coherence plus the selected repair/upgrade/replacement path clears the trusted-publication gate. - the May 14 repair-surface inventory keeps TIN-1147 grounded: the deployed
RustFS image reports
v1.0.0-beta.1, top-level CLI commands are limited toserverandinfo, and the live pod lacksrustfs-admin,rc,mc,aws, ands5cmd. Tagged source contains internal admin heal endpoints, but there is no proved signed operator repair runbook for the live bucket-index recurrence. A May 14 signed RustFS background-heal status probe returned HTTP 200 from/rustfs/admin/v3/background-heal/statuswith valid JSON (bitrotStartCycle,bitrotStartTime,currentScanMode). This is observability evidence, not trusted publication repair. TIN-1147 remains In Progress. A source audit of the tagged RustFS heal handler also found that the bucket/object heal endpoint does not preservedryRun: it parsesHealOpts, then builds the channel request withcreate_heal_request, which defaultsdry_runtoNone; the heal processor defaults that tofalse. Restart recovery is not trusted publication repair, and bucket/object/heal/calls are not safe dry-run probes. A follow-on source audit found that bucket-metadata export/import endpoints are not a proved non-restart repair path either: export depends on the currentlist_bucket/get_bucket_infoAPI view, while import is a mutating zip-archive path that can callmake_bucket(force_create)and does not persist accumulated imported metadata config updates in the current handler. - on 2026-05-10 the
tofu-statebucket-index failure recurred while applying the ARC runner scale-out: disk markers fortofu-stateexisted, but RustFS returnedNoSuchBucket/absentlist-buckets. A controlled restart ofnix-cache/deployment/attic-rustfs-openebsrestored the S3 view; the subsequentarc-runnersapply advancedarc-runners/terraform.tfstateto serial 65 and a no-op post-apply plan passed. This is incident-recovery evidence only; it does not satisfy the non-restart repair gate. - on 2026-05-11, bumble/OpenEBS failed for a different reason: the node booted
an XR kernel without matching ZFS modules, so the OpenEBS ZFS node plugin
could not auto-load ZFS and the bumble-backed PVC plane was unavailable.
Rebooting bumble back into the stock ZFS-compatible kernel restored
zfs,spl, and thetankpool. The RustFS pod then recovered after single-pod cleanup. The repo contract now keeps RustFS data mounts onfsGroupChangePolicy = "OnRootMismatch"and removes recursive restart-timechown -Rfrom the adopted OpenEBS deployment path. This is restart hygiene, not HA state authority and not a fix for the bucket-index recurrence. just ha-state-candidate-static-gate --contract <path>is the pre-migration static gate for any future HA state authority candidate. It rejects the current RustFS singleton, Sting local-path storage, Attic/Bazel cache surfaces, and the activetofu-statebucket as final candidate contracts.docs/contracts/ha-opentofu-state-managed-s3-candidate.jsonis the selected non-secret TIN-1016 proof-target contract for a managed or appliance S3-compatible OpenTofu state service. The validation command isjust ha-state-selected-candidate-static-gate, and it passed again on 2026-05-08. This is a candidate-selection artifact, not a live HA endpoint proof and not protected state migration. TIN-1026 is the active blocker for a real endpoint package plus scopedTOFU_HA_STATE_*proof credentials.just ha-state-candidate-inventory --context honeyreportedNO_LIVE_HA_STATE_CANDIDATEagain on 2026-05-19. It classified currentattic-rustfs-openebsas interim-only, rejected staging S3-compatible test candidates,tcfs/seaweedfs,seaweedfs/seaweedfs-s3, and Sting local-path storage as final state authorities, and noted that Longhorn is not live and is block storage rather than an S3-compatible state authority by itself. Sting is Ready again, but it is not contributing a replicated S3-compatible state authority.- on 2026-05-25 EDT / 2026-05-26 UTC, the read-only PR #805 branch rerun kept
the same blocker shape:
just ha-state-candidate-inventory --context honeystill reportedNO_LIVE_HA_STATE_CANDIDATE, andjust tofu-state-ha-readiness --context honey --expect-interimfailed withtofu-stateabsent from S3list-bucketswhile both/data/tofu-stateand/data/.rustfs.sys/buckets/tofu-statewere present. The newjust ha-state-endpoint-readinessgate is the local TIN-1026 preflight: it reportsTIN-1026_NOT_READYuntil a non-secret endpoint package passesjust ha-state-endpoint-package-gate --package <path>and scopedTOFU_HA_STATE_ACCESS_KEY/TOFU_HA_STATE_SECRET_KEYproof credentials are injected. This is endpoint readiness only; it does not mutate Kubernetes, S3, or OpenTofu state. - on 2026-05-27, repo-level GitHub secret-name readiness still reports
TIN-1026_GITHUB_SECRETS_NOT_READY: visibletinyland-inc/GloriousFlywheelsecrets includeATTIC_TOKEN,RUSTFS_ACCESS_KEY, andRUSTFS_SECRET_KEY, but not the scopedTOFU_HA_STATE_ACCESS_KEY/TOFU_HA_STATE_SECRET_KEYproof authority.RUSTFS_*,ATTIC_*, and broadAWS_*names are substitutes only and must not count as TIN-1026 readiness.just ha-state-github-secret-readinessrecords this without reading secret values or mutating Kubernetes, S3, or OpenTofu state. - on 2026-05-28, the Tinyland owner overlay showed the same blocker from the
consumer side.
tinyland-inc/tinyland-infraPR #15 merged after documenting that manual Plan ARC Runners attempts26534981354and26412207716stop at read-onlyarc-backend-preflight: the liveatticdisk markers exist, but the S3 API omitsatticfromlist-buckets. A freshjust ha-state-candidate-inventory --context honeystill reportedNO_LIVE_HA_STATE_CANDIDATE, and GitHub secret-name metadata fortinyland-infraexposed no scopedTOFU_HA_STATE_*proof authority. That keeps ARC runner enrollment and repo-owned package-runner proof blocked on TIN-1026/TIN-1017 state authority, not on ci-template package workflow shape. - Attic and Bazel are shared acceleration layers for both developer and CI workflows, not publication surfaces
- the live Bazel cache is S3-backed through the
bazel-cachebucket on the OpenEBS-backedattic-rustfs-openebsRustFS service;/datain thebazel-cachepod is a local hot cache, not the durable source of truth - on 2026-05-25,
Source Bazel Prooffailed on a remote-cache digest mismatch for//examples/hello-go-cgo:cgo_test; the recovery started as cache-only: delete the implicatedcas.v2object and roll the statelessbazel-cachepods so local hot caches could not keep serving it. That operation then reproduced the RustFS bucket-index class:list-bucketswent empty while disk markers forattic,bazel-cache,tofu-state, andwas110-public-inputsremained. A controlled restart ofattic-rustfs-openebsrestored bucket API visibility and the guarded OpenTofu state check. Treat this as live proof that the current RustFS-backed Bazel cache remains acceleration-only, not trusted CAS/action-cache authority for broad/default RBE. Note that direct raw S3 body hashes are not the right CAS audit for this deployment becausebazel-remoterunsStorage mode: zstd; integrity checks must hash decoded payload bytes. - the Attic stack now declares a dedicated
was110-public-inputsRustFS bucket for approved public WAS-110 archive mirrors; do not place those pinned source archives in theattic,bazel-cache, ortofu-statebuckets - on 2026-05-06,
was110-public-inputswas created onattic-rustfs-openebsand populated with the two public WAS-110 archive objects,public-source-lock.json, andSHA256SUMS; a direct pod port-forward round trip downloaded the two archive objects, verified their SHA-256 sums, materialized a tempwas110_vendor_blobsrepo offline, and passedpins/verify-vendor-repo.sh - do not run a broad honey Attic stack apply without reviewing the saved plan.
On 2026-05-06 UTC, the TIN-980 state-adoption pass removed stale old
Kubernetes provider-type addresses and imported the live RustFS/Attic
API/GC/secret objects at
_v1addresses. A follow-up pass imported the liveattic-configConfigMap, reads the non-placeholder database URL from the existingattic-secretsSecret, reads the live Attic JWT signing key and RustFS credentials from their existing service Secrets, and keeps the ConfigMap mount as an explicit honey adoption mode. The adoption tfvars also preserve the live honey placement for Attic API, Attic GC, and bazel-cache pods. A freshjust tofu-plan atticpass now produces6 to add, 14 to change, 0 to destroy,just tofu-plan-guard atticpasses, and targeted saved-plan review shows no Kubernetes Secret data-key changes or deployment node-selector changes. The remaining server-config ConfigMap-to-Secret move is a separate hardening cutover, not surprise drift in this adoption plan. - ARC and GitLab runner IaC should keep the same cache-first contract attached:
Nix lanes receive Attic substituter trust through
NIX_CONFIG, while Nix, Docker, and DinD lanes receiveBAZEL_REMOTE_CACHEandGF_BAZEL_SUBSTRATE_MODE=shared-cache-backedwhen the Bazel endpoint is configured. ARC runner modules can now carry optional backend-neutralBAZEL_REMOTE_EXECUTORwiring throughbazel_executor_endpoint, but that input is empty by default, requires a separateBAZEL_REMOTE_CACHE, and does not promote targets outside the checked RBE eligibility manifest. - local
direnv/flake sessions now have a Nix-specific preflight:just cache-contract-nix-strictfails unlessNIX_CONFIGcarries the configured Attic substituter and public trust key;.envrcexports those hints before enteringuse flake, so initial devShell realization and later shell commands use the same cache attachment contract setup-flywheelnow shares the same Nix attachment behavior for workflow consumers: when Attic server/cache/key inputs are present, it writes the substituter and public key intoNIX_CONFIG; on GitHub Actions it also adds the job token as a Nixgithub.comaccess token for flake/source fetches. It also repairs preinstalled self-hosted Nix daemon/build-user runtime before workflows call rawnix develop- Nix proof workflows now use
cache-attachment-contract.sh --strict --strict-nix, while Docker/DinD proof lanes stay Bazel-cache strict only just rbe-boundary-checkkeeps operational surfaces on the current remote-cache default and only allows the repo-managed wrapper to pass a Bazel remote executor after the strict contract classifies the shell asexecutor-backed- the dashboard cache API and cache page surface the same boundary: Bazel is
remote-cachemode and remote execution is not selected - OpenTofu module reference docs are now guarded by
just tofu-module-docs-check, so new modules such asrunner-namespace-policymust be represented alongside the module tree - the root
tofu-apply, stack-local apply helpers, and ARC deploy workflow now run the saved-plan guard before apply, blocking destructive runner namespace, namespace-policy, cache PVC/bucket, ARC controller, repo-scoped ARC registration URL, and repo-shaped runner label drift unless an operator sets an explicit reviewed bypass docs/runners/live-rollout-checklist.mdis the operator checklist for the hardened k8s/Tofu rollout. It records the state-move, import, scale, RBAC, cache, and post-apply verification boundary; it is not a substitute for a recorded live plan/apply proof.- Woodpecker/Codeberg remains future adapter work. The repo has Forgejo proof documentation, but no production Woodpecker stack, shared fleet, or verified Attic/Bazel cache attachment path.
- Bazel remote cache covers action outputs. External repository/archive fetches
have a separate
authority surface. The
Source Bazel Proof now materializes the Linux x64 Node 22.13.1 toolchain
archive plus the critical Bzlmod archives that recently blocked GF
proof/TTFCH analysis into an ephemeral verified
BAZEL_DISTDIRbefore Bazel starts, then validates that runtime materialization manifest againstdocs/contracts/bazel-distdir-source-proof-coverage.jsonbefore requiring the manifest in the external-fetch authority check. Other repository fetches still rely on the repo-managed wrapper path and its configuredBAZEL_REPOSITORY_CACHE/BAZEL_DISTDIRinputs. Source Bazel Proof also runs a bounded public Attic NAR read canary beforenix develop; an unreadable representative NAR is cache-substrate reliability evidence, not a Bazel source-package failure. The workflow records that canary as a warning and artifact, then continues into the Bazel package proof so RustFS/Attic read authority does not hide repository-cache and external-input evidence. The dedicated RustFS State Authority Canary remains the blocking backend-authority lane and now captures read-onlyatticbucket-index plus NAR integrity evidence even whentofu-statefails before the scratch probe. As of 2026-05-20, live evidence still shows the durable incident objectnar/8iv5j0f2difw6wg9vwj9r2raacb08fkv.nar; the earlierxhsczcfkg4gmwk4g9sskwqsn4nlb0294-bazelnarinfo still exists but redirects to a missing RustFS chunk and remains tracked as TIN-1393 repair evidence.just bazel-external-input-manifestnow inventories the lockfile inputs: BCR registry files and generated archive repositories are hash-recorded, while generated Node.js toolchain repositories are still version/template-pinned without a lockfile hash inMODULE.bazel.lock. That is a precise mirror / repository-cache / durable distdir blocker, not RBE evidence.docs/contracts/bazel-external-input-durable-authority.jsonnow names the durable authority promotion gate directly: current status isno-live-durable-authority,covered_inputsis empty, all 23 candidate inputs remain pending for durable coverage, and promotion requires auth, retention, restore, provenance, and consumer exposure evidence. Consumer repos can also pass verified generated local repositories through the copied wrapper or explicit RBE proof wrapper withGF_BAZEL_INJECT_REPOSITORIES, which becomes repeated--inject_repositoryflags. The explicit proof wrapper can also use--workspaceto run from a checked-out consumer workspace while keeping the executor-bearing wrapper in GloriousFlywheel. This is the cache-forward / proof-lane path for public pinned inputs such as a WAS-110 public-community vendor repo; it is not a durable mirror by itself and does not make private blobs public-cache or CAS eligible. - source-built container image publication belongs on the shared
tinyland-dindlane, not on GitHub-hosted runners or repo-specific labels - ARC listener pods currently run on
sting; runner job payload placement is controlled by each shared capability or owner-registration scale set. Baseline Nix, KVM/GPU, and sharedtinyland-dindpayloads remain honey-bound where they need current runtime or host capability, while statelesstinyland-docker, bounded heavy Nix, and some compatibility-residue payloads are deliberately admitted tostingas compute-expansion relief. This is placement for shared capability classes and owner registration overlays, not new runner taxonomy - kube-API workflows use shared capability-class operator or bootstrap lanes,
not hosted runners or repo-specific labels. ARC plan/apply currently uses
ARC_DEPLOY_RUNNER_LABELwithtinyland-nix-heavyas the bootstrap default until the operator lane is available for that workflow; Attic plan/apply and the RustFS canary usetinyland-nix-operator. These jobs rewrite the decoded Honey kubeconfig to the in-cluster Kubernetes service endpoint when running inside ARC. That keeps mutation jobs independent of node-specific tailnet API reachability while preserving the shared runner taxonomy. - raising
maxRunnerson a scale set increases that scale set’s ceiling only. It does not create global concurrency accounting across Tinyland and Jess owner overlays; Kubernetes scheduling remains the final shared physical backpressure. TIN-627 records that current policy boundary; a higher-level cross-overlay capacity controller remains future productization work. - cache-first, verify-later remains the intended methodology for Nix, Bazel, and toolchain-heavy workloads
- capability-class lanes are the normal runner taxonomy
- repo identity is not runner taxonomy
- implementation overlays own owner-specific GitHub App, tfvars, and
GF_CORE_DEPLOY_KEYvalues tinyland-inc/tinyland-infraandJesssullivan/jesssullivan-infraare the current overlay authorities; their stable enrollment pins, backend credentials, and ARC credentials are proved, and the six selected Jess personal-boundary compatibility releases have been moved out of core state and reconciled by the Jess overlay- owner-distinct ARC scale-set names are auth and registration identities; they are not repo-specific workflow labels and they do not yet provide a global concurrency policy above Kubernetes scheduling
- local dev and CI are meant to use the same substrate rather than separate execution models
- cacheable/packageable CI belongs on runner lanes; host-bound machine
characterization still stays direct on
honeyunless it is intentionally modeled as a quarantined manual runner lane - FlakeHub is publication and discovery, not runtime/bootstrap
- current
mainproves shared cache acceleration plus narrow explicit REAPI proofs for//app:build,//app:unit_tests,//:deployment_bundle,//docs-site:build, the WAS-110 public-input handoff, and target-scoped public consumer web proofs includingtinyland-inc/omux.xoxd.ai //:build; it does not prove universal full remote offload for every developer workload - private consumer web proof status on 2026-05-18:
Jesssullivan/MassageIthaca //:booking_operation_unit_testsis proved by run25928429263with repo-scoped deploy-key checkout, forced execution,3319 remoteprocesses, remotetest-setup.sh, and one passing private Vite/Vitest booking-operation test.Jesssullivan/MassageIthaca //:svelte_check_testis proved by run25938855554with the same repo-scoped deploy-key checkout authority, forced execution, proof nonce20260515T200641Z-25938855554-1,3319 remoteprocesses, remotesveltekit_sync_bin_/sveltekit_sync_bin, remotetest-setup.sh svelte_check_test_/svelte_check_test, remotegenerate-xml.sh, and one passing private SvelteKit/svelte-check action.Jesssullivan/MassageIthaca //:tsc_noemit_testis proved by run25948484331with the same checkout authority, forced execution, proof nonce20260516T005553Z-25948484331-1,3319 remoteprocesses, remotesveltekit_sync_bin_/sveltekit_sync_bin, remotetest-setup.sh tsc_noemit_test_/tsc_noemit_test, remotegenerate-xml.sh, and one passing private TypeScript no-emit action in24.2s. This is a third narrow MassageIthaca target class.Jesssullivan/MassageIthaca //:playwright_tmd_smokeis proved by run25953478878with the same checkout authority, consumer commit08555e16b9ee0504b1b23e6373b5b6bbfb799f5f, forced execution, proof nonce20260516T050753Z-25953478878-1,3318 remoteprocesses, remotesveltekit_sync_bin_/sveltekit_sync_bin, remotevite_build_bin_/vite_build_bin, remotetest-setup.sh playwright_tmd_smoke_/playwright_tmd_smoke, remotegenerate-xml.sh, and one passing private Playwright TMD smoke action in4.5s. This is a fourth narrow MassageIthaca target class, not broad private-app RBE.tinyland-inc/tinyland.dev //packages/tinyland-grafana:testis now also proved by run25935041748after tinyland.dev PR #401 fixed the Grafana test’s Kubernetes-environment assumption. The proof used repo-scoped deploy-key checkout, the verified private codeload distdir handoff fortummycrypt_tinyland_schemas:0.2.4, forced execution, proof nonce20260515T184435Z-25935041748-1,1531 processes: 468 remote cache hit, 1059 internal, 4 remote, remotetest-setup.sh packages/tinyland-grafana/test_/test, and artifact verifier success. This is still one narrow private package Vitest target class, not broad tinyland.dev RBE or durable private mirror/repository-cache authority.tinyland-inc/tinyland.dev //:app_typecheckis now proved by run25970619559after tinyland.dev PR #410 made the root typecheck target clean enough for the proof lane. The proof used GitHub App checkout authority, the verified privatetummycrypt_tinyland_schemas:0.2.4distdir handoff, forced execution, proof nonce20260516T191944Z-25970619559-1,5578 processes: 1 action cache hit, 2567 remote cache hit, 2955 internal, 56 remote, remote TypeScripttsc, remote Svelte build tool, remote Vite build tool, remoteapp_typecheck_tool, artifact verifier success, and Kubernetes restart evidence that stayed at0. This is one narrow private root app typecheck target class, not all tinyland.dev builds/tests, browser E2E, Vite production build RBE, durable private mirror/repository-cache authority, broad/default web RBE, or CAS/action-cache backend suitability.tinyland-inc/tinyland.dev //:app_buildis now proved separately by run25978934708after tinyland.dev PR #425 added the bounded root production build target. The proof used GitHub App checkout authority, the verified privatetummycrypt_tinyland_schemas:0.2.4distdir handoff, forced execution, proof nonce20260517T021820Z-25978934708-1,6146 processes: 3125 remote cache hit, 2959 internal, 62 remote, remote TypeScript package fanout, remoteJsRunBinary app_build.log, artifact verifier success, and Kubernetes restart evidence that stayed at0. This is one narrow private root Vite/SvelteKit production-build target class, not all tinyland.dev builds/tests, browser E2E, deployed app behavior, durable private mirror/repository-cache authority, broad/default web RBE, or CAS/action-cache backend suitability.tinyland-inc/tinyland.dev //packages/tinyland-activitypub:testis now proved separately by run25981546207. The proof used GitHub App checkout authority,workspace_path=consumer-workspace, the verified privatetummycrypt_tinyland_schemas:0.2.4distdir handoff, forced execution, proof nonce20260517T044208Z-25981546207-1,728 processes: 1 action cache hit, 299 remote cache hit, 415 internal, 14 remote, remoteesbuildlifecycle-hook execution, remote TypeScripttscforpackages/tinyland-content-types, remotetest-setup.sh packages/tinyland-activitypub/test_/test, remotegenerate-xml.sh, artifact verifier success, and Kubernetes restart evidence that stayed at0. This is one narrow private ActivityPub package Vitest target class, not all tinyland.dev package tests, browser E2E, deployed app behavior, durable private mirror/repository-cache authority, broad/default web RBE, or CAS/action-cache backend suitability.tinyland-inc/tinyland.dev //packages/tinyland-a11y-engine:typecheckis now proved separately by run25984827370. The proof used GitHub App checkout authority,workspace_path=consumer-workspace, consumer commit3730c6966d5e069cff92abc7c606fca9db5b54af, the verified privatetummycrypt_tinyland_schemas:0.2.4distdir handoff, forced execution, proof nonce20260517T073751Z-25984827370-1,553 processes: 223 remote cache hit, 328 internal, 2 remote, remoteesbuildlifecycle-hook execution, remote TypeScripttscforpackages/tinyland-color-utils, artifact verifier success, and Kubernetes restart evidence that stayed at0. This is one narrow private package TypeScript typecheck target class, not all tinyland.dev packages, all TypeScript, Vite/SvelteKit builds, durable private mirror/repository-cache authority, broad/default web RBE, or CAS/action-cache backend suitability.tinyland-inc/tinyland.dev //:puppeteer_local_route_smokeis now proved separately by run26051698671. The proof used GitHub App checkout authority,workspace_path=consumer-workspace, consumer main commitdcb859f658092dc2a6c0f33223cb9ec9a4055c18, the verified privatetummycrypt_tinyland_schemas:0.2.4distdir handoff, forced execution, proof nonce20260518T181314Z-26051698671-1,6319 processes: 1 action cache hit, 3135 remote cache hit, 3052 internal, 132 remote, remote npm lifecycle-hook execution for@tailwindcss/oxide,sharp, andesbuild, remote TypeScripttsc, remote Svelte and Vite build-tool execution, remotetest-setup.sh puppeteer_local_route_smoke_/puppeteer_local_route_smoke, remotegenerate-xml.sh, artifact verifier success, and Kubernetes restart evidence that stayed at0. This is one narrow private local-server Puppeteer route-smoke target class, not all Puppeteer, all tinyland.dev routes, deployed E2E, full app behavior, durable private mirror/repository-cache authority, broad/default web RBE, or CAS/action-cache backend suitability.tinyland-inc/tinyland.dev //:web_package_typecheck_fanoutis now proved separately by run26001030662. The proof used GitHub App checkout authority,workspace_path=consumer-workspace, consumer main commitdcc20d11b8919ae259ce8b3e9b982a37e2d6b56b, the verified privatetummycrypt_tinyland_schemas:0.2.4distdir handoff, forced execution, proof nonce20260517T195322Z-26001030662-1,789 processes: 321 remote cache hit, 465 internal, 3 remote, remoteesbuildlifecycle-hook execution, remote TypeScripttscforpackages/tinyland-color-utilsandpackages/tinyland-auth, artifact verifier success, and Kubernetes restart evidence that stayed at0. This is one bounded private package typecheck fanout target class over tinyland.dev PR #445’s finite//:web_package_typecheck_fanout; it does not prove all tinyland.dev packages, all TypeScript, Vite/SvelteKit builds, Vitest, Playwright/Puppeteer, durable private mirror/repository-cache authority, broad/default web RBE, or CAS/action-cache backend suitability.tinyland-inc/tinyland.dev //:web_package_vitest_fanoutis now proved separately by run26002645581. The proof used GitHub App checkout authority,workspace_path=consumer-workspace, consumer main commit8ee22a2a0130f7241a42c2e3666e310c89a5cfdf, the verified privatetummycrypt_tinyland_schemas:0.2.4distdir handoff, forced execution, proof nonce20260517T210344Z-26002645581-1,1102 processes: 1 action cache hit, 438 remote cache hit, 642 internal, 22 remote, remotetest-setup.shactions forpackages/tinyland-color-utils/test_/test,packages/tinyland-forms/test_/test, andpackages/tinyland-security/test_/test, artifact verifier success, and Kubernetes restart evidence that stayed at0. This is one bounded private package Vitest fanout target class over tinyland.dev PR #447 plus PR #449’s test-suite repair; it does not prove all tinyland.dev package tests, all Vitest, root app tests, Vite/SvelteKit builds, Playwright/Puppeteer, durable private mirror/repository-cache authority, broad/default web RBE, or CAS/action-cache backend suitability.tinyland-inc/tinyland.dev //:playwright_local_route_smokeis now proved separately by run25989829826. The proof used GitHub App checkout authority,workspace_path=consumer-workspace, consumer main commitefa977e701c449dce84065e138f3c8a303ce8334, the verified privatetummycrypt_tinyland_schemas:0.2.4distdir handoff, forced execution, proof nonce20260517T114200Z-25989829826-1,6155 processes: 3139 remote cache hit, 2963 internal, 53 remote, remote TypeScripttsc, remote Vite build-tool execution, remotetest-setup.sh playwright_local_route_smoke_/playwright_local_route_smoke, remotegenerate-xml.sh, artifact verifier success, and Kubernetes restart evidence that stayed at0. This is one narrow private local-server Playwright route-smoke target class, not all Playwright, all tinyland.dev routes, deployed E2E, full app behavior, durable private mirror/repository-cache authority, broad/default web RBE, or CAS/action-cache backend suitability.Jesssullivan/MassageIthaca //:sveltekit_node_buildis now proved separately by run25983800544with repo-scoped deploy-key checkout, consumer commite06a70d12417f04568092a62e225b6c6595c3b39, forced execution, proof nonce20260517T064447Z-25983800544-1,7379 processes: 2 action cache hit, 4186 internal, 3193 remote, remote lifecycle-hook execution foresbuild,msw, andsharp, remotesveltekit_sync_bin_/sveltekit_sync_bin, remotevite_build_bin_/vite_build_bin, artifact verifier success, and Kubernetes restart evidence that stayed at0. This is one narrow private MassageIthaca production-build target class, not all MassageIthaca builds/tests, deployed booking E2E, image publication, durable private mirror/repository-cache authority, broad/default web RBE, or CAS/action-cache backend suitability. - Bazel remote execution now has multiple narrow explicit GF REAPI cell proofs,
but it is not selected as the default product path; Buildbarn, Buildfarm,
BuildBuddy, and NativeLink remain peer projects / possible spike targets, not
GloriousFlywheel dependencies. The committed
gf-reapi-cellmanifest remains idle atreplicas: 0for the TIN-1249 capacity boundary, but GF’s own proof workflow now keeps the live cell resident by default after a successful apply so hourly TTFCH and back-to-back RBE dogfood do not race a missing endpoint. Operators can still dispatch teardown explicitly. TIN-1027,TIN-665,TIN-671,TIN-672, andTIN-882are closed for the minimum REAPI endpoint, first target-class proofs, optional ARC executor endpoint wiring, and related proof surfaces.TIN-668remains the active target-class eligibility umbrella while new classes are proved one by one. Follow-on RBE productization is still durable CAS/action-cache authority, worker lifecycle, benchmarks, and additional target proofs, not broad default executor wiring. The W2 action-cache authority slice is no longer only a design corpus: writer attestation, server-attached platform tags, AC audit rows, the W2.4 nuke-key/quarantine drill, and the W2.5 non-attested writer chaos gate now exist asgf-reapi-cellprimitives. They narrow the poison surface for proof-local AC entries; they do not yet close durable CAS/AC storage, tenant quota, retention/query/dashboard, or default-RBE safety.- External-input authority is also moving from design to enforcement. W3.1
lockfile authority is landed, and the W3.5 source-local guard now rejects
direct
http_archive/http_filecalls without non-emptysha256pins viajust bazel-http-archive-pins-checkand the Validate workflow. This blocks a cheap upstream-poison path before durable distdir/repository-cache authority is complete. It does not yet prove mirror placement, repository-cache prepopulation, or vendor-mode offline completeness. W3.3 control-plane plumbing is now explicit in the repo-managed wrappers:GF_BAZEL_REPOSITORY_DISABLE_DOWNLOAD=truemaps to--repository_disable_download, andBAZEL_OUTPUT_BASEmaps to Bazel startup--output_basefor fresh-output-base repository-cache proofs. That is the switch needed for hermetic CI lanes; it is not itself evidence that a shared repository cache is meeting hit-rate SLOs.Source Bazel Proofnow also passes a run-localBAZEL_REPOSITORY_CACHEalongside the verified distdir, packages the real Node Linux x64 archive through the provider-neutral mirror layout, verifies the package, restores a fresh distdir from it, and uses that restored distdir for the Bazel package/test path. It then reruns//:deployment_bundlefrom a freshBAZEL_OUTPUT_BASEwithGF_BAZEL_REPOSITORY_DISABLE_DOWNLOAD=true. The emittedbazel-repository-cache-evidence.jsonrecords file count, byte count, distdir-manifest linkage, the no-download target, and explicit W3.3 boundaries; the durable external-input authority contract tracks this assource_proof_repository_cache_evidence, still withdurable: falseandhit_rate_slo_proof: false, and tracks the provider-neutral package/restore assource_proof_distdir_mirror_restore_evidence, still withstorage_endpoint: false. That is merged-main warm-cache and local package/restore evidence, not durable mirror, retention, consumer exposure, hit-rate SLO, or broad/default RBE evidence.just e3-external-input-authority-statusnow makes both live-proof setup blockers explicit: the default non-secret authority package pathbazel-external-input-authority-package.jsonis missing, andjust bazel-distdir-mirror-github-readinessreports that endpoint/region must exist asGF_EXTERNAL_INPUT_MIRROR_*variables or secrets while access/secret-key material must exist as scoped GitHub secrets. Until aproof_readypackage and those scoped GitHub names exist, TIN-1468 remains live-proof blocked for authority setup rather than runner capacity. The W3.4 vendor-mode canary is also now resource-envelope honest: main run26223210487failed with Bazel Java heap OOM whilerules_rustextracted the Rust toolchain on the baselinetinyland-nixlane. The canary has moved to the sharedtinyland-nix-heavycapability lane, passes an explicitGF_VENDOR_MODE_BAZEL_HOST_JVM_XMXhost JVM heap, and classifies that failure asrules_rust_toolchain_extract_heap_oomso failed runs still upload machine-readable evidence. The follow-up branch canary26242395403then proved the heap fix was not the end of the W3.4 blocker: the livetinyland-nix-heavypod was evicted because its 16Gi ephemeral-storage limit contradicted the canary’s 40Gi scratch preflight. A managed ARC apply moved the live lane to40Gi/64Gi, and workflow-dispatch canary26245482714then hit the next true envelope boundary: roughly53Giin the vendor temp tree before kubelet eviction at the 64Gi scratch limit, with memory also touching the 64Gi cgroup ceiling. Follow-up canary26246609243passed the 64Gi boundary and was later evicted at the 128Gi scratch limit. The committed heavy-lane ARC envelope moved to96Girequested /160Gilimited memory and192Girequested /256Gilimited ephemeral storage. Managed apply26247461740reconciled that envelope live, and canary26247715938then failed in Bazel vendor mode rather than by kubelet eviction: after completing the Rust/Cargo repository work it hit the BCRrules_pkg@1.1.0internalmappings_test_external_repolocal repository pointing at missingtests/mappings/external_repo. PR #768 now bumpsrules_pkgto1.2.0, whose BCR MODULE removes that local test repository. The same canary also hit GitHub artifact quota during evidence upload, so the workflow mirrors the evidence JSON into the Actions step summary and keeps artifact upload best-effort. Follow-up canary26350919668moved past therules_pkgleak and exposed the next repository-rule authority gap:pybind11_bazelwanted an explicit local Python repository. PR #768 now makespkgs.python3part of the CI devshell and passes--repo_env=PYTHON_BIN_PATH=...into the vendor/build/test phases so the Python path is declared evidence rather than ambient runner state. Workflow-dispatch canary26351062144then passed the full-scope W3.4 lane on branch head27e40ce:bazel vendorcompleted, the materialized graph moved into a roughly170Givendor directory, and//:deployment_bundlebuilt successfully from that vendor directory. Live runner observation showed a141429174272byte memory peak with zero cgroup OOM/max events. Current-main canary26549932671repeated the proof on SHA000ee8e86ce084f245273b63d3b2dbe2ab58b0ef, uploaded downloadable evidence, and showed the live pod staying around14Gimemory while scratch reached roughly170Gi. The committed envelope now keeps the160Gimemory limit for burst headroom, lowers the memory request to64GisomaxRunners = 2is a truthful honey-backed dogfood target under ordinary load, and preserves the192Gi/256Giscratch request/limit. The 2026-05-31 scheduled canary26710742767then exposed a real external-input gap rather than runner capacity:bazel vendortried to fetch ahermetic_launcherprebuilt stub from GitHub and hit a502. PR #855 recorded the fullhermetic_launcherprebuilt stub set in the candidate manifest and staged those verified bytes intoBAZEL_DISTDIR; workflow-dispatch canary26717187299passed on the dogfoodedtinyland-nix-heavylane withclassifier=ok, built//:deployment_bundlefrom the vendored graph, and was merged as3d68e10. Post-merge main canary26718312931repeated the proof onmainat3d68e10, with the vendor step, evidence summary, and artifact upload all green. This is W3.4 vendor-mode proof, not durable external-input authority or broad/default RBE closure. The E3 external-input authority status surface is now executable asjust e3-external-input-authority-status. It rolls up the W3.4 vendor-mode nightly streak, latest on-demand green proof, theBazel Distdir Full Package Proofnightly streak, the scoped GitHub mirror variable/secret-name readiness gate, and the manualBazel Distdir Mirror Live Proofgate. Current May 31 truth remainsE3_EXTERNAL_INPUT_AUTHORITY_NOT_READY: vendor-mode has a green reviewed on-demandmaindogfood proof after the latest scheduled red, but E3 still needs the full 14-night scheduled streak onmain, matching distdir-package streak, scoped mirror authority names, and a reviewed live mirror proof. - BCR/package-authority work is separate from RBE execution authority.
tinyland-inc/bazel-registry#42addedtummycrypt_scheduling_bridge@0.5.11withtummycrypt_scheduling_kit@0.8.0and preserved historicaltummycrypt_scheduling_bridge@0.4.10metadata after verifying the old release really depended on scheduling-kit^0.7.7. The post-merge registry validation run25637800822passed validate, resolve smoke, and Stage 1 consumer smoke. This is package-authority hygiene, not RBE evidence. - runner classes should be described by proved capability lanes and explicit boundaries, not by repo-shaped or aspirational claims
docs/may contain operator-only topology, auth, and planning context; do not treat it as a sanitized public package
Current Proof Reality
- live default-branch status is governed by GitHub Actions; this section records
recent proof packages and should not be read as a claim that a named docs-only
commit will remain the moving
HEAD - the latest audited default-branch proof package is
f0721b58ec4a2e25f525dd7655a77748ec4e3959after PR #695. - the PR #695 post-merge package was green for Secret Detection
25980560631, Validate25980560638, Tranche Proof Status25980560632, Publish to FlakeHub25980560619, Deploy Docs25980560637, RustFS State Authority Canary25980560626, Platform Proof25980560634, and Source Bazel Proof25980560625. The post-PR #695 live read-only burst audit showedtinyland-nix-heavyclear atcurrent=0 pending=0across the two owner overlays, whiletinyland-nixhad one pending runner without declared label-capacity saturation. - PR #587 refreshed the BCR/RBE/RustFS product-reality docs and guard without
expanding RBE claims. PR #596 guarded the remote-test roadmap truth, PR #597
through PR #601 added pilot adoption docs/templates and workflow contract
guards, PR #602 synchronized cache publication reality, PR #604 added Stage
1 rust/c++/go cache-backed test targets to Source Bazel Proof, PR #605 fixed
gf-reapi-cell output inlining so only requested inline outputs are returned
in Execute responses, PR #607 kept the Go proof target pure, PR #608
promoted the pure-Go
//examples/hello-go:hello_testclass, PR #609/#610 hardened the dashboard Docker pnpm/Corepack bootstrap, PR #611 reconciled the May 10 truth documents, PR #624 promoted the trivial C++ REAPI proof, PR #628 added the browser-capable Playwright smoke target, PR #630 hardened the worker scratch/home handling for Chromium, PR #668 promoted the public omux standalone Vite build proof, run25897326537extended omux browser proof to a Playwright static-output smoke class, PR #669 promoted the publicjesssullivan.github.ioVitest proof, PR #670 promoted the publicjesssullivan.github.ioPlaywright runtime proof, and PR #671 recorded the private web consumer checkout-authority blocker, PR #679 added the repo-scoped deploy-key checkout path, PR #682 verified the private tinyland-schemas codeload distdir handoff, PR #683 promoted the private tinyland.dev Grafana Vitest proof, PR #685 through PR #687 promoted the private MassageIthaca svelte-check, TypeScript no-emit, and Playwright TMD proof classes, PR #688 scaled the REAPI proof cell for TypeScript fanout, PR #689 promoted the private tinyland.dev app typecheck proof, PR #690 made the proof cell scale to zero between proofs, PR #693 promoted the private tinyland.dev root app build proof, PR #694 synchronized that post-merge truth, and PR #695 addedShared Label Queue Pressureto the ARC burst audit so scarce capability lanes show holder repositories, pending runner state, and scheduler resource-pressure messages before a capacity mutation. Forced REAPI run25638930305recorded the next C++ target-class blocker: the remote compile action for//examples/hello-cc:hello_testtried to execute a Nixgcc-wrapperpath missing from the worker and produced6 processes: 6 internal. After the worker image carried the C/C++ wrapper closure, run25648975728proved the trivial C++ unit-test class with8 processes: 4 internal, 4 remote. PR #605 also published signed image digestsha256:bb5455a038bdbff2560f22491c131c2163d3089ffafedee08f937d63f35fa848for the GF REAPI cell. PR #574 added the checked target-class eligibility manifest, PR #575 added optional backend-neutral ARCBAZEL_REMOTE_EXECUTORwiring while keeping the default runner posture cache-backed, and PR #586 promoted//docs-site:buildfrom candidate to proved after PR #585 repaired the docs-site Bazel source shape. The forced explicit GF REAPI proof set now includes//app:buildfrom run25581256308with869 remoteprocesses,//:public_vendor_handoff_fixturefrom run25589377905with1 remoteprocess and injected public WAS-110 repository evidence,//app:unit_testsfrom run25601913985withbazel_command=test,527 remoteprocesses, 20 Vitest files, and 168 passing tests,//:deployment_bundlefrom run25602726443with1 remoterules_pkgbuild_taraction, and//docs-site:buildfrom run25608601158with1046 remoteprocesses and remotedocs-site/.svelte-kit/docs-site/buildaction evidence, and pure-Go//examples/hello-go:hello_testfrom run25634296833withbazel_command=test,11 remoteprocesses, remoteGoStdlib/ compile / link /test-setupevidence, and cgo-backed Go//examples/hello-go-cgo:cgo_testfrom run25649628233withbazel_command=test,11 remoteprocesses, remoteruntime/cgo,GoCompilePkg,GoLink, andtest-setupevidence, Rust//examples/hello-rust:hello_testfrom run25648670844with5 remoteprocesses, and C++//examples/hello-cc:hello_testfrom run25648975728with4 remoteprocesses, and//docs-site:playwright_chromium_smokefrom run25712694947withbazel_command=test,1060 remoteprocesses, remotesveltekit_sync, remotevite_build, remotetest-setup, and a passingplaywright-coresmoke with/bin/chromium; andtinyland-inc/omux.xoxd.ai //:puppeteer_chromium_smokefrom run25826953857withbazel_command=test,137 remoteprocesses, remotesveltekit_sync, remotevite_build, remotetest-setup, and a passingpuppeteer-coresmoke with/bin/chromium,tinyland-inc/omux.xoxd.ai //:playwright_chromium_smokefrom run25897326537withbazel_command=test, forced execution, proof nonce20260515T024138Z-25897326537-1,6 remoteprocesses, public main commitd3608a5a6325adee0a5e625cf7ad76b470e7b83f, remote@tailwindcss/oxideandesbuildlifecycle hooks, remotesveltekit_sync, remotevite_build, remotetest-setup.sh playwright_chromium_smoke_/playwright_chromium_smoke, remotegenerate-xml.sh, and a passing Playwright Chromium static-output smoke with/bin/chromium,tinyland-inc/omux.xoxd.ai //:playwright_local_route_smokefrom run26005817853withbazel_command=test, forced execution, proof nonce20260517T232840Z-26005817853-1,13 remoteprocesses, public main commitcd730bdc432b6eb2af4cac7032c040e4ab734da7, GitHub App checkout authority, remote@tailwindcss/oxideandesbuildlifecycle hooks, remotesveltekit_sync, remotevite_build, remotetest-setup.sh playwright_local_route_smoke_/playwright_local_route_smoke, remotegenerate-xml.sh, proof artifact id7047042599, artifact sha2569b4509a1095f707678d2e13a4f78861db74d55cb5af2538e8c277ec3bae1e4c4, and a passing Playwright local route smoke for/agent-snippetwith/bin/chromium,tinyland-inc/omux.xoxd.ai //:puppeteer_local_route_smokefrom run26037732121withbazel_command=test, forced execution, proof nonce20260518T135044Z-26037732121-1,10 remoteprocesses, public main commit50e0b796cbc44bc82de67891b1999e7e48cff473, GitHub App checkout authority, remote@tailwindcss/oxideandesbuildlifecycle hooks, remotesveltekit_sync, remotevite_build, remotetest-setup.sh puppeteer_local_route_smoke_/puppeteer_local_route_smoke, remotegenerate-xml.sh, proof artifact id7059740497, artifact sha256cf768f62b03f84e3246a2adc012fa14b6c7026ede1bcb2e0d8352f8221b1dd4c, and a passing Puppeteer local route smoke for/agent-snippetwith/bin/chromium,tinyland-inc/tinyland.dev //:puppeteer_local_route_smokefrom run26051698671withbazel_command=test, forced execution, proof nonce20260518T181314Z-26051698671-1,132 remoteprocesses, private main commitdcb859f658092dc2a6c0f33223cb9ec9a4055c18, GitHub App checkout authority, verifiedtummycrypt_tinyland_schemas:0.2.4distdir staging, remote@tailwindcss/oxide,sharp, andesbuildlifecycle hooks, remote TypeScripttsc, remote Svelte and Vite build-tool execution, remotetest-setup.sh puppeteer_local_route_smoke_/puppeteer_local_route_smoke, remotegenerate-xml.sh, proof artifact id7065881708, artifact sha256270bcb553348afb4ae8a77f2954bb4f9fa75c2570b6d2d26a6eef9dbc612ea99, and a passing Puppeteer local route smoke for/legal/privacywith/bin/chromium,tinyland-inc/omux.xoxd.ai //:unit_testsfrom run25742782051withbazel_command=test,4 remoteprocesses, and remotetest-setup.sh unit_tests_/unit_tests ... ./vitest.config.tsevidence, plustinyland-inc/omux.xoxd.ai //:buildfrom run25891956165withbazel_command=build, forced execution, proof nonce20260514T234057Z-25891956165-1,4 remoteprocesses, remote@tailwindcss/oxideandesbuildlifecycle hooks, remotesveltekit_sync, and remotevite_buildevidence, plusJesssullivan/jesssullivan.github.io //:puppeteer_chromium_smokefrom run25777472760andJesssullivan/jesssullivan.github.io //:sveltekit_vite_build_smokefrom run25779597385, each withbazel_command=test, forced execution,855 remoteprocesses, and remotetest-setupevidence, plusJesssullivan/jesssullivan.github.io //:playwright_chromium_smokefrom run25894297074withbazel_command=test, forced execution, proof nonce20260515T005745Z-25894297074-1,855 remoteprocesses, remote lifecycle-hook execution without browser download, remotetest-setup.sh playwright_chromium_smoke_/playwright_chromium_smoke, and a passing Playwright Chromium runtime smoke with/bin/chromium, plusJesssullivan/jesssullivan.github.io //:types_unit_testsfrom run25892939448withbazel_command=test, forced execution, proof nonce20260515T001050Z-25892939448-1,855 remoteprocesses, remoteesbuild,sharp, andpuppeteerlifecycle-hook actions without action-time browser downloads, and remotetest-setup.sh types_unit_tests_/types_unit_testswithexit_code=0, plusJesssullivan/MassageIthaca //:svelte_check_testfrom run25938855554withbazel_command=test, forced execution, proof nonce20260515T200641Z-25938855554-1,3319 remoteprocesses, remotesveltekit_sync_bin_/sveltekit_sync_bin, remotetest-setup.sh svelte_check_test_/svelte_check_test, remotegenerate-xml.sh, and one passing SvelteKit/svelte-check action, plusJesssullivan/MassageIthaca //:tsc_noemit_testfrom run25948484331with forced execution, proof nonce20260516T005553Z-25948484331-1,3319 remoteprocesses, remotesveltekit_sync_bin_/sveltekit_sync_bin, remotetest-setup.sh tsc_noemit_test_/tsc_noemit_test, remotegenerate-xml.sh, and one passing TypeScript no-emit action in24.2s, plusJesssullivan/MassageIthaca //:playwright_tmd_smokefrom run25953478878with forced execution, proof nonce20260516T050753Z-25953478878-1,3318 remoteprocesses, remotesveltekit_sync_bin_/sveltekit_sync_bin, remotevite_build_bin_/vite_build_bin, remotetest-setup.sh playwright_tmd_smoke_/playwright_tmd_smoke, remotegenerate-xml.sh, and one passing Playwright TMD smoke action in4.5s.tinyland-inc/tinyland.dev //:app_typecheckfrom run25970619559adds a private root app typecheck proof with GitHub App checkout authority, verifiedtummycrypt_tinyland_schemas:0.2.4distdir staging, forced execution, proof nonce20260516T191944Z-25970619559-1,56 remoteprocesses, remote TypeScripttsc, remote Svelte build tool, remote Vite build tool, remoteapp_typecheck_tool, and Kubernetes restart evidence that stayed at0.tinyland-inc/tinyland.dev //:app_buildfrom run25978934708adds a private root Vite/SvelteKit production-build proof with GitHub App checkout authority, verifiedtummycrypt_tinyland_schemas:0.2.4distdir staging, forced execution, proof nonce20260517T021820Z-25978934708-1,62 remoteprocesses, remote TypeScript package fanout, remoteJsRunBinary app_build.log, proof verifier success, and Kubernetes restart evidence that stayed at0.tinyland-inc/tinyland.dev //packages/tinyland-activitypub:testfrom run25981546207adds a private ActivityPub package Vitest proof with GitHub App checkout authority, verifiedtummycrypt_tinyland_schemas:0.2.4distdir staging, forced execution, proof nonce20260517T044208Z-25981546207-1,14 remoteprocesses, remoteesbuildlifecycle-hook execution, remote TypeScripttsc, remotetest-setup.sh packages/tinyland-activitypub/test_/test, remotegenerate-xml.sh, proof verifier success, and Kubernetes restart evidence that stayed at0.tinyland-inc/tinyland.dev //packages/tinyland-a11y-engine:typecheckfrom run25984827370adds a private package TypeScript typecheck proof with GitHub App checkout authority,workspace_path=consumer-workspace, consumer commit3730c6966d5e069cff92abc7c606fca9db5b54af, verifiedtummycrypt_tinyland_schemas:0.2.4distdir staging, forced execution, proof nonce20260517T073751Z-25984827370-1,2 remoteprocesses, remoteesbuildlifecycle-hook execution, remote TypeScripttscforpackages/tinyland-color-utils, proof verifier success, and Kubernetes restart evidence that stayed at0.tinyland-inc/tinyland.dev //:playwright_local_route_smokefrom run25989829826adds a private local-server Playwright route-smoke proof with GitHub App checkout authority, verifiedtummycrypt_tinyland_schemas:0.2.4distdir staging, forced execution, proof nonce20260517T114200Z-25989829826-1,53 remoteprocesses, remote TypeScripttsc, remote Vite build-tool execution, remotetest-setup.sh playwright_local_route_smoke_/playwright_local_route_smoke, remotegenerate-xml.sh, proof verifier success, and Kubernetes restart evidence that stayed at0.tinyland-inc/tinyland.dev //:puppeteer_local_route_smokefrom run26051698671adds a private local-server Puppeteer route-smoke proof with GitHub App checkout authority, verifiedtummycrypt_tinyland_schemas:0.2.4distdir staging, forced execution, proof nonce20260518T181314Z-26051698671-1,132 remoteprocesses, remote TypeScripttsc, remote Svelte and Vite build-tool execution, remotetest-setup.sh puppeteer_local_route_smoke_/puppeteer_local_route_smoke, remotegenerate-xml.sh, proof verifier success, and Kubernetes restart evidence that stayed at0.Jesssullivan/MassageIthaca //:sveltekit_node_buildfrom run25983800544adds a private SvelteKit/Vite production-build proof with repo-scoped deploy-key checkout, consumer commite06a70d12417f04568092a62e225b6c6595c3b39, forced execution, proof nonce20260517T064447Z-25983800544-1,3193 remoteprocesses, remote lifecycle-hook execution foresbuild,msw, andsharp, remotesveltekit_sync_bin_/sveltekit_sync_bin, remotevite_build_bin_/vite_build_bin, proof verifier success, and Kubernetes restart evidence that stayed at0. This is real proof-lane RBE evidence for narrow target classes, not default product RBE, broad JavaScript/Vitest/Playwright/Puppeteer/E2E RBE, publication/deployment RBE, OpenTofu RBE, image publication RBE, all Go/C++/Rust tests, or RustFS-backed CAS/action-cache authority. The earlier PR #564 checkpoint at259a8fce7e73476270845ba2d7c741e80ab6e6b8established the first successful app-target proof and exposed the warm-run cache-hit gap that PR #565 closed. The earlier PR #491 checkpoint atc4544a65e536bac0576820ed04523e4a82d3701band PR #489 checkpoint at568fad179217251e8c9b4c3c7d80e49965f5fddcadditionally provedDeploy ARC RunnersandBuild Container Images; the PR #479 Source Bazel Proof recordedBAZEL_REMOTE_CACHE=grpc://bazel-cache.nix-cache.svc.cluster.local:9092, external-fetch statusupstream-with-retries, and1 remote cache hit. - the current developer-machine cache attachment policy is explicit and
bounded:
without a routable operator-provided
BAZEL_REMOTE_CACHE,just inforeportscompatibility-local-onlyandjust cache-contract-strictfails.just developer-cache-attachment-proofis the bounded proof command once the endpoint is present. On 2026-04-29, the internal operator proof passed through a Honeysvc/bazel-cachelocalhost port-forward atgrpc://127.0.0.1:19092and built//:deployment_bundlein read-only cache mode. This remains shared cache acceleration, not Bazel remote execution or a stable public Bazel cache hostname. TIN-650 proved the operator-provided developer-machine attachment path; TIN-758 records the supported exposure policy as operator-provided endpoint only. GitHub #417 is closed after lab PR #304 proved the downstream package-canary cache contract. Any future endpoint posture decision needs a fresh infra/auth/network issue. - PR #445 made
just checkthe bounded local hygiene gate, kept heavyweight Nix/OpenTofu validation on explicitjust check-full/just nix-check, and added the repo-ownedscripts/tofuresolver so OpenTofu commands use the flake-managed toolchain instead of ambient host state - the 2026-04-29 read-only ARC runtime audit showed all ARC listener pods
Running on
sting, two activemassageithacarunner pods onsting, no listener-cap drift, and no broker/session continuity errors in scanned active runner logs. Node conditions wereReady=True,DiskPressure=False, andNetworkUnavailable=Falseforbumble,honey, andsting;kubectl topreported about 5% / 25% CPU/memory onbumble, 3% / 13% onhoney, and 18% / 18% onsting. That checkpoint closed the stale listener-placement assumption; network/session continuity incidents now need fresh evidence, not inherited assumptions. - later on 2026-04-29, PR #476 provided fresh TIN-620 evidence: GitHub job
73677624434forProve tinyland-nix contractstayed queued whileAutoscalingRunnerSet/tinyland-nixhadmaxRunners=16and reportedcurrent=0,pending=0, andrunning=0. The listener pod was Running but stopped after deleting broker message100003897. Restarting only thearc-systems/tinyland-nix-ddd868ff-listenerpod caused the replacement listener to scale immediately, the PR job passed, and the post-merge main proof passed. This was a listener/session/broker handling symptom, not a raw capacity cap. - TIN-620 is closed as a repeatable diagnostic and stale-check disposition, not
as a claim that future ARC/CNI incidents cannot happen. PR #484 added offline
arc-runtime-auditcoverage for healthy runtime, broker/session drift, and idle listener-cap drift. The 2026-05-09 live sprint kickoff audit found freshtinyland-nixrunner-control drift:AutoscalingRunnerSet/tinyland-nixreportedphase=Pending,currentRunners=0, no listener-config secret, and two 18-hour active runner pods whose logs contained broker/socket retry evidence. Main remained green through other lanes, but that does not clear the staletinyland-nixcontrol-plane state. Deleting the staleEphemeralRunnerSet/tinyland-nix-xc9zxrestoredAutoscalingRunnerSet/tinyland-nixtophase=Running, recreated thetinyland-nix-ddd868ff-listener/ listener-config path, allowed the PR #588Prove tinyland-nix contractjob to pass, and the strict post-remediation audit found no listener-cap, runner-count, or broker/session drift. The audit now also reports ARS phase, active runner pod counts, missing listener config, and runner-count drift so this class is visible before a queued-job symptom appears. PR #485 addedjust arc-network-continuity-auditto classify API/CNI continuity, kubelet eviction/pressure, and node readiness/network evidence before blaming cache/auth/overlay code. The original Jess overlay push run24941652054is no longer a stale failing check: GitHub reports run attempt 3 assuccess, with the replacementvalidatejob73040148853passing onjesssullivan-nix-runner-2vtrzat2026-04-25T23:54:27Z. The earlier failed job73037946157remains historical API/CNI continuity evidence, not an active failing gate. The 2026-05-27 #805 managed apply showed the same control-plane shape can recur with idle no-job runners: staleEphemeralRunnerSet/tinyland-nix-bg297kept thetinyland-nixlistener from recreating after caps were restored. The runtime audit now has a non-mutating--fail-on-stale-idle-listener-blockerguard that reports the owningEphemeralRunnerSet, candidate no-job runners, and whether GitHub runnerbusy=false/ status evidence has been supplied before any cleanup command is printed. This keeps the dogfooding lane recoverable without normalizing hosted-runner fallback or broad runner deletion. - the 2026-04-29
just kubelet-imagefs-capacity-audit --node bumbleread-only audit kept TIN-613 active:bumblereportedReady=True,DiskPressure=False, andNetworkUnavailable=False, but kubeletrootfs,imagefs, andcontainerfseach had only 11.4 GiB available (16.3%) on a 69.9 GiB filesystem. This is below the 20% warning threshold even though durable OpenEBS/ZFS storage remains storage-biased towardbumble. The offline self-test now preserves that warning boundary and a stricter critical boundary as regression fixtures; it does not claim the live node headroom has been remediated. A fresh 2026-05-02 read-only audit still showedbumblebelow the warning threshold at 12.0 GiB available (17.1%) forrootfs,imagefs, andcontainerfs, withDiskPressure=False; this confirms the selected near-term decision is scheduling avoidance plus a future host maintenance window, not using raw ZFS capacity as runner headroom. - the 2026-05-11 runner availability incident also exposed a scheduler limit that is easy to miss when looking only at cluster CPU and memory: honey can hit its node pod-count ceiling while sting and bumble still have apparent headroom. Honey-pinned runner classes then remain pending because the lane’s selectors and tolerations do not admit them to sting, and bumble is the storage-biased OpenEBS/ZFS node rather than default runner burst capacity. The fix is topology and shared-capability capacity work, not repo-specific labels and not treating raw OpenEBS storage as hot runner scratch.
- the prior audited external-fetch proof package passed on
mainat44f6a948470fc98a5233cb7b835562d01289038c:Validate,Secret Detection,Source Bazel Proof,Platform Proof,Deploy Docs,Publish to FlakeHub, andTranche Proof Statuswere green. The Source Bazel Proof records remote action cache configured, repository cache unset, distdir unset, downloader retries configured, and external-fetch statusupstream-with-retries. - the earlier
Deploy ARC Runnersproof package passed onmainat3c7fdf5261ed74babee54c792c3cce3c9e71112f:Validate,Secret Detection,Source Bazel Proof,Platform Proof,Deploy Docs,Publish to FlakeHub,Deploy ARC Runners, andTranche Proof Statuswere green - the
Deploy ARC Runnersproof for3c7fdf5is intentionally honey-bound for plan/apply/verify. The preceding PR attempt proved the boundary by timing out against the kube API fromsting, while the final PR and default-branch proof kept stateless Docker relief onstingand kube mutation on a honey-reachable capability lane - the Docker placement proof package passed on
mainat371edd20acd4a7009fa38d7425282e646370aeb0:Validate,Secret Detection,Source Bazel Proof,Platform Proof, andPublish to FlakeHubwere green Deploy ARC Runnersworkflow-dispatch apply run25010338517completed withallow_destroy=falseafter PR #443 merged; post-apply verification showed thetinyland-dockerrunner template and listener template onstingwithdedicated.tinyland.dev/compute-expansion=true:NoSchedule- the Docker placement apply proves stateless Docker lane relief for the current honey pod-ceiling pressure; it does not prove global cross-overlay capacity coordination above Kubernetes scheduling
- the first PR #444 plan run after the Docker placement apply exposed the
boundary directly:
tinyland-dockeronstingcould run the Docker proof but timed out reaching the kube API endpoint fromDeploy ARC Runners, so ARC plan/apply must stay on a honey-bound lane until that network path is proved - the post-PR-434 proof package passed on
mainat7ae6b3d653199ec1dc5299f2a541a63225a9aa94:Source Bazel Proof,Platform Proof,Validate,Secret Detection,Deploy Docs, andPublish to FlakeHubwere green - the
Source Bazel Prooffor7ae6b3dran ontinyland-nix, requiredGF_BAZEL_SUBSTRATE_MODE=shared-cache-backed, passedBAZEL_REMOTE_CACHE=grpc://bazel-cache.nix-cache.svc.cluster.local:9092explicitly through the wrapper path, and reported1 remote cache hit - PR #564 added the first audited explicit REAPI proof package for Bazel remote
execution. PR #572 added machine-verified WAS-110 public-input proof
artifacts.
BAZEL_REMOTE_EXECUTORremains opt-in and absent from the default cache-backed wrapper path. - the
Platform Prooffor7ae6b3dshowed the Nix runner fetchingrunner-dashboardfromhttp://attic.nix-cache.svc.cluster.local/mainand pushing a post-job Attic delta; this is real cache acceleration, not universal remote execution/offload - TIN-545 hardening merged as PR #404 at
c48869d16eec6cda3f83894c8edad3a03d47554f; the default-branch proof package passed onmain:Platform Proof24921935297,Source Bazel Proof24921935299,Validate24921935307,Secret Detection24921935309, andPublish to FlakeHub24921935301 - the post-PR-404
tinyland-nix-heavydefault-branch proof passedrunner-dashboardon attempt 1 in job72984921255, so the new bounded retry path did not mask the proof - PR #404 keeps non-signal failures hard-failing while adding diagnostics and a single retry for the known signal-9/Killed class; the recurring class is now handled as bounded proof hardening rather than only a manual rerun recovery
- the TIN-543 parity-loop close merged as PR #402 at
d5bc77780bb2bf9e91802c301a483a5e4104eeee; its default-branch proof package passed after rerunning the failedtinyland-nix-heavyjob inPlatform Proofrun24921370509 - that first
tinyland-nix-heavyattempt failed on the recurringrunner-dashboardsignal-9 class; the rerun passed, the docs-only TIN-543 diff was not identified as the cause, and TIN-545 later landed the bounded hardening described above - the post-PR-401 proof package passed on
main:Source Bazel Proof24921220511,Platform Proof24921220518,Validate24921220516,Secret Detection24921220517,Deploy Docs24921220512,Publish to FlakeHub24921220522, andTranche Proof Status24921220519 - PR #400 remains the latest landed product slice before the status-parity
refresh; its proof package passed on
main:Build Container Images24920942761,Source Bazel Proof24920942766,Platform Proof24920942768,Validate24920942769,Secret Detection24920942763,Deploy Docs24920942770,Publish to FlakeHub24920942760, andTranche Proof Status24920942762 Build Container Imagespassed on the sharedtinyland-dindlane for the runner dashboard, Caddy Tailscale proxy, Nix runner, and browser runner image jobs after PR #400 mergedSource Bazel Proofpassed onmainin run24913926497after the ARC apply; the proof ran ontinyland-nixand kept the source-repo Bazel package on the cache-backed wrapper pathPlatform Proofpassed onmainin run24913926480after the ARC apply;tinyland-docker,tinyland-nix, andtinyland-nix-heavyall completed their shared runner contractsDeploy ARC Runnersapply run24913839623completed successfully with0 added, 1 changed, 10 destroyed, and the post-apply listener-cap audit passedSource Bazel Proofpassed onmainin run24912686706after #388; the log showsBAZEL_REMOTE_CACHE=grpc://bazel-cache.nix-cache.svc.cluster.local:9092and execution throughscripts/bazel-cache-backed.shPlatform Proofpassed onmainin run24912686712;Validate,Secret Detection,Publish to FlakeHub, andDeploy Docsalso passed on the same merge- the
Build Container Imagesworkflow and release image-build matrix now targettinyland-dind; the default-branch image workflow path filter includes the workflow file itself so changes to the image-publication contract trigger a default-branch proof - TIN-514 records the follow-up capacity correction from that proof: bumble’s
current ephemeral-storage envelope caused pending DinD runner payload pods
during image publication bursts, so the shared DinD payload lane now uses
honey’s larger ephemeral-storage envelope while preserving the
tinyland-dindlabel - the earlier green Source Bazel Proof on run
24909959967was not a truthful cache-backed proof: Bazel received literal${BAZEL_REMOTE_CACHE}from.bazelrcand then completed locally; #388 fixed that by passing--remote_cache="$BAZEL_REMOTE_CACHE"explicitly - default-branch proof surfaces on
mainare real operating truth, but they still depend on self-hosted runner/cache injection and bounded cluster-local DNS reachability - runner pressure is still an active boundary: queue pressure, finite runner
envelopes, additive lanes such as
tinyland-nix-heavy, and node hygiene remain part of the current dogfood contract rather than solved background details - PR #419 merged at
cacc9497617f8c2f096afb5152d16e8774dd8d14; the post-merge proof package passed Secret Detection, Validate, Deploy Docs, Publish to FlakeHub, Platform Proof, Source Bazel Proof, Build Container Images, and Tranche Proof Status - PR #420 merged at
b120c99eddf0dbabfce8d07116ac8dfa7c1a7590; it restored the implementation-overlay boundary, preserved compatibility releases through moved state addresses, and kept destructive ARC cleanup guarded - PR #422 merged at
5b0e0b8584a865d3428332070080903c9efd1dbd; it added the implementation-overlay workstream and enrollment-preflight tracking surface justwas added to the flake devshell on 2026-04-25; overlay CI previously failed withjust: not foundbecause the devshell banner referenced Just but the package was not indevToolslibatomic1was added to thetinyland-nixrunner image on 2026-04-25 to support Node 25 runtime requirements fromactions/setup-nodebumbleimage-GC/free-disk follow-up was revisited on 2026-04-25 after ARC listener eviction: the node isReadyandDiskPressure=False, but its kubelet root/image filesystem is a small root volume separate from the ZFS storage pool and remained tight after supported CRI and Nix cleanup. Treat bumble as storage-biased OpenEBS/ZFS infrastructure, not as the default ARC listener or baseline runner scheduling authority.- source-repo dogfooding is stronger after #388 because the source Bazel proof now attaches to the actual shared cache endpoint, but it still proves shared cache acceleration rather than universal full remote offload for every developer workload
- the bounded advanced-runner proof floors are landed: KVM, shared GPU, Dawn/WebGPU userspace proof, and bounded Darwin proof
- the first conservative public-docs package is live in-tree and CI-enforced, but it is not the center of the active productization lane
- several public reference pages are now generated from structured repo metadata instead of maintained only as prose
Current Enrollment And Shared-Lane Reality
- nominal shared-label config, template consumption, and historical exception notes do not count as real default-branch runner authority by themselves
- blocked or partial downstream repos should be described as shared-lane reachability or control-plane debt, not as candidates for repo-shaped runner lanes
- downstream repos such as
Dell-7810remain blocked on shared-lane reachability and owner-scope debt; they are not candidates for productized repo-shaped runner classes XoxdWMremains in the same owner-boundary bucket asDell-7810: its shared fast lanes are toggle-gated,GF_SHARED_RUNNERS_REACHABLEis still unset, and the repo currently sees zero accessible self-hosted runners- personal-account repos cannot become shared pooled-substrate proofs merely by adding repo-scoped ARC anchors; the clean exits are org/enterprise shared scope, an intentional mirror/owner-boundary move, or a blocked state
- the orgwide enrollment queue now encodes #407, #413, and #412 closure-proof policy directly: it names the related issue, blocks or narrows closure canary dispatch, requires assigned-job or post-retirement proof, and names evidence that does not count
Jesssullivan/MassageIthacahas two distinct follow-ups: state ownership for the old repo-shaped ARC residue has been rehomed into the Jess overlay, while the remaining retirement question belongs with #412 and requires migrating or removing compatibility runners; Docker-capable image-publication access is completed in TIN-681 / #438 and proved through shared capability labels rather than a permanent repo-shaped product label- downstream consumer debt is not the same thing as source-repo dogfood integrity; blocked consumer repos should not be treated as proof criteria for whether GloriousFlywheel itself still proves its own substrate honestly
- hosted runners are not part of GloriousFlywheel’s first-party source-repo proof contract; emergency/operator escape hatches must not be counted as product evidence or used for publication/status/release dogfood
- Chapel, Nix, and Bazel-heavy workloads are core reasons the platform exists, not special exceptions
- some named repos still sit in authority-truth or runner-reachability debt:
tinyland-inc/rockies,tinyland-inc/betterkvm,tinyland-inc/tinyland.dev,Jesssullivan/XoxdWM, andJesssullivan/Dell-7810 - current admin/reporting surfaces should describe the platform as a bounded, pooled substrate with active enrollment debt, not as a repo-shaped rollout program
Current Active Route
- restore and keep the default-branch proof package routine and green
- leave a short RCA whenever recovery needs more than a bounded rerun
- retire repo-shaped runner taxonomy from config, docs, and planning surfaces
- retire local-heavy Bazel teaching from canonical product surfaces
- restate and prove the pooled GloriousFlywheel dev-plus-CI substrate contract
- make source-repo dogfooding the first proof point again
- keep
Source Bazel Prooftruthful: endpoint presence alone is not enough; the proof must pass the actualBAZEL_REMOTE_CACHEvalue to Bazel - keep downstream blocked repos framed as shared-lane reachability problems and separate from source-repo dogfood integrity claims
- keep
docs/,README.md, and planning/admin surfaces honest about the internal/public split and the current bounded dogfood state - keep the completed TIN-490 ARC lane retirement visible as live stack truth: destructive apply was explicit, repo-derived scale sets were removed, and post-apply source/platform proofs stayed green
- keep auth authority explicit: access auth, read-side authority, compatibility mutation, and the missing forge-neutral mutation authority must stay separate
- keep the TIN-545 signal-9 handling honest: diagnostics plus a one-shot retry for the known class are product hardening, not proof that runner pressure is solved forever
- keep
aarch64/riscv/Dawn-native dispatch and localized warm-cache guarantees in future-lane research until each one has a named proof surface - keep the public-alpha export route green by keeping
README.md,public-docs/, secret scanning, dogfood contract checks, and the explicit visibility-history gate aligned to the bounded current truth
Biggest Remaining Gaps
- some management surfaces still overread nominal runner intent as current authority unless the queue and scoreboard are kept honest
- the repo only partially proves the stronger remote-backed developer story it intends to tell; current proof is shared cache acceleration, not full remote execution or remote builder offload
- native
aarch64,riscv, and dedicated Dawn dispatch are not current platform contract surfaces; they remain future lane design work, not present product truth - localized warm-cache guarantees for heavy Hackage, Chapel, GPU backends, and similar toolchain surfaces are also future architecture work, not a current runner-level promise
public-docs/is still narrower than the internal operator, topology, and onboarding surface by design- runner pressure, cache reachability, and downstream shared-lane reachability remain bounded operational realities, not solved historical problems
- repo-derived ARC scale-set config has been removed as normal product
structure, but explicit compatibility quarantine remains in the active Jess
overlay; closed
#409recorded the later-observed livemassageithacaresidue, closed #421 / TIN-568 records the completed implementation-overlay rehome, and open #412 now tracks the remaining retirement decision forpersonal-*,personal-package-*, and related compatibility lanes; closed #438 / TIN-681 records the Docker-capable MassageIthaca image-publication proof without reopening repo-shaped workflow labels as product structure bumbleno longer has an active DiskPressure signal after the TIN-495 follow-up, but the April 29 kubelet/imagefs audit still shows only 16.3% available rootfs/imagefs/containerfs headroom. The May 1 fixture coverage makes the warning/critical thresholds repeatable in CI, and the May 2 live audit still shows only 17.1% available. Keep the live remediation as TIN-613 node hygiene, not as the source Bazel proof root cause.- the later cache-first dogfood and advanced-runner board is valid next-horizon productization work, but it should not be treated as the current execution lane until the pooled-substrate reset is complete