Cache Architecture

Cache Architecture

GloriousFlywheel uses a multi-layer caching strategy for Nix derivations and Bazel actions.

Architecture Diagram

Components

Component Purpose Replicas Scaling
Attic API Server Nix binary cache HTTP API 2-10 HPA (CPU/memory)
Attic GC Worker Garbage collection 1 Fixed
Bazel Remote Cache Bazel action/CAS cache 1-2 HPA
MinIO S3-compatible object storage 1 (single) or 4×4 (distributed) Manual
PostgreSQL (CNPG) Attic metadata database 1 (dev) or 3 (HA) Manual

Data Flow

Nix Cache Push

  1. nix build produces derivation in /nix/store
  2. attic watch-store daemon streams new paths to Attic (incremental)
  3. At build end, attic push sends final closure (belt-and-suspenders)
  4. Attic chunks NARs (64 KiB avg), compresses with zstd (level 8)
  5. Chunks stored in MinIO S3, metadata in PostgreSQL

Nix Cache Pull

  1. nix build checks substituters in order
  2. Attic responds to /nix-cache-info and narinfo queries
  3. Matching paths served from MinIO via Attic API
  4. Nix validates against trusted public keys

Bazel Cache

  1. bazel build --config=ci-cached checks remote cache for action results
  2. Cache hits: download minimal outputs (--remote_download_minimal)
  3. Cache misses: build locally, upload results async (--experimental_remote_cache_async)
  4. Storage: MinIO S3 with LRU eviction at 100 GB

Retention & Cleanup

Storage Retention Mechanism
NAR files 90 days MinIO lifecycle rules
Chunk files 90 days MinIO lifecycle rules
Attic metadata 3 months GC worker (12h cycle)
Bazel cache 100 GB max LRU eviction

Network Endpoints

Service Internal Address External Address
Attic API attic-cache.{ns}.svc.cluster.local:8080 https://attic-cache.{domain}
Bazel Cache (gRPC) bazel-cache.{ns}.svc.cluster.local:9092 Optional ingress
MinIO minio.{ns}.svc.cluster.local:9000 Not exposed
PostgreSQL {cluster}-rw.{ns}.svc.cluster.local:5432 Not exposed

GloriousFlywheel