Shared-layer lifecycle: accounting, eviction, recovery, materialization

[chruffins]

summary

Capstone stage of the image-storage project — finishes migration, eviction, accounting, recovery, and observability on the shared-layer path.

• Materialization in builds: after a pull, each manifest layer is materialized into images/layers/<digest>/ (best effort; the composed rootfs still comes from blobs, so an artifact failure is logged and only degrades sharing).
• Platform fallback: native per-layer artifacts are enabled only on Linux when running as root. Rootless Linux and macOS skip background layer materialization and continue using the merged rootfs composition path with userspace whiteout handling. Image contents remain unchanged; unsupported hosts only lose per-layer sharing.
• Reference-protected eviction: image deletion evicts only layer artifacts unreferenced by any remaining manifest model; a 10-minute grace period keeps fresh artifacts safe from racing in-flight builds. Startup also evicts orphans left by an unclean shutdown.
• Interrupted-build recovery: stale .unpack-* / .install-* / .tag-stage-* temp directories are swept at startup (age-gated so live builds are untouched).
• Layer-aware accounting: with materialization wired into the pull path, layer artifact bytes now actually land in the TotalOCICacheBytes bucket alongside OCI blobs, so CurrentImageStorageBytes (ready rootfs + cache/layers) counts them exactly once and capacity admission sees the real footprint.
• Observability: hypeman_images_layer_artifacts_evicted_total counter plus slog eviction summaries and per-layer materialization warnings.
• Legacy flattened images remain readable and bootable; they carry no manifest model and no layer references, and are retired only through the existing delete/promotion paths.

validation

• End-to-end lifecycle test: two images sharing a base layer materialize the base once; deleting one image keeps the shared layer; deleting the last reference evicts everything.
• Accounting, grace-period, reference-protection, stale-temp-dir, and unsupported-platform fallback tests.
• Targeted layer and lifecycle tests pass on a rootless host. The full image package suite is blocked only by Docker Hub anonymous pull rate limits (TOOMANYREQUESTS).
• Earlier broader validation passed except Docker Hub pulls (anonymous rate limit) and VM lifecycle tests, which require network-bridge privileges; both failures were also present on unmodified main.
• CI is unavailable during this work; all validation is local.

rebase and follow-up hardening

Rebased onto the current composition and layer-artifact stack. The lifecycle caller now uses the manager's probed native-artifact capability, so Linux-root hosts whose layer-store filesystem lacks mknod or trusted overlay xattrs take the merged-rootfs fallback instead of repeatedly attempting native materialization.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 33c8ca3. Configure here.}

[cursor]

Eviction can wipe in-flight layers

Medium Severity

Layer eviction decides freshness from the digest directory ModTime, and it runs concurrently with materialization. Unpack and mkfs.erofs write inside child temp dirs, so the parent mtime goes stale; DeleteImage can then RemoveAll that tree while a build is still using it, especially during the later unlocked ExportRootfs window before the manifest is written.

Additional Locations (2)

• lib/images/manager.go#L534-L569
• lib/images/manager.go#L865-L867

 

^{Reviewed by Cursor Bugbot for commit 33c8ca3. Configure here.}

[cursor]

Layer bytes skipped after failed builds

Medium Severity

Materialized layer artifacts are written to the layer store before conversion, but refreshDiskUsageTotals only runs on successful finalize, delete, or startup. After a failed conversion the artifacts remain on disk while TotalImageBytes keeps the stale cached total, so capacity admission undercounts the real footprint.

Additional Locations (2)

• lib/images/manager.go#L900-L907
• lib/images/disk_usage.go#L110-L153

 

^{Reviewed by Cursor Bugbot for commit 33c8ca3. Configure here.}