fix(ci): actually prune the BuildKit layer cache - #7249
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile SummaryThis PR replaces an unsupported
Confidence Score: 5/5The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking issues identified. Current callers provide valid retention values, the fallback handles omitted matrix fields, and diagnostic or prune failures remain non-fatal as intended.
|
| Filename | Overview |
|---|---|
| .github/actions/docker-build/action.yml | Replaces the ignored builder input with a guarded BuildKit prune step and preserves the intended matrix-aware retention fallback. |
Reviews (1): Last reviewed commit: "fix(ci): actually prune the BuildKit lay..." | Re-trigger Greptile
waleedlatif1
force-pushed
the
ci/prune-buildkit-layer-cache-explicitly
branch
from
August 29, 2026 02:21
0638f6b to
de7134d
Compare
The max-cache-size-mb input added in 8fa7f0c never did anything. setup-docker-builder v1 accepted it and pruned in its own post step, but the v2 rewrite dropped the input, and the repo pins v2.1.0. GitHub only WARNS on an unknown composite input, so every build since has logged Unexpected input(s) 'max-cache-size-mb', valid inputs are ['cache-key', ...] and pruned nothing. Scanned every tag to confirm: the input exists in v1.8.0 through v1.12.0 and in none of v2.0.0, v2.0.1, v2.1.0. Rather than downgrade a builder rewrite to reach a config knob, run the prune ourselves — v1's command verbatim, against the fixed address v2 itself uses for `buildctl du` and `debug workers`: sudo buildctl --addr tcp://127.0.0.1:1234 prune --all --keep-storage <MB> Both flags read from BuildKit master rather than assumed. buildctl's --all is not `docker buildx prune --all`: it means "include internal/frontend references", and cache/manager.go shows the only records skipped without it are those typed internal or frontend plus refs shared with an external source. It does not wipe the cache. --keep-storage maps onto the modern MaxUsedSpace field, so it is buildctl's spelling of --max-used-space, not a deprecated alias, and it is the only size flag buildctl exposes. Two guards, both for failure modes that are silent and expensive: Reject a non-positive-integer budget. buildctl parses --keep-storage as a float, and BuildKit treats keepBytes==0 as "no cap" (`gcMode := opt.keepBytes != 0`), pruning everything eligible instead of trimming. A typo like '40GB' — valid in turbo.json, but this flag is a bare MB number — would empty the cache and make every later build cold, costing far more than the storage saved. Wait for `du` to settle after pruning. buildctl prune returns before buildkitd has finished deleting (moby/buildkit#1198), and the builder's post step SIGTERMs buildkitd then SIGKILLs it after 30s (shutdownBuildkitd: `const a=3e4`); on SIGKILL it sets sigkillUsed and skips the sticky disk commit outright, discarding the run's cache and risking a corrupt bbolt metadata DB. The wait is bounded, and a steady-state trim settles almost immediately — it is the first catch-up prune against a 200 GB backlog that would otherwise run into that window. Warn rather than fail throughout, since an oversized cache is not worth failing a deploy over — but print `buildctl du` either side, because a silent no-op is exactly the failure mode that hid this regression for a day.
waleedlatif1
force-pushed
the
ci/prune-buildkit-layer-cache-explicitly
branch
from
August 29, 2026 02:30
de7134d to
753e3b0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
max-cache-size-mbinput I added in8fa7f0cc4enever did anything. Every Docker build since has logged:setup-docker-builderv1 accepted that input and pruned in its own post step. The v2 rewrite dropped it, and we pin v2.1.0. GitHub only warns on an unknown composite input, so it failed silently — the app disk sat above 200 GB for a full day while the config looked correct.Scanned every tag rather than infer:
max-cache-size-mbThe fix
Rather than downgrade a builder rewrite to reach a config knob, run the prune ourselves — v1's command verbatim (from its
dist/index.jspruneBuildkitCache), against the fixed address v2 itself uses forbuildctl duanddebug workers:Both flags verified against BuildKit master, not assumed
buildctl's--allis notdocker buildx prune --all. It means "Include internal/frontend references".cache/manager.goshows the only records skipped without it:--allas being required to reclaimRUN --mount=type=cachedirs — that was wrong. Those are typedexec.cachemountand are reclaimed either way.--allis kept because it's what v1 used and it prunes strictly more.)--keep-storageis not deprecated. Incmd/buildctl/prune.goit maps straight onto the modernMaxUsedSpacefield — it is buildctl's spelling of--max-used-space, and the only size flag buildctl exposes.Why a foreground command
Turborepo's equivalent eviction runs on a detached thread that is never joined, and on a 206 GB cache it makes no measurable progress inside a 150s job (measured: the disk moved 206.2 → 206.7 GB across 17h of builds). A blocking
buildctl pruneeither completes or reports why not.Warns rather than fails — an oversized cache isn't worth failing a deploy over — but prints
buildctl dueither side, because a silent no-op is precisely the failure mode that hid this for a day.Verification
dist/index.js; confirmed v1's exact command and that v2 hardcodestcp://127.0.0.1:1234and already shells out tobuildctl, so binary and address are both present at that point in the job.cmd/buildctl/prune.go,client/prune.go,cache/manager.go,client/diskusage.go).actionlintoutput identical to the staging baseline;shellcheckclean on the new run block.bun run lintand all 39check:auditspass.Effect is not yet measurable — it applies from the first build after merge. I'll report the real
dudeltas from build logs rather than assume.Type of Change
Testing
As above. The real verification is the
before prune -> / after prune ->lines this adds to every Docker build log.Checklist