pre-commit-review: four rule-family hunters (S5)

[buzzdan]

What

Token budget stage S5, the four-hunter arm only. The pre-commit-review skill spawns one rule-hunter per rule family with pre-filter hits, four at most, where it spawned one per rule (up to twelve).

Hunter	Rules	Family
types	R1, R2, R11, R12	primitives, validation, enums and sentinels, options
structure	R3, R4, R5	package, file and function shape
tests and dependencies	R6, R7, R8, R10	tests, globals, dependency injection, dependencies
documentation	R9	comments and the documentation network, beside the comment critic

A hunter gets the absolute paths of its family's rule files that had hits (never a rule with no hit, never two families), reads them all in its first turn (ceiling ~30k tokens, from ~20k), runs every rule's detection commands in one labelled Bash call, and returns one receipt per falsifying question per rule (R<N> Q<n>: …) and one tally per rule. The merged report's Hunters: reconciliation therefore stays per rule and the report format does not change. On --all, a hunter's reading order sorts dirs.txt by the family's combined hits. A rule file that does not read gets R<N>: rule unreadable at <path> in its tally's place, rendered in the header as PARTIAL coverage (Copilot finding, fixed in a8de5c0).

Budget (12955b1): the first turn plus four calls scoped or six on --all, plus one call per rule beyond the first — eight to ten for a four-rule hunter — with the extra calls reserved for the rules' questions before any further reading.

Two changes ride along because they touch the same files:

• Hunter report cap: finding blocks, receipts, tallies and the not reached: line, nothing else, about 3k tokens (was ~10k of narrative the parent drops).
• Review-only command runs no build, tests or linters: /<prefix>-review now invokes the review alone; /<prefix>-analyze still runs all three gates. The review-clean-tree eval runs had spent ~6M tokens per tier on tests and lint before a review that changes nothing.

Files

• core/skills/pre-commit-review/SKILL.md, reference.md — step 2 spawn rule and family table, hunt-focus table gets a Family column, hunter output shape and report cap, cluster-pass wording for family obsession, per-rule reconciliation, unreadable-rule rendering.
• core/agents/rule-hunter.md — family inputs, first turn reads all given rules, one detection loop over every rule, 30k first-turn ceiling, budget scaled by rule count, report cap, per-rule receipts and tallies, unreadable-rule line.
• core/commands/{{.CmdPrefix}}-review.md, -analyze.md — no tests/lint in review; analyze wording.
• READMEs and CHANGELOGs of the three bindings; docs/token-budget.md S5 section rewritten (arm built, gate decision recorded, family table, cap, no-test change).
• Rendered plugins regenerated for go, python and generic.

Gates run

task generate for all three bindings, task check (no drift), lint-core (no hard residue, README section unchanged), the three check-repo-brain_test.sh suites, generator unit tests, scripts/check-docs.sh: all pass on every commit. Rendered sed -n lines stay on one line.

Proof: review tier, Sonnet 5

Two runs. S5 = go-s5-158f75a (fixed 7-call hunter budget; 7 runs, 19.2M tokens, $18.86). S5b = go-s5b-12955b1 (budget scales with rule count; review-full ×3 only, 13.1M tokens, $13.72). Baseline is go-2.11.0-c78b55f (pre-S1); S1–S4 is the run recorded on #61–#63 (its result dirs were not available in this container, so its token figures are the −41% recorded there, ≈6.6M on the baseline's review-full mean).

Gate 1 — graders

case	S5b (head)	S5	S1–S4	baseline
review-full (of 111)	107 · 104 · 101 (mean 104.0)	107 · 98 · 105 (mean 103.3)	110	104 · 103 · 53 (band 103–105)
case-a-retention-review (of 13)	—	11 · 13	13 · 11	13 · 11
case-b-endpoint-review (of 15)	—	13 · 14	15 · 13	13 · 13

• Scoped cases (S5 only; the budget change does not touch them — a one- or two-rule hunter keeps 5–7 calls): one grader flip each against S1–S4, which the reading rules call noise.
• review-full, S5: the structure hunter exhausted its 7 calls on --all in run 2, never ran R4 Q1/Q6, and four R4/R5 plants went missing (98). S5b raised the floor to 101 and every S5b run found every structure plant; the mean did not move (104.0 vs 103.3) because run 3's types hunter (R1, R2, R11, R12 — 20 questions over 68 files) reported not reached on five directories and missed the three CASE-A plants and R1 Q1. The cluster graders (3–5 per run) are the steady miss in every run: the rule-id-on-title convention slips under a shorter report.
• Read: the fixed budget was one cause; the four-rule families are the other, and a budget alone does not close it. Recall sits at the baseline band (101–107), below the twelve-hunter S1–S4 run's 110.

Gate 2 — billed tokens

case	S5b runs	S5b worst	S5 worst	S1–S4	baseline best	baseline runs
review-full	4.63M · 4.44M · 4.00M	4.63M	5.14M	≈6.6M mean	8.45M	8.45M · 13.88M · 11.46M
case-a-retention-review	— (S5: 1.31M · 1.40M)	1.40M	1.40M	n/a	1.34M	1.44M · 1.34M
case-b-endpoint-review	— (S5: 2.11M · 1.69M)	2.11M	2.11M	n/a	1.27M	1.27M · 2.67M

• review-full: S5b's worst run bills 45% less than the baseline's best (S5: 39%) and less than the S1–S4 mean — Gate 2 passes per case, and the larger budget did not cost tokens (means 4.36M vs 4.24M, variance tighter). Agents per run 5–6 (was 14).
• Per agent, S5b: rule-hunter n=12, median 10 turns (inside the 8–10 budget), max 12; skeptic median 24 (max 32 — a new high, worth its own budget line later); critic 9.
• Scoped cases: no saving claimed (worst > baseline best); S5 does not move them.

Options

1. Merge as-is (S5b head): spend −45% worst-vs-best on the whole-repository review, recall at the baseline band and about 6 graders under S1–S4, with the misses named above.
2. Six hunters: split the two four-rule families (types → R1/R2 + R11/R12; tests and dependencies → R6/R7 + R8/R10). Two more agents per run (8 vs 14 baseline), each hunter carries at most 10 questions, and the spend gate should still clear by a wide margin. One more review-full ×3 ($15) proves it.
3. Stop iterating at four hunters and take the recall band as the price of the cut.

Traces: go-s5-158f75a-traces.tar.zst (2.3 MB) and go-s5b-12955b1-traces.tar.zst (1.6 MB), attached in the session.

Not in S5

One-hunter arm, bounded refactor workers, maxTurns on lint-fixer, quickfix case cap, S6 analyzer, Haiku for hunters.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Cc1QSJWHXUUib2t1CyQGk8

[Copilot]

Copilot review overview

🟡 Changes recommended

The unreadable-rule path can violate reconciliation and conceal incomplete family coverage.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1

Open (1)

• Define unreadable-rule reporting and require partial coverage · New

What changed in this PR

Reduces pre-commit review token usage by grouping rules into at most four family hunters while preserving per-rule reporting.

Changes:

• Adds family-based hunters, report caps, and per-rule receipts/tallies.
• Makes review commands skip tests and lint; analyze commands retain all gates.
• Regenerates all bindings and documents S5.

File	Description
core/​agents/​rule-hunter.md	Defines family-hunter behavior and output.
core/​skills/​pre-commit-review/​SKILL.md	Orchestrates hunters by rule family.
core/​skills/​pre-commit-review/​reference.md	Documents family hunting and reconciliation.
core/​commands/​{{.CmdPrefix}}-review.md	Makes review design-only.
core/​commands/​{{.CmdPrefix}}-analyze.md	Updates family-hunter wording.
docs/​token-budget.md	Records S5 design and results.
lang/​go/​passthrough/​README.md	Updates Go source documentation.
lang/​go/​passthrough/​CHANGELOG.md	Records Go binding changes.
lang/​python/​passthrough/​README.md	Updates Python source documentation.
lang/​python/​passthrough/​CHANGELOG.md	Records Python binding changes.
lang/​generic/​passthrough/​README.md	Updates generic source documentation.
lang/​generic/​passthrough/​CHANGELOG.md	Records generic binding changes.
go-linter-driven-development/​agents/​rule-hunter.md	Generated Go hunter instructions.
go-linter-driven-development/​skills/​pre-commit-review/​SKILL.md	Generated Go orchestration.
go-linter-driven-development/​skills/​pre-commit-review/​reference.md	Generated Go reference.
go-linter-driven-development/​commands/​go-ldd-review.md	Generated review-only command.
go-linter-driven-development/​commands/​go-ldd-analyze.md	Generated analyze command update.
go-linter-driven-development/​README.md	Documents Go family hunters.
go-linter-driven-development/​CHANGELOG.md	Records Go plugin changes.
python-linter-driven-development/​agents/​rule-hunter.md	Generated Python hunter instructions.
python-linter-driven-development/​skills/​pre-commit-review/​SKILL.md	Generated Python orchestration.
python-linter-driven-development/​skills/​pre-commit-review/​reference.md	Generated Python reference.
python-linter-driven-development/​commands/​py-ldd-review.md	Generated review-only command.
python-linter-driven-development/​commands/​py-ldd-analyze.md	Generated analyze command update.
python-linter-driven-development/​README.md	Documents Python family hunters.
python-linter-driven-development/​CHANGELOG.md	Records Python plugin changes.
linter-driven-development/​agents/​rule-hunter.md	Generated generic hunter instructions.
linter-driven-development/​skills/​pre-commit-review/​SKILL.md	Generated generic orchestration.
linter-driven-development/​skills/​pre-commit-review/​reference.md	Generated generic reference.
linter-driven-development/​commands/​ldd-review.md	Generated review-only command.
linter-driven-development/​commands/​ldd-analyze.md	Generated analyze command update.
linter-driven-development/​README.md	Documents generic family hunters.
linter-driven-development/​CHANGELOG.md	Records generic plugin changes.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.