Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/design/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -793,6 +793,21 @@ R3's earlier history is under `decisions.md#R2`, which this entry does not repea
- 2026-08-29 — and therefore NOT stated in rules.md, which is a deliberate choice rather than an oversight. The document's examples are keyed on input STRINGS, so any statement of the property invites exactly the test that falsifies it — re-case the input, expect the same output — and the counterexamples are already in the corpora. The property is true of the repair given a parse, and rules.md speaks input-to-output; a rule stating it would be over-broad in the one direction a reader would check. What R5's statement says is enough for the promise that IS kept: a mixed-case name is kept unless repair was asked for anyway. That clause was REWORDED for this, and the reword is the whole point rather than a tidy-up. It read `unless repair regardless of how the name is cased was asked for`, which carries two readings -- the intended one, that the request overrides the keeping, and a second one, that the repair disregards the input's casing, which is this property in nearly this bullet's own words. A reader taking the second reading would run the re-casing test predicted above, land on `Velasquez y Garcia, Dr. Juan Q.` (in the corpus today), and conclude the RULE is wrong when only the phrasing was. Nine words, and they asserted the thing the paragraph exists to deny. The property is pinned in tests/test_capitalization.py instead, over names carrying no single-letter word whose class case decides, with `juan y garcia` beside it as the recorded exception. R5's example block gains `"SHIRLEY MACLAINE" → capitalized="Shirley MacLaine"` from this work, and it earns its place on its own ground rather than as half of a convergence pair: it is the only row in the block that fails when the gate is narrowed to lowercase-only, every other row passing that mutation. Measured three ways — gate deleted (passes, so it does not witness the gate's existence), gate narrowed to accept only all-lowercase (FAILS, and alone in the block), Mac/Mc convention deleted (fails, with the other two rows). Until it was added, R5 stated that repair acts on a name written entirely in one case and witnessed only the lowercase half of it. That lowercase half is still `"juan mcdonald"`, which is byte for byte an R4 row as well, and the duplication is deliberate rather than an editing slip: the two rules make different claims about the same line — R4 that the repair honors the Mac/Mc convention, R5 that an all-lowercase name is acted on at all — and dropping it from R5 would leave the gate's lowercase half unwitnessed inside the rule that states the gate. Five other rows already sit under two rules apiece for the same reason (P5/P6 twice, P5/O5, N3/M4, W1/W3).
- 2026-08-29 — DEBT this extraction leaves, named so the next commit inherits an obligation rather than a rediscovery. Pulling the gate out into R5 leaves R4 carrying ONE falsehood and ONE ambiguity — different defects wanting different repairs, and `interacts: R5` carries neither, the field being advisory. FALSE: R4 promises repair "vocabulary exceptions (McDonald) included", but `str(parse('Juan Mcdonald').capitalized())` is `'Juan Mcdonald'` — the gate refuses before any vocabulary is consulted, and only `str(parse('Juan Mcdonald').capitalized(force=True))`, `'Juan McDonald'`, reaches the exception. R4 needs its promise scoped to names the gate admits. AMBIGUOUS, not false: R4's "an already-correct name comes back unchanged" means correct by the repair's own conventions, i.e. idempotence, and under that meaning it is true; a reader hears correct as the bearer writes it, and under THAT meaning `str(parse('bell hooks').capitalized())` — `'Bell Hooks'` — looks like a counterexample. It is not one, because `bell hooks` is not already-correct in R4's sense. What R4 owes is a disambiguation of "correct", NOT a narrowing to spare deliberately single-cased names: that would be new behavior, and R5's own rationale declines it on the ground that single case leaves the repair nothing to read. Also for that commit, and inert as things stand: R4's boundary row `"Juan McDonald" → capitalized="Juan McDonald"` passes with R5's gate deleted, exactly like the R5 row that was withdrawn above; rewriting it to `capitalized_forced=` makes it discriminate for R4's own subject but still witnesses nothing about the already-correct question. This commit adds R5 and touches R4 only on its pointer line, leaving both defects as found rather than half-fixed by a commit whose subject is something else.

### parse-cost — what a parse is allowed to cost

Every number below is a py3.11 measurement of 2026-08-31, recomputable with `uv run python tools/perf/call_count.py` (add `--modules`, `--stages`, or `--against <ref>`). That harness is in the tree BECAUSE of this entry: its first draft took figures across several sessions with throwaway scripts on whichever interpreter was to hand, and published a module table that spliced one interpreter's before-column onto another's after-column. Four of its eight module numbers were wrong, its four stage numbers were 2x (2000-parse totals labelled per 1000, which its own arithmetic contradicted — four stages summing past the whole parse), and it blamed the wrong PR. Quote nothing here without the interpreter beside it.

- 2026-08-31 #475 — the benchmark bounds FUNCTION CALLS per parse, not seconds, as a per-interpreter BAND of ±2%. A loose 5s wall-clock backstop stays over BOTH entry points for what frame counts cannot see: a compiled regex that starts backtracking emits no `call` event, and neither does C-level work. (A quadratic inside a comprehension or generator IS visible — `call` fires once per generator resume.)
WHY THE CLOCK HAD TO GO. The 1.0s bound failed four times across #466 and #474, always on CI's 3.12 job, always between 1.01s and 1.08s, with master re-running green each time. Three local methods disagreed with CI and with each other: uninstrumented ~1% branch-over-parent (inside the parent's own spread), coverage-instrumented indistinguishable, and the whole benchmark file under `--cov` made the branch FASTER. The same harness measured `origin/master` at 94-96ms and again at 89-92ms twenty minutes later. A check that cannot separate a 1% change from a busy runner does not fail safe; it fails expensively.
WHY A BAND AND NOT A CEILING, which is the correction the review forced. A ceiling with headroom is ANOTHER threshold that happens to break — the failure this replaced. The first draft set 470 against a 408 baseline, and measured, that catches nothing smaller than +61 calls on 3.11 and +84 on 3.12: it would have missed the very regression it was built for on four of five interpreters. The cycle's +67 arrived across a dozen PRs at roughly five calls each, and no ceiling loose enough to be safe can see five. A ±2% band trips at +5, verified by injection. A DROP trips it too, which is intended: an unexplained fall is as much a signal as a rise.
PER INTERPRETER, because the count is not machine-independent — 410 on 3.11, 388 on 3.12, 406 on 3.13-3.15 for `parse`. PEP 709 inlined the comprehension frames 3.11 counts and 3.13 added others back. The first draft claimed "the same number on any machine" and took its baseline on 3.11 by luck: had it been taken on 3.12, the 3.11 job would have gone red with nothing in the file explaining why. A version with no row fails loudly with its own number rather than passing unguarded.
WHAT THE OLD BOUND HAD CAUGHT, which is the substantive half. Measured `--against v2.1.0`: `parse` 343 → 410 and `HumanName` 380 → 447, **+67 calls per parse**; by stage, per 1000 parses, `group` 13.5 → 22.2ms, `classify` 11.6 → 13.5, `assign` 11.1 → 13.0, `post_rules` 5.2 → 8.0. The bound was not too tight. The parser had grown into it, and the clock could not say so with enough confidence to act on.
WHERE THE CALLS WENT, per parse, v2.1.0 → here: `_group.py` 94 → 84, `_pieces.py` 0 → 51, `_post_rules.py` 16 → 33, `_vocab.py` 26 → 32, `_assign.py` 21 → 18, `_classify.py` 14 → 16.
AND WHICH PR SPENT THEM, which the first draft got backwards. It named #439's predicate extraction as "the largest contributor" and built a trade-off on it. Measured across its own merge boundary, #439 cost **zero**: 404 → 404, `_group` shedding 51 exactly as `_pieces` gained them. A pure relocation, and `mechanisms.md#ONE-PREDICATE-PER-QUESTION` says as much — the sharing between `group` and `assign` predated the module. The real largest single contributor is **#424 at +23** (381 → 404), the PR that made group's chain and walk mirror assign's; then #404 at +12, with #361, #367 and #434 at +8 each. So the entry's old conclusion — that recovering the cost means reintroducing the drift #439 removed — deterred work on a PR that spent nothing while +23 went unexamined. Recovering it, if ever wanted, starts at #424's peel walks.
ONE REPAIR WAS TRIED AND REVERTED: collapsing `post_rules`' three separate role-index scans into one pass. It saves **6 calls per parse** on 3.11 (the first draft said one, which is not even reachable — `_idx` is a one-line list comprehension, so each removed call frees two frames). Reverted because it was measured against a noisy timing harness that showed nothing; at 6 calls it is 9% of the cycle's growth and worth reconsidering against the band, which can now see it.
RAISING OR LOWERING A ROW IS A DECISION, not a maintenance chore. Append here with the interpreter and the harness invocation, as above.


### removed-v1-surface

- empty_attribute_default: removed in 2.0 (#255; deprecated in 1.4 per the bridge discipline). Origin #44 (2016): a DB-NULL convenience whose first answer — `name.title or None` — became the migration path. The in-band-signaling bug that sealed it (#254): the 2016 `.replace('None','')` scrub could not tell interpolated None from name text, so "Nonez Smith" rendered
Expand Down
Loading