Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/design/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -446,6 +446,14 @@ Excluded (MAIDEN_MARKERS, per nameparser/config/maiden_markers.py):
R1 gained the separator sentence with this change. That the invariant it states — one comma segment is one suffix entry — was written down in no rule is a large part of why two stages disagreed about it for years: C1 governs routing, R1 governed word ORDER and never mentioned separation, and the only statement of the rule was a code comment.
Left alone, measured and pre-existing: the NO-COMMA path comma-joins a space-separated run too (`John Smith MD PhD` renders 'MD, PhD' at 1.4.0, 2.0.0, 2.1.0 and master alike). The fix requires FAMILY_COMMA and does not reach it; that path has no writer-drawn boundary and is a separate question, recorded here and NOT yet filed. It is load-bearing for one thing: `str()` of a fixed parse is a no-comma string, so re-parsing it re-inserts the comma — round-tripping is not stable on these shapes, and an earlier draft of the release bullet claimed it was.
The name `Smith, MD PhD` entered the corpora as this fix's C1 example — a FIRING one, carrying no `· boundary` annotation, which marks the non-firing example a rule must have — so its ledger entries arrive with it. A fourth written for the 1.4.0 ledger EXPLAINED NOTHING and was dropped: `fix(comma-family) lone post-comma piece routes to suffix/title, not first` already claims the name there, its fields covering all three that move. That is #373's dormant-rule diagnosis doing its job, and #426 is the precedent for dropping rather than keeping a shadowed rule.
- 2026-08-25 #430 / #432 — one boundary, fixed as one: inside a comma segment a ONE-CHARACTER suffix word — the only kind a reader could take for an initial — is read by WHAT STANDS BEFORE IT. Behind another suffix it describes that suffix and continues the run, period included — `Smith, PSM I` and `Smith, PSM I.` are suffix 'PSM I'/'PSM I.' where they read given 'PSM' plus suffix 'I', an initial in that slot being no shape anyone writes. Behind a name word the period is decisive, marking an abbreviation and so name material — `Smith, John V.` is middle 'V.' where it read suffix 'V.', which is what `Smith, John B.` always did; the only thing that made V differ is that V is also suffix vocabulary. The bare form is the boundary and does not move: `Smith, John V` stays suffix 'V' (v1 parity, #144), which is why the test is the PERIOD and not the numeral. SCOPE, stated as the code reads rather than as the intent was argued: the class is one-character SUFFIX VOCABULARY, not "a roman numeral". It sweeps in `2` (`Smith, John 2.` is middle '2.'), misses `X` and `IX` (roman, but not vocabulary — `Smith, PSM X` is unchanged), and never touched `II`/`III`/`IV` at all, which are two characters and so were never initial-shaped and never vetoed. Anyone adding a one-character suffix in a script that has initials extends it. And the strict knob is consulted: under `lenient_comma_suffixes=False` the initial veto stands and the run ends at the numeral, which the first draft silently overrode by reading no policy at all.
#430's cause was rules.md#S2's initial veto in `is_suffix_piece`, which keeps a numeral out of a suffix reading — right after a name, wrong inside a credential run — so the segment did not look like a run at all and fell to the given-and-suffix walk. Two wrong answers came out of that one cause, and a fix verified on `Smith, PSM I` alone would have left the other: where the run's first word is TITLES vocabulary too (`Smith, MD I`, and `Smith, Jr. I` through the period-abbreviation inference) the leading-title peel took it instead, giving title 'MD' plus given 'I'.
THREE readers, one reading, which is the part worth copying. The gate that says a segment is a credential run, assign's router that assigns its roles, and group's one-entry join that renders it must agree PIECE FOR PIECE, and #429 shipped the inverse of its own fix by deriving that agreement twice. `segment_suffix_reading` returns the per-piece verdicts and all three read it (mechanisms.md#ONE-PREDICATE-PER-QUESTION); the intermediate state where only the gate and the router had been converted rendered `Smith, PSM I` as 'PSM, I', the render splitting a run whose roles were already right — the same tell as #429, one predicate along.
Declined, and the v1 suite caught it: asking `is_trailing_numeral_suffix` for #432's half. It answers the period question — `_ROMAN` is anchored and admits no period — but it also refuses a numeral standing behind an initial-shaped piece, which is a NO-COMMA rule and the opposite of this path's parity: `Chang, Andy C I` is first 'Andy', middle 'C', suffix 'I', and asking that predicate here made the numeral a middle. decisions.md's #401/#421 entry records the same fork declining to transfer to this walk under LENIENT. The half that applies is the period alone.
Blast radius, and the corpus IS blind to the moving classes: of the 1069 names on master, seven carry a one-character numeral behind a word inside a comma segment — `Chang, Andy C I`, `Doe, Rev. John V, Jr.`, `Maier, Amy I, Jr.`, `Maier, Amy Lauren I`, `Smith, John I`, `Smith, John V`, `Berg, abdul V` — every one of them BARE, and NONE moves. Zero pre-existing corpus names move at all; the only movers are the three this change adds as C1 examples. So the gate's 0-unexplained is evidence about the corpus, not about the change, and the case table is the whole coverage. `Berg, abdul V` cannot reach the walk at all: the bound-given join makes `abdul V` one piece, which is why the LENIENT mirror #401/#421 declined does not collide here. The moved names entered the corpora as C1's new examples, and carry ledger entries at all three baselines except #430's at 1.4.0, where `fix(comma-family) lone post-comma piece routes to suffix/title, not first` already claims the name and a second rule would be dormant.
Two things the /simplify round settled by measurement rather than argument. The walk asks `is_suffix_piece(pieces[m - 1])` — a VOCABULARY test — where the reading carries the verdict it just recorded, and unifying them looks obviously right until you run it: the walk assigns `PSM` to GIVEN in `Smith, PSM I.`, so a reading-style flag says "behind a name" and sends the numeral to middle, losing the strict parity restored two commits earlier. The two sites ask different questions on purpose, and the vocabulary answer is the load-bearing one. The cost is a reading the walk cannot reach: `Smith, John PSM I I.` moves from middle 'I' + suffix 'PSM, I.' to middle 'I I.' + suffix 'PSM', two consecutive one-character numerals behind a credential — accepted, both readings being defensible for a shape nobody writes, and recorded here rather than chased at the price of two correct pins.
The walk is also strict-blind, pre-existing and unchanged: it calls is_suffix_lenient unconditionally, so `Smith, John V` reads suffix 'V' under `lenient_comma_suffixes=False` as under the default. The reading consults the knob and the walk does not, which is now conspicuous where it was merely unexamined. Not this change's to fix.
Not done, and recorded rather than left implicit: neither half emits an ambiguity, though the no-comma `John Smith V` reports `suffix-or-name` for the same fork. The comma path has never reported it (decisions.md#C1 records PARTICLE_OR_GIVEN suppressed there for a related reason), and adding an emitter needs a trigger, a stage-ownership entry and a case row — more than this fix, and #432 raises it as a secondary observation rather than a requirement.

### T1 — separators, not joiners

Expand Down
2 changes: 1 addition & 1 deletion docs/design/mechanisms.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Problem shape. "Which stage does X?" — asked before attributing behavior in pr

## ONE-PREDICATE-PER-QUESTION — one predicate answers it, and every other site calls that

Problem shape. Two stages need the same answer about the same input, and the one that does not own the decision is about to test for it. Contract statement. Where two sites ask the same question, exactly one predicate answers it and every other site calls that one — never a condition written to match it. The predicate belongs to the QUESTION, not to whichever stage decides: it may sit in a leaf both stages import, and for the leading-title test it must, since the deciding stage is assign and group cannot import assign. How it works. A hand-written mirror agrees with its original only until one of them moves, and the drift is invisible in both directions: each site keeps passing its own tests while they disagree about an input neither covers. Five instances, every one found as a defect before it was found as a pattern — #319 lifted the wholly-suffix predicate into the vocabulary layer "so the comma decision and the honorific peel's segment test cannot drift apart"; #401/#421 lifted the trailing-numeral fork out of assign so the bound-given reserve stopped carrying a copy, its hand-written mirror having been falsified in review more than once — the lesson recorded there being that what must be mirrored is assign's WALK, not merely its condition; #425 replaced that reserve's hand re-derivation of the trailing peel with one function over the view the join would leave; #424 moved assign's leading-title test down because group's own `title()` does not see H2's unlisted abbreviations, so `Xyz. van Johnson` chained where `Dr. van Johnson` did not; #429 moved the no-name-segment test down because group asked by segment INDEX where assign asks by CONTENT. The destination follows the LAYER, not the topic: a predicate over token text goes to `_vocab`, one over pieces and tags to `_pieces`. Both are leaves the stages sit on. The piece layer got its own module only in #439 — until then those predicates collected in `_group`, not because grouping owned them but because `_assign` imports `_group` and cannot be imported back, so group was the one place both stages could reach; five had accumulated across four PRs before the module existed. Stage order is this mechanism's limit, and it forecloses the alternative: where the reader comes AFTER the decider, record the answer on the state instead — `ParseState.order` is that shape, "Recorded rather than recomputed downstream, because the two can differ" — which is unavailable whenever the EARLIER stage is the one asking. (The concrete assign→group import that forced the `_group` collection is gone since #439; what remains is the ordering it was a symptom of, and tests/v2/test_layering.py is where the leaf's contract is now written down.) The cost is a second evaluation of the same predicate, measured for #429 at 1.2–2.2% of a family-comma parse and 0% of every other; recording that number was the right answer there over plumbing a state field the two sites would not otherwise share. Lives in. nameparser/_pipeline/_vocab.py over text (is_wholly_suffix, and is_trailing_numeral_suffix — the #401/#421 instance, whose only caller since #439 is the shared peel rather than a stage) and nameparser/_pipeline/_pieces.py over pieces: is_suffix_piece, is_leading_title, leading_titles, peel_walk, peel_trailing and segment_holds_no_name are called by both stages, is_title_piece and trailing_start by group alone — `trailing_start` being the one to know, since it answers where the trailing run begins and is what P2's chain and M2's walk stop at. tests/v2/test_layering.py holds each module's contract, and a piece predicate growing a dependency on a STAGE shows up there as a widened entry. Reach for it when. You are about to write a condition that mirrors, matches or "does what X does" — or you find a comment saying one does. Grep for the other site's predicate and call it instead.
Problem shape. Two stages need the same answer about the same input, and the one that does not own the decision is about to test for it. Contract statement. Where two sites ask the same question, exactly one predicate answers it and every other site calls that one — never a condition written to match it. The predicate belongs to the QUESTION, not to whichever stage decides: it may sit in a leaf both stages import, and for the leading-title test it must, since the deciding stage is assign and group cannot import assign. How it works. A hand-written mirror agrees with its original only until one of them moves, and the drift is invisible in both directions: each site keeps passing its own tests while they disagree about an input neither covers. Five instances, every one found as a defect before it was found as a pattern — #319 lifted the wholly-suffix predicate into the vocabulary layer "so the comma decision and the honorific peel's segment test cannot drift apart"; #401/#421 lifted the trailing-numeral fork out of assign so the bound-given reserve stopped carrying a copy, its hand-written mirror having been falsified in review more than once — the lesson recorded there being that what must be mirrored is assign's WALK, not merely its condition; #425 replaced that reserve's hand re-derivation of the trailing peel with one function over the view the join would leave; #424 moved assign's leading-title test down because group's own `title()` does not see H2's unlisted abbreviations, so `Xyz. van Johnson` chained where `Dr. van Johnson` did not; #429 moved the no-name-segment test down because group asked by segment INDEX where assign asks by CONTENT. The destination follows the LAYER, not the topic: a predicate over token text goes to `_vocab`, one over pieces and tags to `_pieces`. Both are leaves the stages sit on. The piece layer got its own module only in #439 — until then those predicates collected in `_group`, not because grouping owned them but because `_assign` imports `_group` and cannot be imported back, so group was the one place both stages could reach; five had accumulated across four PRs before the module existed. Stage order is this mechanism's limit, and it forecloses the alternative: where the reader comes AFTER the decider, record the answer on the state instead — `ParseState.order` is that shape, "Recorded rather than recomputed downstream, because the two can differ" — which is unavailable whenever the EARLIER stage is the one asking. (The concrete assign→group import that forced the `_group` collection is gone since #439; what remains is the ordering it was a symptom of, and tests/v2/test_layering.py is where the leaf's contract is now written down.) The cost is a second evaluation of the same predicate, measured for #429 at 1.2–2.2% of a family-comma parse and 0% of every other; recording that number was the right answer there over plumbing a state field the two sites would not otherwise share. Lives in. nameparser/_pipeline/_vocab.py over text (is_wholly_suffix, and is_trailing_numeral_suffix — the #401/#421 instance, whose only caller since #439 is the shared peel rather than a stage) and nameparser/_pipeline/_pieces.py over pieces: is_suffix_piece, is_leading_title, leading_titles, peel_walk, peel_trailing and segment_suffix_reading are called by both stages — the last of those is #430's instance, where THREE readers share one answer, the render join being the third — while is_title_piece and trailing_start are called by group alone — `trailing_start` being the one to know, since it answers where the trailing run begins and is what P2's chain and M2's walk stop at. tests/v2/test_layering.py holds each module's contract, and a piece predicate growing a dependency on a STAGE shows up there as a widened entry. Reach for it when. You are about to write a condition that mirrors, matches or "does what X does" — or you find a comment saying one does. Grep for the other site's predicate and call it instead.

## CLAUSE-CONTENT-OVERRULES-DELIMITER — content wins

Expand Down
15 changes: 15 additions & 0 deletions docs/design/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -613,6 +613,15 @@ C1. Rationale: a credential run after the comma means the name is in
more than one name word keeps its positional read, order and
all. A name word in the part after the comma makes it the
given name, with titles before it and suffixes after.
A one-character suffix word — the only kind a reader could take
for an initial — is read by what stands before it. Behind
another suffix it is describing that suffix and continues the
run, written with a period or without, an initial being no shape
anyone writes there; behind a name word it is the generation
only when written bare, a period marking it the abbreviation of
a name and so a middle initial. Longer suffix words are not in
question either way, and the strict knob above still vetoes the
initial-shaped ones, so the run ends at them there.
"Smith, John" → family="Smith"
"سلمان، محمد" → family="سلمان"
"田中、太郎" → family=""
Expand All @@ -623,6 +632,12 @@ C1. Rationale: a credential run after the comma means the name is in
"Smith, PhD" → suffix="PhD"
"Smith, Jr." → suffix="Jr."
"Smith, Sr." → suffix="Sr."
"Smith, PSM I" → suffix="PSM I"
"Smith, PSM I." → suffix="PSM I."
"Smith, PSM I." strict-comma-suffixes → suffix="I."
"Smith, John V." → middle="V."
"Smith, John PhD I." → suffix="PhD, I."
"Smith, John V" → suffix="V" · boundary
"Smith, Ph. D. Jr." → suffix="Ph. D. Jr."
"Smith, MD PhD" → suffix="MD PhD"
"Smith, Dr." → title="Dr."
Expand Down
Loading