From e836b4cdfb70715278c42a762fb13bd870b940c8 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 30 Aug 2026 22:10:07 -0700 Subject: [PATCH 1/2] fix(group): a suffix never begins a name (#371) `parse("Ph. D. Van Johnson")` read given 'Van Johnson' with an empty family and suffix 'Ph. D.'. It now reads title 'Ph.', given 'D.', family 'Van Johnson'. Position outranks the vocabulary match. The v1 fix_phd merge -- which heals a split `Ph.`/`D.` into one credential -- was applied wherever the pair stood, and that is what made a leading credential possible at all: every other suffix-shaped word standing first already falls out as a title (Jr., MD, Esq., Sr., via TITLES or H2's abbreviation clause) or as an ordinary name word (PhD, III). The pair was the only shape that reached the defect. RESTORES 1.4.0 rather than deviating from it. v1 matched the pair by regex on the raw string and healed it only where it TRAILED; leading, it split them exactly as this does -- measured on the released wheel, all four corpus names of the shape return to their v1 reading. So expected_since_1.4.0.toml LOSES its fix(leading-credential) rule (the gate said EXPLAINED NOTHING, which is how a parity restoration announces itself) while 2.0.0 and 2.1.0 GAIN one claiming the same four names, five roles and digest. What shipped in 2.0 was a regression from applying the merge unconditionally. Opening the NAME is not opening a piece list: a credential run legitimately opens segment 1 after a family comma ("Smith, Ph. D. Jr."), which C1 reads as a listing. A first draft tested `k == 0` alone, broke fifteen tests and took the gate red at three baselines. Accepted: Parser.revise(suffix="Ph. D.") now renders 'Ph., D.'. revise() sub-parses the string it is given and a field value has no head. A draft carved that out by requiring a name to displace, and the carve-out made the head reading depend on what FOLLOWED -- appending a maiden clause changed whether 'Ph.' was a title, which the maiden property test caught. Dropping the carve-out removed both problems. rules.md#S2 states the rule; three mutations verified; gate green at all three baselines. --- docs/design/decisions.md | 6 ++++ docs/design/rules.md | 12 +++++++ docs/release_log.rst | 1 + nameparser/_pipeline/_group.py | 20 ++++++++++-- tests/v2/cases.py | 3 +- tests/v2/pipeline/test_group.py | 19 +++++++++-- tests/v2/test_ledger_guards.py | 11 +++++-- tests/v2/test_parser.py | 10 ++++-- tools/differential/corpus_rules.jsonl | 2 ++ tools/differential/expected_since_1.4.0.toml | 33 ++++++-------------- tools/differential/expected_since_2.0.0.toml | 28 +++++++++++++++++ tools/differential/expected_since_2.1.0.toml | 28 +++++++++++++++++ 12 files changed, 140 insertions(+), 33 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 86dcbcef..e9d07a99 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -305,6 +305,12 @@ Declined (rc1 arc; the full argument is AGENTS.md's gotcha): - 2026-07 (v2 core, PR #288; recorded plan deviation #1 of the core plan) — "Ph. D." tokenizes as two words and is merged back by vocabulary (v1 fix_phd), so the spaced and unspaced spellings read alike. +- 2026-08-31 #371 (the merge declines at the head) — a suffix never BEGINS a name: position outranks the vocabulary match, and `rules.md#S2` now states it. The merge is what made a leading credential possible at all — every other suffix-shaped word standing first already falls out as a title (H2's abbreviation clause, or TITLES membership: `Jr.`, `MD`, `Esq.`, `Sr.`) or as a name word (`PhD`, `III`), so the `Ph.`/`D.` pair was the only shape that reached the defect. It reached it by emptying the family: `Ph. D. Van Johnson` read given 'Van Johnson' with no surname at all, which is #371's report. + THIS RESTORES 1.4.0 RATHER THAN DEVIATING FROM IT, and the ledgers say so in both directions. v1's fix_phd healed the pair by regex on the raw string and matched it only where it TRAILED; leading, v1 split it — measured on the released wheel, `Ph. D. Van Johnson` gives title 'Ph.', first 'D.', last 'Van Johnson', which is exactly what this change gives. So `expected_since_1.4.0.toml` LOSES its fix(leading-credential) rule (the gate reported EXPLAINED NOTHING, which is how a parity restoration announces itself there) and the 2.0.0 and 2.1.0 ledgers GAIN one, claiming the same four names, the same five roles and the same digest. What shipped in 2.0 was a regression from applying the merge unconditionally, not a reading anyone chose. + OPENING THE NAME IS NOT OPENING A PIECE LIST, and the first draft got that wrong: a credential run legitimately opens segment 1 after a family comma (`Smith, Ph. D. Jr.`), which C1 reads as a listing rather than a name beginning. Testing `k == 0` alone broke fifteen tests and took the gate red at all three baselines. The caller passes the distinction in, since `_group_segment` cannot see which segment it holds. + ACCEPTED: `Parser.revise(suffix="Ph. D.")` now renders 'Ph., D.'. revise() runs a full sub-parse of the string it is given, and a field value has no head for a head-position rule to consult; a second draft carved that out by requiring a name to displace, and the carve-out made the head reading depend on what FOLLOWED it — appending a maiden clause changed whether 'Ph.' was a title, which `test_a_maiden_clause_changes_nothing_else` caught. Dropping the carve-out removed the inconsistency with it. The merge exists for a credential someone TYPED after a name; a caller who writes the spaced form into the suffix field is taken at their word (Derek's call, 2026-08-31). + What the leading `Ph.` becomes is H2's business, not this rule's: an abbreviation before a name is almost always a title, which is the same clause that reads `Esq. van Gogh` as title 'Esq.' — so the pair reads title 'Ph.', given 'D.'. The issue proposed given 'Ph.', middle 'D.'; H2 claims the first word before the positional read sees it, and the two answers differ only in which field holds `Ph.`. + ### O4 — positional assignment and declared order - 2026-08-07 #83 — romanized Chinese order is answered by DECLARATION, not detection: pinyin carries no signal and diaspora makes locale no guide (the issue's own 2019/2021 conclusions); Policy(name_order=FAMILY_FIRST) is the answer the thread wanted. diff --git a/docs/design/rules.md b/docs/design/rules.md index c4a67f1c..1cc2b89d 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -515,6 +515,14 @@ S1. Rationale: brackets set off more than nicknames — credentials S2. Rationale: generational suffixes and credentials are recognized by vocabulary; an acronym that is also an ordinary name is only unmistakably a credential when its periods are written. + A suffix never BEGINS a name: position outranks the vocabulary + match, so a suffix-shaped word opening a name reads as whatever + its position and shape make it — a title where it wears the + abbreviation shape (H2), an ordinary name word otherwise — and + never as a credential belonging to a name that has not been + written yet. A family comma changes what "begins" means rather + than the rule: the comma has already named the family, so the + part after it is a listing and a credential run may open it (C1). A trailing word of the suffix vocabulary reads as a suffix — generational forms and credential acronyms alike, and an ambiguous acronym written with its periods, one after each @@ -528,6 +536,10 @@ S2. Rationale: generational suffixes and credentials are recognized "John Smith PhD" → suffix="PhD" "John Ma" → family="Ma" · boundary "Jack Ma." → family="Ma." · boundary + "Ph. D. Van Johnson" → family="Van Johnson" + "Ph. D. Van Johnson" → title="Ph." + "Smith, Ph. D. Jr." → suffix="Ph. D. Jr." + "II Van Johnson" → given="II" · boundary Accepted: with words to spare, a bare ambiguous acronym reads as a suffix even beside an East Asian surname it more likely belongs to; and an unambiguous suffix is consumed even when diff --git a/docs/release_log.rst b/docs/release_log.rst index f6222fdb..d45a72ce 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -71,6 +71,7 @@ Release Log - Change case repair to read the parser's own ``conjunction`` tag instead of re-deciding, from the word's spelling, whether a word is a conjunction or an initial. The parse answers that question already -- ``"Scott E. Werner"`` reads ``E.`` as an initial rather than the Italian conjunction -- and the other views honor the answer; case repair asked again, with a shape test applied to each word of a token's text rather than to the token. Two spellings of one name disagreed because of it: ``"juan e-f smith"`` capitalized to ``Juan e-F Smith`` while ``"JUAN E-F SMITH"`` gave ``Juan E-F Smith``; both give ``Juan E-F Smith`` now, ``e-f`` being a middle name and no conjunction of the parse's reading. A conjunction written as a word of its own is untouched, and so is the one-letter carve-out where it applies -- ``"juan y garcia"`` still repairs to ``Juan y Garcia``, ``"JUAN Y GARCIA"`` still to ``Juan Y Garcia``. A field assigned after the parse is unaffected: its text was never classified, so there is no reading to honor and repair asks the vocabulary, applying v1's own predicate the way every earlier version applied it everywhere -- ``h.last = "velasquez y garcia"`` still repairs to ``Velasquez y Garcia`` and ``h.middle = "e."`` to ``E.``. That is the predicate over TODAY's vocabulary, which is narrower than parity with 1.4.0 and the difference is real: ``h.last = "хосе и мария сантос"`` gives ``Хосе И Мария Сантос`` on 1.4.0 and ``Хосе и Мария Сантос`` here, because the Cyrillic ``и`` is a 2.x conjunction and was not a 1.4.0 one. What decides which path a token takes is a mark the assignment leaves, not the absence of a span: a value revised through ``Parser.revise()`` is classified by a sub-parse and keeps its tags, so it repairs as the parse does. One reading does change for hand-built ``Token``\ s in the 2.0 API: an untagged token whose text is conjunction vocabulary is now an ordinary name word and capitalizes, where 2.1 lowercased it -- tags are what the views read, and a hand-built token that carries none is a token with nothing to declare. Case repair is not one of the seven role fields the differential harness compares, so no gate run can see this change either way and none of its counts move; measured directly instead, no name of the 1094-name differential corpus moves under ``capitalized()`` or ``capitalized(force=True)``, its uppercased and lowercased spellings included -- 6564 name/spelling/lexicon rows and 13128 calls (closes #458) + - Fix a name that opens with a spaced ``Ph. D.`` losing its surname. ``parse("Ph. D. Van Johnson")`` read given ``Van Johnson`` with an empty ``family`` and suffix ``Ph. D.``; it now reads title ``Ph.``, given ``D.``, family ``Van Johnson``. A suffix never begins a name -- position outranks the vocabulary match -- and the merge that heals a split ``Ph.``/``D.`` into one credential is what made a leading credential possible at all: every other suffix-shaped word standing first already falls out as a title (``Jr.``, ``MD``, ``Esq.``, ``Sr.``) or as an ordinary name word (``PhD``, ``III``), so this pair was the only shape that reached the defect. The merge is unchanged everywhere else, and a family comma still opens a listing rather than a name, so ``"John Smith Ph. D."`` keeps suffix ``Ph. D.`` and ``"Smith, Ph. D. Jr."`` keeps suffix ``Ph. D. Jr.``. This RESTORES 1.4.0, which healed the pair only where it TRAILED and split it at the head exactly as this release now does -- measured on the released wheel, all four corpus names of this shape return to their v1 reading, and the 1.4.0 ledger loses the rule that used to excuse the difference. One accepted consequence: ``Parser.revise(suffix="Ph. D.")`` renders ``Ph., D.``, since revise() sub-parses the string it is given and a field value has no head for a head-position rule to read (closes #371) - Fix a trailing surname particle being stranded as a standalone middle name under ``Policy(name_order=FAMILY_FIRST)``, where the same listing written with a comma reads it as part of the surname. A particle ending the name has nothing to link forward to, so what it is doing there is decided by what the writing says: after a family comma it joins the family the comma named and is written before it, and a declared family-first order names the family the same way. ``Parser(policy=Policy(name_order=FAMILY_FIRST)).parse("Jong Anke de")`` gave family ``Jong`` with ``de`` left as a middle name, and now gives family ``de Jong``, given ``Anke`` -- the same answer ``parse("Jong, Anke de")`` has always given. The test is the SLOT the particle landed in, not the word: a middle name is a further given name, which a particle is not, and ``FAMILY_FIRST`` is the only order that puts a trailing piece there. ``FAMILY_FIRST_GIVEN_LAST`` puts it in the given slot, where the caller's own declaration says it is the given name, so ``"Nguyen Thi Van"`` under that order still reads given ``Van``. That one test reads both traditions without asking about the vocabulary at all: ``"Beethoven Ludwig van"`` under ``FAMILY_FIRST`` now gives family ``van Beethoven`` even though ``van`` is one of the 37 particles that are ordinary given names elsewhere. In the same change, a particle standing alone where a family-first order puts the GIVEN name is no longer folded into the family: ``"Ménil de"`` reports given ``de`` under both family-first orders, because that slot holds what the caller declared, and the never-given word list supplies a reading where position leaves the question open rather than overriding one position has already given. Nothing moves under the DEFAULT name order. Measured, 30 of 6606 parses move -- this release's 1101-name corpus under three ``name_order`` values with ``middle_as_family`` off and on -- over twelve names. Those counts cannot come from the differential gate: it parses every corpus name with the default policy and sweeps no policy at all, so no non-default ``name_order`` behavior has ever been compared across versions, and the gate output here is unchanged at all three baselines apart from the corpus names this change's own rules.md examples add. What moves is recomputed by the recipe in the ``P6`` entry of ``docs/design/decisions.md``, which compares the seven role fields against a checkout of the parent commit reading the same corpus files; the count is a dated snapshot, measured 2026-08-30. The ``rules.md#P1`` and ``rules.md#P6`` example lines and ``tests/v2/pipeline/test_post_rules.py`` are what pin the behavior (closes #467) - Fix ``initials()`` reading a name in a different order than the fields of the same name. Two rules fold words into the family name and render them before the rest of it -- ``Policy(middle_as_family=True)``, which sends every middle word to the family, and the tussenvoegsel attachment after a family comma -- and both do it by marking the words rather than moving them, since a parsed word keeps the position it was written at. The ``family`` field reads that mark and ``initials()`` did not, so one parse gave two orders: ``parse("der, y van")`` gave family ``van der`` and initials ``y. d. v.``, and now gives ``y. v. d.``. This RESTORES v1: ``middle_name_as_last`` is v1's spelling of the same option, so most of what moves has a 1.4.0 answer to be measured against, and measured over the 1094-name differential corpus at the default name order, 71 names move under that option, 54 of them back to exactly what 1.4.0 returns and not one of them away from it -- ``"Doe, Dr. John A."`` gives ``J. A. D.`` again where 2.0 through 2.2 gave ``J. D. A.``, and ``"Brundridge, Contessa A"`` gives ``C. A. B.`` where they gave ``C. B. A.``. Of the 17 that match 1.4.0 neither before nor after, 14 now agree with it on the ORDER and differ only in how v1 grouped initials -- 1.4.0 gives one initial per element of its own ``last_list``, so a conjunction-joined surname yields ``V G.`` where the 2.x view, one initial per word, yields ``V. G.``; one more is ``"der, y van"``, whose family is nothing but particles, where 1.4.0 contributes no initial at all and 2.x contributes its words, a difference this release does not touch; and the remaining two parse differently from 1.4.0 for reasons that predate this fix. Without the option, one corpus name moves, the ``"der, y van"`` above -- and it is one of one, since it is the only name in the corpus whose family holds two contributing words with a folded one behind the other, the shape an order change can be seen in at all. ``HumanName.initials()`` was already right and is unchanged, measured: no name of the 1094-name corpus moves through the facade, with the option or without it. It reads ``first_list``/``middle_list``/``last_list``, which prepend the folded words as v1 did, so it is the 2.0 API's ``ParsedName.initials()`` that was out of step -- with the field beside it, with the facade, and with 1.4.0 at once. ``initials()`` is not one of the seven role fields the differential harness compares, so no gate run can see this change: run at all three baselines before and after, the output is identical to the byte -- 1094 corpus names, 229 / 194 / 102 intentional diffs and ``unexplained: 0`` at 1.4.0 / 2.0.0 / 2.1.0, with every per-heading count unchanged. So where the counts in this bullet come from has to be said, the gate's classified summary not being able to supply them and the ``rules.md#R3`` example line witnessing the order without counting anything: what MOVES is recomputed by the recipe in the ``R3`` entry of ``docs/design/decisions.md``, which compares this view against the pre-change rendering over these same four corpora and reproduces the 71 and the one-of-one; the 1.4.0 comparisons and the facade sweep are dated snapshots rather than re-derivable ones, measured 2026-08-30 against the released 1.4.0 wheel and against the pre-change tree, which nothing in the repository re-runs. The ``rules.md#R3`` example line and ``tests/v2/test_render.py`` are what pin the behavior (closes #408) diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index 552f2bbc..59d81a45 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -297,6 +297,7 @@ def _group_segment(seg: tuple[int, ...], additional: int, ambiguities: list[PendingAmbiguity] | None = None, cores: Set[str] = frozenset(), given_name_titles: Set[str] = frozenset(), + opens_the_name: bool = False, ) -> tuple[list[Piece], list[set[str]], MaidenTake | None]: pieces: list[Piece] = [[i] for i in seg] ptags: list[set[str]] = [set() for _ in seg] @@ -367,10 +368,24 @@ def merge(lo: int, hi: int, add: Set[str] = frozenset(), # ph-d merge first: "Ph." "D." adjacent -> one suffix piece # (decisions.md#phd-merge; v1 fix_phd did this by regex on the # raw string) + # A suffix never BEGINS a name: position outranks the vocabulary + # match (#371). This merge is what makes a leading "Ph." "D." a + # credential at all -- every other suffix-shaped word standing + # first already falls out as a title (H2's abbreviation clause, or + # TITLES membership) or as a name word, so the pair is the only + # shape that reaches the defect, and it reached it by emptying the + # family: "Ph. D. Van Johnson" read given 'Van Johnson' with no + # surname at all. + # + # OPENING THE NAME is not "opening a piece list". A credential run + # legitimately opens segment 1 after a family comma ("Smith, + # Ph. D. Jr."), which is C1's listing form rather than a name + # beginning, so the caller passes the distinction in. k = 0 while k < len(pieces) - 1: a, b = pieces[k], pieces[k + 1] - if (len(a) == 1 and len(b) == 1 + if (not (opens_the_name and k == 0) + and len(a) == 1 and len(b) == 1 and PH.fullmatch(tokens[a[0]].text) and D.fullmatch(tokens[b[0]].text)): merge(k, k + 2, add={"suffix"}) @@ -750,7 +765,8 @@ def group(state: ParseState) -> ParseState: seg, additional, tokens, bound_join, None if family_comma else ambiguities, seg_cores, - state.lexicon.given_name_titles) + state.lexicon.given_name_titles, + opens_the_name=(seg_idx == 0 and not family_comma)) # the marker is dropped and the maiden name's tokens become # MAIDEN (#274); which pieces those are was settled in # _group_segment, before the joins diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 3caf5425..ff761b74 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -2044,7 +2044,8 @@ def __post_init__(self) -> None: {"title": "Dr.", "given": "John", "family": "Smith", "suffix": "Ph. D."}), Case("phd_split_leading", "Ph. D. John Smith", - {"given": "John", "family": "Smith", "suffix": "Ph. D."}, + {"title": "Ph.", "given": "D.", "middle": "John", + "family": "Smith"}, classification="fix", notes="v1 healed 'Ph.'+'D.' only when trailing; leading it " "split them (title 'Ph.', given 'D.', real given name " diff --git a/tests/v2/pipeline/test_group.py b/tests/v2/pipeline/test_group.py index 1152bd69..aee79d74 100644 --- a/tests/v2/pipeline/test_group.py +++ b/tests/v2/pipeline/test_group.py @@ -122,11 +122,26 @@ def test_a_suffix_shaped_leading_piece_is_not_stepped_over() -> None: # become three -- which puts Van in the middle name rather than the # family once roles exist. See _group.py for why that reading is # worse rather than merely different. - assert _piece_texts(_grouped("Ph. D. Van Johnson")) == \ - [["Ph. D.", "Van Johnson"]] + # + # "Ph. D. Van Johnson" left this test with #371: the pair is no + # longer merged at the head, so there is no suffix-shaped leading + # piece there to step over or not. The scan's decision is the same + # and "II Van Johnson" still witnesses it -- a numeral needs no + # merge to be suffix vocabulary. assert _piece_texts(_grouped("II Van Johnson")) == [["II", "Van Johnson"]] +def test_the_phd_merge_declines_at_the_head_of_a_name() -> None: + # A suffix never begins a name (S2, and #371). The merge is what + # made a leading "Ph." "D." a credential at all, and it emptied the + # family doing it. Segment 0 only: after a family comma the run + # legitimately opens its segment, which C1 reads as a listing. + assert _piece_texts(_grouped("Ph. D. Van Johnson")) == \ + [["Ph.", "D.", "Van Johnson"]] + assert _piece_texts(_grouped("Smith, Ph. D. Jr.")) == \ + [["Smith"], ["Ph. D.", "Jr."]] + + def test_von_und_zu_bridges() -> None: # conjunction "und" joins two prefixes; the joined piece is a derived # prefix and still chains onto the following name (v1 PR #191) diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 14ffbf2f..295bb567 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -1068,6 +1068,11 @@ class _LatinCopy(NamedTuple): # begins, and the alternation is over ANCHORS, not over words -- # there is no vocabulary here to drift from. frozenset({"^", ",\\s*"}), + # fix(#371)'s alternation is the three corpus names that carry a + # leading 'Ph. D.' -- a list of names, not a copy of any wordlist. + # The vocabulary the rule turns on is `ph`/`d`, which the regex + # spells out in the anchored stem rather than in the alternation. + frozenset({" John Smith", " Van Johnson", ", Jr\\."}), # fix(#445)'s movers, one corpus name per alternative -- a list of # names, not a copy of any wordlist, so there is no vocabulary for # it to drift from. Two sets because the ledgers group the nine @@ -1572,8 +1577,6 @@ def _claim(rule: dict) -> _Claim: _Claim(19, ('family', 'given', 'middle', 'suffix'), "aa475ddd4745"), "feat(#269) non-Latin titles/conjunctions recognized": _Claim(4, ('given', 'middle', 'title'), "e86eeb13eeb2"), - "fix(leading-credential) a split 'Ph. D.' before the name stays one unit": - _Claim(4, ('family', 'given', 'middle', 'suffix', 'title'), "1425d85a2d86"), "fix(#424) an unlisted abbreviation is as transparent as a listed title to the leading particle": _Claim(1, ('family', 'given'), "ca7b37af6cf8"), "fix(#367) a title no longer displaces a leading particle out of the leading position": @@ -1763,8 +1766,12 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('_ambiguities', 'family', 'given'), "e62caedec864"), "fix(#399) a maiden marker bounds the particle chain: the geb. spelling": _Claim(1, ('family', 'maiden'), "2150936a8c55"), + "fix(#371) a suffix never begins a name: the Ph./D. merge declines at the head": + _Claim(4, ('family', 'given', 'middle', 'suffix', 'title'), "1425d85a2d86"), }, "expected_since_2.1.0.toml": { + "fix(#371) a suffix never begins a name: the Ph./D. merge declines at the head": + _Claim(4, ('family', 'given', 'middle', 'suffix', 'title'), "1425d85a2d86"), "fix(#335) a marker-led clause leaves the one name word its bare reading": _Claim(1, ('maiden', 'nickname'), "c09cc7dba88b"), "fix(#434) a multi-word maiden marker takes the maiden name": diff --git a/tests/v2/test_parser.py b/tests/v2/test_parser.py index 14a9e0cf..9c344345 100644 --- a/tests/v2/test_parser.py +++ b/tests/v2/test_parser.py @@ -823,11 +823,17 @@ def test_revise_preserves_particle_tags() -> None: assert r.initials() == "J. V. S." # particles contribute no initial -def test_revise_keeps_multiword_suffix_one_credential() -> None: +def test_revise_takes_a_spaced_credential_literally() -> None: + # The Ph./D. merge is a HEAD-POSITION rule (#371) and a field value + # has no head: revise() runs a full sub-parse of the string it is + # given, so "Ph. D." there is two initials, not one credential. + # Deliberate -- the merge exists for a credential someone TYPED + # after a name, and a caller who writes the spaced form into the + # suffix field is taken at their word. p = Parser() n = p.parse("John Smith Ph.D.") r = p.revise(n, suffix="Ph. D.") - assert r.suffix == "Ph. D." # replace() would render "Ph., D." + assert r.suffix == "Ph., D." def test_revise_views_match_a_fresh_parse() -> None: diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index e2de31d0..a1fb9546 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -44,6 +44,7 @@ "Hans „Erster“ und “Zweiter” Müller" "Hassan Mohamad Ali" "Hassan, Mohamad Ahmad Ali" +"II Van Johnson" "J. Smith" "J. née Jones Smith V" "J.R. Smith" @@ -136,6 +137,7 @@ "Nguyen, Van Le" "Nguyễn Thị Minh Khai" "Nguyễn, Thị Vân" +"Ph. D. Van Johnson" "Rev. John Smith" "SHIRLEY MACLAINE" "Salam, abd Allah" diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index 0188ec86..7c0c71b9 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -1138,30 +1138,15 @@ name_regex = "[\\u0400-\\u04FF]" fields = ["title", "given", "middle"] -[[change]] -issue = "fix(leading-credential) a split 'Ph. D.' before the name stays one unit" -# 'Ph. D. John Smith'. v1 healed the adjacent 'Ph.'/'D.' pair only when -# it TRAILED; leading, it split them -- title 'Ph.', given 'D.', and -# the real given name pushed to middle. 2.0 keeps the credential whole -# and routes it to `suffix`, which is the post-nominal field, so a -# leading credential lands there rather than in `title`. Slightly odd -# positionally, and still strictly better than manufacturing a given -# name out of 'D.'. -# -# Anchored to the START on purpose. Trailing 'Ph. D.' is PARITY and -# stays unclassified below; widening this regex would mask a -# regression in the shape that is the whole reason fix_phd exists. -# -# `family` joined the roles when #413's harvest brought three names -# into this rule's reach, two of which move it: a bare 'Ph. D.' (1.4 -# read family 'D.', now suffix) and 'Ph. D. Van Johnson' (1.4 read -# family 'Van Johnson', now given). Both move `family` for the same reason the other roles -# move -- the credential stops being split into a title and a given -# name, and everything behind it shifts one place. The REGEX is -# untouched; only the roles the same shape was always going to move -# are now recorded. -name_regex = "^Ph\\. ?D\\." -fields = ["title", "given", "middle", "family", "suffix"] +# The fix(leading-credential) rule that stood here is GONE, and its +# absence is the record: it explained 2.0 routing a leading split +# 'Ph. D.' to `suffix` where v1 read title 'Ph.' and given 'D.'. #371 +# restored v1 on all four names of that shape, so the rule explained +# nothing and the gate said so (EXPLAINED NOTHING is an error, which +# is how a parity restoration announces itself here). What v1 did -- +# heal the pair only when it TRAILS -- is what the parser does again; +# rules.md#S2 now states it as a rule rather than a regex accident. + # Deliberately NOT a [[change]] rule: TRAILING 'Ph. D.' split-token # healing ('John Ph. D.', 'John Smith, Ph. D.') is PARITY, not a 2.0 diff --git a/tools/differential/expected_since_2.0.0.toml b/tools/differential/expected_since_2.0.0.toml index 2d1bc78b..e8cffd2d 100644 --- a/tools/differential/expected_since_2.0.0.toml +++ b/tools/differential/expected_since_2.0.0.toml @@ -1240,3 +1240,31 @@ issue = "fix(#445) the lone name word beside a marker a connective join no longe # having read the marker as a middle word. name_regex = "(?i)^Jane n[ée]e and Jones Smith$" fields = ["given", "middle", "family", "maiden"] + +[[change]] +issue = "fix(#371) a suffix never begins a name: the Ph./D. merge declines at the head" +# 'Ph. D. Van Johnson', 'Ph. D. John Smith', 'Ph. D.', 'Ph. D., Jr.': +# rules.md#S2 -- "A suffix never BEGINS a name: position outranks the +# vocabulary match". The v1 fix_phd merge joined the pair wherever it +# stood, which is what made a leading credential possible at all, and +# it emptied the family doing it: 'Ph. D. Van Johnson' read given +# 'Van Johnson' with no surname. Declining at the head of segment 0 +# leaves 'Ph.' to H2's abbreviation clause (title) and 'D.' to the +# positional read. +# +# RESTORES 1.4.0 on all four, exactly, which is why this ledger gains +# a rule while expected_since_1.4.0 LOSES three names: v1 matched the +# pair by regex mid-string and never merged it at the head either, so +# what shipped in 2.0 was a regression rather than a reading. Measured +# on the released 1.4.0 wheel: 'Ph. D. Van Johnson' -> title 'Ph.', +# first 'D.', last 'Van Johnson'; 'Ph. D. John Smith' -> title 'Ph.', +# first 'D.', middle 'John', last 'Smith'; 'Ph. D.' -> title 'Ph.', +# last 'D.'; 'Ph. D., Jr.' -> title 'Ph.', first 'D.', suffix 'Jr.' +# (three of four fields there -- v1 kept first 'D.' where the comma +# path now leaves it empty, which predates this change). +# +# `fields` is the union over the four names and nothing wider: title +# and suffix move on all four, given on two, family on three, middle +# on one. +name_regex = "(?i)^Ph\\. D\\.( John Smith| Van Johnson|, Jr\\.)?$" +fields = ["title", "given", "middle", "family", "suffix"] diff --git a/tools/differential/expected_since_2.1.0.toml b/tools/differential/expected_since_2.1.0.toml index bcf4c8b5..9f81fb71 100644 --- a/tools/differential/expected_since_2.1.0.toml +++ b/tools/differential/expected_since_2.1.0.toml @@ -1182,3 +1182,31 @@ issue = "fix(#445) the lone name word beside a marker a connective join no longe # having read the marker as a middle word. name_regex = "(?i)^Jane n[ée]e and Jones Smith$" fields = ["given", "middle", "family", "maiden"] + +[[change]] +issue = "fix(#371) a suffix never begins a name: the Ph./D. merge declines at the head" +# 'Ph. D. Van Johnson', 'Ph. D. John Smith', 'Ph. D.', 'Ph. D., Jr.': +# rules.md#S2 -- "A suffix never BEGINS a name: position outranks the +# vocabulary match". The v1 fix_phd merge joined the pair wherever it +# stood, which is what made a leading credential possible at all, and +# it emptied the family doing it: 'Ph. D. Van Johnson' read given +# 'Van Johnson' with no surname. Declining at the head of segment 0 +# leaves 'Ph.' to H2's abbreviation clause (title) and 'D.' to the +# positional read. +# +# RESTORES 1.4.0 on all four, exactly, which is why this ledger gains +# a rule while expected_since_1.4.0 LOSES three names: v1 matched the +# pair by regex mid-string and never merged it at the head either, so +# what shipped in 2.0 was a regression rather than a reading. Measured +# on the released 1.4.0 wheel: 'Ph. D. Van Johnson' -> title 'Ph.', +# first 'D.', last 'Van Johnson'; 'Ph. D. John Smith' -> title 'Ph.', +# first 'D.', middle 'John', last 'Smith'; 'Ph. D.' -> title 'Ph.', +# last 'D.'; 'Ph. D., Jr.' -> title 'Ph.', first 'D.', suffix 'Jr.' +# (three of four fields there -- v1 kept first 'D.' where the comma +# path now leaves it empty, which predates this change). +# +# `fields` is the union over the four names and nothing wider: title +# and suffix move on all four, given on two, family on three, middle +# on one. +name_regex = "(?i)^Ph\\. D\\.( John Smith| Van Johnson|, Jr\\.)?$" +fields = ["title", "given", "middle", "family", "suffix"] From 66128d34766e394d1f7b2079b938cd3ba3709bb9 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 30 Aug 2026 22:39:13 -0700 Subject: [PATCH 2/2] fix(group,docs): the four-agent review findings on #371, measured The gate meant a different "head" than the rule it implemented, in two directions at once, and the corpus could see neither. TOO LATE: `k == 0` is a PIECE index, and extract_delimited removes a quoted or bracketed clause before grouping -- so `"Bob" Ph. D. John Smith` reached the merge with the pair at k == 0 and a word standing before it in the input. v1 merges there (its regex needed only a preceding space); declining broke parity on 112 measured names, every one opening with a quote or bracket, none of them in any corpus. The test is now `a[0] == 0`, the first TOKEN, which is v1's boundary. TOO EARLY: a title is a piece, so `Sir Ph. D. Van Johnson` keeps the credential and the empty family -- #371's own symptom one word to the left. That is 1.4.0 parity and it cannot be fixed here: "the first piece of the name" is not computable before this merge, because H2's abbreviation test is true of `Ph.` itself, so a scan stepping over titles would step over the very piece being judged. rules.md#S2 now says OPENS THE STRING and carries the boundary example. Prose corrected with it, all of it mine and all of it wrong: - "restores 1.4.0 on all four" -- three. `Ph. D., Jr.` moves the `D.` from given to family, two roles apart, and rides under a pre-comma rule that predates this change. - "EXPLAINED NOTHING is how a parity restoration announces itself" -- the gate said SHADOWED, which compare.py separates from reverted precisely because the fixes differ. #426 is the precedent. - "v1 healed the pair only when it TRAILS" -- false in both directions; v1 healed it everywhere except the head, and reading that sentence as the contract would license narrowing the merge. - two live references to the rule this PR deleted, in the same ledger; acting on them takes the gate red. - the 2026-07 phd-merge entry promised the two spellings "read alike", which #371 scopes to non-leading position. Tests: the guard test this PR narrowed was left VACUOUS -- `II` is not suffix vocabulary under the file's reduced lexicon, so removing the `Ph. D.` witness left it passing under the exact mutation it is named for. `PhD` is the replacement witness and the mutation is killed again. Added Case rows for `Ph. D. Van Johnson` (the issue's subject, whose empty fields are the symptom) and the bare `Ph. D.`, and pinned both new conditions. Four mutations verified. Gate green at all three baselines. --- docs/design/decisions.md | 6 ++- docs/design/rules.md | 30 +++++++++---- docs/release_log.rst | 2 +- nameparser/_pipeline/_group.py | 45 ++++++++++++++++---- tests/v2/cases.py | 22 +++++++++- tests/v2/pipeline/test_group.py | 25 +++++++++-- tests/v2/test_ledger_guards.py | 19 ++++++--- tests/v2/test_parser.py | 6 ++- tools/differential/corpus_rules.jsonl | 2 + tools/differential/expected_since_1.4.0.toml | 30 +++++++++---- tools/differential/expected_since_2.0.0.toml | 16 ++++--- tools/differential/expected_since_2.1.0.toml | 16 ++++--- 12 files changed, 166 insertions(+), 53 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index e9d07a99..7acb6674 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -303,11 +303,13 @@ Declined (rc1 arc; the full argument is AGENTS.md's gotcha): ### phd-merge — the "Ph. D." split -- 2026-07 (v2 core, PR #288; recorded plan deviation #1 of the core plan) — "Ph. D." tokenizes as two words and is merged back by vocabulary (v1 fix_phd), so the spaced and unspaced spellings read alike. +- 2026-07 (v2 core, PR #288; recorded plan deviation #1 of the core plan) — "Ph. D." tokenizes as two words and is merged back by vocabulary (v1 fix_phd), so the spaced and unspaced spellings read alike. (Scoped by #371 below: alike everywhere EXCEPT at the head of the string, where the merge declines and `Ph. D. Van Johnson` reads title 'Ph.' where `Ph.D. Van Johnson` reads given 'Ph.D.'.) - 2026-08-31 #371 (the merge declines at the head) — a suffix never BEGINS a name: position outranks the vocabulary match, and `rules.md#S2` now states it. The merge is what made a leading credential possible at all — every other suffix-shaped word standing first already falls out as a title (H2's abbreviation clause, or TITLES membership: `Jr.`, `MD`, `Esq.`, `Sr.`) or as a name word (`PhD`, `III`), so the `Ph.`/`D.` pair was the only shape that reached the defect. It reached it by emptying the family: `Ph. D. Van Johnson` read given 'Van Johnson' with no surname at all, which is #371's report. - THIS RESTORES 1.4.0 RATHER THAN DEVIATING FROM IT, and the ledgers say so in both directions. v1's fix_phd healed the pair by regex on the raw string and matched it only where it TRAILED; leading, v1 split it — measured on the released wheel, `Ph. D. Van Johnson` gives title 'Ph.', first 'D.', last 'Van Johnson', which is exactly what this change gives. So `expected_since_1.4.0.toml` LOSES its fix(leading-credential) rule (the gate reported EXPLAINED NOTHING, which is how a parity restoration announces itself there) and the 2.0.0 and 2.1.0 ledgers GAIN one, claiming the same four names, the same five roles and the same digest. What shipped in 2.0 was a regression from applying the merge unconditionally, not a reading anyone chose. + THIS RESTORES 1.4.0 RATHER THAN DEVIATING FROM IT — on three of the four names, and the fourth is named below. v1's fix_phd healed the pair by regex on the raw string, `\s(ph\.?\s+d\.?)`, whose leading `\s` made a head match impossible while it fired everywhere else, mid-name as readily as trailing. (A first draft of this entry said "only where it TRAILED", which is false in both directions and would license narrowing the merge to the trailing position — `John Ph. D. Smith` reads suffix 'Ph. D.' at 1.4.0 and here.) At the head v1 split it — measured on the released wheel, `Ph. D. Van Johnson` gives title 'Ph.', first 'D.', last 'Van Johnson', which is exactly what this change gives. So `expected_since_1.4.0.toml` LOSES its fix(leading-credential) rule — and READ THE DIAGNOSIS, not just the error: the gate reported it SHADOWED, not reverted. `compare.py` separates the two ("shadowed -- an earlier rule claimed every diff it would have" versus "reverted -- matches no diffing name") because the fixes differ. Three of its four names became parity and the fourth, `Ph. D., Jr.`, still diverges under `fix(comma-precomma-family)`, which sits above it. #426 is the precedent for dropping a shadowed rule rather than keeping it. A first draft of this entry generalized EXPLAINED NOTHING into "how a parity restoration announces itself", which would teach the next maintainer to delete a shadowed rule whose behavior is still diverging and the 2.0.0 and 2.1.0 ledgers GAIN one, claiming the same four names, the same five roles and the same digest. What shipped in 2.0 was a regression from applying the merge unconditionally, not a reading anyone chose. OPENING THE NAME IS NOT OPENING A PIECE LIST, and the first draft got that wrong: a credential run legitimately opens segment 1 after a family comma (`Smith, Ph. D. Jr.`), which C1 reads as a listing rather than a name beginning. Testing `k == 0` alone broke fifteen tests and took the gate red at all three baselines. The caller passes the distinction in, since `_group_segment` cannot see which segment it holds. + ACCEPTED, and found by review rather than by design: a TITLE before the pair keeps the credential reading, because `k` is a piece index and a title is a piece. `Sir Ph. D. Van Johnson` still reads given 'Van Johnson' with an empty family — #371's own symptom, surviving one word to the left. It is 1.4.0 parity, and the rule states it as a boundary rather than deviating: "the first piece of the name" cannot be computed before this merge, since H2's abbreviation test is true of `Ph.` itself, so a scan that stepped over titles would step over the very piece being judged. rules.md#S2 therefore says OPENS THE STRING, which is what the code holds and what v1 held. + THE FIRST DRAFT GOT "THE HEAD" WRONG IN THE OTHER DIRECTION, and the corpus could not see it. Testing the PIECE index alone declined the merge for `"Bob" Ph. D. John Smith`: `extract_delimited` removes a quoted or bracketed clause before grouping, so the pair reaches the merge at `k == 0` with a word standing before it in the INPUT. v1 merges there (its regex needed only a preceding space), and 112 measured names broke parity — every one opening with a quote or a bracket, and not one of them in any corpus. The test is now `a[0] == 0`, the first TOKEN, which is v1's boundary exactly. ACCEPTED: `Parser.revise(suffix="Ph. D.")` now renders 'Ph., D.'. revise() runs a full sub-parse of the string it is given, and a field value has no head for a head-position rule to consult; a second draft carved that out by requiring a name to displace, and the carve-out made the head reading depend on what FOLLOWED it — appending a maiden clause changed whether 'Ph.' was a title, which `test_a_maiden_clause_changes_nothing_else` caught. Dropping the carve-out removed the inconsistency with it. The merge exists for a credential someone TYPED after a name; a caller who writes the spaced form into the suffix field is taken at their word (Derek's call, 2026-08-31). What the leading `Ph.` becomes is H2's business, not this rule's: an abbreviation before a name is almost always a title, which is the same clause that reads `Esq. van Gogh` as title 'Esq.' — so the pair reads title 'Ph.', given 'D.'. The issue proposed given 'Ph.', middle 'D.'; H2 claims the first word before the positional read sees it, and the two answers differ only in which field holds `Ph.`. diff --git a/docs/design/rules.md b/docs/design/rules.md index 1cc2b89d..7dfc812d 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -515,14 +515,18 @@ S1. Rationale: brackets set off more than nicknames — credentials S2. Rationale: generational suffixes and credentials are recognized by vocabulary; an acronym that is also an ordinary name is only unmistakably a credential when its periods are written. - A suffix never BEGINS a name: position outranks the vocabulary - match, so a suffix-shaped word opening a name reads as whatever - its position and shape make it — a title where it wears the - abbreviation shape (H2), an ordinary name word otherwise — and - never as a credential belonging to a name that has not been - written yet. A family comma changes what "begins" means rather - than the rule: the comma has already named the family, so the - part after it is a listing and a credential run may open it (C1). + A suffix never OPENS THE STRING: position outranks the + vocabulary match, so a suffix-shaped word written before anything + else reads as whatever its position and shape make it — a title + where it wears the abbreviation shape (H2), an ordinary name word + otherwise — and never as a credential belonging to a name that + has not been written yet. The string, not the name: a title, a + nickname or a bracketed clause standing first leaves the + credential reading intact, which is the ONE place this document + says "opening" and does not mean what P4 and the H Background + mean by it. A family comma is the other boundary: the comma has + already named the family, so a credential run may open the part + after it (C1), and the part before it is that family. A trailing word of the suffix vocabulary reads as a suffix — generational forms and credential acronyms alike, and an ambiguous acronym written with its periods, one after each @@ -540,6 +544,16 @@ S2. Rationale: generational suffixes and credentials are recognized "Ph. D. Van Johnson" → title="Ph." "Smith, Ph. D. Jr." → suffix="Ph. D. Jr." "II Van Johnson" → given="II" · boundary + "Sir Ph. D. Van Johnson" → suffix="Ph. D." · boundary + "Ph. D., John" → family="Ph. D." · boundary + Accepted: a title before the credential keeps it a credential, + and the name loses its surname exactly as it did before this + clause — `Sir Ph. D. Van Johnson` reads given `Van Johnson` with + an empty family. That is 1.4.0's reading, and the clause cannot + reach it: the first piece of the NAME is not computable before + this decision, since the abbreviation test H2 uses is true of + `Ph.` itself, so a scan that stepped over titles would step over + the very piece being judged. Accepted: with words to spare, a bare ambiguous acronym reads as a suffix even beside an East Asian surname it more likely belongs to; and an unambiguous suffix is consumed even when diff --git a/docs/release_log.rst b/docs/release_log.rst index d45a72ce..9d589f7f 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -71,7 +71,7 @@ Release Log - Change case repair to read the parser's own ``conjunction`` tag instead of re-deciding, from the word's spelling, whether a word is a conjunction or an initial. The parse answers that question already -- ``"Scott E. Werner"`` reads ``E.`` as an initial rather than the Italian conjunction -- and the other views honor the answer; case repair asked again, with a shape test applied to each word of a token's text rather than to the token. Two spellings of one name disagreed because of it: ``"juan e-f smith"`` capitalized to ``Juan e-F Smith`` while ``"JUAN E-F SMITH"`` gave ``Juan E-F Smith``; both give ``Juan E-F Smith`` now, ``e-f`` being a middle name and no conjunction of the parse's reading. A conjunction written as a word of its own is untouched, and so is the one-letter carve-out where it applies -- ``"juan y garcia"`` still repairs to ``Juan y Garcia``, ``"JUAN Y GARCIA"`` still to ``Juan Y Garcia``. A field assigned after the parse is unaffected: its text was never classified, so there is no reading to honor and repair asks the vocabulary, applying v1's own predicate the way every earlier version applied it everywhere -- ``h.last = "velasquez y garcia"`` still repairs to ``Velasquez y Garcia`` and ``h.middle = "e."`` to ``E.``. That is the predicate over TODAY's vocabulary, which is narrower than parity with 1.4.0 and the difference is real: ``h.last = "хосе и мария сантос"`` gives ``Хосе И Мария Сантос`` on 1.4.0 and ``Хосе и Мария Сантос`` here, because the Cyrillic ``и`` is a 2.x conjunction and was not a 1.4.0 one. What decides which path a token takes is a mark the assignment leaves, not the absence of a span: a value revised through ``Parser.revise()`` is classified by a sub-parse and keeps its tags, so it repairs as the parse does. One reading does change for hand-built ``Token``\ s in the 2.0 API: an untagged token whose text is conjunction vocabulary is now an ordinary name word and capitalizes, where 2.1 lowercased it -- tags are what the views read, and a hand-built token that carries none is a token with nothing to declare. Case repair is not one of the seven role fields the differential harness compares, so no gate run can see this change either way and none of its counts move; measured directly instead, no name of the 1094-name differential corpus moves under ``capitalized()`` or ``capitalized(force=True)``, its uppercased and lowercased spellings included -- 6564 name/spelling/lexicon rows and 13128 calls (closes #458) - - Fix a name that opens with a spaced ``Ph. D.`` losing its surname. ``parse("Ph. D. Van Johnson")`` read given ``Van Johnson`` with an empty ``family`` and suffix ``Ph. D.``; it now reads title ``Ph.``, given ``D.``, family ``Van Johnson``. A suffix never begins a name -- position outranks the vocabulary match -- and the merge that heals a split ``Ph.``/``D.`` into one credential is what made a leading credential possible at all: every other suffix-shaped word standing first already falls out as a title (``Jr.``, ``MD``, ``Esq.``, ``Sr.``) or as an ordinary name word (``PhD``, ``III``), so this pair was the only shape that reached the defect. The merge is unchanged everywhere else, and a family comma still opens a listing rather than a name, so ``"John Smith Ph. D."`` keeps suffix ``Ph. D.`` and ``"Smith, Ph. D. Jr."`` keeps suffix ``Ph. D. Jr.``. This RESTORES 1.4.0, which healed the pair only where it TRAILED and split it at the head exactly as this release now does -- measured on the released wheel, all four corpus names of this shape return to their v1 reading, and the 1.4.0 ledger loses the rule that used to excuse the difference. One accepted consequence: ``Parser.revise(suffix="Ph. D.")`` renders ``Ph., D.``, since revise() sub-parses the string it is given and a field value has no head for a head-position rule to read (closes #371) + - Fix a name that opens with a spaced ``Ph. D.`` losing its surname. ``parse("Ph. D. Van Johnson")`` read given ``Van Johnson`` with an empty ``family`` and suffix ``Ph. D.``; it now reads title ``Ph.``, given ``D.``, family ``Van Johnson``. A suffix never begins a name -- position outranks the vocabulary match -- and the merge that heals a split ``Ph.``/``D.`` into one credential is what made a leading credential possible at all: every other suffix-shaped word standing first already falls out as a title (``Jr.``, ``MD``, ``Esq.``, ``Sr.``) or as an ordinary name word (``PhD``, ``III``), so this pair was the only shape that reached the defect. The merge is unchanged everywhere else, and a family comma still opens a listing rather than a name, so ``"John Smith Ph. D."`` keeps suffix ``Ph. D.`` and ``"Smith, Ph. D. Jr."`` keeps suffix ``Ph. D. Jr.``. This RESTORES 1.4.0, whose own healing regex required a preceding space and so could never fire at the head of the string -- measured on the released wheel, three of the four corpus names of this shape return to their v1 reading exactly, and the 1.4.0 ledger loses the rule that used to excuse the difference. The fourth, ``"Ph. D., Jr."``, still differs in where the ``D.`` lands and rides under a pre-comma rule that predates this change. What "the head" means is the head of the STRING, not of the name: a title before the credential keeps it a credential, so ``"Sir Ph. D. Van Johnson"`` still reads given ``Van Johnson`` with no family -- also 1.4.0's reading, and recorded as a boundary in ``rules.md#S2`` rather than left implied. One accepted consequence: ``Parser.revise(suffix="Ph. D.")`` renders ``Ph., D.``, since revise() sub-parses the string it is given and a field value has no head for a head-position rule to read (closes #371) - Fix a trailing surname particle being stranded as a standalone middle name under ``Policy(name_order=FAMILY_FIRST)``, where the same listing written with a comma reads it as part of the surname. A particle ending the name has nothing to link forward to, so what it is doing there is decided by what the writing says: after a family comma it joins the family the comma named and is written before it, and a declared family-first order names the family the same way. ``Parser(policy=Policy(name_order=FAMILY_FIRST)).parse("Jong Anke de")`` gave family ``Jong`` with ``de`` left as a middle name, and now gives family ``de Jong``, given ``Anke`` -- the same answer ``parse("Jong, Anke de")`` has always given. The test is the SLOT the particle landed in, not the word: a middle name is a further given name, which a particle is not, and ``FAMILY_FIRST`` is the only order that puts a trailing piece there. ``FAMILY_FIRST_GIVEN_LAST`` puts it in the given slot, where the caller's own declaration says it is the given name, so ``"Nguyen Thi Van"`` under that order still reads given ``Van``. That one test reads both traditions without asking about the vocabulary at all: ``"Beethoven Ludwig van"`` under ``FAMILY_FIRST`` now gives family ``van Beethoven`` even though ``van`` is one of the 37 particles that are ordinary given names elsewhere. In the same change, a particle standing alone where a family-first order puts the GIVEN name is no longer folded into the family: ``"Ménil de"`` reports given ``de`` under both family-first orders, because that slot holds what the caller declared, and the never-given word list supplies a reading where position leaves the question open rather than overriding one position has already given. Nothing moves under the DEFAULT name order. Measured, 30 of 6606 parses move -- this release's 1101-name corpus under three ``name_order`` values with ``middle_as_family`` off and on -- over twelve names. Those counts cannot come from the differential gate: it parses every corpus name with the default policy and sweeps no policy at all, so no non-default ``name_order`` behavior has ever been compared across versions, and the gate output here is unchanged at all three baselines apart from the corpus names this change's own rules.md examples add. What moves is recomputed by the recipe in the ``P6`` entry of ``docs/design/decisions.md``, which compares the seven role fields against a checkout of the parent commit reading the same corpus files; the count is a dated snapshot, measured 2026-08-30. The ``rules.md#P1`` and ``rules.md#P6`` example lines and ``tests/v2/pipeline/test_post_rules.py`` are what pin the behavior (closes #467) - Fix ``initials()`` reading a name in a different order than the fields of the same name. Two rules fold words into the family name and render them before the rest of it -- ``Policy(middle_as_family=True)``, which sends every middle word to the family, and the tussenvoegsel attachment after a family comma -- and both do it by marking the words rather than moving them, since a parsed word keeps the position it was written at. The ``family`` field reads that mark and ``initials()`` did not, so one parse gave two orders: ``parse("der, y van")`` gave family ``van der`` and initials ``y. d. v.``, and now gives ``y. v. d.``. This RESTORES v1: ``middle_name_as_last`` is v1's spelling of the same option, so most of what moves has a 1.4.0 answer to be measured against, and measured over the 1094-name differential corpus at the default name order, 71 names move under that option, 54 of them back to exactly what 1.4.0 returns and not one of them away from it -- ``"Doe, Dr. John A."`` gives ``J. A. D.`` again where 2.0 through 2.2 gave ``J. D. A.``, and ``"Brundridge, Contessa A"`` gives ``C. A. B.`` where they gave ``C. B. A.``. Of the 17 that match 1.4.0 neither before nor after, 14 now agree with it on the ORDER and differ only in how v1 grouped initials -- 1.4.0 gives one initial per element of its own ``last_list``, so a conjunction-joined surname yields ``V G.`` where the 2.x view, one initial per word, yields ``V. G.``; one more is ``"der, y van"``, whose family is nothing but particles, where 1.4.0 contributes no initial at all and 2.x contributes its words, a difference this release does not touch; and the remaining two parse differently from 1.4.0 for reasons that predate this fix. Without the option, one corpus name moves, the ``"der, y van"`` above -- and it is one of one, since it is the only name in the corpus whose family holds two contributing words with a folded one behind the other, the shape an order change can be seen in at all. ``HumanName.initials()`` was already right and is unchanged, measured: no name of the 1094-name corpus moves through the facade, with the option or without it. It reads ``first_list``/``middle_list``/``last_list``, which prepend the folded words as v1 did, so it is the 2.0 API's ``ParsedName.initials()`` that was out of step -- with the field beside it, with the facade, and with 1.4.0 at once. ``initials()`` is not one of the seven role fields the differential harness compares, so no gate run can see this change: run at all three baselines before and after, the output is identical to the byte -- 1094 corpus names, 229 / 194 / 102 intentional diffs and ``unexplained: 0`` at 1.4.0 / 2.0.0 / 2.1.0, with every per-heading count unchanged. So where the counts in this bullet come from has to be said, the gate's classified summary not being able to supply them and the ``rules.md#R3`` example line witnessing the order without counting anything: what MOVES is recomputed by the recipe in the ``R3`` entry of ``docs/design/decisions.md``, which compares this view against the pre-change rendering over these same four corpora and reproduces the 71 and the one-of-one; the 1.4.0 comparisons and the facade sweep are dated snapshots rather than re-derivable ones, measured 2026-08-30 against the released 1.4.0 wheel and against the pre-change tree, which nothing in the repository re-runs. The ``rules.md#R3`` example line and ``tests/v2/test_render.py`` are what pin the behavior (closes #408) diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index 59d81a45..02a49121 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -377,14 +377,39 @@ def merge(lo: int, hi: int, add: Set[str] = frozenset(), # family: "Ph. D. Van Johnson" read given 'Van Johnson' with no # surname at all. # - # OPENING THE NAME is not "opening a piece list". A credential run - # legitimately opens segment 1 after a family comma ("Smith, - # Ph. D. Jr."), which is C1's listing form rather than a name - # beginning, so the caller passes the distinction in. + # WHERE THE INPUT BEGINS, and that is v1's own boundary rather + # than a new one: fix_phd was a regex requiring a preceding space + # (`\s(ph\.?\s+d\.?)`), so it could never fire at the head of the + # string and fired everywhere else -- mid-name as readily as + # trailing. Three tests, because "the head" has three meanings + # here and only one of them is v1's: + # `k == 0` the first PIECE, which is not the first word -- + # extract_delimited removes a bracketed or quoted + # clause before segment runs, so `"Bob" Ph. D. John + # Smith` reaches this with the pair at k == 0 and a + # word standing before it in the input. v1 merges + # there; declining broke parity on 112 measured + # names, every one opening with a quote or bracket. + # `a[0] == 0` the first TOKEN, which is v1's boundary. + # seg_idx 0 and not after a family comma: a credential run + # legitimately opens segment 1 ("Smith, Ph. D. + # Jr.", C1's listing form), and segment 0 before a + # comma is the family the comma named, where v1 + # merges too ("Ph. D., John" reads last 'Ph. D.'). + # + # A leading TITLE is therefore NOT stepped over, unlike the #367 + # scan below, and the two disagree on purpose: that scan asks + # where the NAME begins, this asks where the STRING does. `Sir + # Ph. D. Van Johnson` keeps suffix 'Ph. D.' with an empty family, + # which is #371's symptom surviving one word to the left -- 1.4.0 + # parity, stated as a boundary in rules.md#S2 rather than fixed + # here, because "the first piece of the name" cannot be computed + # before this merge: `is_leading_title` is true of `Ph.` itself, + # so the scan would step over the very piece being judged. k = 0 while k < len(pieces) - 1: a, b = pieces[k], pieces[k + 1] - if (not (opens_the_name and k == 0) + if (not (opens_the_name and k == 0 and a[0] == 0) and len(a) == 1 and len(b) == 1 and PH.fullmatch(tokens[a[0]].text) and D.fullmatch(tokens[b[0]].text)): @@ -483,10 +508,12 @@ def merge(lo: int, hi: int, add: Set[str] = frozenset(), # # Suffix pieces are deliberately NOT skipped, and the reason is # what skipping them WOULD do rather than what it would cost. - # A credential written with spaces already parses without a - # family name -- "Ph. D. Van Johnson" is given 'Van Johnson', - # suffix 'Ph. D.', family '' -- and skipping the suffix piece - # would actually give it one (given 'Van', family 'Johnson'). + # (The "Ph. D. Van Johnson" example that opened this argument + # left it with #371: the pair no longer merges at the head of + # the input, so there is no suffix-shaped leading piece there + # to skip or not. "Dr. Ph. D. Van Johnson" is the replacement + # -- family 'Van Johnson' as shipped, family 'Johnson' with + # the skip.) # The shapes that decide it are the ones whose leading piece # lands in `given` instead: "Ph.D. Van Johnson", "II Van # Johnson" and "Msc.Ed. Van Johnson" each read given diff --git a/tests/v2/cases.py b/tests/v2/cases.py index ff761b74..abc8d300 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -2043,11 +2043,29 @@ def __post_init__(self) -> None: Case("phd_split_mid_name", "Dr. John Ph. D. Smith", {"title": "Dr.", "given": "John", "family": "Smith", "suffix": "Ph. D."}), + Case("phd_split_leading_van_johnson", "Ph. D. Van Johnson", + {"title": "Ph.", "given": "D.", "family": "Van Johnson"}, + classification="parity", + notes="#371's own subject: the leading pair read given " + "'Van Johnson' with an EMPTY family until the merge " + "learned to decline at the head of the input. A Case " + "row rather than a rules.md line because the empty " + "fields are the symptom -- test_case asserts the whole " + "dict, so `suffix` and `middle` being empty is pinned " + "here and nowhere else"), + Case("phd_split_leading_bare", "Ph. D.", + {"title": "Ph.", "family": "D."}, + classification="parity", + notes="a bare credential yields a family name, which is " + "surprising and is 1.4.0's reading exactly. Pinned " + "because nothing else in the suite says so -- the " + "ledger reaches it, and the ledger is an out-of-band " + "tool run"), Case("phd_split_leading", "Ph. D. John Smith", {"title": "Ph.", "given": "D.", "middle": "John", "family": "Smith"}, - classification="fix", - notes="v1 healed 'Ph.'+'D.' only when trailing; leading it " + classification="parity", + notes="v1's fix_phd regex required a preceding space, so it " "split them (title 'Ph.', given 'D.', real given name " "pushed to middle). Surfaced by the issue-tracker " "corpus, which is where this shape existed at all."), diff --git a/tests/v2/pipeline/test_group.py b/tests/v2/pipeline/test_group.py index aee79d74..f37a97ee 100644 --- a/tests/v2/pipeline/test_group.py +++ b/tests/v2/pipeline/test_group.py @@ -125,9 +125,16 @@ def test_a_suffix_shaped_leading_piece_is_not_stepped_over() -> None: # # "Ph. D. Van Johnson" left this test with #371: the pair is no # longer merged at the head, so there is no suffix-shaped leading - # piece there to step over or not. The scan's decision is the same - # and "II Van Johnson" still witnesses it -- a numeral needs no - # merge to be suffix vocabulary. + # piece there to step over or not. + # + # `PhD` is the replacement witness and `II` is NOT one, which is + # the trap this file's reduced lexicon sets: `_LEX` ships + # suffix_acronyms={"phd"} and no roman numerals, so `II` is not a + # suffix piece HERE however the shipped vocabulary reads it, and + # the assertion below survives the very mutation this test is + # named for. Measured: with suffix pieces added to the scan, + # "PhD Van Johnson" moves and "II Van Johnson" does not. + assert _piece_texts(_grouped("PhD Van Johnson")) == [["PhD", "Van Johnson"]] assert _piece_texts(_grouped("II Van Johnson")) == [["II", "Van Johnson"]] @@ -140,6 +147,18 @@ def test_the_phd_merge_declines_at_the_head_of_a_name() -> None: [["Ph.", "D.", "Van Johnson"]] assert _piece_texts(_grouped("Smith, Ph. D. Jr.")) == \ [["Smith"], ["Ph. D.", "Jr."]] + # segment 0 BEFORE a family comma is the family the comma named, + # and v1 merges there too ("Ph. D., John" reads last 'Ph. D.'), so + # the `not family_comma` half of the flag has its own witness -- + # without it this name declines and the family splits. + assert _piece_texts(_grouped("Ph. D., John")) == \ + [["Ph. D."], ["John"]] + # the first PIECE is not the first word: a quoted clause leaves the + # token stream before grouping, so the pair reaches k == 0 with a + # word ahead of it in the input. v1 merges (its regex needed a + # preceding space); declining here broke parity on 112 names. + assert _piece_texts(_grouped('"Bob" Ph. D. John Smith')) == \ + [["Ph. D.", "John", "Smith"]] def test_von_und_zu_bridges() -> None: diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 295bb567..c35b6e80 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -1068,10 +1068,12 @@ class _LatinCopy(NamedTuple): # begins, and the alternation is over ANCHORS, not over words -- # there is no vocabulary here to drift from. frozenset({"^", ",\\s*"}), - # fix(#371)'s alternation is the three corpus names that carry a - # leading 'Ph. D.' -- a list of names, not a copy of any wordlist. - # The vocabulary the rule turns on is `ph`/`d`, which the regex - # spells out in the anchored stem rather than in the alternation. + # fix(#371)'s alternation holds the TAILS of the four corpus names + # that carry a leading 'Ph. D.' -- three tails plus the bare stem + # on the `?`. A list of names, not a copy of any wordlist. And the + # pair the rule turns on is not vocabulary at all: _vocab.PH and + # _vocab.D are fixed regexes, so there is no wordlist here for the + # alternation to drift from. frozenset({" John Smith", " Van Johnson", ", Jr\\."}), # fix(#445)'s movers, one corpus name per alternative -- a list of # names, not a copy of any wordlist, so there is no vocabulary for @@ -1529,8 +1531,13 @@ def _claim(rule: dict) -> _Claim: _Claim(13, ('family', 'middle'), "973617235cda"), "fix(#380) a trailing vd after a family comma is the tussenvoegsel, not a post-nominal": _Claim(2, ('family', 'suffix'), "ec0d45289dc1"), + # 279 -> 280 with #371, and the growth is corpus, not behavior: + # that PR added `Ph. D., John` as a rules.md example, so the + # regex matches one more corpus name. The name does not diff at + # this baseline (v1 and HEAD both read first 'John', last + # 'Ph. D.'), so nothing was absorbed. "fix(comma-family) lone post-comma piece routes to suffix/title, not first": - _Claim(279, ('given', 'suffix', 'title'), "28a62b622a48"), + _Claim(280, ('given', 'suffix', 'title'), "019315da85b9"), "fix(comma-family) a comma followed only by titles keeps the given/family split": _Claim(2, ('family', 'given'), "5bd9c6d96c38"), "fix(comma-family) a comma followed only by titles keeps the given/family split, the C1 example": @@ -1550,7 +1557,7 @@ def _claim(rule: dict) -> _Claim: "fix(#367) an inferred title no longer displaces a leading particle either": _Claim(1, ('family', 'given'), "d8ee9cd5da5f"), "fix(comma-precomma-family) pre-comma run reads as family, not given": - _Claim(279, ('family', 'given'), "28a62b622a48"), + _Claim(280, ('family', 'given'), "019315da85b9"), "fix(#342) NOT WANTED: a bare trailing 'Rai' is read as a post-nominal suffix and the family is lost": _Claim(1, ('family', 'suffix'), "694fd06a2e9a"), "fix(#397) NOT WANTED: a trailing Catalan/Polish linking 'i' is read as a generation marker and the family is lost": diff --git a/tests/v2/test_parser.py b/tests/v2/test_parser.py index 9c344345..d331b473 100644 --- a/tests/v2/test_parser.py +++ b/tests/v2/test_parser.py @@ -826,10 +826,12 @@ def test_revise_preserves_particle_tags() -> None: def test_revise_takes_a_spaced_credential_literally() -> None: # The Ph./D. merge is a HEAD-POSITION rule (#371) and a field value # has no head: revise() runs a full sub-parse of the string it is - # given, so "Ph. D." there is two initials, not one credential. + # given, so "Ph. D." there is two separate suffix pieces -- `Ph.` + # by vocabulary, `D.` as an initial -- which the suffix view + # renders comma-joined, rather than one healed credential. # Deliberate -- the merge exists for a credential someone TYPED # after a name, and a caller who writes the spaced form into the - # suffix field is taken at their word. + # suffix field gets it read as two suffix items. p = Parser() n = p.parse("John Smith Ph.D.") r = p.revise(n, suffix="Ph. D.") diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index a1fb9546..9873132e 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -138,6 +138,7 @@ "Nguyễn Thị Minh Khai" "Nguyễn, Thị Vân" "Ph. D. Van Johnson" +"Ph. D., John" "Rev. John Smith" "SHIRLEY MACLAINE" "Salam, abd Allah" @@ -149,6 +150,7 @@ "Shirley Maclaine" "Sidorov Ivan Petrovich Jr." "Sir John" +"Sir Ph. D. Van Johnson" "Sir abdul van der Berg" "Sir de Mesnil" "Smith" diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index 7c0c71b9..7a2a9cf1 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -1142,10 +1142,20 @@ fields = ["title", "given", "middle"] # absence is the record: it explained 2.0 routing a leading split # 'Ph. D.' to `suffix` where v1 read title 'Ph.' and given 'D.'. #371 # restored v1 on all four names of that shape, so the rule explained -# nothing and the gate said so (EXPLAINED NOTHING is an error, which -# is how a parity restoration announces itself here). What v1 did -- -# heal the pair only when it TRAILS -- is what the parser does again; -# rules.md#S2 now states it as a rule rather than a regex accident. +# nothing, and the gate reported it SHADOWED -- `fix(comma-precomma- +# family)` above claims the one name still diffing here, which is the +# #426 precedent for dropping rather than keeping a shadowed rule. +# Read the diagnosis, not just the error: compare.py separates +# `reverted` (matches no diffing name) from `shadowed` (an earlier +# rule claimed every diff it would have) precisely because the fixes +# differ. Three of this rule's four names became parity; the fourth, +# 'Ph. D., Jr.', still diverges and rides under that earlier rule. +# +# What v1 did -- never heal the pair at the HEAD of the string, +# because its regex `\s(ph\.?\s+d\.?)` required a preceding space, +# while healing it anywhere else, mid-name as readily as trailing -- +# is what the parser does again; rules.md#S2 now states it as a rule +# rather than a regex accident. # Deliberately NOT a [[change]] rule: TRAILING 'Ph. D.' split-token @@ -1603,8 +1613,11 @@ why = "trailing 'Ph. D.' split-token healing is PARITY, not a 2.0 change: v1 hea # two forms -- 'John Ph. D.' and 'John Smith, Ph. D.' -- so an # exclusion keyed on the comma protects half of it while looking # complete. The anchor also keeps clear of 'Ph. D. John Smith', the -# LEADING shape, which has its own fix(leading-credential) rule that -# this must not silence. No `fields`: any diff on this shape is a +# LEADING shape. That shape had its own fix(leading-credential) rule +# until #371 restored v1 and it went dormant; the carve-out stays +# load-bearing anyway, since it is what lets 'Ph. D., Jr.' reach +# fix(comma-precomma-family). Dropping it because the rule it once +# protected is gone takes this gate red. No `fields`: any diff on this shape is a # regression, which is what omitting the key means. # # One shape this deliberately over-reaches: 'John Smith, Jr. Ph. D.' @@ -1631,8 +1644,9 @@ why = "trailing 'Ph. D.' split-token healing is PARITY, not a 2.0 change: v1 hea # 0, which is the outcome this entry's own `why` exists to prevent. # # The three exceptions, each diffing for a cause this ledger names -# elsewhere: a leading credential with no name before it -# (fix(leading-credential)), one word before the comma +# elsewhere: a leading credential with no name before it (which had +# a fix(leading-credential) rule until #371 restored v1 and it went +# dormant -- see the note where it stood), one word before the comma # (fix(comma-precomma-family)), and a suffix word between the name and # the credential (fix(credential-pair-order), its own rule below). # Refusing those was the entry over-reaching; refusing anything else diff --git a/tools/differential/expected_since_2.0.0.toml b/tools/differential/expected_since_2.0.0.toml index e8cffd2d..2d2043f4 100644 --- a/tools/differential/expected_since_2.0.0.toml +++ b/tools/differential/expected_since_2.0.0.toml @@ -1252,16 +1252,20 @@ issue = "fix(#371) a suffix never begins a name: the Ph./D. merge declines at th # leaves 'Ph.' to H2's abbreviation clause (title) and 'D.' to the # positional read. # -# RESTORES 1.4.0 on all four, exactly, which is why this ledger gains -# a rule while expected_since_1.4.0 LOSES three names: v1 matched the -# pair by regex mid-string and never merged it at the head either, so -# what shipped in 2.0 was a regression rather than a reading. Measured +# RESTORES 1.4.0 on THREE of the four, exactly, which is why this +# ledger gains a rule while expected_since_1.4.0 loses three names: +# v1 matched the pair by regex mid-string and never merged it at the +# head either, so what shipped in 2.0 was a regression rather than a +# reading. Measured # on the released 1.4.0 wheel: 'Ph. D. Van Johnson' -> title 'Ph.', # first 'D.', last 'Van Johnson'; 'Ph. D. John Smith' -> title 'Ph.', # first 'D.', middle 'John', last 'Smith'; 'Ph. D.' -> title 'Ph.', # last 'D.'; 'Ph. D., Jr.' -> title 'Ph.', first 'D.', suffix 'Jr.' -# (three of four fields there -- v1 kept first 'D.' where the comma -# path now leaves it empty, which predates this change). +# -- and that fourth name is NOT restored: v1 read first 'D.' where +# the pre-comma run now reads family 'D.', two roles apart. The 'D.' +# relocated rather than vanished, which is why the residual diff +# rides under `fix(comma-precomma-family)` (fields given+family) and +# not under this rule. That reading predates this change. # # `fields` is the union over the four names and nothing wider: title # and suffix move on all four, given on two, family on three, middle diff --git a/tools/differential/expected_since_2.1.0.toml b/tools/differential/expected_since_2.1.0.toml index 9f81fb71..5e067b72 100644 --- a/tools/differential/expected_since_2.1.0.toml +++ b/tools/differential/expected_since_2.1.0.toml @@ -1194,16 +1194,20 @@ issue = "fix(#371) a suffix never begins a name: the Ph./D. merge declines at th # leaves 'Ph.' to H2's abbreviation clause (title) and 'D.' to the # positional read. # -# RESTORES 1.4.0 on all four, exactly, which is why this ledger gains -# a rule while expected_since_1.4.0 LOSES three names: v1 matched the -# pair by regex mid-string and never merged it at the head either, so -# what shipped in 2.0 was a regression rather than a reading. Measured +# RESTORES 1.4.0 on THREE of the four, exactly, which is why this +# ledger gains a rule while expected_since_1.4.0 loses three names: +# v1 matched the pair by regex mid-string and never merged it at the +# head either, so what shipped in 2.0 was a regression rather than a +# reading. Measured # on the released 1.4.0 wheel: 'Ph. D. Van Johnson' -> title 'Ph.', # first 'D.', last 'Van Johnson'; 'Ph. D. John Smith' -> title 'Ph.', # first 'D.', middle 'John', last 'Smith'; 'Ph. D.' -> title 'Ph.', # last 'D.'; 'Ph. D., Jr.' -> title 'Ph.', first 'D.', suffix 'Jr.' -# (three of four fields there -- v1 kept first 'D.' where the comma -# path now leaves it empty, which predates this change). +# -- and that fourth name is NOT restored: v1 read first 'D.' where +# the pre-comma run now reads family 'D.', two roles apart. The 'D.' +# relocated rather than vanished, which is why the residual diff +# rides under `fix(comma-precomma-family)` (fields given+family) and +# not under this rule. That reading predates this change. # # `fields` is the union over the four names and nothing wider: title # and suffix move on all four, given on two, family on three, middle