diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 2f9464c1..377d31c6 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -446,6 +446,14 @@ Excluded (MAIDEN_MARKERS, per nameparser/config/maiden_markers.py): R1 gained the separator sentence with this change. That the invariant it states — one comma segment is one suffix entry — was written down in no rule is a large part of why two stages disagreed about it for years: C1 governs routing, R1 governed word ORDER and never mentioned separation, and the only statement of the rule was a code comment. Left alone, measured and pre-existing: the NO-COMMA path comma-joins a space-separated run too (`John Smith MD PhD` renders 'MD, PhD' at 1.4.0, 2.0.0, 2.1.0 and master alike). The fix requires FAMILY_COMMA and does not reach it; that path has no writer-drawn boundary and is a separate question, recorded here and NOT yet filed. It is load-bearing for one thing: `str()` of a fixed parse is a no-comma string, so re-parsing it re-inserts the comma — round-tripping is not stable on these shapes, and an earlier draft of the release bullet claimed it was. The name `Smith, MD PhD` entered the corpora as this fix's C1 example — a FIRING one, carrying no `· boundary` annotation, which marks the non-firing example a rule must have — so its ledger entries arrive with it. A fourth written for the 1.4.0 ledger EXPLAINED NOTHING and was dropped: `fix(comma-family) lone post-comma piece routes to suffix/title, not first` already claims the name there, its fields covering all three that move. That is #373's dormant-rule diagnosis doing its job, and #426 is the precedent for dropping rather than keeping a shadowed rule. +- 2026-08-25 #430 / #432 — one boundary, fixed as one: inside a comma segment a ONE-CHARACTER suffix word — the only kind a reader could take for an initial — is read by WHAT STANDS BEFORE IT. Behind another suffix it describes that suffix and continues the run, period included — `Smith, PSM I` and `Smith, PSM I.` are suffix 'PSM I'/'PSM I.' where they read given 'PSM' plus suffix 'I', an initial in that slot being no shape anyone writes. Behind a name word the period is decisive, marking an abbreviation and so name material — `Smith, John V.` is middle 'V.' where it read suffix 'V.', which is what `Smith, John B.` always did; the only thing that made V differ is that V is also suffix vocabulary. The bare form is the boundary and does not move: `Smith, John V` stays suffix 'V' (v1 parity, #144), which is why the test is the PERIOD and not the numeral. SCOPE, stated as the code reads rather than as the intent was argued: the class is one-character SUFFIX VOCABULARY, not "a roman numeral". It sweeps in `2` (`Smith, John 2.` is middle '2.'), misses `X` and `IX` (roman, but not vocabulary — `Smith, PSM X` is unchanged), and never touched `II`/`III`/`IV` at all, which are two characters and so were never initial-shaped and never vetoed. Anyone adding a one-character suffix in a script that has initials extends it. And the strict knob is consulted: under `lenient_comma_suffixes=False` the initial veto stands and the run ends at the numeral, which the first draft silently overrode by reading no policy at all. + #430's cause was rules.md#S2's initial veto in `is_suffix_piece`, which keeps a numeral out of a suffix reading — right after a name, wrong inside a credential run — so the segment did not look like a run at all and fell to the given-and-suffix walk. Two wrong answers came out of that one cause, and a fix verified on `Smith, PSM I` alone would have left the other: where the run's first word is TITLES vocabulary too (`Smith, MD I`, and `Smith, Jr. I` through the period-abbreviation inference) the leading-title peel took it instead, giving title 'MD' plus given 'I'. + THREE readers, one reading, which is the part worth copying. The gate that says a segment is a credential run, assign's router that assigns its roles, and group's one-entry join that renders it must agree PIECE FOR PIECE, and #429 shipped the inverse of its own fix by deriving that agreement twice. `segment_suffix_reading` returns the per-piece verdicts and all three read it (mechanisms.md#ONE-PREDICATE-PER-QUESTION); the intermediate state where only the gate and the router had been converted rendered `Smith, PSM I` as 'PSM, I', the render splitting a run whose roles were already right — the same tell as #429, one predicate along. + Declined, and the v1 suite caught it: asking `is_trailing_numeral_suffix` for #432's half. It answers the period question — `_ROMAN` is anchored and admits no period — but it also refuses a numeral standing behind an initial-shaped piece, which is a NO-COMMA rule and the opposite of this path's parity: `Chang, Andy C I` is first 'Andy', middle 'C', suffix 'I', and asking that predicate here made the numeral a middle. decisions.md's #401/#421 entry records the same fork declining to transfer to this walk under LENIENT. The half that applies is the period alone. + Blast radius, and the corpus IS blind to the moving classes: of the 1069 names on master, seven carry a one-character numeral behind a word inside a comma segment — `Chang, Andy C I`, `Doe, Rev. John V, Jr.`, `Maier, Amy I, Jr.`, `Maier, Amy Lauren I`, `Smith, John I`, `Smith, John V`, `Berg, abdul V` — every one of them BARE, and NONE moves. Zero pre-existing corpus names move at all; the only movers are the three this change adds as C1 examples. So the gate's 0-unexplained is evidence about the corpus, not about the change, and the case table is the whole coverage. `Berg, abdul V` cannot reach the walk at all: the bound-given join makes `abdul V` one piece, which is why the LENIENT mirror #401/#421 declined does not collide here. The moved names entered the corpora as C1's new examples, and carry ledger entries at all three baselines except #430's at 1.4.0, where `fix(comma-family) lone post-comma piece routes to suffix/title, not first` already claims the name and a second rule would be dormant. + Two things the /simplify round settled by measurement rather than argument. The walk asks `is_suffix_piece(pieces[m - 1])` — a VOCABULARY test — where the reading carries the verdict it just recorded, and unifying them looks obviously right until you run it: the walk assigns `PSM` to GIVEN in `Smith, PSM I.`, so a reading-style flag says "behind a name" and sends the numeral to middle, losing the strict parity restored two commits earlier. The two sites ask different questions on purpose, and the vocabulary answer is the load-bearing one. The cost is a reading the walk cannot reach: `Smith, John PSM I I.` moves from middle 'I' + suffix 'PSM, I.' to middle 'I I.' + suffix 'PSM', two consecutive one-character numerals behind a credential — accepted, both readings being defensible for a shape nobody writes, and recorded here rather than chased at the price of two correct pins. + The walk is also strict-blind, pre-existing and unchanged: it calls is_suffix_lenient unconditionally, so `Smith, John V` reads suffix 'V' under `lenient_comma_suffixes=False` as under the default. The reading consults the knob and the walk does not, which is now conspicuous where it was merely unexamined. Not this change's to fix. + Not done, and recorded rather than left implicit: neither half emits an ambiguity, though the no-comma `John Smith V` reports `suffix-or-name` for the same fork. The comma path has never reported it (decisions.md#C1 records PARTICLE_OR_GIVEN suppressed there for a related reason), and adding an emitter needs a trigger, a stage-ownership entry and a case row — more than this fix, and #432 raises it as a secondary observation rather than a requirement. ### T1 — separators, not joiners diff --git a/docs/design/mechanisms.md b/docs/design/mechanisms.md index bb470729..f010cde2 100644 --- a/docs/design/mechanisms.md +++ b/docs/design/mechanisms.md @@ -55,7 +55,7 @@ Problem shape. "Which stage does X?" — asked before attributing behavior in pr ## ONE-PREDICATE-PER-QUESTION — one predicate answers it, and every other site calls that -Problem shape. Two stages need the same answer about the same input, and the one that does not own the decision is about to test for it. Contract statement. Where two sites ask the same question, exactly one predicate answers it and every other site calls that one — never a condition written to match it. The predicate belongs to the QUESTION, not to whichever stage decides: it may sit in a leaf both stages import, and for the leading-title test it must, since the deciding stage is assign and group cannot import assign. How it works. A hand-written mirror agrees with its original only until one of them moves, and the drift is invisible in both directions: each site keeps passing its own tests while they disagree about an input neither covers. Five instances, every one found as a defect before it was found as a pattern — #319 lifted the wholly-suffix predicate into the vocabulary layer "so the comma decision and the honorific peel's segment test cannot drift apart"; #401/#421 lifted the trailing-numeral fork out of assign so the bound-given reserve stopped carrying a copy, its hand-written mirror having been falsified in review more than once — the lesson recorded there being that what must be mirrored is assign's WALK, not merely its condition; #425 replaced that reserve's hand re-derivation of the trailing peel with one function over the view the join would leave; #424 moved assign's leading-title test down because group's own `title()` does not see H2's unlisted abbreviations, so `Xyz. van Johnson` chained where `Dr. van Johnson` did not; #429 moved the no-name-segment test down because group asked by segment INDEX where assign asks by CONTENT. The destination follows the LAYER, not the topic: a predicate over token text goes to `_vocab`, one over pieces and tags to `_pieces`. Both are leaves the stages sit on. The piece layer got its own module only in #439 — until then those predicates collected in `_group`, not because grouping owned them but because `_assign` imports `_group` and cannot be imported back, so group was the one place both stages could reach; five had accumulated across four PRs before the module existed. Stage order is this mechanism's limit, and it forecloses the alternative: where the reader comes AFTER the decider, record the answer on the state instead — `ParseState.order` is that shape, "Recorded rather than recomputed downstream, because the two can differ" — which is unavailable whenever the EARLIER stage is the one asking. (The concrete assign→group import that forced the `_group` collection is gone since #439; what remains is the ordering it was a symptom of, and tests/v2/test_layering.py is where the leaf's contract is now written down.) The cost is a second evaluation of the same predicate, measured for #429 at 1.2–2.2% of a family-comma parse and 0% of every other; recording that number was the right answer there over plumbing a state field the two sites would not otherwise share. Lives in. nameparser/_pipeline/_vocab.py over text (is_wholly_suffix, and is_trailing_numeral_suffix — the #401/#421 instance, whose only caller since #439 is the shared peel rather than a stage) and nameparser/_pipeline/_pieces.py over pieces: is_suffix_piece, is_leading_title, leading_titles, peel_walk, peel_trailing and segment_holds_no_name are called by both stages, is_title_piece and trailing_start by group alone — `trailing_start` being the one to know, since it answers where the trailing run begins and is what P2's chain and M2's walk stop at. tests/v2/test_layering.py holds each module's contract, and a piece predicate growing a dependency on a STAGE shows up there as a widened entry. Reach for it when. You are about to write a condition that mirrors, matches or "does what X does" — or you find a comment saying one does. Grep for the other site's predicate and call it instead. +Problem shape. Two stages need the same answer about the same input, and the one that does not own the decision is about to test for it. Contract statement. Where two sites ask the same question, exactly one predicate answers it and every other site calls that one — never a condition written to match it. The predicate belongs to the QUESTION, not to whichever stage decides: it may sit in a leaf both stages import, and for the leading-title test it must, since the deciding stage is assign and group cannot import assign. How it works. A hand-written mirror agrees with its original only until one of them moves, and the drift is invisible in both directions: each site keeps passing its own tests while they disagree about an input neither covers. Five instances, every one found as a defect before it was found as a pattern — #319 lifted the wholly-suffix predicate into the vocabulary layer "so the comma decision and the honorific peel's segment test cannot drift apart"; #401/#421 lifted the trailing-numeral fork out of assign so the bound-given reserve stopped carrying a copy, its hand-written mirror having been falsified in review more than once — the lesson recorded there being that what must be mirrored is assign's WALK, not merely its condition; #425 replaced that reserve's hand re-derivation of the trailing peel with one function over the view the join would leave; #424 moved assign's leading-title test down because group's own `title()` does not see H2's unlisted abbreviations, so `Xyz. van Johnson` chained where `Dr. van Johnson` did not; #429 moved the no-name-segment test down because group asked by segment INDEX where assign asks by CONTENT. The destination follows the LAYER, not the topic: a predicate over token text goes to `_vocab`, one over pieces and tags to `_pieces`. Both are leaves the stages sit on. The piece layer got its own module only in #439 — until then those predicates collected in `_group`, not because grouping owned them but because `_assign` imports `_group` and cannot be imported back, so group was the one place both stages could reach; five had accumulated across four PRs before the module existed. Stage order is this mechanism's limit, and it forecloses the alternative: where the reader comes AFTER the decider, record the answer on the state instead — `ParseState.order` is that shape, "Recorded rather than recomputed downstream, because the two can differ" — which is unavailable whenever the EARLIER stage is the one asking. (The concrete assign→group import that forced the `_group` collection is gone since #439; what remains is the ordering it was a symptom of, and tests/v2/test_layering.py is where the leaf's contract is now written down.) The cost is a second evaluation of the same predicate, measured for #429 at 1.2–2.2% of a family-comma parse and 0% of every other; recording that number was the right answer there over plumbing a state field the two sites would not otherwise share. Lives in. nameparser/_pipeline/_vocab.py over text (is_wholly_suffix, and is_trailing_numeral_suffix — the #401/#421 instance, whose only caller since #439 is the shared peel rather than a stage) and nameparser/_pipeline/_pieces.py over pieces: is_suffix_piece, is_leading_title, leading_titles, peel_walk, peel_trailing and segment_suffix_reading are called by both stages — the last of those is #430's instance, where THREE readers share one answer, the render join being the third — while is_title_piece and trailing_start are called by group alone — `trailing_start` being the one to know, since it answers where the trailing run begins and is what P2's chain and M2's walk stop at. tests/v2/test_layering.py holds each module's contract, and a piece predicate growing a dependency on a STAGE shows up there as a widened entry. Reach for it when. You are about to write a condition that mirrors, matches or "does what X does" — or you find a comment saying one does. Grep for the other site's predicate and call it instead. ## CLAUSE-CONTENT-OVERRULES-DELIMITER — content wins diff --git a/docs/design/rules.md b/docs/design/rules.md index fa7a6480..2f62e2e7 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -613,6 +613,15 @@ C1. Rationale: a credential run after the comma means the name is in more than one name word keeps its positional read, order and all. A name word in the part after the comma makes it the given name, with titles before it and suffixes after. + A one-character suffix word — the only kind a reader could take + for an initial — is read by what stands before it. Behind + another suffix it is describing that suffix and continues the + run, written with a period or without, an initial being no shape + anyone writes there; behind a name word it is the generation + only when written bare, a period marking it the abbreviation of + a name and so a middle initial. Longer suffix words are not in + question either way, and the strict knob above still vetoes the + initial-shaped ones, so the run ends at them there. "Smith, John" → family="Smith" "سلمان، محمد" → family="سلمان" "田中、太郎" → family="" @@ -623,6 +632,12 @@ C1. Rationale: a credential run after the comma means the name is in "Smith, PhD" → suffix="PhD" "Smith, Jr." → suffix="Jr." "Smith, Sr." → suffix="Sr." + "Smith, PSM I" → suffix="PSM I" + "Smith, PSM I." → suffix="PSM I." + "Smith, PSM I." strict-comma-suffixes → suffix="I." + "Smith, John V." → middle="V." + "Smith, John PhD I." → suffix="PhD, I." + "Smith, John V" → suffix="V" · boundary "Smith, Ph. D. Jr." → suffix="Ph. D. Jr." "Smith, MD PhD" → suffix="MD PhD" "Smith, Dr." → title="Dr." diff --git a/docs/release_log.rst b/docs/release_log.rst index ddc6f3b6..92c37019 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -49,6 +49,7 @@ Release Log - Fix a credential run after a one-word family comma reading as a title or a given name: ``"Smith, Jr."`` and ``"Smith, PhD"`` now give suffix ``Jr.``/``PhD`` where they gave title, and ``"Smith, Ph. D. Jr."`` gives suffix ``Ph. D. Jr.`` where the split credential landed in the given name (a regression from 1.4.0, which read suffix ``Ph. D.``). The position right after a family comma is postnominal position, so a run that is nothing but suffix words is read as the credential run it is before the title peel's whole-segment exception can claim it. Vocabulary still decides which words qualify -- ``"Smith, Dr."`` keeps title ``Dr.``, because ``dr`` is not postnominal vocabulary at all -- and the leading readings are untouched (``"Sr. Garcia"`` is still title ``Sr.`` while ``"Smith, Sr."`` is now suffix ``Sr.``); a name word in the run makes it the given-and-suffix reading it always had (``"Smith, John Jr."``) (closes #296, closes #325) - Fix a space-separated credential run after a family comma rendering with a comma the name never had: ``"Smith, MD PhD"`` gives suffix ``MD PhD`` where it gave ``MD, PhD``, and ``"Smith, CBE MC"``, ``"Smith, BSc MBA"``, ``"Smith, PhD Jr."`` and ``"Smith, Dr. MD PhD"`` the same. The roles are unchanged; only the rendered string carried the extra comma. One comma segment is one suffix entry, and grouping decided which segments those were by their position while assignment decided the same segment by its content, so the two disagreed about the segment a family comma leaves after it. The reading follows assignment now, which is where the segment's roles were already decided. This reaches any family comma whose following segment holds no name word, not only a one-word family: ``"John Smith, Jr. III"`` gives suffix ``Jr. III``, which is also what 1.4.0 gave. The full-name comma form was never affected -- ``"John Smith, MD PhD"`` has given suffix ``MD PhD`` since 1.4.0 -- and this brings the rest into line with it. A run written with commas keeps them (``"Smith, MD, PhD"`` is still suffix ``MD, PhD``), a comma between two entries survives (``"Smith Jr., Mr. Jr."`` is still suffix ``Jr., Jr.``), and a name word in the segment still makes it the given-and-suffix reading (``"Smith, John Jr."``). A name written without a comma is unaffected and still renders its run comma-joined, so re-parsing ``str()`` output does not reproduce the run (closes #429) + - Fix a one-character suffix word after a comma being read by the wrong neighbour: ``"Smith, PSM I"`` gives suffix ``PSM I`` where it gave given ``PSM`` and suffix ``I``, and ``"Smith, John V."`` gives middle ``V.`` where it gave suffix ``V.``. Inside a comma part a suffix word short enough to be mistaken for an initial -- ``I``, ``V`` and ``2`` in the shipped vocabulary -- is read by what stands before it. Behind a credential it is describing that credential -- ``PSM I`` is Professional Scrum Master level I -- so the run continues through it whether or not a period is written, since an initial in that position is not a name shape anyone writes; ``"Smith, MD I"`` and ``"Smith, Jr. I"`` reached the same wrong answer by a different route and move with it. Behind a name the period decides, marking an abbreviation and so a middle initial, exactly as ``"Smith, John B."`` always read. A numeral written bare after a name is still the generation it looks like (``"Smith, John V"`` is suffix ``V``), and a name with no comma is untouched (closes #430, closes #432) - Fix a name opening with a particle that is *never* a given name being split at the particle under a family-first name order -- ``Policy(name_order=FAMILY_FIRST)`` and ``Policy(name_order=FAMILY_FIRST_GIVEN_LAST)`` alike, and identically: ``"de Mesnil"`` read as family ``de``, given ``Mesnil``, and ``"de la Vega"`` as family ``de``, given ``la Vega``. Each is now the whole surname, as it has always been in the default order. The rule enforcing it asked for the particle by the ``GIVEN`` role, which under a family-first order belongs to the token *after* the particle, so the test read the wrong word and declined. It now also asks by position -- the piece that opens the name -- so both shapes of the same rule are caught: where such a particle stands alone as a piece, either opening the name or in the given position, the name is left with no given name at all, the given and the middles folding into the family. Standing *alone* is the whole of it, and the rule claims nothing wider: ``"Juan de la Vega"`` under ``FAMILY_FIRST`` still reports given ``de la Vega``, because there the particle chained onto the words after it rather than standing alone, and a bare ``"de"`` with nothing to fold into is still reported as the given name. The decision behind the fix: a word that can never be a given name leaves ``name_order`` nothing to decide, so declaring family-first is not a reason to make ``de`` a surname on its own. A leading particle that *may* be a given name is genuinely order-dependent and is untouched -- ``"van Gogh"`` still reads as family ``van``, given ``Gogh`` under both family-first orders. This is also what gives ``Lexicon.particles_ambiguous`` an effect outside the default order: taking a word out of it now changes the parsed fields under a family-first order, where before it moved only the ambiguity report. Seven of the 751 differential corpus names move, the same seven under each family-first order; default-order output is byte-identical over all 751, at the 1.4.0, 2.0.0 and 2.1.0 differential baselines alike (closes #359) diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 6a0bfdbf..96f5a8ff 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -37,13 +37,15 @@ from __future__ import annotations import dataclasses +from collections.abc import Sequence, Set +from nameparser._lexicon import Lexicon from nameparser._pipeline._vocab import ( effective_script, is_suffix_lenient, resolve_script_set, ) from nameparser._pipeline._pieces import ( is_suffix_piece, leading_titles, peel_trailing, peel_walk, - segment_holds_no_name, + segment_suffix_reading, ) from nameparser._pipeline._state import ( ParseState, PendingAmbiguity, Structure, WorkToken, @@ -256,6 +258,40 @@ def _assign_main(seg_idx: int, state: ParseState, return order +def _reads_as_a_trailing_suffix(piece: Sequence[int], + prev_piece: Sequence[int], + prev_ptags: Set[str], + tokens: Sequence[WorkToken], + lexicon: Lexicon) -> bool: + """The lenient tail test for a trailing one-token piece after a + family comma, and #432's carve-out from it. + + A word that could be a middle initial, written with the period that + marks an abbreviation, is name material -- so 'Smith, John V.' is + middle 'V.' where 'Smith, John V' stays suffix 'V' (v1 parity, + #144). Only behind a NAME word: behind a suffix the credential run + owns it, and 'Smith, John PhD I.' keeps suffix 'PhD, I.'. + + The period is the whole carve-out. is_initial_shaped would be + redundant beside it: the caller only consults this where + is_suffix_piece said no, and a lenient single token it refuses is + one carrying the `initial` tag, so the shape is already implied. + + NOT is_trailing_numeral_suffix, though it answers the period half: + it also refuses a numeral behind an initial-shaped piece, which is + a no-comma rule and the opposite of this path's v1 parity -- + 'Chang, Andy C I' is first Andy, middle C, suffix I, and asking + that predicate here made the numeral a middle. The #401/#421 entry + under decisions.md#P5 records the same fork declining to transfer + to this walk under LENIENT. + """ + text = tokens[piece[0]].text + if text.endswith(".") and not is_suffix_piece( + prev_piece, prev_ptags, tokens): + return False + return is_suffix_lenient(text, lexicon) + + def assign(state: ParseState) -> ParseState: tokens = list(state.tokens) ambiguities = list(state.ambiguities) @@ -300,9 +336,10 @@ def assign(state: ParseState) -> ParseState: # positional read peels a trailing suffix first: 'Smith Jr., # Mr.' has two pieces and one name, and read positionally lost # its family (the code review). - no_name = segment_holds_no_name(state.pieces[1], - state.piece_tags[1], tokens) - if no_name and sum( + reading = segment_suffix_reading( + state.pieces[1], state.piece_tags[1], tokens, + state.policy.lenient_comma_suffixes) + if reading is not None and sum( 1 for k, piece in enumerate(fam_pieces) if not is_suffix_piece(piece, fam_tags[k], tokens)) > 1: order = _assign_main(0, state, tokens, ambiguities) @@ -335,12 +372,11 @@ def assign(state: ParseState) -> ParseState: # duals ('Smith, Sr.' is Senior, 'Sr. Garcia' Señor). A # name word in the segment makes it v1's walk ('Smith, # John Jr.'). - if no_name: + if reading is not None: + # the gate's own reading (#430) for k, piece in enumerate(pieces): _set_roles(tokens, piece, - Role.SUFFIX if is_suffix_piece( - piece, ptags[k], tokens) - else Role.TITLE) + Role.SUFFIX if reading[k] else Role.TITLE) n = len(pieces) else: n = _peel_leading_titles(pieces, ptags, tokens) @@ -365,8 +401,9 @@ def assign(state: ParseState) -> ParseState: and len(state.segments) == 2) if is_suffix_piece(pieces[m], ptags[m], tokens) or ( last_of_two and len(pieces[m]) == 1 - and is_suffix_lenient( - tokens[pieces[m][0]].text, state.lexicon)): + and _reads_as_a_trailing_suffix( + pieces[m], pieces[m - 1], ptags[m - 1], + tokens, state.lexicon)): _set_roles(tokens, pieces[m], Role.SUFFIX) else: _set_roles(tokens, pieces[m], Role.MIDDLE) diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index fa25a5cb..16d39f51 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -42,7 +42,7 @@ from nameparser._lexicon import _title_key from nameparser._pipeline._pieces import ( is_leading_title, is_suffix_piece, is_title_piece, - leading_titles, peel_trailing, peel_walk, segment_holds_no_name, + leading_titles, peel_trailing, peel_walk, segment_suffix_reading, trailing_start, ) from nameparser._pipeline._state import ( @@ -710,9 +710,11 @@ def group(state: ParseState) -> ParseState: # can pin it -- dropping it is an equivalent mutant over the # corpora and 65,725 generated inputs -- so it is documented # rather than tested. - one_entry = tail or ( - family_comma and seg_idx == 1 - and segment_holds_no_name(pieces, ptags, tokens)) + reading = (segment_suffix_reading( + pieces, ptags, tokens, + state.policy.lenient_comma_suffixes) + if family_comma and seg_idx == 1 else None) + one_entry = tail or reading is not None if one_entry: # v1 renders each tail COMMA SEGMENT as one suffix entry # ('Smith, V MD' -> suffix 'V MD'); a delimiter core inside @@ -725,6 +727,10 @@ def group(state: ParseState) -> ParseState: # #206 parity is a TAIL rule, and the one-entry join is the # only half that follows assign's content read. entry_open = False + # non-Optional for the per-piece loop: inside `one_entry`, + # `reading is None` implies `tail`, so the tuple is only + # indexed where it exists + entry = reading or () kept: list[int] = [] for k in range(len(pieces)): is_core = (len(pieces[k]) == 1 @@ -741,20 +747,27 @@ def group(state: ParseState) -> ParseState: # is written role-blind because the merge is (the ph-d # pair reaches GIVEN as one element), though no # multi-token TITLE piece witnesses it -- none turned up - # in 38,892 generated family-comma inputs. BETWEEN pieces it continues an ENTRY, - # and only pieces that render into the same run may do - # that. On a tail segment every kept piece does -- that - # is what `tail` means -- but off it assign routes piece - # by piece (_assign.py), so a title piece is not part of - # the suffix entry. Letting one continue the entry tags - # a token the SUFFIX view never joins and the TITLE view - # does: 'Smith, Rev. Dr.' collapsed title_list to - # ['Rev. Dr.'], and after a pre-comma suffix the tag - # glued across the writer's own comma ('Smith Jr., Mr. - # Jr.' rendered suffix 'Jr. Jr.') -- the inverse of the - # bug this block exists to fix. - in_entry = tail or is_suffix_piece( - pieces[k], ptags[k], tokens) + # in 38,892 generated family-comma inputs. + # + # BETWEEN pieces the tag continues an ENTRY, and only + # pieces that render into the same run may do that. On a + # tail segment every kept piece does -- that is what + # `tail` means -- but off it assign routes piece by + # piece, so a title piece is not part of the suffix + # entry. Letting one continue the entry tags a token the + # SUFFIX view never joins and the TITLE view does: + # 'Smith, Rev. Dr.' collapsed title_list to ['Rev. + # Dr.'], and after a pre-comma suffix the tag glued + # across the writer's own comma ('Smith Jr., Mr. Jr.' + # rendered suffix 'Jr. Jr.') -- the inverse of the bug + # this block exists to fix. + # + # Which pieces those are is assign's reading, computed + # once per segment above rather than re-derived here: + # is_suffix_piece alone refuses a numeral continuing a + # credential run, and would render 'Smith, PSM I' as + # 'PSM, I' (#430). + in_entry = tail or entry[k] for pos, i in enumerate(pieces[k]): if pos > 0 or (in_entry and entry_open): tokens[i] = dataclasses.replace( diff --git a/nameparser/_pipeline/_pieces.py b/nameparser/_pipeline/_pieces.py index e1cddb46..5d0aac96 100644 --- a/nameparser/_pipeline/_pieces.py +++ b/nameparser/_pipeline/_pieces.py @@ -14,7 +14,8 @@ imported back, so group was the only place both stages could reach. They arrived there that way across three PRs -- #424 brought is_leading_title, leading_titles and trailing_start, #425 the peel -(peel_walk, peel_trailing), #429 segment_holds_no_name. +(peel_walk, peel_trailing), #429 the no-name-segment test that +#430 turned into segment_suffix_reading. is_title_piece and is_suffix_piece are older than any of that: they were group's from its first commit, and travel because the others call them. @@ -108,42 +109,94 @@ def is_suffix_piece(piece: Sequence[int], ptags: Set[str], return "vocab:suffix" in tags and "initial" not in tags -def segment_holds_no_name(pieces: Sequence[Sequence[int]], +def _numeral_behind_the_initial_veto(piece: Sequence[int], + tokens: Sequence[WorkToken]) -> bool: + """Suffix vocabulary that is_suffix_piece refuses because it is + also initial-shaped: a ONE-CHARACTER entry, bare or with a period. + + Named for the shape rather than enumerated, because the shape is + what the code tests and the enumeration goes stale -- in the + shipped lexicon it reaches i, v and 2, and NOT x or ix (roman, but + not suffix vocabulary) nor ii/iii/iv (suffix vocabulary, but two + characters, so never initial-shaped and never vetoed in the first + place). A caller adding a one-character suffix in a script that + has initials extends it. + + The veto is right where such a word could be a middle initial, and + wrong where it is describing the suffix in front of it, which is + the only place this is asked from. The len(piece) != 1 guard is + defensive: a merged multi-token piece carries "suffix" in ptags, so + is_suffix_piece claims it one branch earlier and no reachable input + arrives here with one. + """ + if len(piece) != 1: + return False + tags = tokens[piece[0]].tags + return "vocab:suffix" in tags and "initial" in tags + + +def segment_suffix_reading(pieces: Sequence[Sequence[int]], ptags: Sequence[Set[str]], - tokens: Sequence[WorkToken]) -> bool: - """The segment is titles and suffixes only ('John Smith, Dr.', - 'John Smith, Mr. Jr.') -- nothing in it is a name word. - - The FAMILY_COMMA rule "segment 0 is wholly the family name" rests on - the writer having said where the family name ends. A comma followed - by no name word said no such thing -- 'John Smith, Dr.' is 'Dr. John - Smith' with the honorific moved, and 'John Smith, Mr. Jr.' the same - with the postnominal along -- so the pre-comma name keeps its - positional read instead of being merged. Uses the same + tokens: Sequence[WorkToken], + lenient: bool, + ) -> tuple[bool, ...] | None: + """How each piece of a no-name segment reads: True a suffix, False + a title. None when the segment holds a name word and so is not a + credential run at all. + + ONE answer for three readers -- assign's no-name gate, its + router, and group's one-entry join -- because they must agree piece + for piece. #429 shipped the inverse of its own fix by deriving that + agreement twice (mechanisms.md#ONE-PREDICATE-PER-QUESTION). + + rules.md#S2's initial veto keeps a roman numeral out of a suffix + reading, which is right after a NAME word: 'Smith, John V.' is a + middle initial (#432). After a SUFFIX word the numeral is + describing that suffix -- 'PSM I' is Professional Scrum Master + level I -- so the run continues through it, period included, an + initial there being no shape anyone writes (#430). A title resets + that: what follows a bare title is not continuing a credential. + + None covers both ways a segment can fail to be a run: a name word + anywhere in it, and no pieces at all ('Doe,, Jr.', which holds no + title to read by). + + `lenient` is Policy.lenient_comma_suffixes, and only the numeral + continuation consults it. C1: "by default a recognized suffix word + counts even written like an initial, while strict mode vetoes + initial-shaped words" -- so under strict the veto stands and the + run ends where it always did. Reading no policy here silently + overrode the one knob a caller sets to prevent exactly this. + + The FAMILY_COMMA rule "segment 0 is wholly the family name" rests + on the writer having said where the family name ends. A comma + followed by no name word said no such thing -- 'John Smith, Dr.' is + 'Dr. John Smith' with the honorific moved -- so the pre-comma name + keeps its positional read instead of being merged. Uses the same is_leading_title predicate the peel does, period-abbreviation inference included, so the two cannot disagree about what a title - is; a suffix piece counts as what it is, so a mixed run like - 'Smith, Dr. Jr.' is a title and a postnominal, each read where it - stands, and never a title run 'Dr. Jr.'. An empty segment - ('Doe,, Jr.') holds no title to read by. - - TWO callers, asking it for different reasons, and the difference - matters. assign uses it to decide whether the comma fixed the family - name (above). group's one-entry join (#429) uses it to decide - whether the segment is a credential run at all. - - True does NOT mean "every piece is a suffix" -- the title tolerance - is the whole point, and a true segment can still hold pieces assign - routes to TITLE, so a caller rendering the segment as one unit must - ask is_suffix_piece per piece as well. What assuming otherwise cost - is recorded at the one-entry join in group(), the caller that made - the assumption. + is; a mixed run like 'Smith, Dr. Jr.' is a title and a postnominal, + each read where it stands, never a title run 'Dr. Jr.'. """ if not pieces: - return False - return all(is_suffix_piece(pieces[k], ptags[k], tokens) - or is_leading_title(pieces[k], ptags[k], tokens) - for k in range(len(pieces))) + return None + out: list[bool] = [] + for piece, tags in zip(pieces, ptags): + # the verdict just recorded IS "stands behind a suffix" -- keeping + # a separate flag meant maintaining that equality by hand at three + # sites, and a fourth branch that appended without assigning would + # have diverged silently + after_suffix = bool(out) and out[-1] + if is_suffix_piece(piece, tags, tokens): + out.append(True) + elif (lenient and after_suffix + and _numeral_behind_the_initial_veto(piece, tokens)): + out.append(True) + elif is_leading_title(piece, tags, tokens): + out.append(False) + else: + return None + return tuple(out) class Peel(NamedTuple): diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 63e831cc..055a1deb 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -1667,6 +1667,97 @@ def __post_init__(self) -> None: "segment 1 alone. Dropping that conjunct left the whole " "suite green while this shape's suffix silently became " "'PhD Jr.' (the mutation matrix found it)"), + Case("family_comma_run_ending_in_a_numeral", "Smith, PSM I", + {"family": "Smith", "suffix": "PSM I"}, + classification="fix(#430)", + notes="the credential run does not end because its last word " + "is a roman numeral: PSM I is Professional Scrum Master " + "level I, and the numeral describes the credential " + "rather than the person's generation. The run read as " + "given 'PSM' + suffix 'I' because the initial veto in " + "is_suffix_piece keeps a numeral out of a credential " + "run, so the segment did not look like one"), + Case("family_comma_run_numeral_ignores_the_period", "Smith, PSM I.", + {"family": "Smith", "suffix": "PSM I."}, + classification="fix(#430)", + notes="and the period does not end it either. After a suffix " + "word the numeral is describing that suffix, and an " + "initial in that position is not a name shape anyone " + "writes -- so the abbreviation reading that governs " + "#432 does not reach here. The full-name 'John Smith, " + "PSM I.' has read it this way all along"), + Case("family_comma_run_numeral_after_a_dual_word", "Smith, MD I", + {"family": "Smith", "suffix": "MD I"}, + classification="fix(#430)", + notes="the same shape reached through the leading-title peel " + "instead: md is TITLES vocabulary too (the #296 " + "deviation), so this read title 'MD' + given 'I' where " + "'Smith, PSM I' read given 'PSM' + suffix 'I'. Two " + "wrong answers, one cause -- a fix verified on PSM " + "alone would leave this one broken and look green. " + "'Smith, Jr. I' is the same story by the other route, " + "reaching the peel through the period-abbreviation " + "inference rather than TITLES membership, and is the " + "spelling a writer actually produces; it had a row of " + "its own until the mutation matrix showed the two trace " + "identically once fixed -- both heads are suffix pieces " + "now, so is_leading_title is never consulted for either"), + Case("family_comma_numeral_after_a_name_is_an_initial", + "Smith, John V.", + {"given": "John", "middle": "V.", "family": "Smith"}, + classification="fix(#432)", + notes="the other half of the boundary: after a NAME word the " + "period is decisive, because it marks an abbreviation " + "and an abbreviation is name material. 'Smith, John B.' " + "has always read middle 'B.'; the only thing that made " + "V. different is that V is also suffix vocabulary"), + Case("family_comma_bare_numeral_after_a_name_is_the_suffix", + "Smith, John V", + {"given": "John", "family": "Smith", "suffix": "V"}, + notes="THE BOUNDARY, and v1 parity (#144): with no period " + "there is no abbreviation, so the numeral is the " + "generation it looks like. This row is what makes " + "#432's fix a period test rather than a numeral test"), + Case("family_comma_title_resets_the_credential_run", "Smith, PSM Dr. I", + {"given": "PSM", "middle": "Dr.", "family": "Smith", + "suffix": "I"}, + notes="THE RESET, and unchanged since 1.4.0. A title ends the " + "run: what follows a bare title is not continuing a " + "credential, so the numeral behind it does not join and " + "the segment is no run at all. Removing that one line " + "left the whole suite green while this became title " + "'Dr.' + suffix 'PSM I' -- the reset fires 60 times " + "across the suite and until this row no input observed " + "it, which is the inert-measurement shape"), + Case("family_comma_run_numeral_after_a_split_credential", + "Smith, Ph. D. I", + {"family": "Smith", "suffix": "Ph. D. I"}, + classification="fix(#430)", + notes="the numeral continues a run whose head is a MERGED " + "piece -- the Ph./D. pair the #325 split-credential " + "merge builds, which carries 'suffix' in its piece tags " + "rather than on a single token. A structurally different " + "pin on the same three readers as the PSM rows, so an " + "edit to those cannot quietly unpin the render join"), + Case("family_comma_numeral_behind_a_suffix_is_not_an_initial", + "Smith, John PhD I.", + {"given": "John", "family": "Smith", "suffix": "PhD, I."}, + notes="THE OTHER BOUNDARY, and parity at every baseline. The " + "period makes a numeral name material only behind a NAME " + "word; behind a suffix the run owns it, and the first " + "draft of #432 read the piece alone and made this middle " + "'I.'. Rendered with the comma because the writer typed " + "no run here -- the segment holds a name, so it is the " + "walk, not the one-entry join"), + Case("family_comma_strict_keeps_the_initial_veto", + "Smith, PSM I.", + {"given": "PSM", "family": "Smith", "suffix": "I."}, + policy=Policy(lenient_comma_suffixes=False), + notes="C1's strict knob still vetoes initial-shaped words, so " + "the run ends at the numeral where lenient continues " + "through it. #430's first draft read no policy at all " + "and silently overrode the one knob a caller sets to " + "prevent exactly this; nothing in the suite saw it"), Case("family_comma_run_with_a_name_is_not_a_run", "Smith, John Jr.", {"given": "John", "family": "Smith", "suffix": "Jr."}, notes="the non-flip: a name word in the run makes it the " diff --git a/tests/v2/pipeline/test_pieces.py b/tests/v2/pipeline/test_pieces.py new file mode 100644 index 00000000..fe6376dd --- /dev/null +++ b/tests/v2/pipeline/test_pieces.py @@ -0,0 +1,103 @@ +"""Unit tests for the shared piece predicates. + +_pieces has had no unit-test module since #439 moved it out of _group; +its predicates were reached only end to end through the case table. +These pin the two contracts that shape cannot reach: a defensive branch +no parse can produce, and the stability the three readers rest on. +""" +from nameparser._lexicon import Lexicon +from nameparser._pipeline._assign import assign +from nameparser._pipeline._classify import classify +from nameparser._pipeline._group import group +from nameparser._pipeline._pieces import ( + _numeral_behind_the_initial_veto, segment_suffix_reading, +) +from nameparser._pipeline._segment import segment +from nameparser._pipeline._state import ParseState +from nameparser._pipeline._tokenize import tokenize +from nameparser._policy import Policy + + +def _through_group(text: str) -> ParseState: + state = ParseState(original=text, lexicon=Lexicon.default(), + policy=Policy()) + for stage in (tokenize, segment, classify, group): + state = stage(state) + return state + + +def test_the_numeral_veto_refuses_a_multi_token_piece() -> None: + """The len(piece) != 1 guard, which no parse can exercise. + + A merged multi-token piece carries "suffix" in its PIECE tags, so + is_suffix_piece claims it one branch earlier and the reading never + asks this helper about one -- the review instrumented it over the + whole suite and both differential corpora and found no call with a + longer piece. That makes the guard unreachable, not wrong: without + it the helper would read piece[0]'s tags and answer for the FIRST + token of a piece rather than for the piece, which is a wrong answer + where returning False is a safe one. + + Asserted directly because it cannot be asserted through a name -- + and with a piece the TAG test would accept, so that only the guard + can produce the False. A first version of this test used the merged + Ph./D. piece, whose head carries vocab:suffix but not initial: it + COVERED the line and pinned nothing, since the tag test answered + first and deleting the guard changed no result. + """ + state = _through_group("Smith, V PSM") + tokens = list(state.tokens) + head = next(i for i, tok in enumerate(tokens) if tok.text == "V") + assert {"vocab:suffix", "initial"} <= tokens[head].tags, ( + "the probe needs a token the TAG test would accept, or the " + "assertion below passes without reaching the guard") + + # one token: the tag test answers, and answers yes + assert _numeral_behind_the_initial_veto((head,), tokens) is True + # two tokens headed by that same token: only the guard can say no, + # so deleting it flips this + assert _numeral_behind_the_initial_veto((head, head + 1), tokens) is False + + +def test_the_reading_is_positional_and_total() -> None: + """One verdict per piece, in order -- the invariant all three + readers index by, and the only thing that makes reading[k] mean + pieces[k].""" + state = _through_group("Smith, MD PSM I") + reading = segment_suffix_reading( + state.pieces[1], state.piece_tags[1], list(state.tokens), True) + assert reading is not None + assert len(reading) == len(state.pieces[1]) + assert all(isinstance(v, bool) for v in reading) + + +def test_the_reading_does_not_move_when_roles_are_assigned() -> None: + """The stability the shared-predicate design rests on. + + group reads the reading before assign runs and assign reads it + again afterwards; that is only safe because the predicates read + token TAGS and text, which assign never rewrites -- it writes + roles. If a stage ever tagged during assignment the two readers + would silently disagree, which is the drift #429 and #430 are. + """ + state = _through_group("Smith, PSM I") + before = segment_suffix_reading( + state.pieces[1], state.piece_tags[1], list(state.tokens), True) + after_state = assign(state) + after = segment_suffix_reading( + after_state.pieces[1], after_state.piece_tags[1], + list(after_state.tokens), True) + assert before == after == (True, True) + + +def test_strict_ends_the_run_at_the_initial_shaped_numeral() -> None: + """C1's strict knob, at the predicate rather than through a parse. + + Lenient continues the credential run through a one-character + suffix word; strict vetoes initial-shaped words, so the run is no + run at all and the segment falls to the walk. + """ + state = _through_group("Smith, PSM I") + args = (state.pieces[1], state.piece_tags[1], list(state.tokens)) + assert segment_suffix_reading(*args, True) == (True, True) + assert segment_suffix_reading(*args, False) is None diff --git a/tests/v2/test_facade_cases.py b/tests/v2/test_facade_cases.py index 2e51fdf9..926a510e 100644 --- a/tests/v2/test_facade_cases.py +++ b/tests/v2/test_facade_cases.py @@ -84,6 +84,9 @@ "maiden_marker_kyusei_delimited", "ko_honorific_period_under_strict_comma_suffixes", "ja_honorific_glued_family_comma_strict_knob", + # the strict knob has no v1 spelling, so the row that pins C1's + # initial veto surviving #430 is core-only + "family_comma_strict_keeps_the_initial_veto", "ja_honorific_glued_family_comma_credential_pair_strict_knob", }) diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index ecd2bfa2..f81cfa2f 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -1239,6 +1239,8 @@ def _claim(rule: dict) -> _Claim: #: both is growth into names the rule genuinely describes. _CORPUS_CLAIMS: dict[str, dict[str, _Claim]] = { "expected_since_1.4.0.toml": { + "fix(#432) a dotted numeral behind a name is a middle initial, not the generation": + _Claim(1, ('middle', 'suffix'), "e9f282da0d0f"), "fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots": _Claim(108, ('family', 'given', 'middle'), "9a814f70c2dc"), "fix(#274) maiden markers consumed": @@ -1248,7 +1250,7 @@ def _claim(rule: dict) -> _Claim: "fix(#379) a tussenvoegsel after a family comma attaches to the family": _Claim(13, ('family', 'middle'), "973617235cda"), "fix(comma-family) lone post-comma piece routes to suffix/title, not first": - _Claim(275, ('given', 'suffix', 'title'), "58f0d640d1d9"), + _Claim(279, ('given', 'suffix', 'title'), "28a62b622a48"), "fix(comma-family) a comma followed only by titles keeps the given/family split": _Claim(2, ('family', 'given'), "5bd9c6d96c38"), "fix(comma-family) a comma followed only by titles keeps the given/family split, the C1 example": @@ -1268,9 +1270,9 @@ def _claim(rule: dict) -> _Claim: "fix(#367) an inferred title no longer displaces a leading particle either": _Claim(1, ('family', 'given', 'middle'), "d8ee9cd5da5f"), "fix(comma-precomma-family) pre-comma run reads as family, not given": - _Claim(275, ('family', 'given'), "58f0d640d1d9"), + _Claim(279, ('family', 'given'), "28a62b622a48"), "fix(suffix-routing) two-token name with unambiguous trailing suffix stays suffix": - _Claim(1069, ('family', 'given', 'suffix'), "b1ccb5c7532c"), + _Claim(1073, ('family', 'given', 'suffix'), "4109210ea38c"), "fix(suffix-delimiter-rendering) no-space delimiter core token kept whole": _Claim(0, ('suffix',), "e3b0c44298fc"), "ambiguous-surname-acronym data change: parenthesized (MA)/(DO) now stays nickname": @@ -1343,6 +1345,10 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('family', 'given'), "e62caedec864"), }, "expected_since_2.0.0.toml": { + "fix(#430) a credential run does not end at the roman numeral describing it": + _Claim(2, ('given', 'suffix'), "3c8fa6bc827a"), + "fix(#432) a dotted numeral behind a name is a middle initial, not the generation": + _Claim(1, ('middle', 'suffix'), "e9f282da0d0f"), "fix(#429) a wholly-credential segment after a one-word family renders as one entry": _Claim(1, ('suffix', 'title'), "9e0b9e8d5cbe"), "fix(#379) a tussenvoegsel after a family comma attaches to the family": @@ -1441,6 +1447,10 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('family', 'maiden'), "2150936a8c55"), }, "expected_since_2.1.0.toml": { + "fix(#430) a credential run does not end at the roman numeral describing it": + _Claim(2, ('given', 'suffix'), "3c8fa6bc827a"), + "fix(#432) a dotted numeral behind a name is a middle initial, not the generation": + _Claim(1, ('middle', 'suffix'), "e9f282da0d0f"), "fix(#429) a wholly-credential segment after a one-word family renders as one entry": _Claim(1, ('suffix', 'title'), "9e0b9e8d5cbe"), "fix(#379) a tussenvoegsel after a family comma attaches to the family": diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 05009f10..6f44d141 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -134,11 +134,16 @@ "Smith, Dr. Jr." "Smith, Esq." "Smith, John" +"Smith, John PhD I." +"Smith, John V" +"Smith, John V." "Smith, Jr." "Smith, MD PhD" "Smith, Major. John" "Smith, Ms." "Smith, Ms. Jane" +"Smith, PSM I" +"Smith, PSM I." "Smith, Ph. D. Jr." "Smith, PhD" "Smith, Sr." diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index c37cc5b2..a9bcca8c 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -1286,3 +1286,20 @@ issue = "fix(#418) the connective carve-out counts the name the maiden clause le # expected_since_2.1.0.toml copy carries the fuller account. name_regex = "(?i)^\\S+\\s+[ye]\\s+\\S+\\s+n[eé]e\\s+\\S+$" fields = ["given", "middle", "family", "maiden"] + +[[change]] +issue = "fix(#432) a dotted numeral behind a name is a middle initial, not the generation" +# 'Smith, John V.': v1 read suffix 'V.'. Behind a NAME word the period +# is decisive -- it marks an abbreviation, and an abbreviation is name +# material -- so the numeral is the middle initial it looks like, as +# 'Smith, John B.' always was. The BARE form is the boundary and is +# v1 parity either way: 'Smith, John V' stays suffix 'V' (#144), and it +# is a corpus name, so the gate asserts that half rather than this +# entry's silence. +# +# The #430 half needs no entry at this baseline: 'Smith, PSM I' is +# already claimed by "fix(comma-family) lone post-comma piece routes to +# suffix/title, not first", whose fields cover the move. A second rule +# would be dormant, which the gate reports as EXPLAINED NOTHING. +name_regex = "(?i)^smith,\\s*john\\s+v\\.$" +fields = ["middle", "suffix"] diff --git a/tools/differential/expected_since_2.0.0.toml b/tools/differential/expected_since_2.0.0.toml index fed2117a..1ecddf86 100644 --- a/tools/differential/expected_since_2.0.0.toml +++ b/tools/differential/expected_since_2.0.0.toml @@ -858,3 +858,37 @@ issue = "fix(#429) a wholly-credential segment after a one-word family renders a # read once rather than absorbed here. name_regex = "(?i)^smith,\\s*md\\s+phd$" fields = ["title", "suffix"] + +[[change]] +issue = "fix(#430) a credential run does not end at the roman numeral describing it" +# 'Smith, PSM I' and 'Smith, PSM I.': the baseline read given 'PSM', +# suffix 'I' -- the run ended at the numeral because rules.md#S2's +# initial veto keeps a numeral out of a suffix reading, so the segment +# did not look like a credential run and fell to the given-and-suffix +# walk. Behind a SUFFIX the numeral is describing that suffix ('PSM I' +# is Professional Scrum Master level I), so the run continues through +# it, period included -- an initial in that position is no shape anyone +# writes. The full-name 'John Smith, PSM I' has read it this way all +# along; this brings the one-word family into line. +# +# `family` is deliberately OUT of the field list: the family name does +# not move on this rule, so a regression there stays loud. Anchored to +# the two example spellings -- the class has no other corpus name, and +# a harvested one should arrive UNEXPLAINED and be read once. +name_regex = "(?i)^smith,\\s*psm\\s+i\\.?$" +fields = ["given", "suffix"] + +[[change]] +issue = "fix(#432) a dotted numeral behind a name is a middle initial, not the generation" +# 'Smith, John V.': the baseline read suffix 'V.'. Behind a NAME word +# the period is decisive -- it marks an abbreviation, and an +# abbreviation is name material -- so the numeral is the middle initial +# it looks like. 'Smith, John B.' has always read middle 'B.'; the only +# thing that made V. differ is that V is also suffix vocabulary. +# +# The BARE form is the boundary and does not move: 'Smith, John V' is +# still suffix 'V' (v1 parity, #144), and it is a corpus name, so that +# half is asserted by the gate rather than by this entry's silence. +# `given` and `family` are out of the field list; neither moves. +name_regex = "(?i)^smith,\\s*john\\s+v\\.$" +fields = ["middle", "suffix"] diff --git a/tools/differential/expected_since_2.1.0.toml b/tools/differential/expected_since_2.1.0.toml index 125d38f9..6413ce27 100644 --- a/tools/differential/expected_since_2.1.0.toml +++ b/tools/differential/expected_since_2.1.0.toml @@ -795,3 +795,37 @@ issue = "fix(#429) a wholly-credential segment after a one-word family renders a # read once rather than absorbed here. name_regex = "(?i)^smith,\\s*md\\s+phd$" fields = ["title", "suffix"] + +[[change]] +issue = "fix(#430) a credential run does not end at the roman numeral describing it" +# 'Smith, PSM I' and 'Smith, PSM I.': the baseline read given 'PSM', +# suffix 'I' -- the run ended at the numeral because rules.md#S2's +# initial veto keeps a numeral out of a suffix reading, so the segment +# did not look like a credential run and fell to the given-and-suffix +# walk. Behind a SUFFIX the numeral is describing that suffix ('PSM I' +# is Professional Scrum Master level I), so the run continues through +# it, period included -- an initial in that position is no shape anyone +# writes. The full-name 'John Smith, PSM I' has read it this way all +# along; this brings the one-word family into line. +# +# `family` is deliberately OUT of the field list: the family name does +# not move on this rule, so a regression there stays loud. Anchored to +# the two example spellings -- the class has no other corpus name, and +# a harvested one should arrive UNEXPLAINED and be read once. +name_regex = "(?i)^smith,\\s*psm\\s+i\\.?$" +fields = ["given", "suffix"] + +[[change]] +issue = "fix(#432) a dotted numeral behind a name is a middle initial, not the generation" +# 'Smith, John V.': the baseline read suffix 'V.'. Behind a NAME word +# the period is decisive -- it marks an abbreviation, and an +# abbreviation is name material -- so the numeral is the middle initial +# it looks like. 'Smith, John B.' has always read middle 'B.'; the only +# thing that made V. differ is that V is also suffix vocabulary. +# +# The BARE form is the boundary and does not move: 'Smith, John V' is +# still suffix 'V' (v1 parity, #144), and it is a corpus name, so that +# half is asserted by the gate rather than by this entry's silence. +# `given` and `family` are out of the field list; neither moves. +name_regex = "(?i)^smith,\\s*john\\s+v\\.$" +fields = ["middle", "suffix"]