gh-156518: Return the simple case mappings defined by the UCD - #156519
Open
serhiy-storchaka wants to merge 1 commit into
Open
gh-156518: Return the simple case mappings defined by the UCD#156519serhiy-storchaka wants to merge 1 commit into
serhiy-storchaka wants to merge 1 commit into
Conversation
Py_UNICODE_TOUPPER(), Py_UNICODE_TOLOWER() and Py_UNICODE_TOTITLE() returned the first character of the full mapping when it is longer than one character, which the UCD does not define as a simple mapping: the simple uppercase of LATIN SMALL LETTER SHARP S is itself, not "S". Return the character itself in that case, or the simple mapping which Tools/unicode/makeunicodedata.py does not store: the Greek letters with ypogegrammeni are uppercased to the letter with prosgegrammeni, and LATIN CAPITAL LETTER I WITH DOT ABOVE is lowercased to LATIN SMALL LETTER I. _sre.unicode_iscased() no longer infers casedness from the simple mappings, which a cased character can lack, but compares the full mappings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The three macros now return the character itself when its full case mapping is longer than one character, and the simple mapping which
Tools/unicode/makeunicodedata.pydoes not store for the 27 Greek letters with ypogegrammeni and for LATIN CAPITAL LETTER I WITH DOT ABOVE.They agree now with fields 12, 13 and 14 of
UnicodeData.txtfor every code point._sre.unicode_iscased()compared the simple mappings with the character, which no longer detects a cased character having none, so it compares the full mappings instead. It is unchanged for every code point.