Summary
A syntactically valid Python file (passes python -m py_compile on 3.12–3.14)
is dropped from analysis with the generic diagnostic:
A parse error occurred while processing <file>, and as a result this file
could not be analyzed. Check the syntax of the file using the
python -m py_compile command and correct any invalid syntax.
The trigger needs three ingredients in one file:
- any PEP 695 construct (
type X = int, or def f[T: str](...)),
- a string literal containing the emoji variation selector U+FE0F
(e.g. "⚠️" = U+26A0 U+FE0F),
- a
%-format directive (%s) in that same literal.
With (1) and (2) but not (3), the extractor instead logs a non-fatal
value-level warning that reveals the underlying defect — it re-serializes the
selector as a Rust-style escape and then fails to re-parse it as a Python
escape:
[WARN] Error '(unicode error) 'unicodeescape' codec can't decode bytes in
position 12-13: truncated \uXXXX escape' while parsing value '"\"⚠\u{fe0f}\""'
With (3), the corrupted snippet reaches a %-formatting call inside the error
reporter and the extraction of the file fails outright:
[WARN] Exception not enough arguments for format string in tsg-python while
parsing <file> in <file>
Minimal reproducer
type X = int
import logging
logging.getLogger().warning("⚠️ problem %s: %s", "a", "b")codeql database create db --language=python --source-root=repro --overwrite
# → [WARN] Exception not enough arguments for format string in tsg-python ...
# → diagnostic: "A parse error occurred while processing ..."
Negative controls (each is clean):
- remove the
type alias (no PEP 695 in file) → clean
⚠ alone (U+26A0 without U+FE0F) → cleancafé, ✓, → (2- and 3-byte chars), 💾 (4-byte astral) → clean%s without the selector → clean- write the selector as an escape:
"\u26a0\ufe0f problem %s" → clean
So the fault is specific to a raw U+FE0F in the literal, and PEP 695 syntax
anywhere in the file switches the literal-processing path onto the affected
code.
Impact
Any repository combining PEP 695 syntax with emoji-bearing %-style log
strings loses whole files from Python analysis, and the only signal is the
generic "check your syntax" diagnostic pointing at <file>#L0C0 — which sends
users to py_compile, which passes. In our case a ~22k-line file had been
silently unanalyzed for months.
Environment
- CodeQL CLI 2.26.4 (Homebrew), macOS (Apple silicon) — reproduces locally
- Also reproduces in GitHub code scanning default setup (observed on
github.com Actions runners, same two log lines)
- Source file encoding UTF-8;
python -m py_compile passes on CPython
3.12/3.13/3.14
Summary
A syntactically valid Python file (passes
python -m py_compileon 3.12–3.14)is dropped from analysis with the generic diagnostic:
The trigger needs three ingredients in one file:
type X = int, ordef f[T: str](...)),(e.g.
"⚠️"= U+26A0 U+FE0F),%-format directive (%s) in that same literal.With (1) and (2) but not (3), the extractor instead logs a non-fatal
value-level warning that reveals the underlying defect — it re-serializes the
selector as a Rust-style escape and then fails to re-parse it as a Python
escape:
With (3), the corrupted snippet reaches a
%-formatting call inside the errorreporter and the extraction of the file fails outright:
Minimal reproducer
Negative controls (each is clean):
typealias (no PEP 695 in file) → clean⚠alone (U+26A0 without U+FE0F) → cleancafé,✓,→(2- and 3-byte chars),💾(4-byte astral) → clean%swithout the selector → clean"\u26a0\ufe0f problem %s"→ cleanSo the fault is specific to a raw U+FE0F in the literal, and PEP 695 syntax
anywhere in the file switches the literal-processing path onto the affected
code.
Impact
Any repository combining PEP 695 syntax with emoji-bearing
%-style logstrings loses whole files from Python analysis, and the only signal is the
generic "check your syntax" diagnostic pointing at
<file>#L0C0— which sendsusers to
py_compile, which passes. In our case a ~22k-line file had beensilently unanalyzed for months.
Environment
github.com Actions runners, same two log lines)
python -m py_compilepasses on CPython3.12/3.13/3.14