XMLParser._setevents() can use freed items from a mutated list

[Nievesjyl]

Description

XMLParser._setevents() converts events_to_report with PySequence_Fast() and iterates it using PySequence_Fast_GET_SIZE and PySequence_Fast_GET_ITEM. An exact list is returned by reference rather than copied, and the item macro yields an unowned pointer. Concurrent clear or refill can invalidate the saved bound, free an item before its type check, or release its string storage before comparison and Py_NewRef().

Observed Behavior

Each consumer used a private XML parser, TreeBuilder, and queue while one exact event-name list was shared with a mutator. The free-threaded ASan build crashed in _elementtree_XMLParser__setevents_impl() at the first item-use region. The same executable completed normally with the GIL enabled.

Affected Version

CPython 3.14.7 at commit 823f0323ee6ec1402088b73bce1a38473cac36dc, built with --disable-gil --with-address-sanitizer.

Reproduction

ASAN_OPTIONS=detect_leaks=0:halt_on_error=1 PYTHON_GIL=0 python3.14 poc/reproduce.py --seconds 10

Setting PYTHON_GIL=1 provides the GIL-enabled control.

PoC Source Code

poc/reproduce.py

#!/usr/bin/env python3
"""Race XMLParser._setevents() with mutation of its exact-list input."""

import argparse
import threading
import time
import xml.etree.ElementTree as ET


NAMES = (b"start", b"end", b"start-ns", b"end-ns", b"comment", b"pi")


def fresh_events() -> list[bytes]:
    return [bytes(bytearray(NAMES[i % len(NAMES)])) for i in range(256)]


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--seconds", type=float, default=10.0)
    parser.add_argument("--consumers", type=int, default=10)
    args = parser.parse_args()

    shared = fresh_events()
    stop = threading.Event()

    def mutate() -> None:
        while not stop.is_set():
            shared.clear()
            shared.extend(fresh_events())

    def consume() -> None:
        xml_parser = ET.XMLParser()
        queue: list[object] = []
        while not stop.is_set():
            try:
                xml_parser._setevents(queue, shared)
            except ValueError:
                pass

    threads = [threading.Thread(target=mutate)]
    threads.extend(threading.Thread(target=consume) for _ in range(args.consumers))
    for thread in threads:
        thread.start()
    time.sleep(args.seconds)
    stop.set()
    for thread in threads:
        thread.join()
    print("completed")


if __name__ == "__main__":
    main()

[ZeroIntensity]

Refer to #157124 (comment)