Element APIs can use freed children from a mutated input list

[Nievesjyl]

Description

Element.extend() and slice assignment call PySequence_Fast() on caller input and then use raw size and item macros. For an exact list, this aliases the same mutable list instead of creating a snapshot. Element.__setstate__() similarly saves a list size before later borrowed item loads. Concurrent list clearing or replacement can invalidate the saved bound, item array, or child lifetime before type checks and reference increments.

Observed Behavior

Each consumer owned a private destination Element while one shared input list was repeatedly cleared and refilled. The extend mode crashed in Py_INCREF() from _elementtree_Element_extend_impl; slice assignment crashed in element_ass_subscr; and setstate crashed in element_setstate_from_attributes. All three modes completed with the same executable under the GIL.

Affected Version

CPython 3.14.7 at commit 823f0323ee6ec1402088b73bce1a38473cac36dc, built with --disable-gil --with-address-sanitizer.

Reproduction

ASAN_OPTIONS=detect_leaks=0:halt_on_error=1 PYTHON_GIL=0 python3.14 poc/reproduce.py --mode extend --seconds 12

Valid modes are extend, slice, and setstate. Setting PYTHON_GIL=1 provides the GIL-enabled control.

PoC Source Code

poc/reproduce.py

#!/usr/bin/env python3
"""Race Element APIs with mutation of their shared exact-list input."""

import argparse
import threading
import time
import xml.etree.ElementTree as ET


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--seconds", type=float, default=12.0)
    parser.add_argument("--consumers", type=int, default=10)
    parser.add_argument(
        "--mode", choices=("extend", "slice", "setstate"), default="extend"
    )
    args = parser.parse_args()

    shared = [ET.Element("item") for _ in range(256)]
    state = {
        "tag": "root",
        "_children": shared,
        "attrib": {},
        "text": None,
        "tail": None,
    }
    stop = threading.Event()
    counts = [0] * (args.consumers + 1)

    def mutate_list() -> None:
        generation = 0
        while not stop.is_set():
            shared.clear()
            shared.extend(ET.Element(str(generation)) for _ in range(256))
            generation += 1
            counts[-1] += 1

    def consume(slot: int) -> None:
        local = ET.Element("local")
        while not stop.is_set():
            if args.mode == "extend":
                local.extend(shared)
                local.clear()
            elif args.mode == "slice":
                local[:] = shared
            else:
                local.__setstate__(state)
            counts[slot] += 1

    threads = [threading.Thread(target=mutate_list)]
    threads.extend(
        threading.Thread(target=consume, args=(i,)) for i in range(args.consumers)
    )
    for thread in threads:
        thread.start()
    time.sleep(args.seconds)
    stop.set()
    for thread in threads:
        thread.join()
    print(
        {
            "mode": args.mode,
            "consumes": sum(counts[:-1]),
            "mutations": counts[-1],
        }
    )


if __name__ == "__main__":
    main()

[ZeroIntensity]

Refer to #157124 (comment)