gh-124697: Represent inlined comprehensions as subscopes in the symbol table

[iritkatriel]

Resolves #124697

Inlined comprehensions are now represented in the symbol table as block of a new type InlinedComprehensionBlock, which is a subscope of the enclosing scope (not a separate compilation unit).

This moves the complexity of compiling inlined comprehensions from codegen to the symbol table
construction.

It removes the smell of the compiler modifying the symbol table in codegen.

• Issue: refactor the implementation of inlined comprehensions #124697

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #34383228 | 📁 Comparing 3515cc0 against main (d557d64)

  🔍 Preview build  

22 files changed · ± 22 modified

± Modified

• glossary.html
• howto/curses.html
• library/atexit.html
• library/curses.html
• library/functions.html
• library/inspect.html
• library/pyexpat.html
• library/stdtypes.html
• library/symtable.html
• library/tomllib.html
• and 12 more...

[carljm]

This looks great! Thank you for working on this ❤️

[JelleZijlstra]

Very nice! This also fixes #121377 and #156664, but doesn't add tests for their reproducers. Can you add those tests?

I have a PR for the latter almost ready at #156691, but I'm happy to retire it in favor of this PR. @carljm @iritkatriel what do you think about backporting either my change or others? This PR feels a bit much to backport, but we may want to backport fixes for some of the bugs. 156691 is also a somewhat invasive change though.

Codex found that this now fails:

import sys

def outer(x):
    def inner():
        return [(lambda: x, dict(**sys._getframe().f_locals))
                for x in x]
    return inner()

outer([1])

With TypeError: dict() got multiple values for keyword argument 'x'.. #156691 has a larger change to framelocalsproxy to deal with this sort of thing; you may want to incorporate its approach.

[JelleZijlstra]

Another relevant bug is #156091 but that one still crashes under this PR.

[carljm]

I'd be cautious about backporting an invasive fix for #156664. It's clearly a rarely-encountered edge case, given that nobody other than PyPy's test suite discovered the bug since 3.12, and the risk of introducing new bugs in an invasive fix seems high (particularly without this refactor in place.) I think "better the bugs you know than the ones you don't" applies in this case.