Fix embedded Python executable on Windows 3.10 - #820
Conversation
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
|
Good PR but Python 3.10 reaches EOL in a month (https://devguide.python.org/versions/) at which point PythonCall will also drop support for it. So I'm inclined to leave this buggy behaviour and just advise downstream users to require Python >=3.11. |
|
Note that the October 2026 EOL is approximate; there isn’t actually a specific EOL day yet. As of right now, Python 3.10 is still a valid upstream-supported version. For this PR, I have it gated to only be active on Python 3.10: if Sys.iswindows() && startswith(Base.unsafe_string(Py_GetVersion()), "3.10.")So it will become inactive on future Python versions anyway. Could we please merge this? I have some PySR users on older Python versions (I think academic environments often lag), so I would prefer not to drop compatibility with a Python version before its upstream EOL. This issue in particular is hitting my CI right now, since I currently test against the oldest still-supported Python version. I would monkey-patch around it if I could, which is what I did in PySR for #814 -> astroautomata/PySR#1362 while waiting on the new juliacall release. But sadly this one is internal enough that I am unable to patch on my side. |
|
Yeah no problem, I'll just make a note to remove the dead code when I drop support for 3.10 in a month or so. AFK still but will release this week hopefully. Thanks for the PR. |
Windows CPython 3.10 derives
sys.executablefrom the host process even afterPy_SetProgramName. When Julia embeds Python, that leavessys.executablepointing atjulia.exe, so Python subprocess launches use the wrong program.Set the executable path before
Py_InitializeExusing CPython 3.10's exported_Py_SetProgramFullPath. The private API call is restricted to Windows Python 3.10; other versions and already-initialized interpreters keep their existing initialization path.The regression starts a fresh Julia process with the configuration inherited from JuliaCall, checks its executable against the parent Python, and launches a Python subprocess through it. The test does not set the interpreter environment variable. Existing tests are unchanged.
Verification
Before the fix, the regression failed on Windows Python 3.10 with both official CPython and Conda. Windows Python 3.14 and Ubuntu Python 3.10 passed.
With the fix, all four configurations pass.
This addresses the remaining worker-initialization failure exposed by astroautomata/PySR#1366.
The full upstream Tests workflow also passed all 16 Julia/Python jobs. Benchmark execution completed, but its job failed while posting the result comment with HTTP 403; the reporting fix is tracked separately in #819.
PySR integration
The controlled Windows comparison tests the released dependencies before and after PySR #1362, a JuliaCall 0.9.34 control, and this upstream pair with the PySR compatibility assignment removed.
Before #1362, a Python 3.10 parent gets a separate CondaPkg Python 3.14 worker. After #1362, the worker uses Python 3.10 but its
sys.executablelaunches Julia and fails withERROR: unknown option `-c`. The 0.9.34 control reproduces the same Windows 3.10 initialization defect.With this source build and no PySR compatibility assignment, both import orders pass the unchanged worker regression and launch Python successfully. The complete
python -m pysr test main,startupsuites pass on both Windows Conda configurations:The original startup-test file has the same SHA-256 in every configuration. No PySR tests or assertions were changed.
A coordinated release containing #814 and this fix is required before PySR can remove its downstream workaround and require the corrected dependency. This validation uses upstream source builds; the published-package canary continues to test released packages.