Skip to content

fix: wait for output splitters before completing PythonShell - #335

Open
lonecoding wants to merge 1 commit into
extrabacon:masterfrom
lonecoding:fix/wait-output-splitters
Open

fix: wait for output splitters before completing PythonShell#335
lonecoding wants to merge 1 commit into
extrabacon:masterfrom
lonecoding:fix/wait-output-splitters

Conversation

@lonecoding

Copy link
Copy Markdown

When a custom stdout or stderr splitter processes data or flushes asynchronously, PythonShell can emit close and invoke its end() callback before the final output event. A caller that reads its collected messages in the completion callback therefore receives incomplete output. The current order can be close → end callback → message("tail"); it should be message("tail") → close → end callback.

Track completion of each active splitter's readable side alongside the existing source-stream and process-exit checks. Keep waiting for the original streams as well, so a splitter that ends early does not truncate the stderr collected for process errors. No public API, dependency, or newline parsing changes.

Regression tests use real Python processes and a child-process close barrier, without sleeps or mocked process events. They cover stdout/stderr, asynchronous _transform/_flush, exit codes 0/7, and complete stderr collection when a splitter ends before its source.

Validation on macOS arm64, Node.js 22.23.1 and Python 3.9.6:

  • Unmodified master 1cf1bfcac6ccde265bf0ac0e422fc54379941ed0: 43 existing tests pass.
  • New asynchronous-output regressions before the fix: 8 fail with output arriving after completion.
  • Focused suite after the fix: 9 pass.
  • Full npm test: 52 pass.
  • Prettier and git diff --check: pass; generated TypeScript declarations match master.

For the full suite, PYTHONPYCACHEPREFIX was set to a writable local directory for Python's syntax-check cache. Windows and Linux were not run locally.

Related work checked: #332 changes newline splitting; #329 flushes splitters in its timeout path. The eight asynchronous-output regressions still fail on both current PR heads without this fix. This PR addresses normal process completion, including nonzero exit codes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant