refactor(grpc-web): top-level pyodide imports, [grpc-web] extra, in-Pyodide unit tests

[g-despot]

Motivation

Review feedback on #2142 from @tsmith023: the web package kept itself importable on CPython through in-function pyodide imports and force= escape hatches — an import hack applied to the logical implementation. His suggestion: define the package coherently for its one environment (Emscripten/Pyodide) with top-level imports, and make weaviate-client[grpc-web] the documented install path. This branch applies that approach on top of feat/grpc-web-wasm so the delta can be reviewed on its own; if we agree it's the right shape, it gets folded into #2142.

Approach

Moving the pyodide imports to module scope has consequences, and this PR takes all of them rather than half-committing:

1. The package is only importable under Emscripten. _sender.py and _httpx_fetch.py import pyodide.http.pyfetch at the top of the file. The seams that existed only to keep CPython imports working — install(force=), install_fetch_transport(force=), make_httpx_sender — are deleted, and the package __init__ now imports its submodules plainly instead of splitting them around _bootstrap().
2. The unit tests therefore run as pytest inside Pyodide. ci/pyodide-e2e/units.mjs mounts packages/web/tests/ into a real Pyodide interpreter under Node and runs the bundled pytest over it — standard collection, fixtures, parametrize, assertion rewriting. Async tests execute on Pyodide's event loop through JSPI stack switching: pytest's runner is synchronous, so run_until_complete needs --experimental-wasm-jspi on Node plus a callPromising() entrypoint. Without the flag the run fails loudly at startup — it can never produce a false green. pytest's exit code crosses to JS as a plain return value (empty collection is exit 5 and fails the step). conftest.py keeps CPython pytest from collecting the directory. No running Weaviate is needed — everything goes through fake senders / a fake pyfetch.
3. [grpc-web] becomes the install path, with lockstep enforced by the resolver. The extra carries a sys_platform == "emscripten" marker: under Pyodide it pulls the companion, on CPython it is a no-op — one requirements list works everywhere. Both packages derive their version from the same git tag (setuptools_scm), and packages/web/setup.py injects weaviate-client==<version> into the companion's requirements at build time, so a mismatched pair can never resolve — the two packages share private contracts (error-string markers, exception constants) that make skew unsafe.

The base client's import-hook branches (missing companion → install hint naming the extra, broken companion → its own error, real-grpc-present → silent fall-through) are plain CPython logic and are pinned by subprocess tests in test/test_wasm_compat.py across the 3.10–3.14 matrix; the bootstrap scenario in units.mjs keeps the one path that needs real Pyodide, micropip and the wheels.

CI: the grpc-web-tests job (CPython 3.10–3.14 matrix) is deleted — it tested an environment the package no longer supports — and the pyodide-e2e job gains a units step (with the JSPI flag) ahead of the e2e run, plus an assertion that both built wheels carry the same version. The release job builds, version-asserts and publishes both packages.

Key areas for review

• packages/web/src/weaviate_client_web/__init__.py — submodule imports now run before _bootstrap(). Safe because _channel/_httpx_fetch take their grpc base classes from ._shim directly, never via sys.modules["grpc"], but worth a careful look.
• packages/web/setup.py — the build-time == pin. Consequence: every tag must publish both packages (a base-only release would leave the extra unresolvable once a pinned companion is on PyPI). The release job now does exactly that: it builds both distributions, asserts equal versions on the artifacts it actually uploads, and publishes both. The companion is wheel-only — its setup.py resolves the version from git tags, which an unpacked sdist would not have. One thing CI cannot verify: PYPI_API_TOKEN must be allowed to create the new weaviate-client-web project on first publish (a project-scoped token cannot; use an account-scoped token or register a pending publisher on PyPI beforehand).
• ci/pyodide-e2e/units.mjs — the JSPI/callPromising() mechanics, and the pinned pytest==8.4.2 / pytest-asyncio==0.25.3 installed via micropip. That pair is metadata-coherent (pytest-asyncio 0.25.3 declares pytest<9,>=8.2); pytest-asyncio 1.x is not an option — its asyncio.Runner-based execution fails under JSPI stack switching, verified empirically.
• test/test_wasm_compat.py — the three subprocess-based import-hook tests, including the grpc-present fall-through that only CPython can exercise (no grpcio wheel exists for Emscripten).

Risks and mitigations

• --experimental-wasm-jspi on pinned Node 22: an experimental V8 flag in one CI step we fully control (the e2e step needs no flag; production browser code never touches JSPI). A missing or broken flag fails loudly at startup, never silently. Verified stable across 5 consecutive local runs; Node and Pyodide are both version-pinned.
• Losing the 3.10–3.14 CPython matrix for the package tests: intentional — the package only ever executes under Pyodide, whose bundle pins the interpreter (currently CPython 3.14). The base client's hook logic, which does run on all five interpreters, is now tested on all five via test/test_wasm_compat.py.
• Extra unresolvable before publication: under Pyodide, both weaviate-client[grpc-web] and a direct companion install fail until the first release carrying the extra/marker is on PyPI — the same status quo as feat(grpc-web): Pyodide/WASM grpc-web transport for the async client #2142 (wheels built from the branch until release). The README says so explicitly now.
• Dev-loop caveat of the == pin: rebuilding only one wheel on a dirty tree a day apart produces an unresolvable pair (the .dYYYYMMDD suffix diverges). The harness always builds both together.

Testing

node --experimental-wasm-jspi ci/pyodide-e2e/units.mjs dist passes locally: the bootstrap scenario plus 136 pytest cases inside real Pyodide under Node (exit codes verified for pass, fail, and empty-collection runs; 5 consecutive runs stable). The built companion wheel carries Requires-Dist: weaviate-client==<identical version> (inspected in METADATA). On CPython: test/test_wasm_compat.py passes (18 tests, including the three import-hook scenarios) and pytest packages/web/tests collects nothing by design. The e2e suite (run.mjs, Weaviate 1.39.0 over native /v1/grpc-web) is unchanged. ruff, flake8 and pyright pre-commit hooks pass.

Breaking changes

None for released users — weaviate-client-web has never shipped. Within the branch: make_httpx_sender and both force= parameters are gone from the package's public API, and running or importing the package on CPython is no longer supported at all.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Caze9m6PBSfYkt77mMKj2b

[orca-security-eu]

Orca Security Scan Summary

Status	Check	Issues by priority
Passed	Secrets	0   0   0   0	View in Orca

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.77%. Comparing base (61df36c) to head (c3ff780).

Additional details and impacted files

@@                   Coverage Diff                   @@
##           feat/grpc-web-wasm    #2160       +/-   ##
=======================================================
+ Coverage                    0   88.77%   +88.77%     
=======================================================
  Files                       0      305      +305     
  Lines                       0    23787    +23787     
=======================================================
+ Hits                        0    21116    +21116     
- Misses                      0     2671     +2671     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

🟢 Approval recommended

The environment-specific refactor, installation metadata, documentation, tests, and CI changes are coherent with no unresolved correctness issues found.

Pull request overview

Refactors gRPC-Web support to target Pyodide exclusively and moves its unit tests into the actual WASM runtime.

Changes:

• Uses top-level Pyodide imports and removes CPython-only escape hatches.
• Adds the [grpc-web] extra and updates installation guidance.
• Replaces pytest execution with a Pyodide-native runner integrated into CI.

File summaries

File	Description
weaviate/connect/base.py	Updates unsupported-environment guidance.
weaviate/__init__.py	References the new installation extra.
setup.cfg	Adds the platform-gated grpc-web extra.
packages/web/src/weaviate_client_web/__init__.py	Simplifies imports and bootstrap.
packages/web/src/weaviate_client_web/_channel.py	Restricts sender customization wording to tests.
packages/web/src/weaviate_client_web/_httpx_fetch.py	Moves Pyfetch import to module scope.
packages/web/src/weaviate_client_web/_sender.py	Removes the CPython HTTPX sender.
packages/web/src/weaviate_client_web/_shim.py	Removes forced shim installation.
packages/web/tests/conftest.py	Prevents CPython pytest collection.
packages/web/tests/harness.py	Adds lightweight test helpers.
packages/web/tests/runner.py	Adds the async Pyodide test runner.
packages/web/tests/test_framing.py	Migrates framing tests from pytest.
packages/web/tests/test_httpx_fetch.py	Runs fetch tests directly in Pyodide.
packages/web/tests/test_shim_install.py	Tests the real Pyodide shim installation.
packages/web/tests/test_single_import.py	Removes CPython-emulated bootstrap tests.
packages/web/tests/test_transport.py	Converts transport tests to async Pyodide execution.
packages/web/README.md	Documents installation and testing.
ci/pyodide-e2e/units.mjs	Adds unit and bootstrap scenarios.
.github/workflows/main.yaml	Moves package testing into the Pyodide job.

Review details

• Files reviewed: 19/19 changed files
• Comments generated: 0
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

The test dependency pins conflict, and the release job does not publish the newly required companion package.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 2
• Review effort level: Balanced