A worker that needs its task token to finish starting cannot be dispatched - #760
Conversation
There was a problem hiding this comment.
🟢 Approved
The ordering fix is centralized across all spawn drivers and covered by focused regression tests.
Pull request overview
Moves token-socket arming to worker creation, preventing MCP-dependent handshakes from deadlocking.
Changes:
- Adds
SessionConfig.Startedand invokes it centrally fromStartWorker. - Arms the dispatcher’s token socket before driver handshakes.
- Adds regression and ordering tests across dispatcher and ACP paths.
[!TIP]
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or rungh pr ready --undo.
Click "Ready for review" or rungh pr readyto reengage.
File summaries
| File | Description |
|---|---|
internal/connector/tokensocket.go |
Documents the revised token window. |
internal/connector/recovery_acp_test.go |
Updates recovery-test ordering rationale. |
internal/connector/driver/worker.go |
Announces workers immediately after startup. |
internal/connector/driver/worker_other.go |
Updates the non-Unix signature. |
internal/connector/driver/drivertest/drivertest.go |
Adapts test worker startup. |
internal/connector/driver/driver.go |
Defines the new driver invariant and callback. |
internal/connector/driver/driver_test.go |
Updates worker tests for SessionConfig. |
internal/connector/driver/codex/codex.go |
Passes full configuration to StartWorker. |
internal/connector/driver/claude/claude.go |
Passes full configuration to StartWorker. |
internal/connector/driver/announce_test.go |
Tests centralized worker announcements. |
internal/connector/driver/acp/compat_test.go |
Arms the compatibility socket during startup. |
internal/connector/driver/acp/arming_test.go |
Tests ACP handshake ordering. |
internal/connector/driver/acp/acp.go |
Passes full configuration to StartWorker. |
internal/connector/dispatcher.go |
Arms sockets through the startup callback. |
internal/connector/dispatcher_test.go |
Models pre-session handshakes in the fake driver. |
internal/connector/dispatcher_arming_test.go |
Reproduces and verifies the deadlock fix. |
internal/commands/connect_worker_unix_test.go |
Adapts worker test setup. |
Review details
- Files reviewed: 17/17 changed files
- Comments generated: 0
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…shake returns The socket accepts nothing until AllowGroup names the worker's process group, and the connector named it only once Driver.NewSession had returned. For the ACP driver the whole handshake runs inside NewSession, the adapter's own /mcp read-back — a real prompt turn — included. An adapter that will not finish that handshake until its MCP servers have connected therefore waits for a token the connector cannot hand over until the handshake blocking it has finished, and both sides sit there until the bridge's 30-second dial or the driver's two-minute handshake runs out. A fake that bound at session/new hit exactly this while the Codex harness row was being built. SessionConfig.Started announces the worker's process the moment the fork returns. It is called from driver.StartWorker, the one place every driver starts its agent, so no driver can forget it; the dispatcher wires it to AllowGroup. Written up as driver invariant 7. The dispatcher still arms on Session.Process() afterwards as a backstop for a driver that announces nothing — AllowGroup takes only the first group it is given, so where the driver did announce, that changes nothing. What the earlier arming widens is the token window, and that is the one thing that relied on the old order: it starts at AllowGroup, so the handshake is inside it now. The card-23 review's concern was a launcher that takes its time, which is preserved — the arming is after the fork. The window is no smaller for the case it is for, since an agent starts its MCP servers during its handshake and their connection is already waiting in the listener's backlog. And it is deliberately not stretched to cover both: the socket is armed for the worker's whole process group, which is the agent's own tree, so a longer armed window is a longer window in which the agent's tools could ask for the token instead of its MCP server. DefaultTokenWindow now says that rather than the old order. The compat check armed on Session.Process() right after NewSession under either ordering, so it passed under both and would not have caught a regression. It arms from cfg.Started now, where the dispatcher does, and fails outright on the old order. Since it cannot run without the pinned adapters and model quota, its verdict is a small object driven through both orderings against the real driver in the ordinary suite (TestTheArmingCheckTellsTheTwoOrderingsApart).
a1ed6ff to
21df8f8
Compare
|
🤖 It cannot see this change. The test-dependency closure of Reproduced on pristine Idle, the same command passes 30 and 40 iterations without a failure. Why it is load-sensitive. The test sets a 200ms server block against a 220ms budget — 20ms of slack over a 200ms sleep. Filing it here rather than fixing it: it is not in this change's territory, and the fix is someone's call about whether that budget should be a real margin or the clock should be injected the way The other failure on that run, |
An agent that will not finish starting until its MCP servers have connected cannot be dispatched. Its server is waiting for the task token, and the connector will not hand the token over until the start it is blocking has returned. Both sides wait out a timeout — the bridge's 30-second dial, or the driver's two-minute handshake — and every dispatch through such an agent fails.
Nothing in the connector prevented this; only adapter behaviour did. It was hit for real while the Codex harness row was being built: a fake that bound at
session/newmade every ACP dispatch fail after 30 seconds.Originally tracked in The ACP handshake needs a token the socket is not yet armed to hand over, which named the hazard rather than fixing it, because changing dispatch ordering on a guess is worse than documenting it. This is the fix, with the guess retired: the deadlock is a failing test first.
The shape of it
The token socket accepts nothing until it is told the worker's process group, and it was told only once the driver had handed back a ready session. For the ACP driver that is the wrong side of a long wait: the whole handshake — including the adapter's own
/mcpread-back, which is a real prompt turn — runs inside the start.So a worker is now announced when its process exists rather than when its session is ready. Every driver starts its agent through one function, so that is where the announcement is made and no driver can forget it. A new driver invariant says so.
What moving it widens, since something did rely on the old order
The token window starts when the socket is armed, so the handshake is inside it now. Three reasons that is right rather than merely tolerable:
The window now means what its own documentation always said it meant: once the worker exists, not once its session is ready.
The compatibility check measured nothing about this
It armed the socket right after the start returned, which is what it did under either ordering, so it passed under both and could not have caught a regression. It now arms where the dispatcher arms, and fails outright on the old order.
That check needs the pinned adapters and model quota, so it cannot prove its own sensitivity in CI. Its verdict is therefore a small object driven through both orderings against the real driver in the ordinary suite: the new one passes, the old one fails with "after NewSession returned", and removing the announcement from production turns the first half red.
Proof
Every test here was watched failing before it passed. The deadlock reproduces in about five seconds with the dispatcher's wiring removed, and in milliseconds with it.