Skip to content

fix(remote): materialize test suite artifacts against a remote daemon - #2272

Merged
thymikee merged 3 commits into
mainfrom
claude/agent-device-issue-2246-f11dd7
Sep 3, 2026
Merged

fix(remote): materialize test suite artifacts against a remote daemon#2272
thymikee merged 3 commits into
mainfrom
claude/agent-device-issue-2246-f11dd7

Conversation

@thymikee

@thymikee thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member

Summary

Fixes #2246.

Remote agent-device test requests now keep daemon-owned suite artifacts off caller-only paths, transfer the completed suite through the existing tracked-artifact transport, and report the exact caller-local invocation directory for the suite, tests, and failures.

Directory downloads use the existing safe archive extractor. They stage beside the caller's destination, propagate one cancellation signal through download and extraction, clean up before timeout rejection, and publish with one atomic rename. The client-side download owner returns the exact materialized path; generic response materialization no longer reconstructs it independently.

The follow-up reduces the original src/remote/daemon-artifacts.ts growth by extracting its download machinery into the 191-line src/remote/artifact-download.ts. Overall scope is 15 files (+1040/-124), mostly 6 focused test files (+631 lines). Production growth remains below the repository's 700-line escalation threshold, while the shipped bundle is +2.2 kB raw / +784 B gzip on the latest size report.

Validation

  • Planted-red at 99615f818c05a388b7cf1d17fc7285c032857d9e: composed materialization returned the bare caller root instead of <root>/<suite-id>, and directory staging was outside the destination filesystem.
  • pnpm check:affected --run: 484 files and 3,610 tests passed, including build, layering, Fallow, and daemon wire compatibility.
  • HTTP remote-daemon proof on the booted iPhone 17 Pro Simulator at f1bda16b72ad9a141a9f168c1ea1f331e3c5f543: default artifact root 1/1 passed; relative --artifacts-dir relative-artifacts 1/1 passed. Both returned the caller-local <root>/<suite-id> and materialized replay.ad plus result.txt under the per-attempt directory.

…#2246)

`agent-device test` crashed with ENOENT against a remote daemon because the
scheduler resolved `--artifacts-dir` against the caller's `cwd`, sent over the
wire, on the daemon's own filesystem. Mirrors #1802's read-side fix for the
same command: the CLI now redirects `--artifacts-dir` to a temp directory the
daemon owns before the suite runs, and the daemon rewrites every artifact path
in its response back to the caller-local root and registers the suite
directory as one downloadable artifact through the existing screenshot/record
transport, extended here to also support directories via the codebase's
existing safe archive extractor (the archive comes from a remote daemon, a
different trust domain, so a raw `tar` invocation was not enough).
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 2.59 MB 2.59 MB +2.9 kB
JS gzip 871.3 kB 872.8 kB +1.5 kB
npm bundled raw 2.59 MB 2.59 MB +2.9 kB
npm bundled gzip 871.3 kB 872.8 kB +1.5 kB
npm tarball 1.02 MB 1.03 MB +1.4 kB
npm unpacked 3.47 MB 3.47 MB +3.2 kB
npm clean-installed 3.47 MB 3.47 MB +3.2 kB

npm unpacked components

Component Base Current Diff
JS / dist source 2.75 MB 2.75 MB +3.3 kB
Apple runner source/project 544.4 kB 544.4 kB 0 B
Apple snapshot presentation source 33.8 kB 33.8 kB 0 B
macOS helper source 54.8 kB 54.8 kB 0 B
Android helper artifacts 43.3 kB 43.3 kB 0 B
Other package files 46.4 kB 46.3 kB -113 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 25.6 ms 26.5 ms +0.9 ms
CLI --help 65.6 ms 66.9 ms +1.3 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/session2.js +573 B +162 B
dist/src/daemon-client-lifecycle.js -552 B -129 B
dist/src/internal/daemon.js +66 B +14 B

Top changed packed files

Packed file Base Current Diff
dist/src/artifact-download.js 0 B 2.6 kB +2.6 kB
dist/src/http-health.js 1.9 kB 655 B -1.3 kB
dist/src/transport.js 0 B 1.3 kB +1.3 kB
dist/src/session2.js 217.7 kB 218.3 kB +573 B
dist/src/daemon-client-lifecycle.js 42.4 kB 41.9 kB -552 B
dist/src/client-types.d.ts 57.5 kB 57.9 kB +412 B
dist/src/byte-limit-stream.js 0 B 398 B +398 B
dist/src/archive.js 8.2 kB 7.9 kB -272 B
package.json 20.0 kB 19.9 kB -113 B
dist/src/internal/daemon.js 107.6 kB 107.6 kB +66 B

@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

BLOCKED at exact head 99615f8. Two production composition bugs remain despite green CI.

First, attachRemoteReplayTestArtifacts correctly reports <clientRoot>/<suiteInvocationId> but registers localPath: clientRoot; materializeRemoteArtifacts then unconditionally overwrites data.artifactsDir with that bare root. Add a real composed materialization regression proving the final response and extracted replay.ad/result.txt resolve under the invocation directory, with planted-red evidence.

Second, staging under os.tmpdir() can hit EXDEV; the fallback recursively copies directly into the final caller directory, so failure/cancellation/timeout can leave partial or later background writes. Stage on the destination filesystem, propagate cancellation through extraction/copy, then atomically rename; add planted-red EXDEV/partial-copy and timeout coverage.

Also provide the linked issue’s real remote-daemon/device evidence for default and relative --artifacts-dir.

Size review: headline +776 is ~275 production LOC, below the 700-line trigger, and package growth is +2.5 kB. Still, the path bug shows ownership is split between daemon rewriting and generic client materialization. Please itemize growth and explain why one client-side owner returning both download destination and final reported path was rejected.

@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Resolved the blocking review at f1bda16b72ad9a141a9f168c1ea1f331e3c5f543.

  • The composed regression now drives attachRemoteReplayTestArtifacts through the real /artifacts HTTP route and client materializer, then proves data.artifactsDir, artifacts[].localPath, replay.ad, and result.txt all land under <client-root>/<suite-id>. Planted red on 99615f818c: expected the suite directory, received the bare root.
  • Directory downloads now stage in .agent-device-download-* beneath the destination root, pass one abort signal through the response pipeline and safe extractor, await cleanup before timeout rejection, and publish only via same-filesystem rename. The old EXDEV recursive-copy fallback is gone. Planted red on 99615f818c: no destination-side stage existed during an active download; the new timeout regression also proves it is gone before rejection returns.
  • The owning client function returns the exact materialized path. materializeRemoteArtifacts consumes that result rather than independently reconstructing the final path. Download machinery moved to src/remote/artifact-download.ts; daemon-artifacts.ts is 439 lines rather than growing to 637.

Local gate:

pnpm check:affected --run
484 test files passed; 3,610 tests passed; build, layering, Fallow, and daemon wire compatibility passed.

Live acceptance used the pushed source checkout, an isolated HTTP proxy/daemon, and the discovered booted iPhone 17 Pro Simulator (F7D6F9A4-4FCC-4DD7-AC0B-3280C9319CB9). The temporary auth token is redacted:

pnpm ad proxy --port 4317 --state-dir /tmp/agent-device-pr2272.u8WVa1/host-state --daemon-auth-token <temporary-token>

AGENT_DEVICE_DAEMON_AUTH_TOKEN=<temporary-token> node bin/agent-device.mjs test /tmp/agent-device-pr2272.u8WVa1/work/pr2272-remote-proof.ad --platform ios --device "iPhone 17 Pro" --remote-config /tmp/agent-device-pr2272.u8WVa1/direct-remote.json --json
# 1/1 passed; artifactsDir=/private/tmp/agent-device-pr2272.u8WVa1/direct-client-cwd/.agent-device/test-artifacts/f8598a66e7497c85

AGENT_DEVICE_DAEMON_AUTH_TOKEN=<temporary-token> node bin/agent-device.mjs test /tmp/agent-device-pr2272.u8WVa1/work/pr2272-remote-proof.ad --platform ios --device "iPhone 17 Pro" --artifacts-dir relative-artifacts --remote-config /tmp/agent-device-pr2272.u8WVa1/direct-remote.json --json
# 1/1 passed; artifactsDir=/private/tmp/agent-device-pr2272.u8WVa1/direct-client-cwd/relative-artifacts/c4545591309060f7

Both invocation directories contained pr2272-remote-proof.ad/attempt-1/replay.ad and result.txt. session list was empty afterward; the temporary proxy profile was disconnected and the proxy stopped.

CI note: the first Android smoke attempt timed out waiting for an unrelated landscape snapshot with readableCaptures: 0; the failed job has been rerun on the same head.

@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Follow-up exact head 53d30bcf78c2c6d0a2fd63bbfe963766d7392e7f: Coverage correctly caught one extra eager CLI module (383 vs 382). downloadRemoteArtifact now function-scope imports the extracted downloader, so the routine CLI closure is back at the merge-base budget. Focused closure + wire + artifact tests: 455/455 passed. Re-run pnpm check:affected --run: 484 files / 3,610 tests passed, plus build, layering, Fallow, and wire compatibility. Fresh exact-head CI is running.

@thymikee
thymikee merged commit 7bf8d8c into main Sep 3, 2026
18 of 19 checks passed
@thymikee
thymikee deleted the claude/agent-device-issue-2246-f11dd7 branch September 3, 2026 18:25
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-03 18:25 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

agent-device test cannot make its results directory and stops before the first flow

1 participant