Skip to content

viewer: expose depth snapshot channel - #646

Closed
thomas-yanxin wants to merge 1 commit into
pascalorg:mainfrom
thomas-yanxin:codex/expose-depth-snapshot-channel
Closed

viewer: expose depth snapshot channel#646
thomas-yanxin wants to merge 1 commit into
pascalorg:mainfrom
thomas-yanxin:codex/expose-depth-snapshot-channel

Conversation

@thomas-yanxin

@thomas-yanxin thomas-yanxin commented Aug 14, 2026

Copy link
Copy Markdown

What does this PR do?

Adds an optional channel: 'rgb' | 'depth' to SnapshotPipeline.capture, defaulting to the existing RGB behavior.

The depth channel reuses the snapshot pipeline's existing scene pass, render target, crop, GPU readback, and encoder. It emits near geometry as white and far/background pixels as black. Orthographic cameras use the hardware depth attachment directly because Three's current getLinearDepthNode() always applies perspective conversion.

This makes aligned RGB/depth evidence available to viewer consumers without maintaining a second capture implementation.

How to test

  1. From packages/viewer, run bun run build.
  2. From packages/viewer, run bun run test and confirm all 102 tests pass.
  3. Run bunx biome check packages/viewer/src/lib/snapshot-pipeline.ts packages/viewer/src/lib/snapshot-pipeline.test.ts packages/viewer/src/index.ts from the repository root.

Screenshots / screen recording

Not applicable: this adds a capture API channel and does not change the live viewer UI.

Checklist

  • I've tested this locally with the viewer build and full viewer test suite
  • My code follows the existing code style
  • I've documented the depth encoding on the public capture-channel type
  • This PR targets the main branch

Note

Low Risk
Additive API with default RGB behavior unchanged; depth path is isolated but touches the shared snapshot render/dispose flow.

Overview
Adds an optional channel: 'rgb' | 'depth' argument to SnapshotPipeline.capture (default 'rgb'), so consumers can request aligned depth evidence through the same render target, crop modes, and WebP readback path as existing snapshots.

For 'depth', a dedicated TSL pipeline outputs inverted depth (near white, far/background black) with color transforms disabled. snapshotCameraDepthNode picks raw depth for orthographic cameras and linear depth for perspective, avoiding Three’s perspective-only getLinearDepthNode() on ortho shots. The public SnapshotCaptureChannel type documents the encoding; snapshotCameraDepthNode is covered by a small unit test.

Reviewed by Cursor Bugbot for commit 3671362. Bugbot is set up for automated code reviews on this repo. Configure here.

@Aymericr

Copy link
Copy Markdown
Contributor

This is the better-shaped of your two, and one part of it is a genuinely good catch that I want to acknowledge properly: you're right that getLinearDepthNode() can't be used for an orthographic camera. It routes through getViewZNode(), which applies perspectiveDepthToViewZ unconditionally — three's own source has a // TODO: just if ( builder.camera.isPerspectiveCamera ) sitting on that line. Reading the library instead of trusting the method name is the right habit.

I'm still going to close it, for two reasons.

The first is the same as on #644: nothing consumes a depth capture. Both snapshot callers want one WebP for a thumbnail, and no plan on our side asks for depth. Adding channel: 'depth' to a published @pascal-app/viewer API means committing to it before we know what shape a consumer actually needs.

The second is specific to this diff, and it's the one I'd fix first if a consumer appeared. The depth blob goes through the unchanged SNAPSHOT_MIME, so it's encoded as WebP at quality 0.9 — lossy, chroma-subsampled, 8-bit. As a thumbnail that's the right call; as the "aligned depth evidence" this PR is for, the compression destroys the thing being measured. Your own #644 got this right by sending data channels to PNG. Even with PNG, 8 bits across near=0.1far=1000 is coarse enough that a consumer needs to say what precision it wants — which is another argument for the consumer landing first.

Two smaller notes:

  • depthPipeline is constructed eagerly, so both thumbnail callers now compile a second pipeline they never render. viewer: expose snapshot render channels #644's lazy allocation was the better pattern.
  • snapshotCameraDepthNode branches at graph-build time on the constructor's camera, and both callers pass a PerspectiveCamera — so the ortho path, and the reason it exists, can't currently be reached. The test that covers it passes {} sentinels and asserts identity, so it checks the ternary rather than the depth. When there's a real ortho capture (2D stage cameras, feat(editor): add synchronized 2D viewer modes #672), that branch also needs to be chosen per capture rather than once, since applyEnvironment takes a camera each time.

None of this is wasted. If you're interested in the consumer end, I'd rather talk about that first — open an issue describing what you want depth for and what precision it needs, and we can design the channel around it instead of guessing. The ortho finding should go in that issue; it's worth not losing.

@Aymericr Aymericr closed this Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants