Skip to content

fix(drivers): require admission labels for external resources - #3538

Open
drew wants to merge 2 commits into
mainfrom
codex/require-external-resource-admission-labels
Open

drew wants to merge 2 commits into
mainfrom
codex/require-external-resource-admission-labels

Conversation

@drew

@drew drew commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

Require external resources attached to sandboxes to carry explicit operator-controlled admission labels. This creates a consistent application-level boundary across compute drivers while still allowing operators to customize or disable the policy when needed.

Caller-provided driver config is also disabled by default and requires an explicit gateway opt-in.

Changes

  • Add a shared resource_admission configuration with these defaults:
    • openshell.ai/sandbox-attachable = "true"
    • openshell.ai/workspace = "${workspace}"
  • Disable caller-provided driver config by default with allow_driver_config = false.
  • Require admission labels for Kubernetes PVCs, Secrets, ConfigMaps, RuntimeClasses, PriorityClasses, and image-pull Secrets.
  • Apply workspace labels to data-bearing resources while allowing shared infrastructure to use the fixed approval label.
  • Evaluate Kubernetes resources from the OpenShell-owned Pod template rather than the mutated live Pod.
  • Admit managed-workspace image-pull Secrets at their source before copying them with gateway ownership metadata.
  • Require labels on Docker and Podman named volumes.
  • Reject external host resources that do not have a trusted label resolver while admission is enabled.
  • Revalidate admitted resource identities during sandbox lifecycle operations.
  • Keep GPU device attachments exempt from label admission for now.
  • Document configuration, migration behavior, and driver-specific examples.

Example Kubernetes resource:

metadata:
  labels:
    openshell.ai/sandbox-attachable: "true"
    openshell.ai/workspace: "team-a"

Example Docker volume:

docker volume create \
  --label openshell.ai/sandbox-attachable=true \
  --label openshell.ai/workspace=team-a \
  team-a-data

Example Podman volume:

podman volume create \
  --label openshell.ai/sandbox-attachable=true \
  --label openshell.ai/workspace=team-a \
  team-a-data

Gateway configuration

Admission is enabled by default. The default Kubernetes configuration is equivalent to:

[openshell.drivers.kubernetes]
allow_driver_config = false

[openshell.drivers.kubernetes.resource_admission]
enabled = true

[openshell.drivers.kubernetes.resource_admission.required_labels]
"openshell.ai/sandbox-attachable" = "true"
"openshell.ai/workspace" = "${workspace}"

The same configuration shape is available for Docker, Podman, VM, and MXC drivers by replacing kubernetes with the configured driver name.

To customize the required labels:

[openshell.drivers.kubernetes.resource_admission.required_labels]
"platform.example.com/sandbox-attachable" = "approved"
"platform.example.com/workspace" = "${workspace}"

To explicitly permit caller-provided driver config:

[openshell.drivers.kubernetes]
allow_driver_config = true

Admission can be disabled explicitly for deployments that authorize external resources elsewhere:

[openshell.drivers.kubernetes.resource_admission]
enabled = false

Testing

  • mise run pre-commit passes
  • Core, Kubernetes, Docker, and Podman unit tests pass
  • Server admission and remote-driver handshake tests pass
  • E2E tests added/updated (not run)

Checklist

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@github-actions

Copy link
Copy Markdown

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

@drew drew left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gator-agent

PR Review Status

The initial review found one blocking configuration-compatibility defect in the Kubernetes image-pull Secret path. The rest of the reviewed patch has no additional blocking findings.

Action required: prevent configured admission labels from colliding silently with OpenShell ownership labels, and add the regression test described inline.

Blocking findings:

  • GATOR-661321ad-01: copied image-pull Secrets can overwrite a configured required label and make managed-workspace provisioning fail.

Carried findings:

  • None

Non-blocking suggestions:

  • None
Gator metadata
  • Validation: Project-valid security hardening authored by a repository administrator with a detailed operator workflow and migration behavior.
  • Docs: Fern documentation is updated for the new gateway configuration and driver behavior.
  • Checks: Current branch, Helm, Trivy, and DCO checks are green; required runtime suites have not been dispatched.
  • E2E: test:e2e, test:e2e-kubernetes, and test:windows will be required after review feedback is resolved.
  • Head SHA: 661321adb75a576f6dc82a310b26283ca6ff2b13
  • Base SHA: 99ed6a9df09a70981accfd39fe234fa1a648a93c
  • Merge base SHA: 99ed6a9df09a70981accfd39fe234fa1a648a93c
  • Patch ID: 709e2f4ab3b63510db17a0a2be4ccb9f1d7d113b
  • Gator payload: 10
  • Review mode: initial
  • Previous reviewed SHA: none
  • Review budget exhausted: no
  • Maintainer decision required: no
  • Next state: gator:in-review

LABEL_MANAGED_BY_VALUE.to_string(),
);
labels.insert(LABEL_GATEWAY_ID.to_string(), gateway_id.to_string());
labels.insert(LABEL_SANDBOX_WORKSPACE.to_string(), workspace.to_string());

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gator-agent

Warning — GATOR-661321ad-01 · Ownership labels can invalidate an admitted Secret

Summary: An operator can configure openshell.ai/gateway-id or another internal ownership key as a required admission label. The source image-pull Secret passes admission, but this copy path overwrites that value and the later target check rejects it. Every managed-workspace sandbox using the Secret then fails provisioning even though the documented custom policy is valid.

Fix: Either store copy ownership in non-colliding metadata, or reject the three internal ownership keys during configuration startup and document them as reserved. Add coverage for the collision.

Verify: Require openshell.ai/gateway-id=security-approved, admit a source Secret with that label, and copy it. The gateway must either reject the configuration at startup or keep the copied Secret admissible.

Agent context
  • Location: crates/openshell-driver-kubernetes/src/driver.rs:4778
  • Ownership: This PR introduces customizable replacement label maps and the ownership overwrite followed by target revalidation.

@drew drew added gator:in-review Gator is reviewing or awaiting PR review feedback gator:blocked Gator is blocked by process or repository gates and removed gator:in-review Gator is reviewing or awaiting PR review feedback gator:blocked Gator is blocked by process or repository gates labels Sep 22, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gator:blocked Gator is blocked by process or repository gates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant