@W-23928129 - Add NUT tests for all devops commands - #512
Open
ad-shreya wants to merge 12 commits into
Open
Conversation
ad-shreya
force-pushed
the
ashreya-non-unit-tests
branch
from
July 2, 2026 18:56
3853eb5 to
e702079
Compare
…ronment Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rewrite and expand the NUT suite so every sf devops command has a test: - Fix the REAL_ORG guard to use `.some(Boolean)` so empty-string CI env vars (TESTKIT_HUB_USERNAME/TESTKIT_ORG_USERNAME/TESTKIT_AUTH_URL) are treated as falsy. The prior `??` fell through only on null/undefined, so real-org tests never ran in CI. `.some(Boolean)` also satisfies the prefer-nullish-coalescing lint rule (no `||`). - Rewrite stale tests that referenced removed commands/modules (pipeline activate/attach-project, work-item status update, review create referencing a nonexistent pull-request command). - Cover all 25 devops commands; add always-run flag-validation tests (help, invalid-id, missing-target-org, enum values) plus real-org happy/error paths gated behind REAL_ORG. - Drop fragile exit-2 presence/relationship assertions: without a default org, requiredOrg resolution wins and yields exit 1 (NoDefaultEnvError), not exit 2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ad-shreya
force-pushed
the
ashreya-non-unit-tests
branch
2 times, most recently
from
September 1, 2026 07:56
353963e to
bd9b2da
Compare
…W-23251216)
The REAL_ORG fix let real-org NUTs run in CI, but the shared CI dev hub
does not have the DevOps Center feature enabled, so every API-backed
command failed with FUNCTIONALITY_NOT_ENABLED — including `before` hooks
that create fixtures, which took the always-run flag-validation tests
down with them.
Add a shared nutHelpers module that probes the target org once
(`devops pipeline list`, judged by exit code so it is robust to any
disabled-org error) and expose a `dcEnabled` flag. Real-org tests now
self-skip via `this.skip()` and fixture setup is gated on the same flag,
so a non-DC org runs the validation tests and cleanly skips the rest,
while a DC-enabled org exercises the full suite.
Verified the probe against real orgs: DC-disabled orgs return a non-zero
status ("DevOps Center isn't enabled") and a DC-enabled org returns 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ad-shreya
changed the title
Ran the full suite against a real DevOps Center-enabled org, which surfaced
assertion and fixture bugs that only appear when the org-backed tests execute:
- Assert on the actual message text (error.NoBranch, error.NoFieldsProvided)
instead of stale regexes.
- Use ensureExitCode: 'nonZero' for business errors (these commands exit 2,
not 1); the prior bareword `nonZero` threw ReferenceError at runtime.
- Compare 15- vs 18-char Salesforce IDs via .slice(0, 15) on get/list.
- Add createWorkItem helper: the create endpoint returns no id/name, so
fixtures read the seeded item back from work-item list.
- request status: assert the JSON `message` (not stderr) under --json.
- pipeline update: activate/deactivate happy paths require an active pipeline
(env-configured stages via interactive OAuth) and can't run headlessly;
assert their reachable precondition errors instead.
- stage environment add: drop the test that hangs on interactive OAuth; keep
the invalid-stage error path.
- stage branch add: associate an existing stage branch (configurable
STAGE_BRANCH, default `staging`) on the true last stage (NextStageId=null);
`main` is the reserved mainline the server rejects.
- work-item update status: use "Ready to Promote" ("In Progress" needs an
active pipeline).
- Skip work-item update --subject/--description: the update endpoint rejects
those fields ("Unrecognized field") though create accepts them; product bug
to be filed. Re-enable once the endpoint supports them.
- Centralize the fixture repo in GITHUB_REPO (override via DC_NUT_REPO).
Results: real DC org 120 passing / 1 pending / 0 failing;
no-org 73 passing / 48 pending / 0 failing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@W-23928129@
What does this PR do?
What issues does this PR fix or reference?
#, @@
Functionality Before
<insert gif and/or summary>
Functionality After
<insert gif and/or summary>