CLDSRV-992: Capture cloudserver's log on the s3c functional jobs - #6283
anurag4DSB wants to merge 1 commit into
Conversation
The s3c-ft-tests jobs run cloudserver as a container and upload sproxyd, bucketd, repd and vault logs, but never cloudserver's own. Its output goes to the container's stdout and is discarded, so any failure that originates in cloudserver on these jobs is undiagnosable after the fact. That is not hypothetical. Three CLDSRV-992 rows sit on these jobs and are all stuck for exactly this reason: - F8's ServiceUnavailable on "should get the body of part 8 when ordered MPU". Chasing a fresh occurrence showed nginx with zero 5xx, sproxyd with zero errors, and bucketd with zero 503s and its only connection errors at a single startup instant. The 503 was therefore generated by cloudserver itself, and there is no log to say why. - F4 and F8's 40s timeouts in the uploadPartCopy fixtures. The same fixture builds successfully three or four times per run and then stalls once, so it is an intermittent stall rather than slow work, and again nothing records it. Dump the container's log to the job's artifacts directory before teardown, while the containers still exist. The step is if: always() and ends in `|| true`, so it runs on failure -- which is when it matters -- and can never itself fail a job. A passing run pays one docker compose logs call. Issue: CLDSRV-992
Hello anurag4dsb,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Incorrect fix versionThe
Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:
Please check the |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files
@@ Coverage Diff @@
## development/9.3 #6283 +/- ##
===================================================
- Coverage 85.31% 85.28% -0.03%
===================================================
Files 206 206
Lines 13435 13435
===================================================
- Hits 11462 11458 -4
- Misses 1973 1977 +4
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Request integration branchesWaiting for integration branch creation to be requested by the user. To request integration branches, please comment on this pull request with the following command: Alternatively, the |
|
/create_integration_branches |
Integration data createdI have created the integration data for the additional destination branches.
The following branches will NOT be impacted:
You can set option The following options are set: create_integration_branches |
Waiting for approvalThe following approvals are needed before I can proceed with the merge:
The following options are set: create_integration_branches |
|
claude code hallucination, I checked manually we do collect the logs. |
2 participants
The s3c-ft-tests jobs upload sproxyd, bucketd, repd and vault logs but never cloudserver's own, because its output goes to the container's stdout and is discarded. Any failure originating in cloudserver on these jobs is therefore undiagnosable and becomes hypothetical.
Chasing a fresh occurrence of F8's ServiceUnavailable through the artifacts that do exist: nginx logged zero 5xx, sproxyd zero errors, and bucketd zero 503s with its only connection errors at a single startup instant. So cloudserver produced that 503 itself and nothing recorded why. S
The job now dumps cloudserver's container log to its artifacts directory before teardown, while the containers still exist. The step is
if: always()and ends in|| true, so it runs on failure, which is when it matters, and can never itself fail a job; a passing run pays onedocker compose logscall. If it were wrong we would see an empty or missing s3.log in the very next run's artifacts.