[AMD] Port MI355X DeepSeek V4 fixed-sequence disagg to srt-slurm

[cquil11]

Scope

Port MI355X archived DeepSeek V4 SGLang fixed-sequence to the shared AMD-enabled srt-slurm integration.

• Convert the existing port recipes to native schema 2 (engine and roles) and preserve their engine, image, topology, and workload settings.
• Keep the unchanged InferenceX benchmark_serving.py custom-benchmark interface; no engine patches.
• Put the actual fixed-sequence concurrency lists in the master config and consume caller-owned CONC_LIST, replacing placeholder [1] rows and private recipe sweep lists where present. This preserves the existing measured points while matching current result collection.

This workload is deprecated upstream and remains in configs/deprecated/; it is not restored to active sweeps. Retained for the previously requested runtime-path coverage.

Stack

Based directly on shared AMD integration #2542, which is rebased onto InferenceX main at 96bccfe12b637dad23d32f0bf7ceef0ea86de15a. This child contains only its workload configs, schema-2 recipes, and append-only performance changelog entry. Shared launcher/profile/adapter changes live in #2542.

The base selects SemiAnalysisAI/srt-slurm 03863ab23804f8a31811d9aaccc54abc8961a41c: #1 → #11 → #7 → #14, synchronized with NVIDIA srt-slurm 85086d3c1fe7b319a1d0d7db8781047dd1940772. NVIDIA runners retain their existing runtime selection.

Refresh status

Offline matrix validation: 6 topology/workload rows covering 20 benchmark concurrency points; all 12 prepared throughput/eval configurations pass the pinned runtime's schema/topology checks.

No fresh E2E tests or sweeps were started or rerun. Sweep-trigger labels are removed. Historical runtime evidence is listed in the base PR and does not validate these new heads. Full GPU validation remains pending authorization; this refresh makes no new performance or merge-readiness claim.

AI model disclosure

The exact AI model/version for this refresh is not exposed by the runtime and could not be verified. No subagents contributed to this refresh; earlier model identities are not reliably recorded.

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[claude]

Claude finished @cquil11's task in 7m 32s —— View job

---

PR Review

• Gather context and diff vs base branch
• Review 4 new srt-slurm recipe YAMLs
• Review configs/amd-master.yaml changes (image accessibility, model prefixes)
• Validate perf-changelog.yaml entry (appended at end, paired with master config change)
• Check MTP benchmark requirements (--use-chat-template) and server launch formatting
• Post review findings

Summary

Found 1 blocking issue and 1 warning (posted as inline comments):

1. 🔴 BLOCKING (configs/amd-master.yaml): Both DEP8 lanes (dp-attn: true, ep: 8) trigger the DP+EP resize block in runners/launch_mi355x-amds-srt.sh:213, which does int(os.environ["CONC"]) — but fixed-seq multinode jobs only pass CONC_LIST, never CONC, so the launcher crashes with ValueError before submission. Additionally, any fix that supplies a single CONC value conflicts with the recipe's internal c256/c512 loop (the c512 point would run against a server admission-capped at 256).
2. 🟡 WARNING (perf-changelog.yaml): Three different srt-slurm commits are referenced — the changelog says PR [NVIDIA] Add DSR1 TensorRT Support and Enhanced Plotting #7 @ 10a58d2e, the PR body says [NVIDIA] Add TRT-LLM 70B FP8 via slurm #1 @ c609754b, and the launcher (unchanged in this PR) pins c87d7b34. If the DP-aware routing features the DEP8 recipes rely on aren't in the pinned commit, those lanes fail at runtime.

Everything else checked out: changelog entry is correctly appended at the end and paired with the master-config change, the container image is a public Docker Hub reference, both MTP recipes include --use-chat-template (with --dsv4, which requires it), all benchmark flags exist in utils/bench_serving/benchmark_serving.py, result filenames match the launcher's -c<N>.json parsing, and the TP8 lanes follow the established qwen3-0.6b srt-disagg pattern cleanly.

[cquil11]

Native full-sweep validation started on head 58cfe39: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/33533164273

Expected scheduled production scope: six DSV4 8k1k throughput lanes (STP/MTP TP8 c1–128 and DEP8 c256/c512) plus four standard accuracy lanes. This follows restoration of upstream image pins, server context, router policy, decode chunk-size semantics and benchmark defaults. No other recipe port is being tested in parallel. Results will be staged only after the complete native run is green.

[cquil11]

Current native full sweep: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/33533933546

Head 84b6055 additionally restores the legacy explicit --disaggregation-ib-device rdma0,...,rdma7 flag on both roles. The old run 33533164273 was superseded while every hardware lane was still queued; this is a source correction, not a flake rerun or fill-in run. All eight generated native worker commands were checked. The PR description now also explicitly documents the shared runtime's startup warmup bypass and allocated-port differences.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33533164273
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33533164273

[cursor]

Cache flush uses metrics URLs

Medium Severity

clear_agentic_worker_caches reads AIPERF_SERVER_METRICS_URLS and returns success when that variable is unset. The retired replay flushed workers via SERVER_FLUSH_URLS_CSV, which was always populated and distinct from optional metrics scrape URLs. Cache isolation between concurrency points is skipped whenever metrics URLs are absent.

Additional Locations (1)

• benchmarks/multi_node/agentic_srt.sh#L185-L188

 

^{Reviewed by Cursor Bugbot for commit 84b6055. Configure here.}

[cquil11]

Fixed the workload parity error and rebased on latest main. Head: c38fb02; shared #2542: cdb9e5f.

The original successful MTP run used RANDOM_RANGE_RATIO=0.8, but these ported recipes hardcoded 1.0. Restored workflow forwarding and the legacy 0.8 fallback in all four recipes. No context increase, image change, or MTP change; benchmark_serving.py is untouched.

Original evidence: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/30189328462/job/89759633413
Corrected native full sweep: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/33554199275

The new source requires a new full sweep; rerunning the failed jobs of 33533933546 would still execute the old incorrect workload. Local validation: 246 matrix tests, 126 eval/logging tests, four pinned srt dry-runs, and 12 actual launcher/benchmark-command cases passed. Hardware validation remains pending.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33533933546
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33533933546

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

There are 3 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit c38fb02. Configure here.}

[cursor]

AMD disagg runtime deleted for remaining recipes

High Severity

This commit removes the entire amd_utils Slurm/runtime tree, but active AMD entries such as dsv4-fp4-mi355x-atom-disagg and dsr1-fp4-mi355x-sglang-disagg-mtp still use the old PREFILL_NODES / DECODE_NODES contract that those scripts implemented. Those lanes no longer have a launcher, so they cannot allocate nodes, start workers, or produce results.

Additional Locations (1)

• configs/amd-master.yaml#L663-L675

 

^{Reviewed by Cursor Bugbot for commit c38fb02. Configure here.}

[cquil11]

This is a real integration/merge-order constraint, not a claim that the remaining legacy entries already work on this branch. The amd_utils deletion belongs to shared base #2542, as requested; this child only ports the DSV4 SGLang fixed-sequence entries. The remaining eligible AMD entries are covered by separate child ports (#2627, #2628, #2629, #2631, #2633, plus AgentX #2555). The shared deletion must not be released on its own while those active entries still depend on the retired launcher. We are validating the ports one at a time and are not merging the incomplete migration. This PR currently targets main solely for the native full-sweep trigger; its commits remain based on #2542.

[cursor]

Shared DEP8 recipe loses per-point pins

Medium Severity

Both DEP8 concurrency points (256 and 512) reuse one CONFIG_FILE for STP and one for MTP. The retired path set per-point max-running-requests to that point’s concurrency and retuned the MoRI inter-kernel switch from it. A single static recipe cannot keep those admission and dispatch pins different per allocation.

Additional Locations (1)

• configs/amd-master.yaml#L1166-L1196

 

^{Reviewed by Cursor Bugbot for commit c38fb02. Configure here.}

[cquil11]

Checked against the current head: the recipe is transformed per allocation by the shared launcher before srtctl loads it. runners/launch_mi355x-amds-srt.sh lines 231–266 read CONC_LIST (or CONC), set both roles’ max-running-requests to the allocation concurrency, and set the decode inter-kernel switch threshold to 2 * (concurrency // DECODE_TP). The matrix keeps c256 and c512 as separate entries, so these resolve to admission 256 / threshold 64 and admission 512 / threshold 128 respectively. The explicit model-specific per-rank dispatch pin is preserved via setdefault. The static YAML value is not the realized launch value. No source change is needed for this comment; complete hardware validation of this head is still pending in run 33554199275.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33554199275
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33554199275

[cquil11]

/stage-results 33554199275

[github-actions]

@cquil11 staged run 33554199275: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-09-01~r33554199275

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

[cquil11]

Validation complete: native label-triggered full sweep https://github.com/SemiAnalysisAI/InferenceX/actions/runs/33554199275 is fully green, attempt 1, at current head c38fb02.

Audited all 20 expected throughput points across six jobs, plus all four dedicated GSM8K evaluations (1,319 distinct questions each). Strict accuracies: STP TP8 96.36%, MTP TP8 96.66%, STP DEP8 95.83%, MTP DEP8 95.68%; all pass the existing 91% threshold. Result collection, comparison, changelog metadata, success-rate calculation, and evaluation collection also passed.

Request-count caveat: 20,471/20,472 measured requests completed. STP DEP8 c256 had one incomplete request and one upstream router stream error; its existing failure threshold was not changed. Every other throughput point completed every request, with no recurrence in the later router logs. This is CI-green, not a claim of zero request errors. Optional power telemetry was not validated.

The corrected sweep retains the original 0.8 random-length ratio, 9,217 context, images, MTP settings, and engine options documented above; benchmark_serving.py remains unchanged. No config edits or failed-only reruns were needed during this sweep. Staging was requested in #2630 (comment).