feat: enable DSpark for DeepSeek-V4-Pro-0813 AgentX disagg on MI355X - #3188
Merged
Merged
Conversation
Add a DSpark arm to the AMD multi-node SGLang disaggregated path and switch dsv4-fp4-mi355x-sglang-disagg-agentic-hicache-mtp onto it. models.yaml: DeepSeek-V4-Pro-AgentX gains dspark_flags alongside (not replacing) mtp_flags, so recipes that stay on spec-decoding: mtp keep the EAGLE arm untouched. dp_flags gains --enable-dp-lm-head, which SGLang requires for DSpark under DP attention and which is harmless for EAGLE. server_sglang.sh: the config loader exports MODEL_DSPARK_FLAGS, and build_server_config branches on SPEC_DECODING == draft_model. DECODE_MTP_SIZE carries the draft length for both algorithms but they spend it differently: EAGLE runs that many sequential draft passes, while DSpark emits a whole block in one pass, so num-steps is pinned to 1 and the draft length becomes --speculative-dspark-block-size. The verify window (num-draft-tokens = draft length + 1) is identical, which is why the MORI decode dispatch scaling by (DECODE_MTP_SIZE + 1) needs no change. A model configured with draft_model but no dspark_flags now fails hard instead of silently falling back to EAGLE. amd-master.yaml: all four arms move to spec-decoding: draft_model at DECODE_MTP_SIZE=3 (golden AL 3.01), and the image moves to v0.5.19-rocm720-mi35x-20260913, the first tag carrying the DSpark optimizations. CLIENT_IMAGE stays at 20260907; it only runs the load generator. 在 AMD 多节点 SGLang 分离式路径上启用 DSpark，并将 dsv4-fp4-mi355x-sglang-disagg-agentic-hicache-mtp 切换过去。 models.yaml：DeepSeek-V4-Pro-AgentX 新增 dspark_flags，与 mtp_flags 并列而非 替换，因此仍使用 spec-decoding: mtp 的配方的 EAGLE 分支不受影响。dp_flags 增加 --enable-dp-lm-head，这是 SGLang 在 DP attention 下运行 DSpark 的必需项，对 EAGLE 无副作用。 server_sglang.sh：配置加载器新导出 MODEL_DSPARK_FLAGS，build_server_config 按 SPEC_DECODING == draft_model 分流。DECODE_MTP_SIZE 对两种算法都表示草稿长度， 但花法不同：EAGLE 顺序执行同样次数的草稿前向，而 DSpark 一次前向产出一整块， 因此 num-steps 固定为 1，草稿长度改为 --speculative-dspark-block-size。验证窗口 （num-draft-tokens = 草稿长度 + 1）两者相同，这正是 MORI 解码 dispatch 的 ×(DECODE_MTP_SIZE + 1) 缩放无需改动的原因。若模型配置了 draft_model 却没有 dspark_flags，现在直接硬失败，而不是静默退回 EAGLE。 amd-master.yaml：四条臂全部切到 spec-decoding: draft_model，DECODE_MTP_SIZE=3 （黄金 AL 3.01），镜像升级到 v0.5.19-rocm720-mi35x-20260913，即首个带 DSpark 优化的标签。CLIENT_IMAGE 保持 20260907，该容器只运行负载生成器。
Duyi-Wang
requested review from
1am9trash,
billishyahao,
chunfangamd,
seungrokj and
yctseng0211
as code owners
September 16, 2026 13:30
Contributor
|
Thanks for the contribution!
中文感谢你的贡献！
|
Append the perf-changelog entry for dsv4-fp4-mi355x-sglang-disagg-agentic-hicache-mtp moving to spec-decoding: draft_model on image -20260913. 为 dsv4-fp4-mi355x-sglang-disagg-agentic-hicache-mtp 切换到 spec-decoding: draft_model 及镜像 -20260913 追加 perf-changelog 条目。
Duyi-Wang
changed the title
Rename dsv4-fp4-mi355x-sglang-disagg-agentic-hicache-mtp to dsv4-fp4-mi355x-sglang-disagg-agentic-umbp-dspark per review, so the key names the algorithm the arm now serves (DSpark) instead of the retired MTP label, matching the dsv41flash-*-agentic-dspark precedent. The rename touches only the config key. The emitted matrix is byte-identical to the pre-rename run: 7 points, same exp-names (derived from model-prefix and topology, not the key), image, spec-decoding, topology, concurrency, and offload backends. Historical perf-changelog entries keep the old key, which the append-only contract requires; the new entry records the rename so the lineage stays traceable. 按评审意见，将 dsv4-fp4-mi355x-sglang-disagg-agentic-hicache-mtp 改名为 dsv4-fp4-mi355x-sglang-disagg-agentic-umbp-dspark，使配置键反映该分支实际 使用的算法（DSpark），而非已停用的 MTP 标签，与 dsv41flash-*-agentic-dspark 的命名惯例一致。 改名仅涉及配置键。生成的 matrix 与改名前逐字节一致：7 个点，exp-name （由 model-prefix 与拓扑推导，与键名无关）、镜像、spec-decoding、拓扑、 并发列表、offload 后端均不变。perf-changelog 的历史条目保留旧键名（append-only 契约的要求），新条目记录了此次改名以保持谱系可追溯。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
The guard lived inside build_server_config, which is only ever invoked as PREFILL_SERVER_CONFIG=$(build_server_config ...). exit there terminates the command-substitution subshell, and with no set -e and no status check the script carried on with an empty config string -- dropping base, ep, dp, and parallel flags for that mode, a worse failure than the silent EAGLE fallback the guard was meant to prevent. Move the check to top level, immediately before both build_server_config calls, where exit 1 actually stops the script. The in-function branch keeps a comment explaining why the validation is not co-located with the flag it guards. Verified: with SPEC_DECODING=draft_model and empty MODEL_DSPARK_FLAGS the old shape exits 0 and continues with an empty config, the new shape exits 1 before either assignment. Populated dspark_flags, spec-decoding mtp, empty SPEC_DECODING, and DECODE_MTP_SIZE=0 all still pass through unchanged. 原先的守卫位于 build_server_config 内，而该函数只以 PREFILL_SERVER_CONFIG=$(build_server_config ...) 的形式调用。子 shell 里的 exit 只会终止命令替换本身；脚本没有 set -e，也没有检查退出码，因此会带着空配置字符串 继续执行——该模式下的 base、ep、dp、parallel 参数全部丢失，比守卫本要防止的 "静默退回 EAGLE" 更糟。 将检查移到顶层、两次 build_server_config 调用之前，此处 exit 1 才真正终止脚本。 函数内分支保留注释，说明校验为何没有与它所保护的 flag 放在一起。 已验证：当 SPEC_DECODING=draft_model 且 MODEL_DSPARK_FLAGS 为空时，旧写法退出码 为 0 并带空配置继续，新写法在两次赋值之前即以退出码 1 终止。dspark_flags 非空、 spec-decoding 为 mtp、SPEC_DECODING 为空、以及 DECODE_MTP_SIZE=0 四种情况均保持 原有行为。
Duyi-Wang
added
AMD
full-sweep-enabled
agentx
Contributor
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=35166686551 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=35166686551 |
functionstackx
previously requested changes
Sep 16, 2026
functionstackx
left a comment
functionstackx
left a comment
Collaborator
There was a problem hiding this comment.
@Duyi-Wang can u port over cam's changes to collect node logs from all nodes plz #3170
This was referenced Sep 17, 2026
Collaborator
|
/stage-results 35166686551 |
Contributor
|
@cquil11 staged run 35166686551: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-09-17~r35166686551 This run remains available across future |
Collaborator
|
/use 35166686551 |
cquil11
approved these changes
Sep 17, 2026
cquil11
dismissed
functionstackx’s stale review
September 17, 2026 14:49
I told Seungrok to wait for me since it's still WIP
kedarpotdar-nv
added a commit
that referenced
this pull request
Sep 17, 2026
Resolves the perf-changelog.yaml conflict where this branch and PR #3188 both appended a new entry after the same GLM-5.2 entry, leaving the shared "- config-keys:" line contested. Both entries are kept verbatim, with main's entry ordered first so the branch remains a pure append over main. That ordering is load-bearing: infx/matrix/plan.py rejects any deletion in perf-changelog.yaml, and get_added_lines diffs pull.base.sha against pull.head.sha (two-dot), not the merge base. While the branch was behind main, main-only entries appeared as deletions and the check failed with ValueError: Deletions are not allowed in perf-changelog.yaml. Found deleted line: - config-keys: naming kimik3-fp4-mi355x-vllm-agentic-mtp from PR #3124, an entry this branch never touched. No B300 recipe, master-config, or launcher content changes. <details><summary>中文</summary> 解决 perf-changelog.yaml 的合并冲突：本分支与 PR #3188 都在同一条 GLM-5.2 条目之后追加了新条目，导致共享的 "- config-keys:" 行产生冲突。两条条目均 原样保留，并将 main 的条目排在前面，使本分支相对 main 保持纯追加。 该顺序是必要的：infx/matrix/plan.py 不允许 perf-changelog.yaml 出现任何 删除行，而 get_added_lines 使用 pull.base.sha 与 pull.head.sha 的两点 diff （而非 merge base）。分支落后于 main 时，仅存在于 main 的条目会被判定为 删除，从而使检查失败。 本次合并未改动任何 B300 配方、master config 或 launcher 内容。 </details> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enable DSPARK for DeepSeek-V4-Pro-0813 on the MI355X disaggregated AgentX path. The recipe uses block size 3 / four verification tokens and the measured golden throughput AL of 3.01; evaluation uses natural verification. The 0813 checkpoint supplies the DSpark draft weights.
Both prefill and decode node logs are collected into the uploaded server artifact. All nodes now finish the existing preflight checks before any serving container starts. Serving process groups are tracked and cleaned up on normal completion or failed startup, with bounded teardown that preserves the original benchmark failure status. Benchmark duration, serving parameters, acceptance settings, accuracy thresholds and offload budgets are unchanged by these lifecycle fixes.
Validation:
/use 35166686551workflow.The full-sweep-enabled label remains enabled. No benchmark or accuracy threshold was relaxed.