[AMD][AgentX] Add MiniMax-M3 MXFP4 ATOM AgentX MTP on MI355X / 在 MI355X 上添加 MiniMax-M3 MXFP4 ATOM AgentX MTP recipe - #2816
Conversation
新增 MiniMax-M3 ATOM AgentX recipe，固定可复现镜像并覆盖 MI355X TP4 concurrency sweep。 Align the ATOM serving entrypoint, prefix-cache settings, and generated matrix with the validated MI355X smoke path. 对齐 ATOM 服务入口、prefix cache 参数以及已验证的 MI355X smoke 路径矩阵。 Co-authored-by: Cursor <cursoragent@cursor.com>
修复 MiniMax ATOM prefix cache 默认关闭的问题。 Enable prefix caching by default so generated AgentX runs match the validated smoke configuration. 默认开启 prefix caching，使生成的 AgentX 运行与已验证 smoke 配置一致。 Co-authored-by: Cursor <cursoragent@cursor.com>
按 AgentX 并发度动态设置 ATOM max-num-seqs。 Scale ATOM scheduler capacity to two sequence slots per configured concurrency lane. 根据配置的并发 lane 数量为每个 lane 分配两个 ATOM scheduler sequence slot。 Co-authored-by: Cursor <cursoragent@cursor.com>
为 MiniMax AgentX 适配 ATOM PR #2106 补丁，并修复 served model 名称不匹配导致的 warmup 失败。 Co-authored-by: Cursor <cursoragent@cursor.com>
为 MiniMax ATOM AgentX MTP recipe 增加无 offloading 的 TP2 并发搜索空间，并更新矩阵快照校验。 Co-authored-by: Cursor <cursoragent@cursor.com>
让 MiniMax ATOM AgentX launcher 接受 TP2 search-space 配置。 Co-authored-by: Cursor <cursoragent@cursor.com>
将 MiniMax ATOM AgentX launcher 的 GPU memory utilization 提升到 0.9。 Co-authored-by: Cursor <cursoragent@cursor.com>
charxwu
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline findings, I also checked whether the per-concurrency case "$CONC" block's MAX_NUM_SEQS/MAX_NUM_BATCHED_TOKENS/GPU_MEM_UTIL settings are dead code now that they're overwritten right after the case statement — that overwrite is intentional (the adjacent comment states the fixed values are the official validated MiniMax-M3 launch settings), so it isn't a separate bug beyond what's already flagged inline.
Extended reasoning...
Confirmed findings already posted inline cover real, non-trivial bugs: the patch idempotency check in apply_atom_pr2106_patch.sh can never validate correctly against the excluded-file patch; SPEC_ARGS unconditionally enables EAGLE3 with 3 draft tokens even when the CONC=40/56 branches intend to disable it (NUM_SPEC_TOKENS=0); STATE_CKPT_ARGS is computed but never appended to ATOM_CMD, silently dropping --state-checkpoint-slots; and the waiver doc is misnamed relative to docs/PR_REVIEW_CHECKLIST.md's requirement. These are exactly the kind of functional discrepancies between comments/intent and actual behavior that warrant a human look, and the bug-hunt exit reason (max_bugs) means the search was cut off rather than exhausted, so no additional approval signal should be inferred. I verified the case block in minimaxm3_fp4_mi355x_atom_mtp.sh directly and confirmed the described overwrite pattern (MAX_NUM_SEQS=$((2*CONC)), fixed MAX_NUM_BATCHED_TOKENS=32768, GPU_MEM_UTIL=0.9 right after the case) is real, but distinguished it from a bug since the surrounding comment documents it as intentional — while the state-checkpoint-slots and spec-token gating issues in that same block are genuine bugs already captured inline. Given the number and severity of confirmed findings, this does not meet the bar for approval, and no prior review body exists on this PR to avoid duplicating, so a short informational defer note is appropriate.
Findings marked 🟡 are optional suggestions and need no follow-up push.
| if git -C "$ATOM_ROOT" apply --reverse --check "$PATCH_FILE" >/dev/null 2>&1; then | ||
| echo "ATOM PR #2106 already applied" | ||
| exit 0 | ||
| fi | ||
|
|
||
| # These files were structurally rewritten after the pinned image's ATOM | ||
| # commit. Their freshness guard is orthogonal to the draft-KV OOM fix and | ||
| # is intentionally omitted until it can be ported against that revision. | ||
| git -C "$ATOM_ROOT" apply --check "${PATCH_EXCLUDES[@]}" "$PATCH_FILE" | ||
| git -C "$ATOM_ROOT" apply "${PATCH_EXCLUDES[@]}" "$PATCH_FILE" |
There was a problem hiding this comment.
🔴 The 'already applied' idempotency check at line 27 runs git apply --reverse --check on the FULL patch, but the patch is only ever forward-applied with PATCH_EXCLUDES (backends.py, eagle_proposer.py excluded at lines 14-17). Since those two files never get patched, the whole-patch reverse-check can never succeed once already applied, so every re-run against an already-patched ATOM_ROOT falls through to the excludes-scoped apply --check at line 35, which now fails (hunks already applied) and aborts under set -e. Fix: scope the idempotency check to the same exclude set used to apply the patch (or detect a marker unaffected by the excludes), so an already-patched tree is recognized on every invocation, not just the first.
Extended reasoning...
First run: ATOM_ROOT is unpatched, reverse --check fails as expected, so the script applies the patch with PATCH_EXCLUDES (lines 35-36), leaving backends.py and eagle_proposer.py unmodified by design. Second run against the same ATOM_ROOT (e.g. a persistent/shared AMD runner container reused across the matrix's per-concurrency jobs, or a retried/rerun job) — line 27's reverse --check on the unfiltered patch requires ALL hunks including the excluded files' to reverse cleanly; those files are still in pre-patch state so reversal fails, so the 'already applied' fast path is never taken. The script proceeds to line 35 git apply --check (with excludes) against files that are already forward-patched, which fails because the expected pre-patch context no longer matches. Under set -euo pipefail this aborts the entire script with a raw git error instead of the friendly already-applied message, killing the whole benchmark launch for that concurrency point.
Verification: normal (conditional on re-run against a persistent ATOM_ROOT — the very scenario the idempotency block exists for). Line 27 git apply --reverse --check "$PATCH_FILE" checks the UNFILTERED patch (no PATCH_EXCLUDES), but the patch is only ever forward-applied with atom/model_ops/attentions/backends.py and atom/spec_decode/eagle_proposer.py excluded (lines 14-17, 35-36). Those two files are…
| SPEC_ARGS=( | ||
| --method eagle3 | ||
| --draft-model Inferact/MiniMax-M3-EAGLE3 | ||
| --num-speculative-tokens 3 | ||
| ) |
There was a problem hiding this comment.
🟡 (optional) SPEC_ARGS unconditionally enables eagle3 with 3 speculative tokens regardless of NUM_SPEC_TOKENS, so the CONC=40/56 case branches that set NUM_SPEC_TOKENS=0 (meant to disable the draft model past the throughput knee, per the comment above SPEC_ARGS) have no effect. Fix: gate SPEC_ARGS on NUM_SPEC_TOKENS -gt 0, omitting --method/--draft-model/--num-speculative-tokens when it is 0.
Extended reasoning...
Lines 157/167 set NUM_SPEC_TOKENS=0, SPEC_DECODE_AL=0 for CONC 40 and 56. Line 279 only echoes these values; SPEC_ARGS at lines 274-278 is a fixed array always containing --method eagle3 --draft-model Inferact/MiniMax-M3-EAGLE3 --num-speculative-tokens 3, unconditionally appended to ATOM_CMD. Any manual/TP8 run at CONC=40 or 56 (the script's header says TP8 is accepted for larger-memory variants/manual smoke tests) launches with the draft model enabled contrary to the documented design, wasting the draft forward pass past the intended throughput knee.
Verification: nit. Real dead-variable/logic defect, but it does not degrade any shipped configuration. SPEC_ARGS (lines 274-278) is a fixed array hardcoding --method eagle3 --draft-model Inferact/MiniMax-M3-EAGLE3 --num-speculative-tokens 3, appended unconditionally at line 298. Grep confirms NUM_SPEC_TOKENS is referenced only in the echo at line 279 — never used to gate/build SPEC_ARGS. Therefore the…
functionstackx
changed the title
将 MiniMax ATOM AgentX launcher 与 MI355X vLLM 配置对齐：EAGLE3-GQA draft、synthetic acceptance length 2.78，并启用 thinking_mode。 Co-authored-by: Cursor <cursoragent@cursor.com>
Use rocm/atom-dev:nightly_202609080825-lirzhang-triton-build, which includes merged ROCm/ATOM PR #2147 KV-pool fixes validated by MI355X EAGLE3 dummy smoke without runtime patching. 中文：将 MiniMax ATOM AgentX recipe 镜像升级到 rocm/atom-dev:nightly_202609080825-lirzhang-triton-build（已包含合并后的 PR #2147 KV pool 修复），并移除运行时 patch 脚本与豁免文档。 Co-authored-by: Cursor <cursoragent@cursor.com>
charxwu
changed the title
Enable CPU-only LMCache offload (256 GiB/rank, 256-token chunks) at TP4 conc 40 and 48 for the MI355X MiniMax ATOM agentic recipe. 为 MI355X MiniMax ATOM agentic recipe 在 TP4 并发 40 和 48 启用仅 CPU 的 LMCache offload（每 rank 256 GiB，256 token chunk）。 Co-authored-by: Cursor <cursoragent@cursor.com>
charxwu
changed the title
Resolve perf-changelog and matrix test conflicts after the infx matrix refactor on main; keep MiniMax ATOM recipe entries appended at the tail. 将 main 上的 infx matrix 重构变更与 MiniMax ATOM 分支合并，解决 perf-changelog 与 matrix 测试冲突，并保留 recipe 条目追加在文件末尾。 Co-authored-by: Cursor <cursoragent@cursor.com>
The merge left the first MiniMax ATOM changelog block without its `- config-keys:` line, which broke check-changelog YAML parsing. 合并后第一条 MiniMax ATOM changelog 缺少 `- config-keys:` 行，导致 check-changelog 的 YAML 解析失败。 Co-authored-by: Cursor <cursoragent@cursor.com>
Drop the TP4 conc 40/48 LMCache search-space points for now and remove the launcher knobs dedicated to that arm; the recipe stays GPU-resident through conc 32. 暂缓 TP4 并发 40/48 的 LMCache 配置点，并移除 launcher 中对应分支；recipe 在 conc 32 及以下保持纯 GPU resident。 Co-authored-by: Cursor <cursoragent@cursor.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34462316872 |
billishyahao
left a comment
billishyahao
left a comment
There was a problem hiding this comment.
As a PR reviewer and CODEOWNER, I have reviewed this and have:
- Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
- Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
- Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this. Link: https://inferencex.semianalysis.com/inference?unofficialRun=34462316872
- Verified that this PR passes evals. Please link to GitHub Action workflow that shows this. Link: https://inferencex.semianalysis.com/evaluation?unofficialRun=34462316872
- Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
- For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
- Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
- Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; FLOPs at lower precisions is fine, given that the config passes private evals. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
- If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
- If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
- Verified that every single-node vLLM/SGLang recipe in this PR is documented in the official vLLM recipes and/or the SGLang cookbook:
- I linked the corresponding upstream PR in the vLLM recipe repo or SGLang repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
- Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/
<PR_NUMBER>.md— named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section. - If this PR uses
append-only: true, verified that it only adds generated points or recipe variants inside a selected existing config/scenario and existing same-image visual curve: every previously generated point remains present with the same recipe, no prior point is removed or rerun, and every benchmark-affecting change in the complete diff can affect only the corresponding newly appended points (never an existing point), regardless of which file contains it. - If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.
Additional detail section:
Signed: @billishyahao
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
在 conc 40/48 LMCache 路径增加 ATOM/LMCache SLRU 相关环境变量， 并添加 MI355X 临时测试脚本（镜像 nightly_202609140645）。 Co-authored-by: Cursor <cursoragent@cursor.com>
将 minimaxm3-fp4-mi355x-atom-agentic-mtp 镜像更新为 rocm/atom-dev:nightly_202609140645-lirzhang-triton-build，并同步 perf-changelog。 Co-authored-by: Cursor <cursoragent@cursor.com>
在 perf-changelog 单条 entry 中写明 conc 40/48 的 SLRU 环境变量名称。 Co-authored-by: Cursor <cursoragent@cursor.com>
空提交以重新触发 PR 流水线。 Co-authored-by: Cursor <cursoragent@cursor.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34834202106 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34834280359 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34834280359 |
|
/reuse-sweep-run 34834280359 |
billishyahao
left a comment
billishyahao
left a comment
There was a problem hiding this comment.
As a PR reviewer and CODEOWNER, I have reviewed this and have:
- Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
- Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
- Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this. Link: https://inferencex.semianalysis.com/inference?unofficialRun=34834280359
- Verified that this PR passes evals. Please link to GitHub Action workflow that shows this. Link: https://inferencex.semianalysis.com/evaluation?unofficialRun=34834280359
- Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
- For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
- Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
- Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; FLOPs at lower precisions is fine, given that the config passes private evals. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
- If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
- If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
- Verified that every single-node vLLM/SGLang recipe in this PR is documented in the official vLLM recipes and/or the SGLang cookbook:
- I linked the corresponding upstream PR in the vLLM recipe repo or SGLang repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
- Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/
<PR_NUMBER>.md— named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section. - If this PR uses
append-only: true, verified that it only adds generated points or recipe variants inside a selected existing config/scenario and existing same-image visual curve: every previously generated point remains present with the same recipe, no prior point is removed or rerun, and every benchmark-affecting change in the complete diff can affect only the corresponding newly appended points (never an existing point), regardless of which file contains it. - If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.
Additional detail section:
Signed: @billishyahao
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
There was a problem hiding this comment.
As a PR reviewer and CODEOWNER, I have reviewed this and have:
- Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
- Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
- Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this.
- Verified that this PR passes evals. Please link to GitHub Action workflow that shows this.
- Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
- For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
- Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
- Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; FLOPs at lower precisions is fine, given that the config passes private evals. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
- If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
- If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
- Verified that every single-node vLLM/SGLang recipe in this PR is documented in the official vLLM recipes and/or the SGLang cookbook:
- I linked the corresponding upstream PR in the vLLM recipe repo or SGLang repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
- Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/
<PR_NUMBER>.md— named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section. - If this PR uses
append-only: true, verified that it only adds generated points or recipe variants inside a selected existing config/scenario and existing same-image visual curve: every previously generated point remains present with the same recipe, no prior point is removed or rerun, and every benchmark-affecting change in the complete diff can affect only the corresponding newly appended points (never an existing point), regardless of which file contains it. - If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.
Additional detail section:
- insert any additional info here
recipe at https://github.com/ROCm/ATOM/blob/main/recipes/MiniMax-M3-Agentic-InferenceX.md
Signed: seungrokj
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 122c83e. Configure here.
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
|
@cquil11 @functionstackx can you plz review this ? |
|
/stage-results 34834280359 |
|
@cquil11 staged run 34834280359: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-09-14~r34834280359 This run remains available across future |
7 participants
Summary / 摘要
Add the MiniMax-M3 MXFP4 ATOM AgentX MTP recipe on MI355X (
minimaxm3-fp4-mi355x-atom-agentic-mtp), aligned with the existing vLLM arm (EAGLE3-GQA draft, synthetic acceptance length 2.78,thinking_modeenabled). The recipe uses upstreamrocm/atom-dev:nightly_202609080825-lirzhang-triton-buildas shipped — no runtime ATOM patch or waiver.本 PR 在 MI355X 上新增 MiniMax-M3 MXFP4 ATOM AgentX MTP recipe（
minimaxm3-fp4-mi355x-atom-agentic-mtp），与现有 vLLM 分支对齐（EAGLE3-GQA draft、合成接受长度 2.78、thinking_mode启用）。recipe 直接使用上游镜像rocm/atom-dev:nightly_202609080825-lirzhang-triton-build，无需运行时 ATOM patch 或豁免文档。Recipe / 配置
rocm/atom-dev:nightly_202609080825-lirzhang-triton-buildbenchmarks/single_node/agentic/minimaxm3_fp4_mi355x_atom_mtp.sh[1, 2, 4, 5, 8, 10, 12, 15, 20, 24, 28, 32]; TP2 conc[1, 2, 5];kv-offloading: noneInferact/MiniMax-M3-EAGLE3-GQA), 3 draft tokens, synthetic AL 2.78 on throughput runsmax-num-seqs = 2×CONC, official MiniMax MXFP4 online quant settingsValidation / 验证
E2E sweep (15 configs,
--no-evals): Actions run 34237169926 — successVultri dummy smoke (no patch): TP4 EAGLE3 + block-size 128 bring-up SMOKE_OK on the same image
GSM8K (manual, AIM): flexible-extract 96.29%, strict-match 96.36%
Matrix tests:
127 passedE2E 扫描（15 个配置，
--no-evals）： Actions run 34237169926 — 通过Vultri dummy smoke（无 patch）： 同一镜像上 TP4 EAGLE3 + block-size 128 启动 SMOKE_OK
GSM8K（手动，AIM）： flexible-extract 96.29%，strict-match 96.36%
matrix tests：
127 passedTest plan / 测试计划
generate_sweep_configs.py test-configforminimaxm3-fp4-mi355x-atom-agentic-mtp --no-evals(15 configs)--no-evals)Note
Low Risk
Benchmark/CI configuration and a new agentic launcher only; no changes to core inference libraries or auth/data paths.
Overview
Adds a new MI355X AgentX benchmark recipe for MiniMax-M3 MXFP4 on ATOM (
minimaxm3-fp4-mi355x-atom-agentic-mtp), paired with the existing vLLM arm for apples-to-apples agentic trace replay.The new launcher
minimaxm3_fp4_mi355x_atom_mtp.shstarts the ATOM OpenAI server with official MiniMax MXFP4 settings (FP8 KV/index cache, EAGLE3-GQA MTP with synthetic AL 2.78 on throughput runs,thinking_mode), concurrency-specific ReplaySSM/spec knobs, and optional LMCache DRAM offload—including hybrid CPU attention state offload at higher conc and a separate high-conc path (c40/c48) with SLRU prefix/LMCache policy and spec decoding disabled.amd-master.yamldefines the sweep (TP4 conc 1–32, TP2 1/2/5, plus TP4 c40/c48 with LMCache).launch_mi355x-amds.shpins defaults for this script (MODEL_PATH, prefix caching,AITER_LOG_LEVEL, GSM8K eval tasks dir).Reviewed by Cursor Bugbot for commit 5a101bb. Bugbot is set up for automated code reviews on this repo. Configure here.