Require both modalities to opt in before enabling CFG in LTX2 pipelines - #14650
Open
rootonchair wants to merge 3 commits into
Open
Require both modalities to opt in before enabling CFG in LTX2 pipelines#14650rootonchair wants to merge 3 commits into
rootonchair wants to merge 3 commits into
Conversation
PR huggingface#14447 changed the audio_guidance_scale default to 7.0, which silently kept classifier-free guidance (and the negative prompt) active for every distilled recipe that passes guidance_scale=1.0 - at double the compute and with measurably worse output on distilled checkpoints. LTX-2 has no use case for guiding one modality while the other is disabled, so gate CFG on both scales: guidance_scale=1.0 is a master off-switch again, while the recommended base-model settings (3.0 video / 7.0 audio) still enable it. Applied to LTX2Pipeline, condition, image2video, and IC-LoRA variants. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X4NMaGamTgkHjL9EYqNZ7K
PR huggingface#14447 also flipped stg_scale/audio_stg_scale defaults from 0.0 to 1.0 (with spatio_temporal_guidance_blocks defaulting to [28]), so distilled recipes run a perturbed-attention guidance pass per step even with guidance disabled. Revert both defaults to 0.0: STG is opt-in again, and since audio_stg_scale resolves via `audio_stg_scale or stg_scale`, a 0.0 default also restores the pre-huggingface#14447 "audio follows video" behavior. The base-model doc examples already pass stg_scale=1.0 explicitly and are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X4NMaGamTgkHjL9EYqNZ7K
PR huggingface#14447 flipped modality_scale from 1.0 to 3.0 (and audio_modality_scale from None to 3.0), so distilled recipes run extra modality-isolated transformer passes per step even with CFG and STG disabled - measured at ~40% additional compute and reduced fidelity on the distilled checkpoint. Revert both defaults (audio back to None restores "follow video"), making guidance_scale=1.0 a complete off-switch together with the CFG gate and STG defaults. The docs' 2.5-distilled examples, which state they run unguided, actually do so again; the two base-model snippets that relied on the default now pass modality_scale=3.0 explicitly, matching the other base-model examples. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X4NMaGamTgkHjL9EYqNZ7K
github-actions
Bot
added
documentation
rootonchair
force-pushed
the
fix-ltx2-distilled-guidance-gate
branch
from
August 30, 2026 17:41
ae7d532 to
5b4613c
Compare
github-actions
Bot
added
size/S
Contributor
|
Hi @rootonchair, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice. Once the PR links an issue (or gets the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fix #14649
#14447 aligned the shared LTX-2 pipeline defaults with the LTX-2.5 base-model reference. As a side effect, distilled recipes that pass
guidance_scale=1.0(the documented "8 steps with CFG = 1" usage) silently started running CFG, STG, and modality-isolation guidance — at ~3x the compute and with worse output on distilled checkpoints.This PR restores
guidance_scale=1.0as a complete off-switch, one commit per mechanism:do_classifier_free_guidance:or→andon the video/audio scalesaudio_guidance_scale=7.0default kept CFG (and the negative prompt) active atguidance_scale=1.0. There is no use case that guides one modality while the other is disabled, so require both. Recommended base settings (3.0 / 7.0) still enable CFG.stg_scale,audio_stg_scale:1.0→0.0stg_scale > 0) with default blocks[28]— a perturbed-attention pass ran even "unguided". Back to opt-in; base-model doc examples already passstg_scale=1.0explicitly.modality_scale:3.0→1.0,audio_modality_scale:3.0→Nonemodality_scale > 1), running extra modality-isolated transformer passes each step. With the3.0default it stayed active even after CFG and STG were disabled — the LTX-2.5 distilled docs describe their inference as unguided, yet the example right below still executed it. Reverting the default to1.0makes unguided runs truly unguided; the base-model doc snippets that depended on the old default now setmodality_scale=3.0explicitly.Measured on LTX-2.3-Distilled (5 seed-matched clips, 768x512x121, 8 steps): per-clip inference drops 22 s → 11 s → 6.9 s as each mechanism is removed (a true single pass per step), while fidelity to the bf16 reference improves and temporal flicker returns to bf16 levels.
guidance_rescale(0.0 → 0.7 in #14447) is deliberately untouched: it only applies while CFG is active, so it is inert for distilled runs after this fix.Applied to
LTX2Pipeline,LTX2ConditionPipeline,LTX2ImageToVideoPipeline, and the IC-LoRA variant.Fixes the stale LTX-2.3 distilled examples in
docs/source/en/api/pipelines/ltx2.mdimplicitly; the docs touched here are only the snippets that relied on the reverted defaults.