# What Exactly Is the Flaw in Current CFG Distillation?

According to the paper "Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation" (arXiv, July 2026), the core problem is that existing OPD methods extend velocity matching to the CFG-composed prediction, directly matching teacher and student guided velocities. The authors show that this objective is "under-identified at the branch level": the composed velocity is a linear combination of conditional and unconditional components, and matching the sum alone does not force the individual components to align. This means the student can learn incorrect branch dynamics that still satisfy the loss, leading to poor sample quality.

# How Does the Proposed Fix Resolve This Ambiguity?

The authors propose a simple reparameterization: instead of matching the composed velocity, distill the conditional and unconditional components separately. They call this Branch-Level Distillation (BLD). According to the paper, this ensures that each branch is constrained independently, eliminating the under-identification. The approach requires no additional teacher queries—the same teacher call that produces the composed velocity also yields the individual components. This makes BLD a drop-in replacement for existing OPD pipelines.

# What Evidence Supports That Branch-Level Distillation Works?

The paper provides both theoretical and empirical evidence. Theoretically, they prove that the standard composed-velocity loss has a degenerate solution space, while the branch-level loss has a unique optimum. Empirically, they demonstrate on ImageNet 64x64 and 256x256 that BLD consistently outperforms the standard approach: FID scores improve by 10-20% across guidance scales, and training is more stable, with fewer loss spikes. The authors also show that BLD generalizes to different teacher architectures, including EDM and DDPM.

# Which Current Systems Are Affected by This Flaw?

The flaw affects all on-policy distillation methods that use CFG, including prominent models like SDXL-Turbo (Stability AI, 2023), Latent Consistency Models (Luo et al., 2023), and Progressive Distillation (Salimans & Ho, 2022). According to the paper, any system that matches the composed velocity directly is subject to the under-identification issue. This is a broad class that includes many of the fastest inference models in production today.

# What Are the Practical Implications for Distillation Practitioners?

For practitioners, the fix is straightforward: modify the loss function to compute separate losses for the conditional and unconditional branches, then sum them. The paper reports that this change requires no hyperparameter tuning and adds negligible computational overhead. However, it does require access to the teacher's unconditional predictions, which most modern diffusion models already provide. The authors provide pseudocode and a reference implementation in the appendix.

# What Remains Uncertain About This Approach?

While the theoretical and empirical results are strong, the paper only tests on class-conditional ImageNet. It remains to be seen whether BLD generalizes to text-conditional models like Stable Diffusion, where the unconditional branch is less well-defined. Additionally, the paper does not explore how BLD interacts with other distillation techniques like adversarial losses or score matching. According to the authors, these are directions for future work.

Aspect	Standard Composed-Velocity Distillation	Branch-Level Distillation (BLD)
Loss function	MSE on composed velocity	Sum of MSE on conditional and unconditional velocities
Theoretical uniqueness	Under-identified (multiple solutions)	Unique optimum
Training stability	Frequent loss spikes	Stable convergence
FID (ImageNet 256x256, CFG=3.0)	8.2	6.9
Computational overhead	Baseline	Negligible (same teacher queries)
Implementation complexity	Standard	Drop-in replacement
Verdict	Flawed; should be replaced	Recommended for all OPD pipelines

1. By Q1 2027, Stability AI will release an updated SDXL-Turbo that uses branch-level distillation, citing a 15% FID improvement on ImageNet.
2. By Q2 2027, the Hugging Face Diffusers library will make branch-level distillation the default in its training scripts, deprecating the composed-velocity approach.
3. By Q3 2027, at least two major cloud inference providers (e.g., Replicate, Banana) will offer distilled models trained with BLD, advertising them as "CFG-correct."

• Insight 1: The under-identification flaw is not a bug in CFG itself, but in how it is composed with velocity matching—a subtle but critical distinction.
• Insight 2: Branch-level distillation requires no additional teacher queries, meaning it is a free improvement for any existing OPD pipeline.
• Insight 3: The paper's principle—constrain components not just sums—may apply to other areas of diffusion model training, such as guidance scaling and adversarial training.