Beyond the CFG Crutch: How Correction Training is Rewiring Diffusion Architectures

The Exposure Bias Trap in Diffusion Trajectories

Diffusion models have long suffered from a fundamental training-inference gap. During training, the denoiser is fed clean images corrupted by noise, but during sampling, it is forced to operate on its own imperfect, noisy predictions. This creates an exposure bias where early structural errors—like a malformed facial feature—are not corrected but rather amplified as the model adds fine-grained detail around the initial mistake.

WORKFLOW_TIMELINE: [Input Noise] -> [Denoiser Prediction] -> [Accumulation of Artifacts] -> [Correction Intervention Point] -> [Refined Output].

This shift toward iterative correction mirrors broader industry trends in closed-loop learning where models refine their own outputs to avoid the pitfalls of static training. By introducing a corrector that acts on the denoiser's output at every step, we effectively force the model to 'see' its own errors before they become permanent features of the final image.

Breaking the CFG Monopoly on Image Fidelity

For years, Classifier-Free Guidance (CFG) has been the industry's blunt instrument for improving image quality. While effective, it is notorious for causing oversaturation and significantly reducing the diversity of generated outputs, effectively forcing the model into a narrow, high-contrast aesthetic.

Correction Training offers a more surgical alternative, allowing for high-fidelity generation without the aggressive saturation inherent in CFG. By decoupling the guidance mechanism from the base model's training objective, developers can achieve superior FID scores while maintaining a broader, more natural distribution of outputs.

Amortizing the Cost of Iterative Refinement

Critics often point to the inference-time overhead of applying a corrector multiple times as a barrier to adoption. However, the research suggests that the emergent properties of this method are too significant to ignore, as 'applying correction beyond the single step supervised during training is key to the significant quality gains.'

Just as modern tools prioritize AI signal verification to ensure output integrity, the correction framework treats the denoiser's output as a signal requiring immediate validation. To solve the latency issue, the framework proposes distilling these refined, multi-step corrections back into the base model. This effectively bakes the 'correction' logic into the primary architecture, allowing for high-quality inference without the need for repeated corrector passes.

Beyond DiT: Architectural Agnosticism

One of the most compelling aspects of Correction Training is its versatility across different model families. Whether applied to DiT, SiT, or other diffusion architectures, the framework consistently outperforms unguided baselines, proving that this is a fundamental architectural improvement rather than a model-specific hack.

• DiT/SiT Performance: Significant FID improvements over unguided baselines, often surpassing CFG in both fidelity and diversity.
• JiT Limitations: While still effective, the gains are more modest compared to DiT, suggesting that architectural depth plays a role in how well the corrector can refine the latent space.
• Universal Applicability: The framework functions as a plug-and-play module, demonstrating that the 'correction' logic is a universal requirement for modern, high-resolution diffusion models.