EMA: A Quiet Hyperparameter That Moves Diffusion Leaderboards

Summary

Controlled EMA-decay sweeps on a fixed training run for representation-space RAE-DiT, pixel-space JiT, and latent-space SiT. Same RAE-DiT-XL ImageNet-256 80-epoch run: gFID 4.14 at β=0.9999 vs 3.21 at β=0.999; EPFID@4 can move from 32 epochs to beyond 80. Larger decay raises precision and drops recall (soft collapse on a 2D tree toy). Recommends reporting and matching EMA in short-schedule comparisons. No code release in the post.

Keywords

ema · diffusion · fid · rae · sit · jit · evaluation · blog · x

Topics

diffusion training, EMA, evaluation protocol

Research notes

• Primary: technical blog (cite wang2026ema). Discord posted https://t.co/xr7CNC3Ag7 which resolves to this URL. Rice CRC acknowledged. No repo linked. Blog/analysis, not a hosted corpus, so no datasets_local row.