Taming Outlier Tokens in Diffusion Transformers
Authors Xiaoyu Wu†*, Yifei Wang†*, Tsu-Jui Fu, Liang-Chieh Chen, Zhe Gan, Chen Wei†
View publication：https://arxiv.org/abs/2605.05206
DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture Design in Text to Image Generation
December 11, 2025 research area Computer Vision：/research/?domain=Computer%20Vision
In this work, we empirically study Diffusion Transformers (DiTs) for text-to-image generation, focusing on architectural choices, text-conditioning strategies, and training protocols. We evaluate a range of DiT-based architectures—including PixArt-style and MMDiT variants—and compare them with a standard DiT variant which directly processes concatenated text and noise inputs. Surprisingly, our findings reveal that the performance of standard…
On Inductive Biases That Enable Generalization of Diffusion Transformers
September 22, 2025 research area Computer Vision：/research/?domain=Computer%20Vision conference NeurIPS：/research/?event=NeurIPS
Recent work studying the generalization of diffusion models with UNet-based denoisers reveals inductive biases that can be expressed via geometry-adaptive harmonic bases. However, in practice, more recent denoising networks are often based on transformers, e.g., the diffusion transformer (DiT). This raises the question: do transformer-based denoising networks exhibit inductive biases that can also be expressed via geometry-adaptive harmonic…
Our research in machine learning breaks new ground every day.
Scan to open this article