PixelMoE: Scaling Pixel-Space Diffusion with LLM-Style Mixture-of-Experts

   

📖 Introduction

PixelMoE brings standard LLM-style Mixture-of-Experts (MoE) to pixel-space diffusion for class-conditional ImageNet generation at 256×256 and 512×512 resolution. Each image token activates 4 of 32 routed experts, alongside a shared expert, with bias-based load balancing. Pixel-Space Hierarchical Flow Matching generates images in two stages, first establishing the overall structure at low resolution, then refining details at the target resolution.

PixelMoE achieves FID 1.47 at 256×256 and 1.53 at 512×512 on class-conditional ImageNet-1K, without representation alignment or perceptual losses.

📦 Checkpoints

Model	256×256	FID ↓	512×512	FID ↓
PixelMoE-B-E32A4	Download	1.88	Download	2.05
PixelMoE-L-E32A4	Download	1.54	Download	1.58
PixelMoE-H-E32A4	Download	1.47	Download	1.53

FID scores are the ImageNet-1K results reported in the paper. B, L, and H denote Base, Large, and Huge; E32A4 denotes 32 routed experts with 4 activated per token.

🚀 Download Models

Download a checkpoint with the Hugging Face CLI:

pip install -U huggingface_hub
hf download KlingTeam/PixelMoE pixelmoe_h_e32a4_256.pth --local-dir checkpoints

To download the entire model repository instead:

hf download KlingTeam/PixelMoE --local-dir checkpoints

📌 Note

This repository hosts the pretrained checkpoints of PixelMoE.

📖 Citation

If you find PixelMoE useful in your research, please cite our paper:

@misc{dong2026pixelmoescalingpixelspacediffusion,
  title={PixelMoE: Scaling Pixel-Space Diffusion with LLM-Style Mixture-of-Experts}, 
  author={Ruixiao Dong and Jianmin Bao and Bozhi Luan and Zijie Meng and Xin Tao and Pengfei Wan and Kun Gai and Li Li and Houqiang Li},
  year={2026},
  eprint={2610.18857},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2610.18857}, 
}

Contact

For questions, please contact dongruixiao@kuaishou.com.