Timestep-Aware SVDQuant-GPTQ for W4A4 Quantization of Wan2.2-I2V

Resource information

Authors

Junhao Wu, Dezhong Yao, Hai Jin

Research area

Computer vision

Application domain

General AI research

AI methods

Transformers

Research topics

cs.CV, cs.AI

arXiv record date

2026-05-26

Publication

arXiv

Upstream last modified

May 27, 2026

Last record update

Nov 4, 2026

Abstract

W4A4 quantization of large video diffusion Transformers offers substantial memory savings but is hindered by two main challenges: sparse large-magnitude activation outliers, and strongly timestep-dependent activation distributions across the multi-step denoising trajectory. These difficulties are compounded by Wan2.2-I2V's two-expert Mixture-of-Experts DiT design, whose high-noise and low-noise experts exhibit distinct quantization sensitivities that a single global calibration policy cannot capture. We propose a post-training quantization framework combining SVDQuant-based low-rank outlier compensation, GPTQ-based reconstruction-aware residual weight quantization, and timestep-bin-wise per-layer activation clipping-ratio search conducted independently for each expert. On the OpenS2V-Eval benchmark, our method reduces peak GPU memory by 59.3% relative to the BF16 baseline while incurring only a 0.9% drop in VBench average score and a 2.3% drop in Imaging Quality, demonstrating that expert- and timestep-aware calibration is essential for high-fidelity W4A4 inference on MoE video DiTs.