TLDR: Q-Sched is a new method that makes text-to-image diffusion models more efficient by modifying their noise scheduler instead of their weights. It achieves a 4x reduction in model size while maintaining or improving image quality, as shown by FID scores and user studies. Q-Sched uses a novel ‘Joint Alignment-Quality (JAQ)’ loss for optimization, balancing text-image compatibility and visual quality with minimal calibration data. This approach demonstrates that quantization and few-step distillation can work together to create high-fidelity, compressed generative models.
Text-to-image diffusion models, like Stable Diffusion XL, have revolutionized image generation, but they come with a significant computational cost. These models often require many processing steps through large neural networks, making them expensive to run, especially without specialized hardware like datacenter GPUs.
To address this, researchers have developed ‘few-step diffusion models’ that reduce the number of denoising steps, significantly speeding up inference. However, these models still rely on large, uncompressed backbones (like U-Nets or diffusion transformers), which limits the application of further compression techniques such as post-training quantization.
A new research paper, titled “Q-Sched: Pushing the Boundaries of Few-Step Diffusion Models with Quantization-Aware Scheduling” by Natalia Frumkin and Diana Marculescu from The University of Texas at Austin, introduces a novel approach to overcome these limitations. Q-Sched is a post-training quantization method that modifies the diffusion model’s scheduler rather than directly altering the model’s weights. This innovative strategy allows for a substantial 4x reduction in model size while maintaining, and often improving, image fidelity.
The core idea behind Q-Sched is to adjust the few-step sampling trajectory to be ‘quantization-aware’. Instead of trying to make a quantized model perfectly mimic a full-precision model, Q-Sched learns a new, optimized sampling path. This is achieved through two learnable scalar preconditioning coefficients, ‘cx’ and ‘cϵ’, which are applied to the input image and the predicted noise, respectively. These coefficients are optimized using a new metric called the Joint Alignment-Quality (JAQ) loss.
The JAQ loss is a crucial innovation. Unlike traditional metrics that might focus solely on text-image compatibility (like CLIPScore) or pure image quality, JAQ combines both. This allows Q-Sched to balance how well an image matches the text prompt with its overall visual quality, preventing issues like color distortion or the generation of irrelevant details. Importantly, JAQ is ‘reference-free’, meaning it doesn’t need a ground truth image for comparison, making it highly practical for generative tasks. It also requires only a handful of calibration prompts, avoiding the need for extensive full-precision inference during calibration, which is a common bottleneck for other methods.
The results presented in the paper are compelling. Q-Sched achieved a 15.5% FID improvement over the FP16 4-step Latent Consistency Model and a 16.6% improvement over the FP16 8-step Phased Consistency Model. FID (Fréchet Inception Distance) is a widely used metric for evaluating the quality of generated images, with lower scores indicating better quality. These improvements demonstrate that quantization and few-step distillation are not conflicting but rather complementary strategies for achieving high-fidelity image generation with greater efficiency.
Furthermore, extensive user studies involving over 80,000 human annotations confirmed Q-Sched’s effectiveness on popular models like FLUX.1[schnell] and SDXL-Turbo. Users consistently preferred images generated by Q-Sched over those from other quantization methods, even in aggressive compression settings where other metrics might suggest a decline in quality. The research also highlights that jointly optimizing both pre-conditioning coefficients (cx and cϵ) yields the best image quality.
Also Read:
- Draw-In-Mind: Enhancing Image Editing Precision Through Explicit Design Blueprints
- Conditional-t3VAE: A New Approach for Fair Image Generation in Imbalanced Datasets
In summary, Q-Sched represents a significant step forward in making powerful text-to-image diffusion models more accessible and efficient. By intelligently modifying the noise scheduler and introducing a robust, reference-free optimization loss, it enables substantial model size reductions and faster inference without sacrificing the impressive image quality these models are known for. The code for Q-Sched will be released upon publication and is available at https://github.com/enyac-group/q-sched.
