TLDR: A new method called REQuant improves post-training quantization for deep neural networks. It addresses accuracy loss in low-bit quantization by optimizing the range of weights and reshaping their distribution. The method, which uses an efficient search algorithm, significantly boosts accuracy in 4-bit, 6-bit, and 8-bit settings for models like ResNet and Inception-v3 on image classification tasks, making neural networks more efficient for resource-constrained devices.
Deep neural networks (DNNs) have achieved remarkable success in various fields like image classification and natural language processing. However, their increasing complexity and number of layers demand substantial computing and storage resources. This creates significant challenges for deploying these powerful models in resource-constrained environments such as mobile devices, IoT devices, and edge computing nodes.
To address this, researchers have developed various model compression techniques. Among these, quantization stands out as a highly promising method due to its ability to achieve high compression rates with minimal accuracy reduction. Quantization involves approximating the standard 32-bit floating-point parameters of a model with lower bit-width representations, effectively reducing memory footprint and speeding up execution.
There are two main approaches to quantization: Quantization-Aware Training (QAT) and Post-Training Quantization (PTQ). While QAT integrates quantization into the training process, PTQ applies quantization after the model has been fully trained. This paper focuses on Post-Training Quantization, which has become a standard procedure for creating efficient low-precision neural networks without the need for retraining.
A significant challenge in PTQ is maintaining model accuracy, especially when quantizing to very low bit-widths. The distribution of full-precision weights and activations is crucial, and efficient quantization must preserve the informative parts of the original data. When multiple values are mapped to a single quantized value, determining the optimal quantization levels is key to minimizing accuracy degradation.
Researchers Bingtao Yang, Yujia Wang, Mengzhi Jiao, and Hongwei Huo from Xidian University have introduced a novel method called REQuant (Range Estimation for Quantization) to tackle this problem. Their approach aims to improve the performance of post-training quantization by focusing on two core strategies: range estimation and weight transformation.
Range Estimation for Quantization
The first strategy, range estimation, addresses the issue that simply using the maximum weight values to determine quantization intervals can lead to significant accuracy drops, particularly in low-bit settings. This is because most deep neural network weights are densely distributed around zero, and a large quantization interval (derived from maximum values) fails to distinguish their subtle differences. REQuant introduces a parameter to effectively reduce the range of weights that need to be quantized. It maps outlier weights (those at the rim of the distribution) inward, thereby mitigating quantization loss. This mapping process is framed as an optimization problem, minimizing quantization errors for each layer. The researchers proved that this problem is locally convex and developed an efficient search algorithm, specifically the Golden Section Search, to find the optimal solution.
Reshaping Weight Distribution
The second strategy further optimizes the quantization model by reshaping the distribution of weights. The paper notes that power functions are effective for redistributing weights, especially given the bell-shaped distribution often seen in convolutional neural networks. By applying a square root function to the absolute values of the weights, REQuant can achieve non-uniform quantization in the original weight space. This adaptive allocation of quantization intervals means smaller intervals are used for densely distributed weights and larger intervals for sparsely distributed ones, further enhancing accuracy. Similar to range estimation, this process is also modeled as a locally convex optimization problem, allowing the same efficient search algorithm to be applied.
Also Read:
- Scaling Neural Networks to Millions of Classes with Latent Space Configuration
- Optimizing Generative AI: A Lossless Compression Breakthrough for Model Weights
Experimental Validation
The effectiveness of REQuant was rigorously tested on CIFAR-10 and CIFAR-100 datasets using popular deep learning architectures, including the ResNet series (ResNet-18, ResNet-34, ResNet-50, ResNet-101) and Inception-v3. The results demonstrated that REQuant generally outperforms state-of-the-art post-training quantization methods in terms of top-1 accuracy. For 8-bit and 6-bit quantization, REQuant achieved almost no loss in top-1 accuracy compared to full-precision models. Crucially, it significantly improved the accuracy of 4-bit quantization, a notoriously challenging low-bit setting.
An ablation study confirmed the individual contributions of both the clipping (range estimation) and reshaping operations. Each component was shown to independently improve accuracy, with their combination yielding the best performance, especially in low-bit scenarios. For instance, ResNet-50 at 4-bit precision saw a dramatic improvement from 78.70% (baseline) to 94.47% with REQuant on CIFAR-10. The Golden Section Search algorithm was also identified as the most efficient for solving the optimization problems, offering minimum quantization loss with the fastest search time.
In conclusion, REQuant offers a robust and effective method for post-training quantization, significantly preserving model accuracy even at very low bit-widths. This advancement is crucial for deploying high-performing deep neural networks on devices with limited computational and storage capabilities. The code for REQuant is publicly available for further exploration and use. You can read the full research paper here.
