TLDR: This paper introduces Adjustable Sequence Length (ASL), a novel method for making Stochastic Computing (SC) neural networks more energy-efficient and faster for Internet of Things (IoT) devices. ASL achieves this by intelligently adjusting the length of bitstreams in different layers of the neural network, reducing energy and latency by over 60% with minimal accuracy loss. The research also provides a theoretical model for noise propagation and validates the use of Sobol sequences for maintaining accuracy with truncated bitstreams.
The rapid expansion of neural networks (NNs) into various domains, including robot control, recommender systems, and natural language processing, has highlighted a critical challenge: their demanding computational requirements, especially for resource-limited devices like those found in the Internet of Things (IoT).
Stochastic computing (SC) has emerged as a promising alternative to traditional floating-point (FP) designs for deploying NNs in these energy-constrained environments. SC works by encoding numerical values as serial bitstreams, which significantly reduces energy consumption. However, the potential for further improvements through layer-wise mixed-precision implementation in SC has remained largely unexplored until now.
A new scheme called Adjustable Sequence Length (ASL) has been introduced to address this gap. ASL applies the concept of mixed-precision specifically to SC NNs. The core idea is to intelligently truncate the bitstreams used in different layers of the neural network. This approach is particularly effective because, in SC, all bits in a sequence are considered equivalent, meaning that shortening the sequence primarily affects the precision (increasing variance) rather than fundamentally altering the encoded value’s expectation.
The researchers developed a theoretical model based on operator norms to understand how truncation noise propagates and accumulates through the layers of an SC NN. This model helps estimate how much this noise might be amplified. To validate these theoretical predictions, an extended sensitivity analysis was performed using Random Forest (RF) regression. This analysis confirmed that truncation noise can indeed propagate cumulatively, and importantly, that earlier layers in the network are more sensitive to this noise. This means that maintaining longer bitstreams in the initial layers is crucial for preserving overall accuracy.
To accommodate diverse application needs, ASL proposes two distinct truncation strategies: a coarse-grained approach and a fine-grained approach. The coarse-grained strategy applies generalized configurations, typically preserving full sequence length in early layers and progressively truncating later ones. For instance, a common configuration might be to use full length for the first layer, half for the second, and a quarter for subsequent layers. The fine-grained strategy, on the other hand, involves a more detailed exploration, often using a grid search on a subset of the data to find the optimal balance between accuracy loss and energy/latency savings for specific scenarios.
A key aspect of ASL’s effectiveness lies in its choice of random number generators (RNGs). The paper highlights that Sobol sequences are preferred over traditional Linear Feedback Shift Registers (LFSRs) for generating stochastic bitstreams. This is because Sobol sequences maintain a low correlation even after truncation, ensuring consistent accuracy, whereas LFSRs can suffer from disrupted uniform distributions when truncated.
Evaluations on a pipelined SC Multi-Layer Perceptron (MLP) synthesized at a 32nm technology node demonstrated impressive results. ASL was shown to reduce energy and latency overheads by over 60% with only negligible accuracy loss. For example, with a full model sequence length of 1024, the fine-grained strategy achieved energy savings of up to 49.83% and latency savings of up to 64.94% on datasets like Fashion-MNIST, SVHN, and CIFAR10, all while keeping accuracy loss below 0.1%.
Also Read:
- Dynamic Quantization Training: A Dequantization-Free Path to Efficient AI
- New Framework Converts AI Models to Energy-Efficient Spiking Networks Without Retraining
This research confirms the practical feasibility of the ASL scheme for IoT applications and underscores the significant advantages of mixed-precision truncation in SC designs. It offers a powerful approach for balancing efficiency and accuracy, making SC NNs even more suitable for edge and embedded systems where conventional FP hardware might be too costly or power-intensive. For more technical details, you can refer to the full research paper: Energy-Efficient Stochastic Computing (SC) Neural Networks for Internet of Things Devices with Layer-Wise Adjustable Sequence Length (ASL).
