TLDR: BitParticle is a novel hardware architecture for deep neural network (DNN) acceleration that significantly improves energy and area efficiency. It achieves this by introducing a new MAC unit that effectively exploits bit-level sparsity in both weights and activations, overcoming the “partial product explosion” issue through smart grouping and concatenation. Additionally, it features a “quasi-synchronous” scheduling scheme with intra- and inter-group elasticity and adaptable dataflows, which reduces pipeline stalls and maximizes MAC unit utilization, leading to substantial performance gains over existing solutions.
Deep neural networks (DNNs) are becoming increasingly common in everyday devices, from smartphones to smart home gadgets. However, these “edge devices” often have limited computing power, making it challenging to run complex DNNs efficiently. One promising way to make DNNs more energy-efficient is to exploit “sparsity,” which refers to the presence of many zero values or zero bits within the network’s data.
There are two main types of sparsity: value-level sparsity (where entire numbers are zero) and bit-level sparsity (where individual bits within numbers are zero). While value-level sparsity has been widely studied, bit-level sparsity offers even greater potential for optimization. However, fully leveraging bit-level sparsity comes with two significant hurdles. First, most existing methods can only exploit sparsity in either the weights or the activations, not both simultaneously. This leads to wasted optimization opportunities. Second, the varying number of zero bits means that different computations can take different amounts of time, making it difficult to synchronize operations across many computing units, leading to inefficiencies.
A new research paper, “BitParticle: Partializing Sparse Dual-Factors to Build Quasi-Synchronizing MAC Arrays for Energy-efficient DNNs”, introduces a novel approach to tackle these challenges. The core of their solution is a new Multiply-Accumulate (MAC) unit, the fundamental building block for DNN computations, designed to efficiently exploit bit-level sparsity in both weights and activations.
A Smarter MAC Unit for Dual-Factor Sparsity
The BitParticle MAC unit addresses the first challenge by using a technique called “particlization.” Instead of processing entire numbers, it breaks down 8-bit numbers (representing weights and activations) into smaller “particles” of 1 or 2 bits. These particles are then cross-multiplied, generating up to 16 intermediate results (IRs). The clever part is how it handles these IRs. Unlike previous methods that might lead to an explosion of partial products, BitParticle groups these IRs based on their bit positions. This allows many of them to be “concatenated” rather than accumulated, significantly reducing the computational overhead. Even in the worst-case scenario (no sparsity), the number of partial products remains manageable, similar to a conventional multiplier.
The design also includes an “approximate” variant. Recognizing that some least significant bits might be discarded later in the DNN process, this variant intentionally discards certain IRs. This leads to a minor loss in accuracy (around 3.6% for ResNet-18 on CIFAR-10) but offers substantial reductions in hardware area (20%) and power consumption (13.6%-15.1%), providing a valuable trade-off for designers.
Flexible Synchronization for Efficient Operation
To address the second challenge of variable computation cycles and synchronization, the paper proposes a “quasi-synchronous” scheduling scheme for MAC arrays. Traditional MAC arrays often force all units to wait for the slowest one, leading to idle time. BitParticle introduces “elasticity” both within and across groups of MAC units.
Within each group (a column of MAC units), buffering queues are used. These queues allow individual MAC units to buffer incoming operations, smoothing out latency variations. A significant benefit of these queues is “zero-value filtering,” where zero-valued operands are prevented from entering the queue, effectively reducing their processing time to zero. This dramatically improves throughput, especially in networks with high value sparsity.
Between groups, the scheme allows for a certain degree of “step divergence,” meaning faster groups can advance ahead of slower ones. To maintain data consistency, a small buffer stores older weights for slower groups. This combined approach significantly reduces pipeline stalls and improves the overall utilization of the MAC units.
Adaptable Dataflows for Diverse Networks
The architecture also incorporates two switchable dataflows. Different layers within a DNN can have varying shapes (e.g., early layers might have large spatial dimensions, while later layers have more channels). These adaptable dataflows ensure that the MAC array can efficiently process different layer shapes, maximizing PE (Processing Element, equivalent to a MAC unit) utilization and reducing idle time caused by insufficient parallelism.
Also Read:
- Optimizing AI Energy Use in 5G Networks: A DeepRx Study on Efficiency and Knowledge Distillation
- Metropolis-Hastings Sampling Unlocks Efficient AI Control
Impressive Performance Gains
The evaluation results are compelling. The exact version of the BitParticle MAC array architecture achieves a 29.2% improvement in area efficiency compared to state-of-the-art bit-sparsity-driven architectures, while maintaining comparable energy efficiency. For typical neural network sparsity levels (60%-70%), it shows 23%-14% better area efficiency and 31%-25% better energy efficiency compared to AdaS, another leading approach. Compared to BitWave, it improves energy efficiency by 5.6% to 25.0% across various sparsity levels and significantly boosts area efficiency (38.2%-58.3%). The approximate variant further enhances area and energy efficiency by 2.1% and 7.5% respectively, offering a compelling option for hardware-constrained applications.
In essence, BitParticle offers a comprehensive solution for building highly energy-efficient DNN accelerators by intelligently exploiting bit-level sparsity in both weights and activations, coupled with a flexible synchronization mechanism and adaptable dataflows. This research paves the way for more powerful and efficient AI capabilities on edge devices.
