TLDR: This research introduces an innovative in-memory computing accelerator for Convolutional Neural Networks (CNNs) utilizing Racetrack Memory (RM). It tackles the memory bottleneck in embedded AI by designing RM-optimized arithmetic circuits and proposing a ‘write-shift transformation’ to convert energy-costly write operations into efficient shifts. Additionally, a shift-based multiplier leverages RM’s intrinsic capabilities for logarithmically-quantized models, yielding significant energy savings (up to 83.5x) and performance improvements (up to 1.68x) while maintaining model accuracy. The work demonstrates a promising hardware-software co-design approach for high-performance, energy-efficient AI in resource-constrained environments.
In the rapidly evolving world of Artificial Intelligence, Deep Neural Networks (DNNs) are at the forefront, driving advancements in various applications. However, these powerful networks demand significant computational resources and memory, posing a considerable challenge, especially for deployment on compact, low-resource embedded systems. The traditional architecture, where processing units and memory are separate, often leads to a ‘memory wall’ bottleneck, as data constantly needs to be moved back and forth, consuming energy and time.
A promising solution to this challenge is In-Memory Computing (IMC), an innovative paradigm where computations are performed directly within the memory units. This approach drastically reduces the need for data transfers, thereby enhancing efficiency. Among the emerging memory technologies, Racetrack Memory (RM) stands out due to its high data density, non-volatility, and potential for high speed and low power consumption. RM stores data in magnetic domains that can be shifted along nanowires, acting like a shift register.
Despite its advantages, integrating arithmetic circuits directly into Racetrack Memory presents its own set of hurdles. Conventional memory designs and data layouts are not optimized for RM’s unique shifting access mechanism, leading to inefficiencies in area and power. This research paper, titled “Hardware-software co-exploration with racetrack memory based in-memory computing for CNN inference in embedded systems”, addresses these challenges head-on by proposing an efficient in-memory Convolutional Neural Network (CNN) accelerator specifically optimized for Racetrack Memory.
Key Innovations for Efficient In-Memory Computing
The researchers, Benjamin Chen Ming Choong, Tao Luo, Cheng Liu, Bingsheng He, Wei Zhang, and Joey Tianyi Zhou, have introduced several key innovations:
-
Efficient Arithmetic Circuits: They designed fundamental arithmetic circuits, such as adders and multipliers, specifically tailored for in-memory computing with Racetrack Memory. These circuits are optimized to minimize resource usage and the number of costly write operations.
-
Write-Shift Transformation: A novel energy optimization technique was developed to convert energy-intensive write operations into more efficient shift operations within the RM-based arithmetic circuits. A single write operation in RM can consume about 20 times more energy than a shift operation. By transforming these writes to shifts, the full adder circuit, for instance, saw its energy consumption reduced by an impressive 94.4%.
-
Shift-Based Multiplier: Leveraging the inherent shifting capability of Racetrack Memory, the paper proposes a shift-based multiplier. This is particularly effective for accelerating logarithmically-quantized CNN models, where multiplications can be simplified to shift-and-add operations. This approach avoids the generation and accumulation of partial products, which are common in traditional multiplication methods like Booth multiplication, leading to significant throughput increases and reduced resource consumption.
Accelerator Architecture and Dataflow
The proposed accelerator integrates these optimized arithmetic units into a modular RM-based memory system, organized into banks, mats, and subarrays. The architecture is designed to exploit parallelism and data reuse, especially for CNN operations. For instance, weight values are spatially reused across multiple memory tracks, and partial sums are accumulated immediately to keep activation storage small.
The paper details how various CNN layers, including convolutional, fully-connected, batch normalization, and pooling layers, are mapped onto this accelerator. It highlights strategies like alternating access patterns to hide sequential access latency and distributing input channels across multiple mat groups for parallel processing.
Also Read:
- Boosting Recommender System Performance Through Optimized Data Flow
- Boosting AI Performance on RISC-V: A New Approach to Vector Unit Optimization
Performance and Efficiency Gains
Experimental results demonstrate the significant impact of these innovations. The write-shift transformation alone dramatically improves system-level energy efficiency, reducing total energy consumption by 67% for 4-bit models and up to 83% for 16-bit models. When comparing the shift-based multiplication to traditional Booth multiplication, the shift-based approach leads to over 70% less energy for LeNet-5 and over 90% for ResNet-20. Furthermore, it offers performance improvements, with up to 1.68 times higher performance for ResNet-20.
The research also explores the trade-offs between inference accuracy, energy, and latency with different quantization schemes and bit-widths. For smaller models like LeNet-5, 4-bit shift-based models maintain high accuracy with substantial energy and latency reductions. For more complex models like ResNet-20, an 8-bit shift-based model achieves significant energy savings and latency reduction with a reasonable accuracy loss.
The scalability of the accelerator was also evaluated using large models like VGG-16, demonstrating how batching inputs can effectively reduce external DRAM accesses, especially for model parameters. This research underscores the potential of hardware-software co-design in unlocking the full capabilities of emerging memory technologies for energy-efficient and high-performance AI in embedded systems.
