TLDR: This research introduces the Load-Balance-Longest-Path (LBLP) algorithm for efficiently deploying Convolutional Neural Network (CNN) models on multi-processing In-Memory Computing (IMC) hardware. By dynamically assigning CNN nodes to available processing units based on execution times and load balancing, LBLP significantly improves processing rate and reduces latency compared to other scheduling strategies, as demonstrated through experiments on an IMC Emulator using ResNet and YOLO models.
Deep learning models, especially Convolutional Neural Networks (CNNs), are becoming increasingly vital across various computing platforms, from small edge devices to large data centers. However, the sheer computational complexity of these networks often leads to significant challenges, particularly concerning data movement and energy consumption in traditional hardware accelerators like GPUs and TPUs.
A promising solution to these challenges is In-Memory Computing (IMC). This innovative approach performs computations directly within memory arrays, drastically reducing the need for data transfers. This leads to improved throughput and higher energy efficiency, making it an attractive alternative for accelerating deep learning.
While IMC offers substantial benefits, efficiently deploying CNN workloads on IMC-based hardware, especially systems with multiple processing units (PUs), remains a complex task. The way computational tasks are mapped to these PUs directly impacts performance metrics such as latency and processing rate. The goal is to distribute the workload optimally to maximize efficiency.
Researchers Eleni Bougioukou and Theodore Antonakopoulos from the University of Patras, Greece, have addressed this challenge by introducing a novel task allocation strategy. Their work, titled “Efficient Deployment of CNN Models on Multiple In-Memory Computing Units,” focuses on maximizing processing rate and minimizing latency in multi-processing IMC systems. You can read their full research paper here.
The core of their contribution is the Load-Balance-Longest-Path (LBLP) algorithm. This algorithm dynamically assigns all CNN nodes (individual operations within the network) to the available IMC Emulator (IMCE) Processing Units. LBLP works by first identifying the “longest path” – the sequence of nodes with the highest total execution time. It then sorts these nodes and assigns them to PUs in a way that balances the workload, always picking the PU with the smallest total assigned execution time. This process is repeated for non-longest path nodes, ensuring that parallel operations are assigned to different PUs whenever possible to maximize parallel execution.
To evaluate LBLP, the researchers utilized an In-Memory Computing Emulator (IMCE), a modular platform designed to mimic the behavior of IMC integrated circuits. The IMCE consists of a Front-End Unit, multiple Processing Units (PUs), and a Configuration and Data Analytics Server. The PUs themselves can be of two types: IMC-PUs, optimized for matrix-vector multiplications and convolutions, and DPU-PUs, which handle digital operations like pooling and concatenation. This hybrid architecture reflects the complexity of real-world IMC chips.
The LBLP algorithm was benchmarked against three other scheduling strategies: Weights Balance (WB), Round-Robin (RR), and Random (RD). These algorithms represent different approaches to task distribution, from balancing based on weights or execution times to simple cyclic or random assignments.
Experimental results, using popular CNN models like ResNet8, ResNet18, and YOLOv8n, consistently demonstrated the superior effectiveness of the LBLP algorithm. For instance, with ResNet8 and ResNet18, LBLP consistently delivered the highest processing rates and lowest latencies across various configurations of available PUs. In one comparison with ResNet18, LBLP achieved more than double the processing rate and 1.4 times less latency compared to the Weights Balance algorithm, primarily due to a much more uniform utilization of all PUs.
Even in scenarios where the mix of IMC and DPU cores varied, LBLP maintained its significant performance advantage over other methods. This indicates its robustness and adaptability to different hardware configurations, which is crucial for designing efficient edge AI chips.
Also Read:
- Optimizing Compute-in-Memory for AI: The Manhattan Distance Mapping Approach
- Boosting Hybrid AI Models with Custom RISC-V GPU Instructions
In conclusion, this research highlights that the performance of CNN inference engines isn’t just about raw processing power; it’s also critically dependent on how efficiently CNN models are mapped to the internal architecture, especially in systems with multiple parallel processing units. The LBLP algorithm offers a low-complexity, effective solution that leverages the inherent parallelism of CNNs to balance resource utilization and maximize performance during inference. This work paves the way for more optimized AI model deployment on next-generation in-memory computing hardware.
