TLDR: AIM is a novel software and hardware co-design approach to mitigate IR-drop issues in high-performance Processing-in-Memory (PIM) chips. It introduces metrics like Rtog and HR to correlate workloads with IR-drop. Software methods, Lower Hamming Rate (LHR) and Weight Distribution Shift (WDS), optimize data to reduce IR-drop potential with minimal accuracy loss. Hardware component, IR-Booster, dynamically adjusts voltage-frequency pairs based on HR and real-time monitoring, ensuring reliability. An HR-aware task mapping strategy further optimizes performance for mixed workloads. The system achieves up to 69.2% IR-drop mitigation, 2.29x energy efficiency improvement, and 1.152x speedup on a 7nm 256-TOPS PIM chip.
High-performance computing is constantly pushing the boundaries of what’s possible, and a technology called Processing-in-Memory (PIM) is at the forefront. PIM seamlessly integrates computing capabilities directly within memory units, offering impressive gains in computing density, energy efficiency, and precision. This approach is particularly promising for demanding applications like artificial intelligence.
However, as PIM chips become more powerful and operate at higher speeds, they face a significant challenge known as IR-drop. Imagine a complex city with many buildings drawing power simultaneously; if the power lines aren’t robust enough, some buildings might experience a dip in voltage. Similarly, in a PIM chip, the simultaneous operation of numerous computing units can cause the supply voltage to drop, leading to performance degradation and even reliability issues. Traditional methods to combat IR-drop, often involving complex circuit-level adjustments, are resource-intensive and can compromise the chip’s overall power, performance, and area (PPA).
To address this critical issue, researchers have introduced a novel approach called AIM: Software and Hardware Co-design for Architecture-level IR-drop Mitigation in High-performance PIM. This comprehensive solution tackles IR-drop from a system-wide perspective, combining intelligent software optimizations with dynamic hardware adjustments. You can read the full research paper for more technical details here: AIM: Software and Hardware Co-design for Architecture-level IR-drop Mitigation in High-performance PIM.
Understanding the Problem: Workloads and IR-drop
A key insight behind AIM is recognizing the direct link between the tasks a PIM chip performs (its workload) and the severity of IR-drop. Unlike general-purpose processors, PIM architectures often handle predictable and regular workloads, primarily neural networks. This predictability creates an opportunity for architecture-level optimization.
AIM introduces two crucial metrics to quantify this relationship: Rtog (instantaneous toggle rate) and HR (Hamming Rate). Rtog measures the rapid switching of bits within a PIM bank, which directly correlates with dynamic current draw and thus IR-drop. HR, derived from Rtog, simplifies this by focusing on the average ‘1’ bits in the pre-loaded memory data (weights). Crucially, HR is independent of the input data, making it an optimizable metric for software-based mitigation.
Software Innovations: Optimizing for Lower IR-drop
AIM’s software component focuses on reducing the Hamming Rate (HR) of the data stored in memory, thereby lowering the potential for IR-drop. It achieves this through two main methods:
-
Lower Hamming Rate (LHR): This technique integrates a regularization term into the quantization process, which is how neural network weights are converted into a more compact, lower-precision format. LHR encourages weights to align with values that naturally have lower HR, such as 0 or 8 in an 8-bit system, without significantly impacting the accuracy of the neural network. This is like subtly nudging the data to be more ‘IR-drop friendly’ during its creation.
-
Weight Distribution Shift (WDS): Building on LHR, WDS further refines the weight distribution. It shifts the entire distribution of weights towards positive values, concentrating them around smaller positive numbers that inherently have lower HR. To maintain computational accuracy, a clever hardware mechanism called a ‘shift compensator’ is used to correct for this shift after calculations, ensuring the final results are correct without adding delays to critical operations.
Hardware Intelligence: Dynamic Adaptation
While software optimizations prepare the data, AIM’s hardware component, IR-Booster, provides real-time, dynamic adjustments to mitigate IR-drop during operation. PIM chips are often divided into ‘Macro Groups’ that share power supplies and operate at the same frequency.
-
IR-Booster V-f Pair Adjustment: IR-Booster extends traditional Dynamic Voltage and Frequency Scaling (DVFS) by offering more granular control. Instead of just adjusting voltage and frequency based on general workload, it uses the pre-determined HR information for each Macro Group to select optimal voltage-frequency (V-f) pairs. This allows for lower voltages or higher frequencies (or both) when the HR is low, significantly improving energy efficiency and performance. It operates in ‘sprint mode’ for maximum throughput or ‘low-power mode’ for energy savings.
-
IRFailure-driven Recomputing: To ensure reliability, IR-Booster includes an ‘IR Monitor’ that constantly checks for voltage drops. If an IR-drop violation (IRFailure) occurs, the system quickly adjusts the V-f pair to a safer level and initiates a recomputation for the affected tasks. This mechanism ensures that even aggressive optimizations don’t compromise the chip’s functional integrity.
Co-optimization: HR-aware Task Mapping
To maximize the benefits of both software and hardware, AIM introduces HR-aware task mapping. In complex scenarios where multiple tasks with varying HR values run concurrently, simply assigning them randomly can lead to inefficiencies. HR-aware task mapping intelligently assigns tasks to PIM macros, considering their HR values. This prevents tasks with very different IR-drop characteristics from interfering with each other, ensuring that Macro Groups can operate at their most efficient V-f pairs. This optimization is crucial for maintaining high performance and energy efficiency across diverse workloads.
Also Read:
- Optimizing Neural Radio Receivers with Fibbinary Compression and Quantization
- SnapStream: Boosting LLM Performance and Memory Efficiency for Extended Contexts
Impressive Results
Evaluations on a 7nm 256-TOPS PIM chip design demonstrate the significant impact of AIM. The system achieved up to 69.2% IR-drop mitigation within a macro. This translates to a remarkable 2.29 times improvement in energy efficiency and a 1.152 times speedup in overall chip performance. The study also showed that while IR-Booster provides the primary benefits for transformer-based AI models (due to their dynamic nature), LHR and WDS are particularly effective for convolution-based networks where weights can be pre-optimized.
AIM represents a significant step forward in designing high-performance PIM chips. By intelligently co-designing software and hardware, it effectively addresses the critical challenge of IR-drop, paving the way for more powerful, efficient, and reliable AI accelerators.
