TLDR: This research introduces novel data flow optimization strategies for Deep Learning-based Recommender Models (DLRMs) to overcome performance bottlenecks in embedding layers. By proposing four tailored look-up strategies and an asymmetric table mapping framework for multi-core AI accelerators, the method achieves significant speed-ups (1.5x to 6.5x, over 20x for unbalanced data) and improved robustness compared to baselines, enhancing DLRM inference efficiency and throughput.
Deep Learning-based Recommender Models (DLRMs) are at the heart of personalized experiences on major online platforms like Amazon, Facebook, YouTube, and Netflix. These models are crucial for business success, influencing a significant portion of revenue and content consumption. However, their widespread use comes with a challenge: DLRM inference, the process of using these models to make predictions, accounts for a large portion of AI workloads in data centers, often facing performance bottlenecks.
The primary bottleneck in DLRMs lies within their embedding layers. These layers are responsible for converting categorical features (like user IDs or item categories) into dense, fixed-sized numerical vectors. This process involves numerous random memory accesses to large lookup tables, which can significantly slow down the overall inference process, especially when dealing with vast amounts of data stored in memory.
To address this, a new research proposes innovative data flow strategies designed to accelerate these critical embedding look-ups. The core idea is to optimize how data moves and is accessed, particularly on multi-core AI accelerators and GPUs. The researchers introduce four specific strategies for efficient embedding table look-ups on a single processing core. Furthermore, they present a framework that automatically maps these tables in an asymmetric manner across multiple cores of a system-on-chip (SoC).
The proposed methods were rigorously tested using Huawei Ascend AI accelerators, with comparisons made against the default Ascend compiler and high-level estimations against Nvidia A100 GPUs. The results are promising, demonstrating a significant speed-up ranging from 1.5 times to 6.5 times for real-world workload distributions. For extremely unbalanced data distributions, the speed-up was even more dramatic, exceeding 20 times. A key advantage of this new approach is its reduced dependence on the specific query distribution, making it more robust for diverse real-world scenarios.
The research highlights that efficiently gathering embedding vectors, especially from smaller tables, into on-chip buffers is vital. This approach helps conserve memory bandwidth for larger tables that cannot fit entirely on-chip. By achieving better performance trade-offs, the method allows for higher average throughput under given latency constraints, ultimately leading to increased revenue and improved user satisfaction for platforms relying on DLRMs.
Also Read:
- FlowSpec: Revolutionizing LLM Inference at the Edge for Faster, Smarter AI
- Revolutionizing mmWave MIMO: How AI is Making Intelligent Surfaces Smarter and Faster
For more in-depth technical details, you can refer to the full research paper: Deep Recommender Models Inference: Automatic Asymmetric Data Flow Optimization.
