TLDR: BBOPlace-Bench is the first benchmark for evaluating Black-Box Optimization (BBO) algorithms in chip placement. It provides a flexible framework with three problem formulations (Sequence Pair, Mask-Guided Optimization, Hyperparameter Optimization), integrates various BBO algorithms (SA, EAs, BO), and uses multiple evaluation metrics (MP-HPWL, GP-HPWL, PPA) on industrial chip cases. Experiments show that Mask-Guided and Hyperparameter Optimization formulations, particularly with Evolutionary Algorithms, can outperform existing analytical and reinforcement learning methods, highlighting BBO’s potential in chip design.
Chip placement is a crucial and complex step in designing modern microchips. It significantly influences the chip’s power consumption, performance, and overall physical size (PPA metrics). For decades, engineers have used various optimization techniques to arrange millions of tiny components, like memory blocks (macros) and logic gates (standard cells), onto a silicon canvas. This process is inherently challenging because the objective function – how good a placement is – cannot be easily described mathematically; it’s a ‘black-box’ problem that relies on simulations or real manufacturing evaluations.
Historically, early attempts to use black-box optimization (BBO) for chip placement faced limitations due to immature problem definitions and inefficient algorithms. This led to analytical methods becoming more prevalent. However, recent advancements have shown that BBO has the potential to achieve state-of-the-art results, even surpassing traditional and reinforcement learning approaches.
Introducing BBOPlace-Bench: A Unified Platform
Despite these breakthroughs, the field lacked a standardized benchmark specifically designed for BBO in chip placement. This gap made it difficult for researchers to thoroughly evaluate and compare different BBO problem formulations and algorithms. To address this, a team of researchers proposed BBOPlace-Bench, the first benchmark tailored for this purpose. You can find the full research paper here: BBOPlace-Bench: Benchmarking Black-Box Optimization for Chip Placement.
BBOPlace-Bench offers a modular, decoupled, and flexible framework that allows users to easily implement, test, and compare their own BBO algorithms. It integrates three distinct ways of formulating the chip placement problem for BBO:
- Sequence Pair (SP): A traditional method that uses pairs of permutations to define the relative positions of modules.
- Mask-Guided Optimization (MGO): Represents modules using grid coordinates and employs a ‘wire-mask-guided’ greedy procedure to adjust positions for minimal wirelength and no overlaps.
- Hyperparameter Optimization (HPO): Optimizes the settings (hyperparameters) of advanced analytical placers like DREAMPlace, which then generate the chip layout.
The benchmark also includes a wide variety of existing BBO algorithms, such as Simulated Annealing (SA), various Evolutionary Algorithms (EAs) like Vanilla-EA, Evolution Strategies (ES), and Particle Swarm Optimization (PSO), as well as Bayesian Optimization (BO). These algorithms interact with the problem formulations to find optimal layouts.
Comprehensive Evaluation and Key Findings
BBOPlace-Bench standardizes modern chip cases from industrial benchmarks like ISPD 2005 and ICCAD 2015. It evaluates performance using multiple metrics:
- Macro Placement Wirelength (MP-HPWL): A faster proxy metric focusing on the wirelength of larger components (macros).
- Global Placement Wirelength (GP-HPWL): A more accurate but computationally expensive metric that considers the wirelength of both macros and standard cells.
- PPA Evaluation: Industrial chip metrics (power, performance, area) obtained using commercial tools, providing a comprehensive assessment of chip quality.
The experimental results yielded several important insights:
- The Sequence Pair (SP) formulation generally performed the poorest due to its exponentially growing search space and scalability issues.
- Mask-Guided Optimization (MGO) and Hyperparameter Optimization (HPO) formulations consistently showed superior performance.
- Among the BBO algorithms, Evolutionary Algorithms (especially Vanilla-EA and PSO) demonstrated better overall performance than Simulated Annealing and Bayesian Optimization, particularly in high-dimensional search spaces.
- Notably, the top-performing BBO methods, such as HPO-Vanilla-EA and MGO-Vanilla-EA, often surpassed state-of-the-art reinforcement learning methods (like EfficientPlace) and analytical placers (like DREAMPlace) in both macro and global placement tasks.
- For Global Placement Wirelength, the HPO formulation, which tunes the analytical placer, proved particularly effective, achieving the best results.
- When evaluating real-world PPA metrics, no single BBO algorithm dominated across all objectives, highlighting the complex, multi-objective nature of chip design.
- The study also found that increasing the number of macros (which increases the problem dimension) generally made optimization more difficult, though some algorithms could still yield good results with sufficient evaluation budgets.
Also Read:
- REvolution: A New Era for Hardware Design with AI and Evolution
- Efficient Data Generation for Integrated Circuit Thermal Simulations
Future Directions
While BBOPlace-Bench marks a significant step forward, the authors acknowledge limitations, such as the reliance on commercial software for PPA evaluation. Future plans include integrating open-source EDA tools and incorporating more advanced chip designs. The research also points to promising future directions for BBO in chip placement, including multi-objective optimization, expensive optimization techniques (using surrogate models), high-dimensional optimization strategies, and learning-enhanced optimization approaches.
This benchmark not only facilitates the development of efficient BBO-driven solutions for chip placement but also expands the practical application scenarios for the broader BBO community, tackling a critical real-world engineering challenge.
