TLDR: ForgeHLS is a new, large-scale, open-source dataset for High-Level Synthesis (HLS), featuring over 400,000 diverse hardware designs. It addresses the shortage of data for applying machine learning to HLS optimization by providing systematically generated designs with automated pragma insertions. The dataset’s utility is demonstrated through improved Quality of Result (QoR) prediction and automated pragma exploration, particularly highlighting the superior performance of Large Language Models (LLMs) in these tasks compared to traditional methods, paving the way for more efficient hardware design.
High-Level Synthesis (HLS) is a vital technology in modern hardware design, acting as a bridge between software and hardware. It allows designers to transform high-level programming code, like C, C++, or SystemC, into optimized hardware implementations, significantly reducing the time and complexity traditionally associated with hardware development. A key aspect of HLS involves the use of “pragmas,” which are directives that guide hardware optimizations without changing the core algorithmic code. These pragmas influence critical factors like performance, resource utilization, and power consumption.
Despite its importance, the application of machine learning (ML) to HLS optimization has faced a significant hurdle: the lack of large and diverse datasets. Existing datasets have often been limited in scope and size, restricting progress in crucial research areas like Quality of Result (QoR) prediction and Design Space Exploration (DSE). QoR prediction aims to estimate performance and resource usage early in the design cycle, while DSE focuses on finding optimal pragma configurations to balance performance, resource use, and power.
Introducing ForgeHLS: A New Era for HLS Datasets
To address this critical gap, researchers have introduced ForgeHLS, a groundbreaking large-scale, open-source dataset specifically designed to accelerate ML-driven HLS research. ForgeHLS stands out with over 400,000 diverse designs generated from 536 unique “kernels” (core algorithmic units), covering a wide array of application domains. This makes ForgeHLS approximately ten times larger than previous HLS datasets, offering unprecedented scale and diversity.
The creation of ForgeHLS involved a dual approach to ensure comprehensive coverage. It aggregates real-world HLS kernels from established benchmarks such as PolyBench, MachSuite, CHStone, Rosetta, and official Xilinx Vitis HLS examples. Additionally, it incorporates a significant amount of synthetic code generated using advanced AI models like GPT-4o. This synthetic code is rigorously cross-verified for correctness, ensuring its quality and relevance. The algorithms covered range from fundamental operators and computation-intensive tasks (like matrix multiplication and deep learning layers) to classical Leetcode algorithms and Register-Transfer Level (RTL) designs.
Automated Pragma Insertion and Design Space Exploration
A unique feature of ForgeHLS is its automated workflow for pragma insertion. Unlike previous datasets where pragma combinations were often defined in separate scripts, ForgeHLS directly embeds pragmas into the source code. This direct insertion simplifies usage, especially for Large Language Models (LLMs) and other downstream tasks. The dataset leverages extensive Design Space Exploration (DSE) techniques, including both full DSE for smaller kernels and Bayesian optimization for more complex ones, to systematically explore various pragma configurations and generate a vast number of design instances. This process ensures that the dataset captures both the functional and structural complexities of HLS code, providing feasible HLS designs with corresponding performance metrics.
Demonstrating Utility Through Downstream Tasks
The utility of ForgeHLS is clearly demonstrated through its application to two representative downstream tasks:
HLS Quality of Result (QoR) Prediction: This task involves training machine learning models to accurately predict the latency and resource usage (like LUT, DSP, and FF) of HLS designs. Evaluations showed that while Graph Neural Networks (GNNs) performed well on simpler, smaller datasets, Large Language Models (LLMs) fine-tuned on ForgeHLS exhibited superior generalization ability, especially when predicting QoR for unseen kernel structures. This highlights the LLMs’ capacity to capture complex patterns due to their larger number of learnable parameters and extensive pretraining on diverse data.
Automatic HLS Pragmas Exploration: This task explores the ability of LLMs to automatically insert optimal pragmas into HLS code. By fine-tuning models like LLaMA3, Qwen, and Mistral on ForgeHLS, researchers observed significant improvements in the HLS synthesis pass rate for generated code. For instance, the LLaMA3-7B model trained on ForgeHLS achieved a 69% pass rate, a substantial increase from its original 19%. Furthermore, the “Adjusted Distance to Reference Set” (ADRS) metric, which measures how close generated designs are to true optimal designs, was significantly reduced. This indicates that LLMs, when trained on ForgeHLS, become much more adept at generating valid and near-optimal pragma configurations, showcasing their potential for automating and optimizing HLS design space exploration.
Also Read:
- ChipSeek-R1: Advancing RTL Code Generation with LLMs Through Integrated Hardware Feedback
- Large Language Models Reshape Combinatorial Optimization: A Comprehensive Review
A Foundation for Future Hardware Design Innovation
ForgeHLS represents a significant advancement for the Electronic Design Automation (EDA) community. By providing a large-scale, diverse, and open-source dataset with automated pragma insertion, it lays a valuable foundation for developing more robust and generalizable machine learning models for HLS optimization. Future work aims to further expand the dataset and explore more advanced models and new downstream tasks, such as automated pragma optimization using reinforcement learning or evolutionary algorithms. This research, detailed in the paper available at this link, promises to accelerate machine learning-driven innovation in hardware design.
