TLDR: PEFA-AI introduces an agentic AI system that automates Register-Transfer Level (RTL) code generation for hardware design. It uses multiple specialized AI agents and hardware simulation tools, featuring a Progressive Error Feedback System (PEFA) for self-correction. This approach iteratively refines code based on summarized error logs, significantly improving pass rates for both open-source and closed-source LLMs, bridging their performance gap, and achieving state-of-the-art results with efficient token usage.
The world of hardware design is incredibly complex, with the demand for more powerful chips growing exponentially. Traditionally, creating the intricate blueprints for these chips, known as Register-Transfer Level (RTL) code, has been a time-consuming process heavily reliant on human expertise and iterative corrections. This often leads to long development cycles and escalating costs.
Artificial intelligence, particularly large language models (LLMs), has shown promise in automating code generation. However, applying LLMs to hardware design faces unique challenges. Current methods often require significant human intervention to check for errors, refine prompts, and guide the LLM towards the correct output. Even self-correcting LLMs can struggle with large error logs, leading to confusion or ‘hallucinations’ where the model generates incorrect but plausible-sounding code.
A new research paper introduces PEFA-AI, an innovative agentic flow designed to tackle these challenges head-on. PEFA-AI stands for Progressive Error Feedback Agentic-AI, and it presents a system where multiple specialized AI agents work together with hardware simulation tools to automatically generate RTL code without human intervention. This collaborative approach aims to streamline the hardware design process significantly.
How PEFA-AI Works
At the heart of PEFA-AI is its Progressive Error Feedback System (PEFA). This is a self-correcting mechanism that uses iterative error feedback to progressively refine the generated code. Instead of overwhelming the LLM with massive error logs, PEFA-AI intelligently filters and summarizes the most relevant insights from compilation and simulation results. This focused feedback helps the model concentrate on specific errors, preventing it from getting sidetracked or generating irrelevant code.
The system integrates various agents, including a master agent that parses user prompts and test benches, a code generator agent that leverages LLMs to convert natural language into RTL code, and a code executor agent that compiles and runs the code against test benches. A crucial innovation is the ‘hybrid agent,’ which uses tools like Verilator for linting (checking for syntax errors) and Icarus Verilog for compilation and execution. If any stage fails, the system collects a stack trace, but instead of feeding the full log, a ‘log summarizer’ agent distills the errors into concise messages for the LLM.
This progressive feedback loop allows the system to check for compilation success, functional correctness, and synthesizable constructs. It iteratively refines the code based on these checks, updating the input prompt or logic until the tests pass or a maximum number of feedback loops is reached. This modular design also allows for future expansion to include agents that optimize for power, performance, and area (PPA) constraints.
Also Read:
- AI Agents Bring Clarity and Efficiency to Analog Circuit Design
- CudaForge: AI Agents Streamline CUDA Kernel Optimization with Hardware Insights
Impressive Results and Bridging the Gap
The researchers rigorously benchmarked PEFA-AI using two open-source natural language-to-RTL datasets: VerilogEval and RTLLM1.1. They tested the approach with both open-source LLMs (like Llama 3.1:70B and DeepSeekCoder:33B) and closed-source models (such as GPT-4o and Claude-3.5-Sonnet).
The results are compelling. PEFA-AI significantly improves test pass rates across all models, effectively bridging the performance gap between open-source and closed-source LLMs. For instance, open-source models saw substantial increases in pass rates, demonstrating the power of agentic reasoning. The approach also sets a new benchmark, achieving state-of-the-art pass rates while being efficient in terms of token counts, meaning it uses fewer computational resources compared to previous methods.
This work highlights a promising direction for autonomous hardware design, reducing the need for human intervention and accelerating development cycles. For more in-depth technical details, you can refer to the full research paper: PEFA-AI: Advancing Open-source LLMs for RTL generation using Progressive Error Feedback Agentic-AI.
