TLDR: RefAgent is a novel multi-agent LLM-based framework for automated software refactoring. It uses specialized agents for planning, executing, testing, and refining code changes, leveraging iterative feedback loops and tool-calling capabilities. The framework significantly improves software quality by reducing code smells (median 52.5%) and enhancing attributes like reusability (median 8.6%), while maintaining high unit test (median 90%) and compilation pass rates (median 87%). RefAgent outperforms single-agent LLM approaches and aligns well with human and search-based refactoring efforts, demonstrating the power of multi-agent collaboration and contextual awareness in complex software engineering tasks.
Software refactoring, the process of restructuring existing code without changing its external behavior, is a crucial practice in software development. It helps improve code design, reduce complexity, and enhance maintainability. However, this task is often time-consuming, error-prone, and challenging, especially in large and complex software systems.
Traditional methods for refactoring, including manual efforts and built-in IDE tools, often fall short. Manual refactoring is tedious and can introduce new errors, while IDE tools offer limited refactoring options. Fully automated techniques, such as search-based refactoring, aim to optimize code quality but can sometimes radically change program design, lack guarantees of behavior preservation, and are computationally intensive for large projects.
Recent advancements in Large Language Models (LLMs) have opened new avenues for automating various software engineering tasks, including code generation and repair. While LLMs have shown promise in refactoring by reducing development time and improving code quality, they often rely on static instructions. This limitation means they struggle to adapt to dynamic contexts or make autonomous decisions.
Introducing RefAgent: A Multi-Agent Approach to Software Refactoring
A new research paper titled RefAgent: A Multi-agent LLM-based Framework for Automatic Software Refactoring introduces an innovative solution: RefAgent. Developed by Khouloud Oueslati, Maxime Lamothe, and Foutse Khomh, RefAgent is a multi-agent LLM-based framework designed for end-to-end software refactoring. Unlike single, static LLMs, RefAgent leverages specialized LLM-based agents that can dynamically adapt to evolving contexts, interact with software tools, and execute complex workflows.
The core idea behind RefAgent is to simulate the iterative and collaborative nature of human developers. By distributing tasks among specialized agents, the framework aims to reduce development time, mitigate issues like “hallucinations” (where LLMs generate incorrect or nonsensical code), and ensure that the refactored code maintains its original behavior.
How RefAgent Works: The Specialized Agents
RefAgent consists of four key agents, each with a distinct role in the refactoring process:
-
Context-Aware Planner Agent: This agent is responsible for identifying opportunities for refactoring. It analyzes code dependencies and metrics to generate a structured refactoring plan, deciding which parts of the code need improvement and how.
-
Refactoring Generator Agent: Following the plan from the Planner Agent, this agent executes the refactoring on the target code. It produces a refactored version of the class and updates the code. This agent also receives feedback from other agents to refine its output.
-
Compiler Agent: This agent interacts with the compilation environment to ensure the refactored code is syntactically correct and compiles successfully. If compilation errors occur, it analyzes the error messages, summarizes them, and sends them back to the Refactoring Generator Agent for correction, creating an iterative feedback loop.
-
Tester Agent: Crucially, refactoring should not change the observable behavior of the code. The Tester Agent ensures this by running existing developer-written tests and generating additional regression tests using tools like EvoSuite. If tests fail, it provides feedback to the Refactoring Generator Agent to fix the issues, again in an iterative loop.
This multi-agent collaboration, with its iterative feedback loops, allows RefAgent to dynamically adapt and refine its refactoring suggestions, ensuring both correctness and functionality preservation.
Key Findings and Impact
The researchers evaluated RefAgent on eight open-source Java projects, comparing its effectiveness against single-agent approaches, a search-based refactoring tool (RefGen), and historical developer refactorings. The results are highly promising:
-
High Success Rates: RefAgent achieved a median unit test pass rate of 90% and a median compilation success rate of 87-88%. This indicates its strong ability to preserve functionality and syntactic correctness during refactoring.
-
Code Quality Improvement: The framework reduced code smells by a median of 52.5% and improved key quality attributes like reusability by a median of 8.6%.
-
Alignment with Human and Automated Efforts: RefAgent closely aligned with developer refactorings, achieving a median F1-score of 79.15% in identifying refactoring opportunities. It also showed strong alignment with the search-based tool RefGen, with a median F1-score of 72.7%.
-
Outperforming Single-Agent Approaches: Compared to single-agent LLM methods, RefAgent significantly improved the median unit test pass rate by 64.7% and the median compilation success rate by 40.1%. This highlights the substantial benefits of a multi-agent architecture.
-
Importance of Context and Iteration: An ablation study revealed that context retrieval and iterative refinement are critical for maintaining high test and compilation pass rates, demonstrating how these components contribute to the framework’s robustness.
RefAgent’s performance remained consistent across different underlying LLMs, including GPT-4o, DeepSeekCoder, and StarCoder2, suggesting that the effectiveness stems from the multi-agent collaboration rather than a specific LLM.
Also Read:
- EvoDev: An Iterative Framework for AI-Powered Software Development
- OpenHands Software Agent SDK: A Robust Framework for Production-Ready AI Agents
The Future of Automated Refactoring
The findings from this research underscore the significant potential of multi-agent LLM architectures in advancing automated software refactoring. By breaking down complex refactoring tasks into specialized roles and enabling dynamic feedback loops, RefAgent offers a robust and effective solution to a long-standing challenge in software engineering. This approach paves the way for more autonomous, reliable, and high-quality code maintenance in the future.
