TLDR: MM-ASSIST is the first fully LLM-powered assistant for “Move Method” refactoring, automating the entire process from recommendation to execution. It addresses LLM limitations like hallucinations and limited context by combining static analysis from IDEs, semantic embeddings (VoyageAI), and a novel refactoring-aware Retrieval Augmented Generation (RAG) workflow. Empirical evaluations show MM-ASSIST significantly outperforms previous state-of-the-art tools in accuracy and speed, with a user study confirming its effectiveness and usefulness for developers.
In the world of software development, keeping code clean, organized, and efficient is crucial. One common task developers undertake is called “Move Method” refactoring. This involves relocating a method (a piece of code that performs a specific action) from one class (a blueprint for objects) to another where it logically belongs better. This seemingly simple task is vital for improving code quality, making it easier to understand, and reducing potential issues down the line. However, identifying which methods to move and where to move them can be a complex and time-consuming challenge, even for expert developers.
Traditional tools and approaches for recommending “Move Method” refactorings often fall short. Some rely on static analysis, which can be slow and struggle with large projects, while others use machine learning but require constant retraining and don’t always align with how human experts think. The recent rise of Large Language Models (LLMs) like GPT-4o offered a promising new avenue, given their extensive training on vast amounts of code. Initial studies showed that LLMs could provide expert-level suggestions, but they came with a significant drawback: a high rate of “hallucinations” – recommendations that seemed plausible but were incorrect or impossible to implement, sometimes up to 80% of suggestions.
Introducing MM-ASSIST: An End-to-End Automated Solution
A new research paper introduces MM-ASSIST, the first fully LLM-powered assistant designed to automate the entire “Move Method” refactoring process, from identifying misplaced methods to executing the move. This innovative tool addresses the major limitations of LLMs by combining their creative reasoning with the rigorous, deterministic power of Integrated Development Environments (IDEs) and advanced semantic analysis.
MM-ASSIST’s approach is built on several key innovations:
- Filtering Hallucinations: It uses static analysis from IDEs to automatically filter out LLM-generated recommendations that are mechanically infeasible or would break the code. This ensures that only valid suggestions are presented to the developer.
- Overcoming Context Limitations: “Move Method” refactoring often requires understanding the entire project, which can be too much information for an LLM’s limited context window. MM-ASSIST employs a novel “refactoring-aware Retrieval Augmented Generation (RAG)” system. This intelligent retrieval process fetches only the most relevant parts of the codebase, based on both mechanical feasibility and semantic relevance, to augment the LLM’s input. This allows the LLM to make informed, project-level decisions without being overwhelmed.
- Self-Consistency and Ranking: The system prompts LLMs to be self-consistent, critique their own suggestions, and rank them, leading to higher quality recommendations.
How MM-ASSIST Works
The MM-ASSIST workflow is a synergistic blend of different technologies:
- Initial Filtering: It first filters out methods that cannot be safely moved, such as constructors or getters/setters that would require additional changes.
- Identifying Least Compatible Methods: Using code-trained vector embeddings (from VoyageAI), MM-ASSIST calculates the semantic similarity between a method and its host class. Methods with low similarity are flagged as potentially misplaced.
- LLM Prioritization: The LLM then analyzes these candidate methods, considering their purpose, cohesion, and dependencies, to prioritize the most promising ones for relocation.
- Target Class Identification (Refactoring-Aware RAG): For instance methods, it looks at field types and parameter types in the host class. For static methods, it considers package proximity and identifies utility classes. Static analysis from the IDE further validates these potential targets.
- Semantic Relevance Ranking: VoyageAI embeddings are again used to calculate semantic relevance scores between the candidate method and potential target classes, ranking them.
- Final LLM Selection: A concise summary of the top target classes is fed to the LLM, which then picks the best-suited destination for the method.
- Automated Execution: Once a developer approves a suggestion, MM-ASSIST uses the IDE’s refactoring APIs to safely execute the move, updating all call sites and references automatically.
Also Read:
- Navigating the Future of Software Creation: A Deep Dive into Vibe Coding with Large Language Models
- AI-Generated Software Contracts Enhance Bug Detection and Verification
Impressive Results and Developer Endorsement
MM-ASSIST underwent a thorough evaluation, comparing it against previous state-of-the-art tools like JMOVE, FETRUTH, and HMOVE, as well as a vanilla LLM (GPT-4o) without MM-ASSIST’s enhancements. The results were compelling:
- Superior Accuracy: On a widely used benchmark, MM-ASSIST showed a 1.7x improvement in Recall@1 and Recall@3 (metrics for how often the correct recommendation is among the top 1 or 3 suggestions). On a new dataset of 210 real-world refactorings from open-source software, its Recall rates improved by at least 2.4x.
- Faster Performance: MM-ASSIST generates recommendations in about 30 seconds, significantly faster than baseline tools which could take hours or even days for large projects.
- Developer Approval: A user study with 30 experienced developers who used MM-ASSIST on their own code for a week rated 82.8% of its recommendations positively. On average, each user accepted and applied 7 refactorings to their code. Participants praised the tool for giving them a sense of control and delegating “grunt work” to AI.
This research marks a significant leap forward in automated code refactoring. By intelligently combining the strengths of LLMs, IDEs, and semantic embeddings, MM-ASSIST provides a practical, effective, and useful tool for developers to maintain high-quality code. The paper, titled “Leveraging LLMs, IDEs, and Semantic Embeddings for Automated Move Method Refactoring,” was authored by Fraol Batole, Abhiram Bellur, Malinda Dilhara, Mohammed Raihan Ullah, Yaroslav Zharov, Timofey Bryksin, Kai Ishikawa, Haifeng Chen, Masaharu Morimoto, Shota Motoura, Takeo Hosomi, Tien N. Nguyen, Hridesh Rajan, Nikolaos Tsantalis, and Danny Dig. You can read the full paper here.
