TLDR: The Kimi-Dev research introduces a novel approach to developing advanced AI for software engineering. It demonstrates that training Large Language Models (LLMs) using an ‘Agentless’ workflow, which focuses on structured, verifiable steps like bug localization and code editing, can build essential ‘skill priors’. These foundational skills significantly enhance the efficiency and effectiveness of more complex ‘Agent-based’ LLMs, enabling them to achieve state-of-the-art performance on benchmarks like SWE-bench Verified, even matching proprietary models after minimal adaptation.
Large Language Models (LLMs) are rapidly transforming the field of software engineering (SWE), taking on complex tasks like bug fixing and code generation. A key benchmark for evaluating these AI systems is SWE-bench, which challenges models to resolve real-world bugs in GitHub repositories by producing correct code patches.
Historically, two main approaches have emerged for tackling SWE tasks with LLMs: Agent-based solutions and workflow-based Agentless methods. Agent-based solutions, like SWE-Agent, are highly interactive, mimicking human developers by engaging in multi-turn conversations and using tools within an executable environment. They offer great flexibility and potential but are notoriously difficult to train due to their complex, end-to-end nature.
In contrast, Agentless methods, such as the original Agentless framework, break down SWE tasks into a series of predefined, verifiable steps like bug localization, code repair, and test generation. This modularity makes them more stable and easier to train, often using techniques like Reinforcement Learning with Verifiable Rewards (RLVR). However, their rigid structure can limit adaptability when faced with scenarios requiring multiple rounds of incremental updates.
A new research paper, titled “KIMI-DEV: AGENTLESSTRAINING ASSKILLPRIOR FORSWE-AGENTS,” challenges the idea that these two paradigms are mutually exclusive. Authored by Zonghan Yang, Shengjie Wang, Kelin Fu, Wenyang He, Weimin Xiong, Yibo Liu, Yibo Miao, Bofei Gao, Yejie Wang, Yingwei Ma, Yanhao Li, Yue Liu, Zhenxing Hu, Kaitai Zhang, Shuyi Wang, Huarong Chen, Flood Sung, Yang Liu, Yang Gao, Zhilin Yang, and Tianyu Liu from Moonshot AI, THU, PKU, UCAS, BUPT, and NUS, the paper argues that Agentless training can serve as a powerful foundation for building more capable SWE-Agents.
The core idea behind Kimi-Dev is that reasoning-intensive Agentless training can instill crucial “skill priors” in an LLM. These foundational skills include bug localization (identifying where the bug is), code editing (implementing the fix), and self-reflection (evaluating and refining the solution). By first mastering these atomic capabilities through Agentless training, an LLM can then adapt more efficiently and effectively to the dynamic, multi-turn interactions required by SWE-Agents.
The researchers developed a comprehensive Agentless training recipe for Kimi-Dev, an open-source SWE LLM. This recipe involves several stages: mid-training with a massive dataset of GitHub issues and pull request commits, a “cold-start” phase using supervised finetuning with reasoning trajectories, reinforcement learning focused on refining code editing, and a test-time self-play mechanism where the model generates and evaluates multiple patches and tests to select the best solution.
Kimi-Dev achieved impressive results, reaching 60.4% on SWE-bench Verified, making it the top performer among workflow-based approaches. What’s even more significant is how these Agentless-induced skills transferred to agentic frameworks. With just a lightweight supervised finetuning adaptation on 5,000 publicly available SWE-Agent trajectories, Kimi-Dev powered SWE-Agents to a 48.6% pass@1 score, a performance comparable to that of advanced proprietary models like Claude 3.5 Sonnet.
The study further demonstrated that the skills acquired through Agentless training generalize beyond the specific SWE-bench Verified dataset. Kimi-Dev showed strong performance on broader benchmarks like SWE-bench-live and SWE-bench Multilingual, which include tasks in various programming languages beyond Python. This indicates that the structured skill priors are robust and transferable.
Also Read:
- LLM Agents Tackle Real-World Software Compilation Challenges with BUILD-BENCH
- Lita: Unveiling LLM Coding Prowess with Minimalist Agent Design
In essence, the Kimi-Dev research reframes the relationship between Agentless and agentic frameworks not as competing alternatives, but as complementary stages in developing highly capable coding LLMs. By first building a strong foundation of structured skills through Agentless training, we can scaffold the efficient adaptation of more autonomous and generalizable SWE-Agents. This approach offers a principled pathway towards creating advanced AI systems for software development. You can read the full research paper here: KIMI-DEV: AGENTLESSTRAINING ASSKILLPRIOR FORSWE-AGENTS.
