TLDR: CoComposer is a multi-agent AI system designed to overcome limitations in existing AI music composition tools. It uses five specialized LLM agents (Leader, Melody, Accompaniment, Revision, Review) that collaborate following a traditional music workflow to create symbolic music in ABC notation. Evaluations show CoComposer improves music quality and production complexity compared to other LLM-based systems, offering better interpretability and editability, though dedicated models like MusicFX still lead in absolute aesthetic quality.
The world of artificial intelligence is constantly pushing boundaries, and music composition is no exception. While AI has made significant strides in generating music, existing tools often face limitations in terms of generation duration, overall musical quality, and the ability for users to precisely control the creative process. Many platforms struggle with long text prompts, and the generated music can sometimes lack the professional touch desired by musicians.
Addressing these challenges, researchers Peiwen Xing, Aske Plaat, and Niki van Stein from LIACS, Leiden University, Netherlands, have introduced CoComposer, an innovative multi-agent system designed for collaborative music composition. CoComposer aims to enhance the quality, controllability, and interpretability of AI-generated music by mimicking a traditional music composition workflow.
The CoComposer System: A Collaborative Ensemble of AI Agents
CoComposer is built around five specialized, role-playing language agents that work together to create symbolic music. This multi-agent approach allows for complex tasks like polyphonic music composition to be broken down and handled by dedicated entities, fostering interaction and collaboration. The agents are:
- Leader Agent: Acts as the team leader, interpreting user requirements, decomposing tasks, and assigning them to other agents.
- Melody Agent: Responsible for composing the main melody, selecting instruments, and outputting the music in ABC notation.
- Accompaniment Agent: Designs the accompaniment based on the melody, selecting instruments, and also outputs in ABC notation.
- Revision Agent: Focuses on correcting format and rhythm errors in the ABC notation without altering the creative content.
- Review Agent: Evaluates the compositions from a music theory perspective, providing optimization suggestions across various dimensions like melodic structure, harmony, and rhythm.
The creative process in CoComposer unfolds in two main phases: an Initialization Creation Phase and an Iterative Creation Phase. Initially, the Leader Agent analyzes the user’s prompt and assigns melody and accompaniment tasks. The Melody and Accompaniment Agents then independently create their parts, which are subsequently checked by the Revision Agent for errors. Finally, the Review Agent provides feedback and suggestions. In the iterative phase, the Leader Agent uses this feedback to guide the Melody and Accompaniment Agents in refining their compositions, with further checks by the Revision Agent, until the desired quality is achieved.
This structured collaboration, enabled by frameworks like AutoGen, allows CoComposer to streamline the composition process, requiring only five agents compared to other systems like ComposerX, which uses six. This reduction is based on a more efficient division of labor that closely mirrors real-world music composition practices.
Evaluating Musical Quality and Complexity
To objectively assess CoComposer’s performance, the researchers utilized Meta’s AudioBox-Aesthetics system, a score prediction model that evaluates music aesthetics across four dimensions: Production Quality (PQ), Production Complexity (PC), Content Enjoyment (CE), and Content Usefulness (CU). CoComposer was tested with various large language models, including GPT-4o, DeepSeek-V3-0324, and Gemini-2.5-Flash, and compared against ComposerX and Google’s MusicFX.
The evaluations revealed that CoComposer significantly outperforms existing multi-agent LLM-based systems like ComposerX in terms of subjective aesthetic experience (CE, CU), creative complexity (PC), and production quality (PQ). It also showed a notable advantage in production complexity compared to single-agent systems. Among the LLMs tested, GPT-4o consistently delivered the best overall performance within CoComposer.
While CoComposer demonstrated strong performance against other LLM-based systems, it was also compared to dedicated music generation models like MusicFX. MusicFX, being a specialized model, still leads in subjective experience and complexity. However, CoComposer offers distinct advantages in “interpretability” and “editability” because it is open-source and generates music using the ABC notation, which users can directly view and modify. This makes it a low-threshold, high-controllability tool for music creation, reducing the need for specialized musical theory knowledge.
Also Read:
- Unlocking Deeper Understanding: How Multi-Agent LLMs Are Revolutionizing Causal AI
- AImoclips: Measuring How AI Music Conveys Feelings
The Future of AI Music Composition
The CoComposer project highlights the potential of multi-agent systems to improve the quality and accessibility of AI-generated music. By adopting a collaborative approach and leveraging general-purpose LLMs, CoComposer significantly reduces research, development, and deployment costs, as it doesn’t require extensive pre-training on large music datasets.
Future work for CoComposer includes addressing limitations such as the reliance on the MIDI standard sound library, which struggles with modern computer-synthesized sounds. The system also aims to enhance its understanding of complex musical structures and subjective creativity. Suggestions for improvement include designing a specialized feedback analysis agent to translate fragmented user feedback into structured instructions and incorporating a “memory mechanism” to personalize the creative experience over time.
For those interested in delving deeper into the technical details, the full research paper can be accessed at arxiv.org/pdf/2509.00132.
