TLDR: A research paper by Krol, Llano, and McCormack explores how a deep learning-based music variation tool, Rhapsody Refiner, helps musicians maintain creative ownership. Through a four-week ecological evaluation, the study found that the tool’s reliance on the musician’s skill to provide initial input and refine “moments” into complete ideas fostered a strong sense of ownership over both the creative process and the final composition. This approach highlights how AI can support human creativity by encouraging active involvement rather than automation, preserving the musician’s identity and control.
In an era where artificial intelligence is rapidly transforming industries, the music sector is no exception. While some AI tools promise complete song generation, a new wave of innovation focuses on using AI to support, rather than replace, human creativity. A recent research paper delves into this promising direction, exploring how AI can help musicians maintain personal ownership and creative control over their compositional process.
The paper, titled “Supporting Creative Ownership through Deep Learning-Based Music Variation,” was authored by Stephen James Krol and Jon McCormack from Monash University, Australia, and Maria Teresa Llano from the University of Sussex, United Kingdom. Their work investigates the critical role of personal ownership in musical AI design, aiming to understand how practicing musicians can remain the primary creative force.
Introducing Rhapsody Refiner
At the heart of their study is a deep learning-based music variation system called Rhapsody Refiner. Co-designed with practicing musicians, this tool is distinct from systems that generate entirely new ideas. Instead, Rhapsody Refiner relies on the musician to first provide a musical phrase, which it then explores and varies. This design philosophy ensures the musician remains actively involved in both the initial ideation and the final creation of the song.
Rhapsody Refiner utilizes MusicBert, a bidirectional transformer model, to perform masked prediction on selected notes. Users upload a MIDI file and set parameters to guide the masking of random song sections. The system then predicts variations for these masked parts. Musicians can choose specific aspects to vary, such as pitch or dynamics, and generate multiple variations from a single input. This approach forces the musician to be an active participant, shaping the AI’s output to fit their vision.
A Four-Week Exploration of Creativity
To understand how such a music variation tool functions in a real-world composition setting, the researchers conducted a four-week ecological evaluation. Eight practicing musicians participated, experimenting with Rhapsody Refiner in their own creative environments. Unlike a lab setting, this extended period allowed participants to develop a deep understanding of the tool. They were instructed to compose a song using the system, keep a journal, and later took part in semi-structured interviews.
Key Findings: Ownership, Moments, and Active Co-Creation
The evaluation revealed several crucial themes:
A Tool for Moments: Participants described Rhapsody Refiner as a “tool for moments.” They found that it didn’t generate perfect, complete variations, but rather “parts of them, that were really, really nice.” These moments served as inspiration, taking their compositions in unexpected and interesting directions. The imperfect outputs meant musicians had to actively filter and refine variations, making the system reliant on their skill to produce anything valuable. As one participant noted, “you have to create and be creative for it to work for you.”
A Tool for Ownership: A major finding was the strong sense of ownership participants felt over both the creative process and the final musical piece. They highlighted feeling “fully in control” because the system depended on their input and effort. While some felt complete ownership, others acknowledged a “joint ownership” with Rhapsody Refiner, attributing a small percentage of the song to the AI, but emphasizing that the majority of the emotion and feel came from them.
A Tool for Creative Involvement: Musicians expressed the importance of being the “driving force” behind the project. They found other AI systems that offered more creative agency uncomfortable, as it diminished their sense of worth. With Rhapsody Refiner, they appreciated having the “final say” and controlling the minutiae, contrasting it with prompt-based systems where they felt less involved in the entire creative process. The ability to start with their own initial idea was paramount to feeling that the song was truly “theirs.”
A Tool for Support: Ultimately, participants found the tool highly useful for ideation, providing sounds and ideas they wouldn’t have conceived on their own. It acted as a “sounding board” and an ideation partner, helping them refine their creative goals and even identify what they didn’t like. Crucially, participants did not believe they could become over-reliant on Rhapsody Refiner because it only provided “bits” and not a complete song, reinforcing its role as a supportive aid rather than a replacement.
Also Read:
- Unmasking the Machine: How Listeners Perceive AI-Generated Music
- Unveiling Music AI Decisions: Introducing MUSE-Explainer for Interpretable Music Analysis
Preserving Humanity in Music
The study underscores the importance of designing AI tools that support artistic ownership. By requiring effort from users and embracing imperfections, Rhapsody Refiner encourages active co-creation. This contrasts with the prevailing AI paradigm that often prioritizes consistent, ‘clean’ outputs. The researchers suggest that for practicing musicians who value authorship, systems that encourage effort can help preserve their creative agency.
Furthermore, the paper touches on deeper anxieties surrounding AI’s role in music, questioning what it means to be human when technology can automate creative tasks. However, Rhapsody Refiner alleviated these concerns because its supportive role left creative control firmly in the user’s hands. By not generating the initial idea, it allowed participants to feel that the essence of the music originated from them, maintaining the unique human practice of musical expression.
This research provides valuable insights into how musical AI can genuinely support human creativity, highlighting the importance of designing tools that preserve the humanness of musical expression. For more details, you can read the full paper here.
