Generative Music Model Evaluation: Metrics, Misalignments, Meanings
Generative Music Model Evaluation: Metrics, Misalignments, Meanings
Presenter: Dr. Alexander Sigman – Engineer and musician based in Luxembourg, with 6+ years of experience working at generative music AI startups. Prior to (and even after) transitioning into industry, he was active in academia in the US, Korea, and Japan. His music has been commercially released on the New Focus, Carrier, and Innova labels.
What to Expect: In every article on generative AI models, there is an evaluation section. In the music domain, there is a relatively standard "Swiss army knife" of objective metrics and subjective assessments. But what do these acronyms and scores actually mean, and to what extent can they be trusted? In this presentation, Dr. Sigman will examine the insights that the most commonly used evaluation techniques shed on model performance, and their respective limitations. The design and analysis of listening tests will subsequently be discussed. After analysing these methods individually, the synthesis of objective and subjective indicators to approach a complete picture of model capabilities, as well as the application of "reinforcement learning with AI feedback" to steer the model in the direction of subjective preferences will be addressed.
How to join:
This session is public and will be live-streamed on our Discord channel: Sign-up to get the link.
Future Sesssions:
If you are not a member, join Munich Music Labs to get full access to our knowledge-sharing sessions:
Subscribe to our Luma calendar to stay up to date with upcoming events!