

MuSE::Source separation and embedding inversion
Research talk by Marios Glytsos, PhD student in Music Technology, Music and Audio Research Laboratory, NYU, on two papers: "Classical Guitar Duet Separation using GuitarDuets - a Dataset of Real and Synthesized Guitar Recordings" by Marios Glytsos, Christos Garoufis, Athanasia Zlatintsi and Petros Maragos, and "How Much Audio Is Left in an Embedding? An Inversion Audit of Audio Encoders" by Marios Glytsos and Brian McFee.
Marios presents two projects. The first is on source separation, the task of splitting a recording into its individual instruments, applied to a hard case: two classical guitars with very similar timbres. He introduces GuitarDuets, a dataset of real and synthesized duet recordings, and deep learning approaches to separating the two guitars, including ones guided by the notes being played. The second starts from audio embeddings, the representations that models compute from a recording and pass on to other systems. How much of the original audio is still inside one? He uses a generative model to reconstruct music from the embeddings of several audio encoders and examines what can be recovered. No background needed. Talk and Q&A, open to all NYU students.