Daniel Ajisafe - Making Video Models Adhere to User Intent with Minor Adjustments

​Recent advances in text-to-video diffusion models have significantly improved video generation, but reliably controlling these models remains challenging. Existing methods often struggle to capture precise user intent when following spatial controls such as bounding boxes.

In this session, Ajisafe et al. present an optimization framework that improves control adherence in pre-trained video models through minimal adjustments to user-provided control boxes. The method aligns these controls with the model’s internal attention mechanisms and uses smooth masking to enable stable optimization. This leads to better overall performance than existing baselines on the AnimalKingdom dataset.



Daniel Ajisafe is a Ph.D. student in Computer Science at The University of British Columbia, advised by Prof. Kwang Moo Yi and Prof. Helge Rhodin. His research focuses on controllable generative models, with an emphasis on aligning model behaviour with user intent.

He is a recipient of the RBC Borealis AI Fellowship and received a Best Paper Award for his work on “Mirror-Aware Neural Humans” at the Weakly Supervised Computer Vision Workshop at Deep Learning Indaba 2023. He was also a Google and Facebook Scholar at the African Masters in Machine Intelligence program and previously worked as a Data Scientist at KPMG Nigeria.

Beyond research, Daniel is passionate about impact and community-building. He led the first member-led Black in AI social at CVPR 2023, helping reduce barriers to participation for underrepresented communities.