Understanding Human Gaze in Everyday Social Scenes using Multimodal Deep Learning

• ​Gaze is a powerful cue for understanding attention, intention, and social interaction. It plays a key role in language acquisition and can also signal neurodevelopmental disorders such as autism.

• ​In this talk, we explore computer vision and multimodal approaches (e.g. vision + depth + VLM outputs) for modeling human gaze behaviour.

• ​Beyond spatial localization, we discuss models, datasets, and evaluation protocols that capture the semantic and social aspects of gaze (e.g. who or what is being looked at).

​About the speaker:
Anshul is a postdoctoral researcher at the Idiap Research Institute. He recently completed his PhD at Idiap and EPFL, where he received the EPFL EDEE distinction award for his thesis. His research focuses on human behaviour understanding, with a particular emphasis on gaze estimation using computer vision and multimodal machine learning. His work has been published at top venues including ICCV, CVPR, and NeurIPS, with some papers receiving best paper awards. During his PhD, he also interned at Meta Reality Labs Research, USA.
https://anshul-gupta24.github.io/

​
To attend online:

• ​⁠Add to calendar: https://bit.ly/4pKteTc

• ​Gmeet link: https://meet.google.com/jvw-uwpd-cks?hs=122&authuser=0

​Pre-read:

• ​GazeFollow: A Dataset and Model for Gaze Following
  http://gazefollow.csail.mit.edu/

• ​ChildPlay: A New Benchmark for Understanding Children’s Gaze Behaviour

• ​Sharingan: A Transformer Architecture for Multi-Person Gaze Following

• ​MTGS: A Novel Framework for Multi-Person Temporal Gaze Following and Social Gaze Prediction