Demystifying Attention Sink in LLMs and its Applications to Architecture Design

​Personal website: https://guxm2021.github.io/
Google scholar: https://scholar.google.com/citations?user=BkxEuIoAAAAJ&hl

I am a final-year Ph.D. candidate from National University of Singapore, supervised by Prof. Ye Wang. I earned my B.E. degree of Electronic Engineering and B.S. degree of Finance from Tsinghua University in 2021.

I am a student researcher at Google Deepmind in London, United Kingdom, where I am hosted by Petar Veličković and Larisa Markeeva. I was lucky to be mentored by Tianyu Pang and Chao Du at Sea AI lab (SAIL), Singapore, where I also worked closely with Qian Liu and Min Lin.

My current research focus is to understand, advance and safely deploy generative models and agents. One of my long-term goals is to use test-time-scaling/agents for challenging problems, such as scientific discovery.