From Machine Translation to AI-Mediated Communication - Cohere Labs in Conversation with Marine Carpuat

​From Machine Translation to AI-Mediated Communication
Talk by Marine Carpuat, Associate Professor, University of Maryland
Hosted by Julia Kreutzer, Senior Research Scientist, Cohere Labs

​Multilingual AI systems have never performed better, with steady progress across an increasing variety of languages and tasks on standard benchmarks. But how well do they actually help people understand one another, particularly across language and cultural boundaries?

​This talk argues that achieving this goal requires studying output quality and user perception jointly, and that doing so changes what we build and how we evaluate it. I will draw on my group's machine translation results to motivate a broader agenda for communication-grounded multilingual AI.

​The first part examines what translation taught us about where AI-mediated communication breaks down. In user studies, we found that people are misled by fluency and lack strategies to assess translation quality, creating an illusion of understanding. To mitigate this, we introduce a framework that uses language models not just to translate but to help people assess whether messages meet their communicative goals.

​The second part extends these lessons to cross-cultural communication. Single models exhibit systematic disparities across cultural groups, and benchmarks focus on evaluating model knowledge rather than communication success. We introduce the task of culturally adapted artwork description to illustrate what centering the cultural context and comprehension needs of the listener looks like in practice, and a pragmatic speaker model to improve generation accordingly.

​Together these results motivate a shift from single-model answers toward multi-model scaffolding, and from in vitro benchmarks toward in vivo evaluation of communicative success.

Marine Carpuat
Marine Carpuat is an Associate Professor in the Department of Computer Science at the University of Maryland. Her research focuses on developing AI techniques that help people communicate across languages, and studying whether those systems actually succeed. Her work spans foundational NLP methods, evaluation methodology, and human-centered studies of how people perceive and rely on AI-generated translations. She has published extensively at venues including ACL, EMNLP, NAACL, and CHI, and has received paper awards from *SEM, TALN, and EMNLP. She served as Program Co-Chair of NAACL 2022. Before joining UMD, she was a Research Officer at the National Research Council Canada. She received her PhD from the Hong Kong University of Science and Technology and a diplôme d'ingénieur from the French grande école Supélec.

Webpage: https://www.cs.umd.edu/~marine

​---

​This talk is part of Cohere Labs in Conversation, a limited series of talks in which Cohere Labs scientists and engineers host external researchers for technical talks and Q&A discussions on subjects related to our current explorations at Cohere Labs. We look forward to sharing these talks with you, giving you a glimpse into the problems we're exploring, and learning together from some of the greatest minds in the field.