Translation Is Not Evaluation: How We Measure Agent Performance Across Languages & Cultures
Join AI Circle and LILT for an intimate dinner with LILT
VP of Research Joern Wuebker and a small group of researchers, evaluation leads, and technical leaders for an off-the-record conversation about what breaks when agents cross languages, and what it would take to measure them honestly.
This is a conversation among peers, not a panel. No slides, no pitch, just the people building and evaluating these systems.
Agenda
7:00 - 7:30 - Arrival, refreshments, and mingling
7:30 - 7:45 - Seated for dinner
7:45 - 8:00 - Fireside chat led by VP of Research Joern Wuebker
8:00 - 9:00 - Mingling Dinner and Connecting
Multilingual benchmarks are English benchmarks in translation. The sentences come back fluent, the grammar is clean, and the score still tells you almost nothing about whether the agent works for the person using it.
Who will be there
Research scientists, evaluation and benchmarking leads, multilingual and speech teams, localization and international product owners, data and annotation leads, founders, and operators, drawn from frontier labs, enterprise AI teams, and startups.
Who should come
If you own whether a model works outside of English, this table is for you. That includes anyone building or evaluating multilingual systems, running benchmarks that have to hold up in more than one language, shipping product into non-English markets, or sourcing the data those systems train and get measured on...or industry this room makes sense.
Feel free to invite colleagues who would find this conversation relevant. Plus ones but apply to RSVP and must be approved as space is limited.
_____________________________________________
About LILT
LILT is an AI-native multilingual platform working across frontier AI data, enterprise localization, language-grounded alignment, and multimodal evaluation.
About AI Circle
AI Circle is an invite-only community of AI practitioners advancing the frontier, with chapters in San Francisco, New York, Seattle, Toronto, London, Paris, and campus circles at MIT and Stanford.