Beyond WER: Building and Evaluating Multilingual Voice Intelligence
Beyond WER: What Makes Multilingual Voice AI Good?
An AI Circle x LILT panel + happy hour · Tuesday 17 November · Google, 345 Spear St, San Francisco
Voice AI no longer just transcribes and speaks. Today’s systems listen, reason, respond and act, across dozens of languages, in real time. But we still measure them the old way.
Word error rate can mark down a transcript that got the meaning right. It can also miss a tonal error that changes the meaning completely. And it often moves more with formatting conventions than with real differences between models. Meanwhile, the gaps that matter most in production go unmeasured: accents and dialects, code-switching, noisy real-world audio, domain terminology, and whether a voice agent actually gets the task done.
LILT CEO Spence Green joins researchers from frontier labs and enterprise AI teams to ask what should come after WER. No slides, no pitch. Just the people building and evaluating these systems.
What we’ll cover
Evaluation
Semantic and human evaluation beyond word error rate
Benchmarks built natively in each language, not translated from English
Improving long-tail languages without regressing the rest
Data
Real-world vs. synthetic audio
Expressive, rights-cleared TTS voices
Responsible low-resource speech collection
The big question
What does it mean for a multilingual voice system to be truly good?
Agenda
4:00 – 4:30pm · Arrival, check-in and drinks
4:30 – 5:15pm · Panel: Spence Green (LILT) + [panellists TBC]
5:15 – 7:00pm · Happy hour and networking
7:00pm onwards · Drinks continue at a nearby bar [Location TBC]
Who will be there
Research scientists, evaluation and benchmarking leads, speech and multilingual teams, international product owners, and data and annotation leads. They come from frontier labs, enterprise AI teams and startups.
Who should come
This room is for you if you’re responsible for whether a voice or language model works outside English. That includes building or evaluating multilingual speech systems, running benchmarks that have to hold up across languages, shipping voice products into non-English markets, or sourcing the audio data these systems train on and are measured against.
Space is limited and every registration is reviewed. Feel free to share this with colleagues. Plus-ones need to apply separately.
About LILT
LILT is an AI-native multilingual platform working across frontier AI data, enterprise localization, language-grounded alignment and multimodal evaluation.
About AI Circle
AI Circle is an invite-only community of AI practitioners advancing the frontier, with chapters in San Francisco, New York, Seattle, Toronto, London and Paris, plus campus circles at MIT and Stanford.