Lorong AI x IMDA TSS: AI Wednesdays – The One About Culturally-Grounded Benchmarks (ft. Google Deepmind & MLCommons)
As AI systems scale globally, safety benchmarks need to reflect diverse languages and cultural contexts. Come explore how culturally grounded, multilingual, and multimodal benchmarks are being developed and why this matters for robust AI safety evaluation.
More About the Sharings
Dr Lora Aroyo (Research Scientist, Google DeepMind) will share "Towards Globally Inclusive AI Safety: Launching AILuminate's First Multimodal Safety Benchmark"
AI safety evaluations today are heavily skewed towards English-language data, making it difficult to assess how models behave across different languages and cultural contexts. Lora will introduce the evolution of the MLCommons AILuminate Suite, which aims to address this gap by expanding global representation in safety benchmarking. Explore how AILuminate is building a scalable, resource-efficient framework that enables regional partners to develop multilingual, culturally grounded safety benchmarks, and gain insights from AILuminate’s first multimodal safety benchmark incorporating text and image-to-text evaluations. Learn how cultural specificity is being treated as a core dimension of AI safety, and what this means for building more robust and trustworthy AI systems globally. (Technical Level: 200)
More About the Speaker
Dr Lora Aroyo is a Research Scientist at Google DeepMind, specialising in responsible AI, data quality for generative AI, and AI safety evaluation. Lora is also co-lead for the MLCommons Multimodal Workstream and currently leads the expansion of AILuminate with multimodal safety bechmark. She previously served as Research Scientist at Google Research (2018-2024) and Professor of Human-Computer Interaction at VU University Amsterdam. Lora is renowned for developing the CrowdTruth crowdsourcing method and her pioneering work in user-centric data science. She serves as co-chair of the NeurIPS Datasets & Benchmarks Track, was a keynote speaker at NeurIPS 2023, and previously served as President of User Modeling Inc. Her research focuses on hybrid human-AI systems, with applications spanning digital humanities, cultural heritage, and multimedia analysis.
In Collaboration with IMDA’s Technical Sharing Sessions
IMDA’s Technical Sharing Sessions (TSS) is a monthly forum that convenes practitioners in Singapore’s emerging technology community to tech-spar on AI, Digital Trust, 5G, Quantum Technologies, and Green Software. IMDA has featured 85 speakers across 7 countries, including leaders from IBM, Cohere, SAP, Samsung and Microsoft Research.
More About the Series
AI Wednesdays is Lorong AI’s weekly gathering, bringing together practitioners, researchers and innovators for technical discussions on research insights, product development and engineering practices.
Get involved: Learn more about Lorong AI | Speaker Sign-up | WhatsApp Community | LinkedIn | X