The One About LLMs in Practice II (ft. OCBC AI Lab)
From multilingual model capabilities to production-scale deployment challenges, join us to explore both the theoretical foundations and practical realities of working with LLMs at scale.
More About the Sharings
LLMs have demonstrated impressive translation capabilities even without being explicitly trained on parallel data. This remarkable property has led some to believe that parallel data is no longer necessary for building multilingual LLMs. Is that really true? Reza will share more about previous research on how parallel data actually impacts LLMs' multilingual capabilities, examining translation performance and multilingual reasoning through controlled experiments that reveal parallel data's significant benefits. (Technical Level: 100 - 200)
How do you process a batch of 1,000 requests with a self-hosted LLM, without slowing everything down? If you send them one by one, it takes forever, but if you send them all at once, you risk overwhelming the system. So, what's the right balance? Hear from the Machine Learning Engineering platform team at OCBC's Group Data Office, who developed an LLM batching engine to help achieve this balance. In this Lorong AI session, Isaac will share about OCBC’s on-premises GenAI platform, the challenges of serving both batch and real-time workloads and how OCBC’s internal LLM batching engine aims to addresses this challenge. (Technical Level: 200)
More About the Speakers
Muhammad Reza Qorib is a research fellow at the National University of Singapore. He completed his PhD at NUS under Professor Hwee Tou Ng, focusing on system combinations and mixtures of experts for grammatical error correction. His current research explores the multilinguality of large language models. He actively contributes to the community through the SEACrowd projects and by serving as a reviewer for ACL Rolling Review as well as other NLP conferences and journals.
Isaac is a Machine Learning Engineer at OCBC's Group Data Office. He has contributed to OCBC's internal Large Language Model Serving platform, AI Gateway, and LLM Evaluation platform. He previously worked at SAP as a data scientist, developing Legal AI and AIOps solutions for internal stakeholders.
Psst....Interested in becoming a speaker for this session? Sign up here!