Open Science Robotics Foundation Models for Real-world Deployment with MolmoAct
Vision-language-action (VLA) models promise a future where a single generalist controller can enable robots to perform a wide range of tasks, from folding laundry and assembling parts to tidying wet labs, without requiring a separate hand-engineered policy for each one.
Despite rapid progress, deploying these models in the real world remains challenging. Many frontier models are closed, making them difficult to reproduce or adapt. Open-weight alternatives are often tied to expensive robot hardware, while reasoning-augmented policies can struggle to meet the latency requirements of closed-loop control. Even with task-specific fine-tuning, reliability still falls short of what dependable deployment demands.
In this talk, Prof Jiafei Duan will present two recent projects from the Allen Institute for AI and the University of Washington: MolmoAct and MolmoAct2.
MolmoAct introduces Action Reasoning Models, an interpretable three-stage pipeline that transforms observations and language instructions into depth-aware perception tokens, editable trajectory traces, and low-level robot actions. Building on this work, MolmoAct2 moves closer to practical deployment by introducing open datasets, an embodied reasoning vision-language model, an open action tokenizer, support for low- to medium-cost robot platforms, and adaptive-depth reasoning that enables faster closed-loop control.
Beyond the models themselves, Prof Jiafei will discuss why openness has become a key ingredient for progress in robotics foundation models. Reproducible research depends not only on model weights, but also on access to the datasets, tokenizers, architectures, and evaluation protocols needed to measure robustness, latency, and trajectory quality.
More About the Speaker
Jiafei Duan is an Assistant Professor at the National University of Singapore under the Presidential Young Professorship (PYP) scheme. He received his Ph.D. in Computer Science & Engineering from the University of Washington, where he was advised by Dieter Fox and Ranjay Krishna.
His research centers on robotics foundation models, with a focus on scalable data collection and generation, grounding vision-language models in robotic reasoning, and achieving robust generalization in robot learning.
His work has appeared at leading AI and robotics venues, including ICLR, ICML, RSS, CoRL, ECCV, IJCAI, CoLM, and EMNLP, and has been recognized with the Ubiquitous Robots 2023 Best Paper Award, the CoRL RememberRL Workshop 2025 Best Paper Award, and an ICLR 2024 Spotlight.
More About the Series
Praxis is a focused technical sharing series built around researcher-driven discussions on advanced AI and applied research topics.
Get involved: Learn more about Lorong AI | Speaker Sign-up | WhatsApp Community | LinkedIn | X