Name: MLn Club (ML Reading Group) #16: Vision Language Action Models: Foundation Models for Robotics
Start: 2026-09-07T19:00:00.000-04:00
End: 2026-09-07T21:00:00.000-04:00
Location: 201 S Craig St

Personal

Leo Wang

Welcome to Week 16: π0.7: A Steerable Generalist Robotic Foundation Model

When language models became generalists by training on enormous, diverse datasets, what is the robotics equivalent when useful robot data is scarce, expensive, and fragmented across different machines?

If a robot’s training set contains demonstrations, autonomous rollouts, failures, human video, and Internet data, how does the model learn which behaviors to imitate, which to avoid, and which pieces can be recombined into something new?

The central problem behind general-purpose robotics is not just collecting more data, but making heterogeneous data usable. Prior vision-language-action models can perform many trained behaviors, yet still struggle to compose skills into new tasks without task-specific fine-tuning. π0.7 attacks this by expanding the model’s prompt beyond a simple language instruction: each trajectory can be conditioned on detailed subtask descriptions, generated subgoal images showing what the near-future world should look like, and episode metadata describing speed, quality, mistakes, and control mode. Architecturally, π0.7 is a 5B-parameter VLA with a 4B Gemma 3 vision-language backbone, a memory-style history encoder, and an 860M-parameter flow-matching action expert that predicts chunks of continuous robot actions.

Each of those context signals is a lever on data diversity: metadata lets π0.7 learn from failed and suboptimal autonomous rollouts without blindly imitating them, subgoal images import knowledge from web-scale image-generation pretraining, and cross-embodiment and human data expose the model to behaviors beyond any single robot. The payoff is striking: one model can match task-specific specialists on dexterous tasks, follow unusual instructions in unseen environments, transfer skills between substantially different robots, and compose previously learned behaviors to solve new tasks without task-specific post-training. π0.7 asks whether richer context can substitute, at least partly, for pristine robot datasets.

 for discussion at 8 pm, and (optional) quiet reading from 7 pm.

📖 Reading Recommendations, Questions, or Comments? Contact us 

People who've been wanting to read up on the latest papers in ML and other fields but just haven't been able to find the time/motivation.