Hosts: Junfan Zhu, Aurora Feng
discord.gg/WH7DrTHRXK
Robotics & World Models Reading Club 08: Embodied Human Data as the “Internet of Motion and Behavior” — San Francisco 0516
Robotics & World Models Reading Club 08: — San Francisco
A high-signal reading group for AI researchers & builders pushing the frontiers of robotic world models, WAMs, and embodied intelligence. In our previous sessions, we brought together researchers and engineers from Boston Dynamics, Google, NVIDIA, Stanford, UC Berkeley, CMU, Dyna, ByteDance, Tesla, Generalist, Rhoda AI, and leading Bay Area robotics startups.
Hosted by Junfan Zhu & Aurora Feng.
Supported by Neural Motion, a universal cross-embodiment data representation layer for embodied AI.
Reading Club 08's Core Theme
Embodied Human Data as the “Internet of Motion and Behavior”
Keynote by Ryan Punamiya (NVIDIA Gear, Georgia Tech)
Embodied human data as the internet of motion and behavior
Vision language models have a vast internet to learn tasks like scene understanding and language modeling. However, robotics still is bottlenecked by teleoperation. Similar to supervised learning paradigms in adjacent machine learning fields, we need a more scalable source of rich and diverse data. Recently, a shift towards using what we call “embodied human data” has begun. This new frontier opens up many interesting questions on how we model human behavior in a robotics context, address the various challenges of cross-embodiment learning and how we can scale this paradigm to build an “internet” of motion and behavior. In this talk, I will talk about some of the recent works in learning from egocentric human data, large-scale human pre-training, human-robot co-design and world modeling from human experience. I will conclude with some future directions and food for thought in this exciting new direction.
Pre-Readings
1. Learning Dexterous Manipulation from Egocentric Human Videos
https://arxiv.org/abs/2410.24221
Learns robot policies from egocentric human video via latent action inference + temporal alignment.
→ Replaces teleoperation with weakly-supervised video signals.
2. Scaling Robot Learning with Human Behavior Priors
https://arxiv.org/abs/2509.19626
Uses large-scale human behavior as pretraining, with Transformer policies + low-shot robot finetuning.
→ Human data = pretraining corpus for robotics.
3. Cross-Embodiment Policy Learning via Representation Alignment
https://arxiv.org/abs/2509.04443v1
Aligns human and robot via shared latent action space (contrastive + cycle consistency).
→ Key bottleneck: action representation, not perception.
4. Human-Robot Co-Design for Scalable Data Collection
https://arxiv.org/abs/2512.22414
Co-designs robot morphology + data interfaces to reduce teleop cost and improve alignment.
→ Scaling requires redesigning the data pipeline itself.
5. World Models from Human Experience
https://arxiv.org/abs/2602.16710
Builds action-conditioned world models from human data for planning (latent dynamics + video prediction).
→ Turns human experience into robot imagination.
6. Ego-Exo Transfer: Learning Action from First- and Third-Person Data
https://arxiv.org/abs/2505.21864
Combines egocentric + exocentric views via view-invariant representations.
→ Expands scale while preserving action semantics.
7. Internet-Scale Human Motion Pretraining for Robotics
https://arxiv.org/abs/2604.07607
Trains foundation models of behavior on large-scale human motion data.
→ Toward an “internet of motion” scaling law.
8. Learning Generalist Robot Policies from Human Demonstrations
https://arxiv.org/abs/2602.06949
Learns multi-task generalist policies via sequence modeling over diverse human demos.
→ From imitation → unified behavior models.
Location
San Francisco (Downtown)
Date & Time
Saturday, May 16, 2026 | 2:00 PM – 5:00 PM
Join Discord Community
https://discord.gg/WH7DrTHRXK
Follow Saturday Robotics on X
https://x.com/saturdayrobotic
Agenda
2:00 PM – 2:30 PM Door Opens & Social
Food 😋, beverages🧋 and UNLIMITED strawberries 🍓 (our official reading club fruits ☺️😄).
2:30 PM – 3:30 PM Keynote by Ryan Punamiya (NVIDIA Gear, Georgia Tech) (https://www.rpunamiya.dev/)
Online access via Zoom: TBD
YouTube Recording: TBD (We are looking for recording volunteers)
3:30 PM – 5:00 PM Q&A, open-floor roundtable (10–20 min per topic) on spotlight papers or any paper you’d like to highlight. Feel free to share why the paper matters and its technical details.
Future events
#cvpr-denver-meetup-0606: Saturday Robotics CVPR & World Models Researchers Meetup—Denver 0606
CVPR Denver Luma: https://luma.com/zamm9g2g
Past events
#reading-club-08-0516: Embodied Human Data as the “Internet of Motion and Behavior”
Session 08 Luma: https://luma.com/qoxioge7
#reading-club-07-0509: Learning to Dream: World Models, Imagination, Path to Foundation Models for Control
Session 07 Luma: https://luma.com/srhe0vuo
#reading-club-06-0502: Evolution of Video World Models for Robotics
Session 06 Luma: https://luma.com/sdrd4zwr
Reading Club 06 Review: https://x.com/junfanzhu98/status/2050834699275383008?s=20
#reading-club-05-0425: World Models for Physical Intelligence: From Predictive Brains to Embodied Robots
Session 05 Luma: https://luma.com/p7zvpyvg
Reading Club 05 Review: https://x.com/junfanzhu98/status/2048315020946317710?s=20
YouTube Recording: https://youtu.be/RVy6oQXNDgc?si=u2VLtCBjfdMvXaf-
#reading-club-04-0418: Abstractions of the Physical World for Decision-Making
Session 04 Luma: https://luma.com/atv7bm3i
Reading Club 04 Review: https://x.com/junfanzhu98/status/2045770010979905862
YouTube Recording: https://www.youtube.com/@saturdayrobotic
#reading-club-03-0411: Robotic Policy Adaptation
Session 03 Luma: https://luma.com/561xgirg
Reading Club 03 Review: https://x.com/junfanzhu98/status/2043243484568768519?s=20
YouTube Recording: https://www.youtube.com/@saturdayrobotic
#reading-club-02-0404: JEPA Zoo
Session 02 Luma: https://luma.com/g3qrrti0
Reading Club 02 Review (liked by Yann LeCun on X): https://x.com/junfanzhu98/status/2040716119259164673?s=20
#reading-club-01-0328
Session 01 Luma: https://luma.com/8s4w1wu6
Reading Club 01 Review (liked by Yann LeCun on X): https://x.com/junfanzhu98/status/2038153945219305812
Logistics
Spots are limited. Please arrive by 2:00 PM for check-in. Keynote will begin promptly at 2:30 PM.
We currently do not have volunteers available to assist with late check-ins. Given the high volume of inquiries and 100+ attendees (both online and onsite), we kindly ask that you arrive on time to ensure smooth entry.
Hosts: Junfan Zhu, Aurora Feng
discord.gg/WH7DrTHRXK