Hosts: Junfan Zhu, Aurora Feng
discord.gg/WH7DrTHRXK
Robotics & World Models Reading Club 08: Embodied Human Data as the “Internet of Motion and Behavior” — San Francisco 0516
Robotics & World Models Reading Club 08: — San Francisco
A high-signal reading group for AI researchers & builders pushing the frontiers of robotic world models, WAMs, and embodied intelligence. In our previous sessions, we brought together researchers and engineers from Boston Dynamics, Google, NVIDIA, Stanford, UC Berkeley, CMU, Dyna, ByteDance, Tesla, Generalist, Rhoda AI, and leading Bay Area robotics startups.
Hosted by Junfan Zhu & Aurora Feng.
Supported by Neural Motion, a universal cross-embodiment data representation layer for embodied AI.
Reading Club 08's Core Theme
Embodied Human Data as the “Internet of Motion and Behavior”
Keynote by Ryan Punamiya (NVIDIA Gear, Georgia Tech)
Embodied human data as the internet of motion and behavior
Vision language models have a vast internet to learn tasks like scene understanding and language modeling. However, robotics still is bottlenecked by teleoperation. Similar to supervised learning paradigms in adjacent machine learning fields, we need a more scalable source of rich and diverse data. Recently, a shift towards using what we call “embodied human data” has begun. This new frontier opens up many interesting questions on how we model human behavior in a robotics context, address the various challenges of cross-embodiment learning and how we can scale this paradigm to build an “internet” of motion and behavior. In this talk, I will talk about some of the recent works in learning from egocentric human data, large-scale human pre-training, human-robot co-design and world modeling from human experience. I will conclude with some future directions and food for thought in this exciting new direction.
Pre-Readings
1. Learning Dexterous Manipulation from Egocentric Human Videos
https://arxiv.org/abs/2410.24221
Learns robot policies from egocentric human video via latent action inference + temporal alignment.
→ Replaces teleoperation with weakly-supervised video signals.
2. Scaling Robot Learning with Human Behavior Priors
https://arxiv.org/abs/2509.19626
Pretrains Transformer robot policies on large-scale human behavioral data, then adapts to downstream robotic tasks with low-shot finetuning.
→ Human behavior becomes the robotics equivalent of web-scale pretraining corpora.
3. Cross-Embodiment Policy Learning via Representation Alignment
https://arxiv.org/abs/2509.04443v1
Bridges humans and robots through shared latent action representations using contrastive alignment and cycle-consistency objectives.
→ The main transfer bottleneck is action representation, not visual perception.
4. Human-Robot Co-Design for Scalable Data Collection
https://arxiv.org/abs/2512.22414
Jointly optimizes robot morphology, interfaces, and teleoperation pipelines to improve scalability and reduce human demonstration cost.
→ Scaling robot learning requires redesigning the data pipeline itself.
5. World Models from Human Experience
https://arxiv.org/abs/2602.16710
Constructs action-conditioned latent world models from human experience data for prediction, planning, and imagination-based control.
→ Converts human experience into robot imagination.
6. Ego-Exo Transfer: Learning Action from First- and Third-Person Data
https://arxiv.org/abs/2505.21864
Learns view-invariant action representations by jointly leveraging egocentric and exocentric video data.
→ Expands scalable supervision while preserving action semantics.
7. Internet-Scale Human Motion Pretraining for Robotics
https://arxiv.org/abs/2604.07607
Pretrains behavior foundation models on massive internet-scale human motion datasets for downstream robotic policy transfer.
→ Suggests an emerging “internet-scale motion” scaling law for robotics.
8. Learning Generalist Robot Policies from Human Demonstrations
https://arxiv.org/abs/2602.06949
Trains multitask generalist robot policies through sequence modeling over heterogeneous human demonstrations.
→ Shifts imitation learning toward unified behavior foundation models.
Location
San Francisco (Downtown)
Date & Time
Saturday, May 16, 2026 | 2:00 PM – 5:00 PM
Join Discord Community
https://discord.gg/WH7DrTHRXK
Follow Saturday Robotics on X
https://x.com/saturdayrobotic
Agenda
2:00 PM – 2:30 PM Door Opens & Social
Food 😋, beverages🧋 and UNLIMITED strawberries 🍓 (our official reading club fruits ☺️😄).
2:30 PM – 3:30 PM Keynote by Ryan Punamiya (NVIDIA Gear, Georgia Tech) (https://www.rpunamiya.dev/)
Online access via Zoom: TBD
YouTube Recording: TBD (We are looking for recording volunteers)
3:30 PM – 5:00 PM Q&A, open-floor roundtable (10–20 min per topic) on spotlight papers or any paper you’d like to highlight. Feel free to share why the paper matters and its technical details.
Future events
#cvpr-denver-meetup-0606: Saturday Robotics CVPR & World Models Researchers Meetup—Denver 0606
CVPR Denver Luma: https://luma.com/zamm9g2g
Past events
#reading-club-08-0516: Embodied Human Data as the “Internet of Motion and Behavior”
Session 08 Luma: https://luma.com/qoxioge7
#reading-club-07-0509: Learning to Dream: World Models, Imagination, Path to Foundation Models for Control
Session 07 Luma: https://luma.com/srhe0vuo
#reading-club-06-0502: Evolution of Video World Models for Robotics
Session 06 Luma: https://luma.com/sdrd4zwr
Reading Club 06 Review: https://x.com/junfanzhu98/status/2050834699275383008?s=20
#reading-club-05-0425: World Models for Physical Intelligence: From Predictive Brains to Embodied Robots
Session 05 Luma: https://luma.com/p7zvpyvg
Reading Club 05 Review: https://x.com/junfanzhu98/status/2048315020946317710?s=20
YouTube Recording: https://youtu.be/RVy6oQXNDgc?si=u2VLtCBjfdMvXaf-
#reading-club-04-0418: Abstractions of the Physical World for Decision-Making
Session 04 Luma: https://luma.com/atv7bm3i
Reading Club 04 Review: https://x.com/junfanzhu98/status/2045770010979905862
YouTube Recording: https://www.youtube.com/@saturdayrobotic
#reading-club-03-0411: Robotic Policy Adaptation
Session 03 Luma: https://luma.com/561xgirg
Reading Club 03 Review: https://x.com/junfanzhu98/status/2043243484568768519?s=20
YouTube Recording: https://www.youtube.com/@saturdayrobotic
#reading-club-02-0404: JEPA Zoo
Session 02 Luma: https://luma.com/g3qrrti0
Reading Club 02 Review (liked by Yann LeCun on X): https://x.com/junfanzhu98/status/2040716119259164673?s=20
#reading-club-01-0328
Session 01 Luma: https://luma.com/8s4w1wu6
Reading Club 01 Review (liked by Yann LeCun on X): https://x.com/junfanzhu98/status/2038153945219305812
Logistics
Spots are limited. Please arrive by 2:00 PM for check-in. Keynote will begin promptly at 2:30 PM.
We currently do not have volunteers available to assist with late check-ins. Given the high volume of inquiries and 100+ attendees (both online and onsite), we kindly ask that you arrive on time to ensure smooth entry.
Hosts: Junfan Zhu, Aurora Feng
discord.gg/WH7DrTHRXK