Embodied AI: World Models from Scratch Session 5
Unlock the mystery of world models—no one fully understands them yet, but they're transforming how we interact with AI. Join the embodied AI group as they guide you through this open, hands-on technical series to build world models from scratch and gain deep expertise in architecture, training, and implementation.
Each session pairs technical concepts with practical tools: runnable code, clear diagrams, and interactive explorables that let you experiment in real time. Progress from leveraging pretrained models to constructing your own from the ground up, covering critical aspects like training optimization, model adaptation, computational efficiency, and distillation techniques. Whether you're a researcher, engineer, or curious builder, this series will equip you with the skills to understand and create world models that power the next generation of AI systems.
This session will cover:
Chapter 6: Pretraining World Models
6.1 Understanding World-Model Pretraining
6.2 Preparing Sequential Datasets
6.3 Training Representations
6.4 Training Dynamics
6.5 Training Future Prediction
6.6 Training Action-Conditioned Models
6.7 World-Model Training Objectives
6.8 Building the Pretraining Loop
6.9 Evaluating Pretrained Checkpoints