MLn Club (ML Reading Group) #2: LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics

​Week 2: LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics

​Paper Link

​Most self-supervised learning methods work by carefully balancing instability. Remove stop-gradients, momentum encoders, or augmentation tricks, and they collapse. We’ve gotten strong results, but not a clean understanding of whythey work.

​LeJEPA pushes in the opposite direction: instead of stabilizing training with heuristics, it builds a system where the objective itself prevents collapse.

​The core idea is simple but sharp: learn representations by predicting latent structure across views, while designing the objective so that trivial solutions (e.g., constant embeddings) are provably suboptimal. This removes the need for asymmetric updates used in methods like BYOL or contrastive negatives from SimCLR.

​Join us at CASI to explore:

• ​What fundamentally causes collapse in self-supervised learning, and why most existing methods need architectural asymmetry to avoid it.

• ​Why predicting in latent space changes the game: moving away from pixel reconstruction or contrastive alignment toward structured representation prediction.

​Discussion at 20:00, (optional) quiet reading from 19:00.