Touch Weights - Continual Learning Hack
Continual Learning Hackathon Berlin
Hosted by Construct Labs x Alexandria
August 1st, Berlin.
One day, ~20 of Europe's strongest ML researchers and engineers, one focus: models that keep learning after deployment.
Today's LLMs live in a perpetual present. They emerge from training with vast knowledge frozen into their parameters, then never form new memories. We compensate with scaffolding: retrieval, longer context, agent harnesses. But retrieval is not learning. Real learning requires compression, and compression happens in the weights. This hackathon is about doing exactly that: updating parameters, not prompts.
Teams will receive a curated dataset and dedicated compute. Possible topics include:
Recursive self-improvement: bootstrapping models from their own generated reasoning and feedback, in the spirit of STaR and AlphaEvolve
Synthetic RL environments: building environments that turn deployment-style signals into stable weight updates
On-policy distillation: using a model's own outputs as training signal to sidestep catastrophic forgetting
KV-cache compaction: attachable knowledge modules that compact trajectories into infinite context lengths.
Travel stipends are available for strong participants based outside Germany, particularly Zurich, London and Paris.
With support from Telli, Lyceum and Tech Europe.
Apply below. Selection is based on demonstrated work: papers, repos, or training runs.