Autoscaling RL Paper Club — DSec: Sandbox Infrastructure for Environments at Scale

​Last session DeepSeek-V4.1-Flash gave its sandbox platform one paragraph. This week we read the full systems paper, because the question we care about is what infrastructure has to exist underneath before environment creation can scale.

​DeepSeek Elastic Compute (DSec) serves every sandbox in DeepSeek's RL training and evaluation, from V3.2 to V4.1: function calls, containers, Firecracker microVMs and full VMs behind one SDK. Its argument is that agentic sandboxes are not a serverless workload: they arrive in bursts of up to 32K, most use about 5% of the CPU they ask for, they stay alive and stateful for minutes to hours, and they draw on 130+ TB of images that are each reused only a handful of times. Hence composable read-only image layers, on-demand image loading, memory and CPU overcommit, and rollouts that survive preempted GPU jobs. One scale unit of ~160 CPU nodes serves ~3M sandboxes a day at ~380K peak concurrency.

​Two threads we want to pull on: agents build environments on the same platform by snapshotting a live sandbox, and the paper's catalogue of agents hacking the sandbox itself — forging RPCs to its daemons, reading its logs, overwriting /bin/bash — with the conclusion that "final-output checks alone cannot reliably establish whether the agent solved the task as intended."

​Paper

​DeepSeek Elastic Compute (DSec), Huang et al., DeepSeek, 2026: https://arxiv.org/abs/2609.22978

​Notes: https://research.vibrantlabs.com/club/sandbox-infrastructure-for-environments-at-scale

​Format

​An hour. A short intro on the paper, then open discussion on what it means for anyone building environments. You do not need to have finished the paper, and you do not need to work on this full time. Sessions are recorded — tell us and we keep you out of the recording.