MLn Club (ML Reading Group) #21: How to Run Turing-Complete Malware in PyTorch's Backward Pass
Welcome to Week 21: How to Run Turing-Complete Malware in PyTorch's Backward Pass
We'll read this recent (and important) blog post from Elon Litman (Stanford) discussing a new way to run malware in PyTorch, and what threats it allows. Then, we might also chat a bit about how we might respond to this vulnerability.
This paper demonstrates a novel and dangerous supply-chain exploit in modern deep learning frameworks: running arbitrary, Turing-complete programs (such as Conway's Game of Life or hidden malware payloads) entirely inside PyTorch's
.backward()pass using a mathematically zero-valued loss function. The vulnerability arises from a fundamental disagreement between hardware-level floating-point arithmetic (specifically IEEE 754 truncation under large offsets like $2^{30}$) and the symbolic differentiation rules used by autodiff engines. By constructing a regularization term that evaluates to a strict0.0on the GPU and in all logged training metrics, an attacker can trick autograd into producing a targeted gradient vector.As a result, a routine call to
total_loss.backward()andoptimizer.step()forces the optimizer to silently overwrite model weights with arbitrary state updates—all without ever modifying the base model checkpoint, architecture, dataset, or optimizer. The paper highlights a critical blind spot in current AI security: verifying a model's file hashes, scanning code for malicious hooks, or monitoring training loss curves and auxiliary metrics is insufficient, because malicious execution can successfully hide behind completely flat, zero-valued regularization losses.
Join us at CASI for discussion at 8 pm, and (optional) quiet reading from 7 pm.
📖 Reading Recommendations, Questions, or Comments? Contact us here!
🔎 View past meeting notes here.
What's this?
A super warm group of folks discussing their favorite topics!
In the first half, we host an optional quiet reading space
In the second half, we have a discussion where people can talk about what they found interesting about the reading and ask questions about things they didn't understand
When/Where:
CMU AI Safety Initiative's Office, 201 Craig Street, right across the PNC bank. Look for the open door up the stairs.
8pm discussion, 7pm optional quiet reading time.
Here's how it usually goes:
7:00 PM — arrival and settling in
8:00 PM — introductions
8:10 PM — discussion time
9:00 PM — wrap up then open discussion
Who's it for?
People who've been wanting to read up on the latest papers in ML and other fields but just haven't been able to find the time/motivation.
Why:
We've been procrastinating too much on our readings, even though we have so much fun doing them. We know we're not alone in this and want to keep others accountable for learning more about what they're passionate about!
We've also met a ton of really fun friends by discussing what we care about!
Rules/guidelines on how to act:
Act like a host, include people in conversations, talk to people even if they're strangers, offer to explain what you know, and keep an open mind! come to read stuff and find super fun friends :)
Bring snacks if you're feeling kind!