Rokib Al Dhin Raadh - When the Update Is Not a Gradient: Input-to-State Stability of Self-Repairing Neural Network Training

​Every standard convergence proof for gradient descent assumes that each update is a gradient. Self-repairing training violates that assumption by design, and it still converges. This session shows why, and exactly where the proof stops.

Zero out a converged convolutional layer at epoch 100. A parameter-matched baseline remains at chance on every seed (10.0% ± 0.0%). A self-repairing network recovers to 80.3% ± 1.6% through 85 bounded, logged corrections. Those corrections are state-dependent, diagnostic-triggered and discontinuous; they are the gradient of no objective. Why the trajectory does not diverge is a genuinely mathematical question.

We model training as a switched dynamical system (contractive gradient flow, bounded disturbance, isolated resets) and derive a practical input-to-state stability bound under local L-smoothness and α-strong convexity:

limsup ‖θₜ − θ*‖ ≤ γ = 2βM/α + ε_max/(ηα)

The bound is non-vacuous only if correction magnitude decays, so the stability claim becomes a falsifiable prediction. Within a 383-run evaluation, it holds: correction magnitude falls by 83–93%.

The most instructive part is the gap between theory and observation. The per-step contraction the proof relies on holds on only 31.6% of steps, yet the run converges. The offset term never vanishes, so the result is practical rather than strict ISS. Forward invariance of the neighbourhood remains unproven.

We close on the open problems: deriving ε(t) from first principles, relaxing strong convexity to Polyak–Łojasiewicz, and dwell-time conditions for discrete reinitializations. Attendees are invited to challenge the bound directly.
preprint: https://zenodo.org/records/21730363

​Bio:
Rokib Al Dhin Raadh is an 18y old researcher working on the stability and auditability of neural network training dynamics, without institutional affiliation. He is the sole author of ORMAS, an architecture that localizes, repairs and logs its own component-level failures during training, validated across 383 controlled experiments on four architectures. His input-to-state stability analysis of self-repairing training was accepted at DeepMath 2026 (Conference on the Mathematical Theory of Deep Neural Networks) after double-blind review. He serves as a reviewer for the NeurIPS 2026 Trustworthy AI for Good workshop and is a member of the Cohere Labs Open Science Community. He is the founder of Oxiedo.