K-Steering: A Framework for Behavioral Steering of Language Models
Steering is an inference-time intervention technique that allows you to control behavioural attributes of LLMs.
But, historically, it has always been viewed from a linear lens, which assumes linear directions for concept representations and additive behaviour when combining multiple attributes at once, which has been a limitation due to interference between attributes and separate per-attribute tuning.
We propose a new framework for non-linear steering of language models, which is a unified and flexible approach that trains a single non-linear multi-label classifier on hidden activations and computes intervention directions via gradients at inference time.
About the speaker:
Shreyans is an Independent Interpretability Researcher based out of Bangalore.
His research interests are feature decomposition, multi-lingual alignment, geometry-aware steering and manifold geometry. He was part of the first batch of Lossfunk (then Turing's Dream) Residency.
This is the project he worked on during his 3-month fellowship at Martian. He has worked in Applied ML for 8+ years, building solutions like recommender systems, OCR engines, and search engines.
LinkedIn: shreyans-jain-4b063667
Pre-read: https://arxiv.org/abs/2505.24535
To attend online:
Add to calendar: https://shorturl.at/aQkRQ
Gmeet link: meet.google.com/wju-zvpw-div
Look forward to seeing you!