ML@Berkeley Reading Group: S6/State Space Models

​Come and hear from Sohom Paul and Michael Lutz about Mamba, a structured state space sequence (S4) model. Mamba is a novel language modeling technique that promises to be far more efficient than modern transformer architectures and scales to remarkably large sequences (DNA, etc). We'll walk through prior work, the mechanism behind state space models, Mamba's uniqueness in both its implementation and hardware acceleration, downstream evaluation, and future applications.

​Location Soda Hall, Room 373.

​Paper link: https://arxiv.org/pdf/2312.00752.pdf