Paper: Attention Is All You Need (2017)

​Our next AI Reading Club session is confirmed for March 26, 2026 (Thursday).

​Paper: Attention Is All You Need (2017)
Link: https://arxiv.org/abs/1706.03762

​This session will explore how the Transformer architecture reshaped NLP and laid the foundation for modern LLMs by replacing recurrence with attention, and why that shift remains so important today.

​We will cover:

• ​Why recurrent models were a bottleneck

• ​Scaled dot-product attention and multi-head attention

• ​Positional encodings without RNNs

• ​Residual connections, layer normalization, and training stability

• ​The long-term impact of Transformer design choices on modern models

​We’re also very happy to welcome Arshia Afzal for a guest presentation on the paper. Arshia is a Research Intern at Mistral and co-author of Linear Attention for Efficient Bidirectional Sequence Modeling.

​Scholar profile:
https://scholar.google.com/citations?view_op=list_works&hl=en&hl=en&user=OJ45nEQAAAAJ

​Format:

• ​10–15 minute summary

• ​Guest presentation by Arshia Afzal

• ​40–45 minute group discussion