90/30 Club (ML reading) #24: Fundamental Limitations of Single-Vector Embeddings

​Week 24: Fundamental Limitations of Single-Vector Embeddings

​On the Theoretical Limitations of Embedding-Based Retrieval

This research establishes a fundamental mathematical constraint in dense retrieval: embedding models cannot represent all possible top-k combinations of relevant documents simultaneously.

Weller et al. demonstrate this through both theory and a clever benchmark (LIMIT) where even advanced state-of-the-art models struggle with trivially simple queries, suggesting important implications for instruction-following retrieval systems.

​Join us to explore:

- Why does something as simple as "finding people who like apples" break our best models?

- What do alternatives like multi-vector models or cross-encoders mean for real products?

​🔎Analyzed Papers

​​Discussion at 20:00, (optional) quiet reading from 19:00.