Bigger is not always better: RAG with small LLM

​LLMs are pretty amazing at understanding natural language. When you combine them with RAG (Retrieval-Augmented Generation), it’s like a match made in heaven. You can use this combo for all sorts of things like answering questions on a knowledge base, interactive chatbots, smart coding assistants, and more.

​

​But if you’re worried about privacy, using cloud-based LLMs like GPT or Gemini can be a problem.

​

​What if you could get an astonishingly good RAG using a small LLM?

​

​In this talk, Ariya Hidayat will walk through some code that uses Phi 2, a tiny but mighty LLM, to do RAG on your laptop. He will show you how to use some tricks like query transformation, classifier, and hypothetical embedding to make Phi 2 work like a charm for RAG, even without an internet connection.

​

​Talk will begin around 6pm

​Beer & pizza provided!