vLLM Inference Meetup Bengaluru
Join us for a deep dive into the engine room of vLLM and llm-d AI inferencing, where we will focus on the architecture, optimizations, and raw engineering required to run inference at scale.
Whether you’re looking to squeeze every last token out of your GPU cluster or you're curious about the latest commits to the vLLM and llm-d ecosystems, this is the room you want to be in.
What to Expect
Deep Technical Sessions: Hear directly from the maintainers and core committers of vLLM and llm-d
Scale in Production: Learn from industry leaders about deploying LLMs in production
Hands-on learning & demos
Networking: Stick around for food and soft drinks. It’s a great chance to chat with the speakers and exchange ideas with fellow developers and engineers.
Who Should Attend
vLLM and llm-d users and contributors
ML and infra engineers working on inference and serving
Platform teams running GenAI in production
Anyone curious about efficient inference across local, cloud, and Kubernetes
Event Agenda
1:00 PM – 1:30 PM | Registration & Welcome Registration, Keynote session.
2:00 PM – 2:30 PM | vLLM & llm-d Updates Prasad Mukhedkar, Red Hat
An overview of the latest updates and developments across vLLM and llm-d.
2:30 PM – 3:00 PM | llm-d for Sovereign & Agentic Workloads Pravein Govindan Kannan, IBM
An overview of llm-d’s distributed inference stack, along with ongoing optimization and benchmarking efforts for sovereign and agentic workloads.
3:00 PM – 3:30 PM | Distributed Inference on ROCm with WideEP on vLLM & llm-d by Chaitanya Sri Krishna Lolla & Sirra Ajith, AMD
Explore distributed inference on AMD ROCm using WideEP with vLLM and llm-d.
3:30 PM – 4:00 PM | Tea/Coffee and Snacks
4:00 PM – 4:30 PM | Inside vLLM Semantic Router: Intelligent Routing for LLM Inference Aayush Saini, Red Hat
Explore how vLLM Semantic Router uses intelligent, cache-aware routing based on request semantics, model capabilities, workload characteristics, and infrastructure constraints to optimize inference.
4:30 PM – 5:00 PM | Scaling Inference at NxtGen Using the vLLM Ecosystem Abhishek Kumar, NxtGen
Learn how NxtGen is leveraging the vLLM ecosystem to scale LLM inference workloads.
5:00 PM – 5:15 PM | Break
5:15 PM – 6:15 PM | Hands-on Workshop: vLLM Inference with AMD GPUs - Get hands-on experience running vLLM inference on AMD GPUs using open-source ROCm platforms. Aditya Sivagnanam
Important information
Registration closes 24 hours before the event. We cannot admit unregistered attendees.
Bring your laptop with SSH installed (GPU instances provided by the organizers)
Please bring a photo ID to verify your registration on arrival.