Classifying User Intent with LLMs & Decision models
Session #2 of Data-driven AI Engineering Fundamentals Community Course
Classification is one of the bread-and-butter tasks of AI Engineering. But how do we know if our classification system is actually good enough? And should we always use an LLM?
In this hands-on session, we'll explore these questions using ATIS, a dataset of airline travel queries.
As always, we'll take a data-driven approach.
We'll start by getting familiar with the dataset, understanding the different intents, and looking at real examples.
Then we'll build a simple LLM prompt as our baseline and evaluate its performance.
Once we have our initial results, we'll introduce Jev, a specialized decision model, and see how it compares.
Finally, we'll examine the results, discuss where each approach succeeds or fails, and wrap up with some practical takeaways.
Dataset: https://huggingface.co/datasets/tuetschek/atis
Experience is what makes us grow 🌱 Visit https://buildatadrivenai.com for a visual and friendly agent evals course, and for a full course that gives you the right practice.
Join our Discord and share your questions to help me prep.
If you missed the previous session, you can find it here: