✨Build & Learn with Coffee☕ AI Engineering Build Week 6

​📅 Week 6: Monitor and evaluate your LLM application

​This event is part of our seven-week AI Engineering program, where we build an end-to-end AI application step by step and present our finished projects during Demo Day in Week 7.

​This week, we’ll focus on understanding how our LLM application behaves after deployment and how to measure whether its outputs are actually useful.

• ​Monitor and debug LLM calls with Langfuse tracing.

• ​Manage and version prompts with Langfuse Prompt Management.

• ​Introduce the main concepts behind LLM evaluation.

• ​Write focused unit tests for predictable application behavior.

• ​Build evaluation datasets from representative examples.

• ​Add human annotations and feedback using Langfuse scores.

• ​Build an LLM-as-a-judge evaluator

​By the end of the 7 week cycle, you should have a working AI application in your own GitHub repository and practical experience with AI pipelines, APIs, databases, retrieval, evaluation and deployment.

​You are welcome to follow the shared project or apply the weekly concepts to your own project.

​✨ Who is this for?

​Anyone interested in transitioning towards AI engineering.

​No previous LLM or AI engineering experience is required. However, I recommend some basic programming experiences or be to help with your first build session.

​💻 Bring: Your laptop and curiosity. Grab ☕ at the coffeeshop

​💬 Our slides and meeting link with be shared on our Discord → Join Discord Server