MLflow: ML & LLM Experiment Tracking and Evaluation - AI Build & Learn #12
Welcome to AI Build & Learn a weekly AI engineering stream where we pick a new topic and learn by building together.
This event is about experiment tracking and model evaluation with MLflow, an open-source platform for managing the end-to-end machine learning lifecycle. MLflow covers experiment tracking, model registry, serving, and evaluation tools for both traditional ML and LLM applications.
We'll explore MLflow's tracing and evaluation features for LLM workflows, tracking experiments and metrics, and how MLflow compares to other observability tools like Arize Phoenix (from last event).
Some things to look up to get started:
MLflow GitHub: https://github.com/mlflow/mlflow
MLflow docs: https://mlflow.org/docs/latest/index.html
Reources
Events Calendar: https://luma.com/ai-builders-and-learners
Slack (Discuss during the week): Flyte Slack Group
Hosted by Sage Elliott: https://www.linkedin.com/in/sageelliott/
In this stream
Intro to topic
Community Discussion
Practical examples
Community challenge (optional)
Try spending 30–90 minutes during the week learning or building something related to the topic, then share what you’re working on in Slack.
Note on Flyte / Union
You may see Flyte used in some demos. Flyte is an open-source AI orchestration platform maintained by Union (where I work) for building scalable, durable, and observable AI workflows. You do not need to use Flyte to participate.
Union: https://www.union.ai/
Flyte: https://flyte.org/
Drop a comment with ideas for future topics (agents, RAG, MLOps, robotics, frameworks, and more).