Data and DevOps Tools for Evaluating and Productionizing LLMs
Hamel Husain is a machine learning engineer who loves building machine learning infrastructure and tools 👷🏼♂️. He leads or contributes to many popular open-source machine learning projects. He also has extensive experience (20+ years) as a machine learning engineer across various industries, including large tech companies like Airbnb and GitHub. At GitHub, he led CodeSearchNet, a large language model for semantic search that was a precursor to CoPilot.
Hamel is the founder of Parlance-Labs, a research and consultancy focused on LLMs. Parlance Labs is working with tech-forward companies like Honeycomb and Rechat to accelerate AI-powered features in their products. Hamel has a wealth of experience with end-to-end commercial deployments of LLMs, along with hard-won perspectives from working with these technologies in the wild.
Hamel recently joined your host Hugo Bowne-Anderson for a livestreamed recording of Vanishing Gradients to talk about generative AI, large language models, the business value they can generate, and how to get started.
It was so much fun they’re returning just a week later to walk you through an actual industrial use case Hamel is currently working on with Rechat, a real estate CRM, taking you through LLM workflows and tools, including (but not limited to!)
Prompt engineering
Data quality checks,
LLM evaluation for both human and synthetically generated prompts,
Creating a human-centric UX for interacting with LLMs and their outputs,
Unit tests for LLM output and integrating this into a CI pipeline,
Fine-tuning ChatGPT with your results,
Training classifiers and using RLHF (Reinforcement learning from human feedback).
