LLMOps
Not Started0 / 10 complete
Open a video or resource link, watch it, then check its box to track your progress and unlock what comes next.
10 lessons3 levels1 capstone project
Operate production LLM and agent systems through evaluation, tracing, observability, release gates, safety, cost/latency control, monitoring, and continuous improvement.
BASIC
- Confidently Iterate on LLM Apps — W&B Weave (15 min)
Fast introduction to eval- driven iteration for probabilistic apps. - Debug, Evaluate & Ship Reliable AI Agents — LangChain (30 min selected)
Shows tracing, datasets, experiments and debugging in practice. - LLM Observability, Evaluation & Experimentation — Arize (20 min selected)
Connects production traces to quality and regression detection.
INTERMEDIATE
- LangSmith Tutorial: LLM Evaluation for Beginners — Dave Ebbelaar
Apply LLMOps through the adjacent capability: AI Evals & Monitoring. - AI Observability Overview — Arize AI
Apply LLMOps through the adjacent capability: AI Evals & Monitoring. - Building AI Agents on Google Cloud — Google Cloud
Apply LLMOps through the adjacent capability: AI Platform.
ADVANCED
- Evaluate a RAG System Step by Step — RAGAS
Extend LLMOps into senior/production practice through AI Evals & Monitoring. - AI Infrastructure on Kubernetes: Run:ai & KAITO — Microsoft / NVIDIA
Extend LLMOps into senior/production practice through AI Factory. - Instrumenting & Evaluating LLMs — Hamel Husain
Extend LLMOps into senior/production practice through AI Evals & Monitoring.