LLM Inference & Model Serving Engineering Bootcamp Certificate
Open a video or resource link, watch it, then check its box to track your progress and unlock what comes next.
Search Live Openings
We don’t have a direct job-board feed matched to this one yet, so here are real, live US job searches for “LLM Inference & Model Serving Engineering Bootcamp Certificate” instead.
Bootcamp Mastery Certificate B44
Engineer the serving layer behind production LLMs: GPU memory and KV cache, vLLM, continuous batching, PagedAttention, prefix caching, quantization, Triton/TensorRT-LLM, Ray Serve, benchmarking, autoscaling and observability.
Benchmark: Public learning videos are provided by DevOps & AI Toolkit, vLLM/PyTorch and UC Berkeley contributors, DigitalOcean, NVIDIA Developer, and Anyscale/Apple. Zuyini independently curates them and applies its own build-and-proof completion rubric; this is not a provider-issued certification or endorsement.
Core Video Sequence
01 — Current inference mental model: 12 concepts that matter
02 — VRAM, KV cache, quantization + real concurrency limits
03 — vLLM 2025: architecture, optimizations + roadmap
04 — vLLM technical foundation: PagedAttention + serving
05 — Deploy an OpenAI-compatible vLLM server
06 — Model serving foundation with NVIDIA Triton
07 — Tune latency/throughput with Triton Model Analyzer
08 — TensorRT-LLM optimization + open-development workflow
09 — Enterprise serving platform: Ray Serve + vLLM at Apple
10 — Multi-tenancy, autoscaling, metering + observability
Build-Along Checkpoints
- Checkpoint 1: Serve an open model with vLLM behind an OpenAI-compatible API; capture baseline TTFT, inter-token latency, output tokens/sec, request throughput and GPU memory.
- Checkpoint 2: Calculate weight plus KV-cache memory, then load-test increasing context and concurrency until queueing appears; reconcile measured capacity with the engine startup/runtime evidence.
- Checkpoint 3: Tune continuous batching, max sequences/batched tokens, prefix caching and one quantization option; compare quality, TTFT, throughput, VRAM and cost tradeoffs.
- Checkpoint 4: Deploy the model or a second model through Triton; configure model repository/instances and dynamic batching, then compare serving behavior with the vLLM baseline.
- Checkpoint 5: Benchmark with engine/Triton tooling and a reproducible load generator; report p50/p95 latency, TTFT, TPOT, throughput, GPU utilization and cost per useful token/request.
- Checkpoint 6: Add Ray Serve or equivalent orchestration for replicas/autoscaling, metrics/traces and routing; inject an engine/replica failure or cold start and demonstrate controlled recovery.
7-Day Independent Capstone
- Day 1: Choose model, hardware budget and workload distribution; define latency, throughput, availability and cost SLOs plus acceptance thresholds.
- Day 2: Deploy vLLM and build a load generator; establish single-user and concurrent baselines with TTFT/TPOT, throughput, VRAM and queue-depth evidence.
- Day 3: Tune batching, KV-cache/prefix-cache and quantization; measure at least three configurations and preserve an explicit quality/capacity tradeoff table.
- Day 4: Add Triton or TensorRT-LLM serving for one path; measure dynamic batching/instance configuration and identify where each serving stack fits best.
- Day 5: Add Ray Serve or equivalent routing/autoscaling; test scale-out, multi-replica behavior, warm/cold starts and one multi-tenant or rate-limit boundary.
- Day 6: Instrument dashboards and alerts; inject replica/engine failure or overload, recover, and document rollback/fallback behavior plus SLO impact.
- Day 7: Publish architecture, configs, load generator, benchmark report, dashboards, cost model, failure proof, demo and a production operations runbook.
LLM Serving - Benchmark, Tune, Scale & Operate
Ship a production-style LLM endpoint and prove its economics: establish serving SLOs, model GPU memory, tune batching/cache/quantization, benchmark vLLM and Triton, scale with orchestration and recover from failure.
Proof Rubric — What Must Be Shown
- LATENCY: TTFT/TPOT + p50/p95
- SERVING: vLLM + Triton comparison
- THROUGHPUT: tokens/sec + req/sec
- SCALING: replicas, routing + autoscale
- MEMORY: weights/KV cache + VRAM math
- ECONOMICS: cost per request/token
- TUNING: batch/cache/quant evidence
- OPERATIONS: telemetry, failure + recovery