Conversational AI Engineer
Open a video or resource link, watch it, then check its box to track your progress and unlock what comes next.
- Build natural real-time conversations and the software foundations behind them
- Connect conversations to context, tools and telephony without losing responsiveness
- Evaluate, secure and operate conversational AI at production scale
Benchmark: Current production Voice / Conversational AI roles + OpenAI Realtime/Agents + LiveKit and Google CCAI patterns (current Sep 2026) Current bar: real-time STT/LLM/TTS; low-latency streaming; dialogue state, turn detection and interruption handling; memory/RAG/tools; WebRTC/SIP/telephony; multilingual voice; evals and observability; safe actions, human escalation and production reliability.
Who’s Hiring Right Now
US openings posted in the last 7 days at OpenAI — this role’s own benchmark employers — refreshed daily, as of 2026-11-01. Pulled directly from each company’s public job board; not exhaustive, and postings close fast — worth confirming the details on the employer’s own site before applying.
Build natural real-time conversations and the software foundations behind them
Real-time voice pipeline + speech architecture
Coverage: VAD / STT-ASR / LLM / TTS / streaming audio / Python async / latency
Conversation design, state + agent orchestration
Coverage: Dialogue state / turn-taking / barge-in / prompts / handoffs / multi-step workflows
Connect conversations to context, tools and telephony without losing responsiveness
Context, memory, RAG + tool calling
Coverage: Context engineering / session memory / retrieval / functions / MCP / backend actions
Telephony, WebRTC + production service integration
Coverage: SIP/PSTN / WebRTC / APIs / retries / provider fallback / observability / escalation
Evaluate, secure and operate conversational AI at production scale
Voice-agent evals, observability + failure analysis
Coverage: Task success / WER / latency / interruptions / tool accuracy / traces / regression sets
Safety, human escalation + SLO/incident readiness
Coverage: Guardrails / consent / PII / action confirmation / handoff / SLOs / incident response
Role-Readiness Proof / Capstone
Build a production conversational voice agent for a real workflow. Implement a Python service with streaming audio, STT/LLM/TTS or speech-to-speech, semantic turn detection and barge-in; maintain session state and memory; ground answers with RAG; call at least two tools; connect a telephony or WebRTC channel; add confirmation before consequential actions and human handoff; instrument latency, task success, transcription/tool errors and cost; create a regression eval set; load-test, simulate provider failure and document recovery.
Proof Rubric
- Prove natural interaction rather than a chatbot demo: low and measured latency
- clean turn-taking and interruption recovery
- accurate context and tool use
- safe action boundaries
- resilient telephony/streaming behavior
- reproducible evals
- useful traces and dashboards
- multilingual/noisy-environment handling where relevant
- reliable human escalation
- security/privacy controls
- and evidence that failures or user feedback changed the design