Problem
DoorDash's support chatbot answers a large volume of questions from couriers (Dashers) and customers. New policies, product changes and edge cases need fresh answers, and writing knowledge-base (KB) articles by hand does not keep up. When the KB has no answer, the chat is escalated to a live agent.
Example from the article: a Dasher asks whether to wait for a red card (a payment card for orders) in the mail. The chatbot cannot explain what the welcome kit is and hands the chat to an agent, whose answer becomes the material for a new article.
Solution in brief
Cluster the escalated chats by embedding similarity to rank KB gaps by frequency. An LLM labels each cluster as actionable or informational and drafts an article for the informational ones from the issue summary and sample agent resolutions. Specialists review the drafts, and the chatbot serves them through RAG, matching the live issue summary against stored issue embeddings.
Questions this case answers
- How to find the highest-impact content gaps in a retrieval-backed assistant from the conversations it failed to resolve
- How to turn human-resolved support cases into reviewed knowledge articles at scale while keeping retrieval accurate
- How to find which knowledge-base articles a support chatbot is missing?
- How to draft new help articles from past live-agent resolutions with an LLM?
Tools and methods
- Incremental embedding clustering — group escalated chat summaries into issue clusters with a cosine threshold
- LLM classifier — separates actionable problems from informational queries
- LLM article drafting — writes KB articles from issue summary plus agent resolutions
- Human review queue — content specialists fix policy, tone and conditional logic
- RAG over a vector database — serves the generated articles to the chatbot
- LLM judge and A/B test — offline relevance check, online escalation measurement
Google or a one-time email code. No password.