Problem
Wealthfront's phone support is staffed by people, with no phone tree. The only hard data kept for a call is time, incoming number and duration, plus a free-text note the specialist may write. The notes vary in length and structure. The team wants to know which client needs are most common, so it can remove the biggest ones with automation. Hand labeling does not scale, and an LLM labeling every note gives inconsistent names at high cost.
Example from the article: two notes about the same cash-account interest-rate question could come back from an LLM as "Interest rate question" and "Question about interest rates", so counts split.
Solution in brief
Embed each note, compress the vectors with UMAP, cluster them with HDBSCAN, then repeat inside each cluster to get sub-topics. An LLM names each cluster from notes sampled near sub-cluster centroids. A k-nearest-neighbors classifier trained on the labeled notes then tags new notes, and periodic re-clustering catches new topics.
Questions this case answers
- How to discover a stable multi-level topic taxonomy in a large pile of short free-text records when no label list exists and labeling every record with an LLM is too slow and expensive
- How to assign new free-text records to discovered topics and detect emerging topics when the clustering method cannot score unseen data
- How to find out what clients call support about when the only record is free-text agent notes?
- How to build a topic taxonomy from unstructured text without defining labels up front?
Tools and methods
- OpenAI text-embedding-3-small — turns each note into a 1,536-dimension vector
- UMAP — compresses vectors to 25 dimensions before clustering and to 2 for plots
- HDBSCAN — finds flat clusters with a minimum size
- LLM (frontier model) — names clusters from sampled notes and can merge near-duplicate labels
- k-nearest-neighbors classifier — assigns new notes to the discovered topics
Google or a one-time email code. No password.
