Problem
Airbnb's Categories product lets guests browse homes grouped by a theme such as Lakefront, Golf or Desert. A category is defined in product terms ("Lakefront: less than 100 meters from a lake"), but no single data signal captures that definition. Points of interest (POI) are imperfect and host text is inconsistent. There are no labels at the start, and wrong listings hurt the browse experience. Human review of category fit, cover photo and listing quality also has to scale.
Example from the article: the Lakefront category, where rule-based assignment failed on imperfect lake boundaries, highways between lake and house, and reservoirs mistaken for lakes.
Solution in brief
Start with rules over existing listing signals and send the highest-confidence candidates to human agents. Train one XGBoost model per category on their decisions, plus models for cover photo and quality. Publish automatically only above a score threshold that gives 90% precision on a test set, and keep sending the most valuable candidates to humans.
Questions this case answers
- How to assign items to a theme-based catalog category when no single signal defines membership and no labels exist at the start
- How to move from rule-based candidates and human review to an ML classifier while keeping the precision of automatically published items above a fixed level
- How to build thematic collections of listings (Lakefront, Golf, Desert) when no single signal defines membership?
- How to start labeling with rules and move to ML while keeping the precision of what goes live under control?
Tools and methods
- Rule-based candidate generation — weighted confidence score over listing signals.
- Human review — labels category fit, cover photo and quality tier.
- XGBoost, one binary model per category — controls precision through a score threshold.
- Feature dropout — hides the dominant feature in training so the model finds what rules miss.
- Vision Transformer (ViT), fine-tuned — scores listing photos to pick a cover photo.
Google or a one-time email code. No password.