Building for Inclusivity: The Technical Blueprint of Pinterest’s Multidimensional Diversification

Pedro Silva, Bhawna Juneja, Rohan Mahadev, Sujay Khandagale, Abhay Varmaraja21 Sep 2023

Problem

On a visual platform, fashion and wedding results can show only a narrow range of body types. The ranking stack already diversifies by skin tone, but it has no body type signal, so it has to be built from scratch. A second dimension must be added without breaking relevance or engagement. Pinterest faced this in women's fashion search and in the related feed shown under a Pin (closeup recommendations).

Example from the article: for fashion queries in the US, Pinterest needed the top results to show all body types, on top of the skin tone diversification it already had.

Solution in brief

Train a body type classifier on top of an existing image embedding and store its output on each Pin as a discrete feature. At serving time, add one diversity term per dimension to the weighted sum that a Determinantal Point Process (DPP) re-ranker already optimizes against the utility score. Widen candidate retrieval so diverse items reach the re-ranker in the first place.

Questions this case answers
  • How to diversify a ranked list across several item attributes at once without sacrificing relevance
  • How to create a content-side attribute signal for images when labels are subjective and must be guided by domain experts
  • How to make search results and related-item feeds represent several attributes of the people shown (skin tone, body type) at once?
  • How to add a new diversity dimension to a ranker that already diversifies on another one without retraining it?

Tools and methods

  • Determinantal Point Process (DPP) re-ranking — picks a list that balances utility scores against similarity on the diversity dimensions.
  • Body type signal — classifier on a unified visual embedding, indexed per Pin.
  • Strong-OR retrieval logic — candidate generation that surfaces more diverse items before ranking.
  • Expert-guided labeling — bias-aware guidelines, outside organizations and internal fashion specialists.
Continue reading — it's free

Google or a one-time email code. No password.