Method

Near-duplicate detection

Perceptual hashing, MinHash/LSH and semantic deduplication.

13 cases from 11 companies apply Near-duplicate detection. 2 were checked in an online A/B test; 5 report results in production.

Typical tasks: Moderate content at scale, Catch fraud and abuse, Rank search results, Match entities and 6 more. Companies: Celsus, Sber, LY Corporation, DoorDash, Wildberries, Yelp and 5 more. Often combined with: Embedding similarity matching, Threshold selection, Human in the loop, Word2Vec, Weak supervision and 48 more.