Problem
Keyword search matches the words of a query, not its meaning. A phrase is used in many contexts, so exact matching often misses relevant content or returns the wrong page. A site that wants users to stay and explore needs results that fit the intent of the query even when the wording differs. Semantic search does this, but a brute-force vector search can become the latency bottleneck under concurrent load.
Example from the article: the query "working desk" should find content about a "study table", although the two phrases share no words. Wingify built Kroomsa, a question-answering search engine over Reddit, to test this.
Solution in brief
Encode every question and every query with a sentence encoder (Universal Sentence Encoder, 512 dimensions). Index the question vectors in an exact FAISS index and fetch the 20 nearest questions per query. The top hit supplies the answer and the other hits are shown as related content. Moving the index to a GPU cut latency about 10x, and batching concurrent requests cut it about 225x.
Questions this case answers
- How to match a free-text query to documents by meaning when keyword overlap fails and exact results are required
- How to keep exhaustive vector search fast enough for interactive use when a CPU index saturates under concurrent requests
- How to return relevant answers when the query and the content share no exact words?
- How to build a semantic search service over a few hundred thousand texts with a sentence encoder and a vector index?
Tools and methods
- Universal Sentence Encoder (USE) — turns questions and queries into 512-dimensional vectors that capture intent
- FAISS — exhaustive ("Flat") nearest-neighbor index with Euclidean distance, on CPU or GPU
- MongoDB — stores full posts and answers, looked up by integer id after the search
- Siege — load-testing tool that simulates concurrent users
Google or a one-time email code. No password.