Building the AI Retrieval Infrastructure Behind 20 Billion+ Vectors at HubSpot

Oleg Tereshin, Xin Liu25 Jun 2026

Problem

HubSpot runs semantic search for agents, RAG and contact deduplication, used by 38+ teams. A proof of concept in 2023 had about a dozen Qdrant clusters, managed with Helm and manual tuning of replicas, disk, RAM and CPU. That stopped working as the fleet grew to 140+ clusters. Helm cannot call APIs, autoscale on external metrics or run state-aware lifecycle logic. Open-source Qdrant leaves shard transfers and peer removal to the user. Scaling down was a four-step manual job, and uneven shards forced premature scale-ups.

Example from the article: a BM42 sparse-vector collection with over 3 billion points and shards unevenly spread across pods, so the heaviest pod set the memory limit for the whole cluster.

Solution in brief

HubSpot wraps Qdrant in a Kubernetes operator. Each cluster is one custom resource, and "translators" re-check the cluster every 60 seconds. The same loop adds scale-down with safe shard eviction, greedy shard balancing and replica recovery, so a small team can run hundreds of clusters.

Questions this case answers
  • How to operate a fleet of stateful distributed search clusters when the open-source engine has no automated scaling, rebalancing or replica recovery
  • How to keep per-node memory even in a sharded in-memory index so that one skewed node does not force an early scale-up
  • How to run a self-hosted vector database for dozens of teams without operations work growing with every new cluster?
  • How to scale a Qdrant cluster down without losing data or breaking the replication factor?

Tools and methods

  • Qdrant — open-source vector database with HNSW index, named vectors, hybrid search, quantization
  • Kubernetes operator pattern — keeps actual cluster state equal to the desired state in a custom resource
  • Kafka — asynchronous write path; indexers consume topics created by the operator
  • Vector as a Service (VaaS) — in-house API layer: access control, embeddings, versioning, feedback
Continue reading — it's free

Google or a one-time email code. No password.