Problem
A CRM holds billions of contacts and deals that change all day. HubSpot has about 11B contact objects, with 18M added daily and 18K updates per second (peaks of 60K). If every update triggers a model call and a score write, the inference server and the CRM database overload, and users see scores jump by tiny amounts. Time-driven features such as "days since last update" also mean untouched objects need re-scoring.
Example from the article: deal scoring handles about 20K requests per second online but writes only about 60 updates per second, and scans over 3.5B objects offline.
Solution in brief
Put a shared scoring platform between the update stream and the model. It drops requests that cannot change a score (non-critical properties, update bursts, very low-scoring objects) and writes a score only when it moves by more than a delta threshold. Costly explanations are requested only for scores that pass it.
Questions this case answers
- How to keep model scores for a very large set of entities up to date when every entity change can trigger an inference call and most new scores barely differ from the old ones
- How to cut the cost of model explanations in a batch scoring job when most scores will not be written anyway
- How to keep ML scores of millions of CRM records fresh without overloading the inference server and the CRM database?
- How to avoid re-scoring an object on every burst of property updates?
Tools and methods
- Debouncer — scores each object at most once per time window
- Delta threshold — writes a new score only if it moves by more than delta
- Split score and explanation calls — explanations only for scores that pass the threshold
- Critical-property filter — listens only to properties that feed the model
- Icebox threshold — skips objects with very low scores in the batch job
- Kafka, Hadoop jobs, S3 snapshots — event input, offline re-scoring, daily CRM snapshot
Google or a one-time email code. No password.