Problem
A team has to build a recommender service from nothing. The usual mistake is to start from a model instead of requirements, metrics and data flow. Without an agreed load, latency budget and metrics, nobody can say which model is affordable. The talk is a live workshop that designs the whole service on one invented task, with the ML part in focus.
Example from the article: the audience chose a real-time joke recommender for a mobile app that listens to a conversation and pushes a fitting joke. The assumed load is 100 thousand daily users in Saint Petersburg, 1k RPS at the Friday peak and a 200 ms SLA at the 95th percentile.
Solution in brief
Fix requirements, then business metrics, then ML metrics that correlate with each. Draw the pipeline as small services connected by Kafka topics: preprocessing, cheap vectorization with key phrases, candidate retrieval from similar users and by BM25, boosting re-ranking, enrichment and push. Cache candidates per session and monitor technical and ML metrics in real time.
Questions this case answers
- How to design a real-time recommender service end to end, from requirements through metrics to architecture, under a fixed latency and throughput budget
- How to pair each business metric with an offline ML metric that correlates with it and can be measured with standard means
- How to design a real-time recommender service end to end, from requirements to services and queues
- How to pick ML metrics that correlate with business metrics such as session length, subscription conversion and user count
Tools and methods
- Kafka — queue between services; only item IDs travel after retrieval.
- PostgreSQL — user metadata, cosine search for similar users, jokes, history.
- FastText and key phrases — cheap vectorization that holds 1k RPS.
- BM25 (custom) — text search of jokes by key phrases.
- Gradient boosting, pointwise — re-ranking of about 100 candidates.
- Redis — fallback recommendations and a session candidate cache.
- Prometheus and Grafana — technical monitoring.
Google or a one-time email code. No password.
