Experience in Creating a Design Document When Designing an ML System: A Store Clustering Service

Lyubava Tkachenko27 May 2023

Problem

Several departments at Lenta, a Russian retailer with 259 hypermarkets and 568 smaller stores, need store clusters for different tasks. They build them by hand: 3 to 5 hours of pure work per run, plus waiting for other teams to export data. Manual splitting ignores some factors and cannot be reproduced. The business gave the team only a broad goal, a web service for clustering stores, with no clear steps.

Example from the article: the team estimated 3 to 5 hours per manual run against about 20 minutes with the service, and wrote a design doc to plan the work.

Solution in brief

The team wrote an ML system design doc before development. Its block diagram split the work between DS and MLE so they ran in parallel, and a data sources table showed data gaps and owners. The planning also showed that a Streamlit MVP was enough.

Questions this case answers
  • How to plan an ML service from a broad business request when the steps between idea and result are missing, and split the work so that data scientists and engineers run in parallel
  • How to estimate the payoff of automating a manual recurring analytics task before building it, and choose between a full web service and a lightweight prototype
  • How to plan an ML service when the business gives only a broad goal such as «build a clustering service»
  • How to estimate the payoff of automating a manual analytics task before building it

Tools and methods

  • ML System Design Doc template — structure for goals, methodology, block diagram and data sources
  • Block diagram with roles — marks which stages belong to the product owner, DS and MLE
  • Data sources table — what data exists, where it lives, how complete it is, who to ask for the rest
  • Streamlit — dashboard for the MVP of the clustering service
  • FastAPI — backend for complex logic behind a Streamlit interface (suggested in Q&A)
  • Decision record — a wiki page per important decision (suggested in Q&A)
Continue reading — it's free

Google or a one-time email code. No password.