Data Strategy in a Large Group of Companies

Viktor Kantor25 May 2024

Problem

A large group of companies with many businesses (telecom, bank, cinema, ads) runs hundreds of ML tasks on shared data. Teams take a task first and discover the data problems later: the needed signal was never logged, a reference table was never collected, the labels do not exist. At MTS this stretched projects by months. Models that could have shipped in about a month waited for data.

Example from the article: the base station placement model (Smart Rollout) took long to build because station parameters were never collected in one format. Without that delay, something working could have been deployed in about a month.

Solution in brief

Before starting tasks, decide four things: what data to collect and in what form, where the money is (which monetization directions and which tasks can be templated), what level of data quality is needed, and how the plan gets enforced, top-down through KPIs and bottom-up through people who care. The talk is a warm-up, not a recipe for writing the document.

Questions this case answers
  • How to decide what data a large group of business units must collect and how to prioritize monetization use cases before starting ML tasks
  • How to turn repeated ML tasks across business units into a shared platform so that each team does not rediscover missing data
  • How to decide whether a large company needs a data strategy and what it should cover
  • How to decide which data to collect first and which use cases to prioritize for monetization

Tools and methods

  • Monetization map — directions colored by the money they bring; sets data priorities
  • Platformization — a shared platform for repeating tasks (recommendations, ad serving, CVM targeting)
  • Four-level data quality assessment — event counts, corrupted data and adequacy, accuracy against a golden set, usefulness for the model
  • MTS DQ Framework, JIRA/Remedy, data catalog — automated checks, incidents, quality status for data users
Continue reading — it's free

Google or a one-time email code. No password.