Three Pillars of Successful Data Science Implementation

Evgeny Smirnov25 May 2024

Problem

A business asks a data science team for "a model" or "generative AI", and a good share of even simple models never pay off after rollout. The failures come from outside the model: data that does not explain the target, no way to run the model in production, a cost larger than the benefit, or a business process that ignores the output. The team spends weeks building something that is dropped at the pilot stage.

Example from the article: a team built a good churn model, and the pilot still failed because nobody had thought of an action that could retain the client.

Solution in brief

Before building, check three things: economic and technical feasibility, readiness of data and infrastructure, and readiness of the business process. Pick the simplest model class that works, compare the money benefit with all costs, and set the task together with the process owner.

Questions this case answers
  • How to decide before building a model whether its benefit exceeds all costs and whether the data, infrastructure and business process can actually use its output
  • How to choose the simplest model class that is good enough for a business decision, comparing a baseline with boosting, neural networks and LLMs
  • How to decide whether a data science project is worth implementing
  • How to check that data and infrastructure are ready for an ML project

Tools and methods

  • Four data-source requirements — correlation with the target, enough volume, long enough history, availability in production
  • Model class ladder — baseline, gradient boosting for tables, neural networks for images, audio and text, LLMs only for generation
  • Analytics to ML to DL pyramid — each level is credited only with its delta over the previous one
  • Economic feasibility estimate — benefit in money against development, implementation, support and hardware costs
Continue reading — it's free

Google or a one-time email code. No password.