Problem
Managers who know classical software development plan an ML team like a software team. Strong people can spend three months and end with a model that returns random output on the available data, which is a normal outcome of R&D. Neighbouring teams that planned around the first model version then fail with it, and the speaker says she was once fired for exactly this. Task framing, dataset collection and quality evaluation suffer from the same gap.
Example from the article: when the speaker joined Skyeng, everyone believed the company had one speech recognition (ASR) system. In fact there were about as many as there were tasks, for example Russian with many output parameters, Russian high quality, and English with pronunciation scoring.
Solution in brief
Explain how ML development works to management before the project starts, and record the talk. Ask customers about likely changes in scope, and keep a model inventory with GUI test stands for reuse. Plan data collection and annotation with a named owner, an early annotation check and a timeline multiplied by 3.
Questions this case answers
- How to set management expectations and timelines for research projects whose outcome is uncertain and whose stages depend on hidden data and labeling dependencies
- How to decide whether a generative model is good enough for production when developers, users and domain experts give conflicting judgments
- How to plan ML R&D projects so that management expectations and timelines match how ML work actually goes
- How to avoid building many overlapping ML models for similar tasks in one company
Tools and methods
- Discovery or deep-dive workshop — explains ML stages and trade-offs to management
- Model inventory with GUI test stands — other teams try existing models themselves
- Dataset checklist — owner, time, annotation quality check
- Timeline multiplier ×3 and conditional deadlines — buffer for data work
- Toloka, Amazon Mechanical Turk — external annotators
Google or a one-time email code. No password.
