Problem
After years of strong growth, Duolingo's daily active users (DAU) stalled in 2018. The team that owned DAU could not build A/B tests with a visible effect, because DAU is one lump that many behaviors feed into. Teams need a smaller metric that still matters to the business and is easier to move.
Example from the article: in 2018 Duolingo split DAU into user states, found the rate that moved DAU most, staffed a team on it, and DAU has grown about 4x since 2019.
Solution in brief
Put every user, past or present, into one of 7 mutually exclusive activity states each day and track the daily rates of moving between states (a Markov model). Raise each rate by the same amount in a simulation and see which lifts DAU most. Give that rate to a team and confirm with A/B tests.
Questions this case answers
- How to decompose a stagnating top-line engagement metric into mutually exclusive user states with transition rates, so that teams can optimize a lower-level metric they can move
- How to rank candidate lower-level metrics by their downstream effect on the top-line metric before spending experiment capacity
- How to find metrics a product team can actually move when the top-line daily active users (DAU) metric has stagnated?
- How to break DAU into lifecycle segments and see which retention or reactivation rate matters most?
Tools and methods
- Markov model of activity states — splits DAU into 7 states with daily transition rates
- Growth simulation — raises each rate in turn and measures the effect on DAU
- A/B tests on the chosen rate — check that the rate is movable and that it moves DAU
- User-identity resolution (aliasing) — keeps one person from appearing as a new user on a new device
Google or a one-time email code. No password.