Problem
Duolingo teaches in more than 20 languages, and every interface message and push notification is translated from English. A tone that works in one language can feel flat or off-putting in another, so a direct translation may motivate learners less than it should. Previously, each copy A/B test needed engineering time, so changing a localized string was a small edit with a large cost and made testing inefficient.
Example from the article: the Spanish message shown when a learner quits a lesson ("Do you want to end this session?") struck Duolingo as a bit demotivating.
Solution in brief
Duolingo built Expurrimenter, a tool inside its translation system that lets localization teams set up copy A/B tests directly on strings, linked to the company's own experiment analytics. Teams pick high-traffic or friction screens, write a hypothesis about tone, and test 2-3 versions per language or globally. A winner in one language may not carry over, as the Spanish rerun of the German test showed.
Questions this case answers
- How to test interface text across many languages and markets when each variant needs engineering work to ship
- How to decide whether a copy change that won in one market transfers to another
- How to find out whether an in-app message works equally well in every language the product supports?
- How to make copy A/B tests cheap enough that localization teams can run them without engineers?
Tools and methods
- A/B/n test on UI strings — 2-3 copy versions compared against the current text, per language or globally
- Copy testing tool (Expurrimenter) — Duolingo's in-house tool for creating copy experiments without engineers
- String IDs in code — each piece of copy is referenced by an ID so it can be swapped by an experiment
Google or a one-time email code. No password.