How we adjust in-app messaging for learners in different languages

Eva Raimondi14 Jan 2022

Problem

Duolingo teaches in more than 20 languages, and every interface message and push notification is translated from English. A tone that works in one language can feel flat or off-putting in another, so a direct translation may motivate learners less than it should. Previously, each copy A/B test needed engineering time, so changing a localized string was a small edit with a large cost and made testing inefficient.

Example from the article: the Spanish message shown when a learner quits a lesson ("Do you want to end this session?") struck Duolingo as a bit demotivating.

Solution in brief

Duolingo built Expurrimenter, a tool inside its translation system that lets localization teams set up copy A/B tests directly on strings, linked to the company's own experiment analytics. Teams pick high-traffic or friction screens, write a hypothesis about tone, and test 2-3 versions per language or globally. A winner in one language may not carry over, as the Spanish rerun of the German test showed.

Questions this case answers
  • How to test interface text across many languages and markets when each variant needs engineering work to ship
  • How to decide whether a copy change that won in one market transfers to another
  • How to find out whether an in-app message works equally well in every language the product supports?
  • How to make copy A/B tests cheap enough that localization teams can run them without engineers?

Tools and methods

  • A/B/n test on UI strings — 2-3 copy versions compared against the current text, per language or globally
  • Copy testing tool (Expurrimenter) — Duolingo's in-house tool for creating copy experiments without engineers
  • String IDs in code — each piece of copy is referenced by an ID so it can be swapped by an experiment
Continue reading — it's free

Google or a one-time email code. No password.