Problem
The Games product analysts at The New York Times worked straight from raw event data. After Wordle joined the portfolio in 2022, tens of millions of players per day flowed into the pipeline. Each request from a product manager meant a long, slow query and a new scheduled job. Jobs were built without a plan, so practices differed, aggregations were incomplete and some jobs were duplicated.
Example from the article: a simple question about trends or usage across the site required a long, inefficient query on raw data.
Solution in brief
The team built tables with explicit aggregation layers on top of raw events, under one naming convention and one table design for every game. Dashboards sit on the last layer, so they load quickly and can be copied to a new game. Work ran at 20% of team capacity in ticketed sprints, and a data observability tool watches the tables.
Questions this case answers
- How to restructure raw event tables into layered aggregates so reports stay fast and consistent as event volume and the number of products grow
- How to fund and run a multi-month data-model rebuild inside a small analytics team that also serves daily requests
- How to stop writing long ad hoc queries on raw event data for every stakeholder request
- How to make dashboards for each new game launch quickly and with the same metrics
Tools and methods
- Layered aggregation tables — base, aggregate, summary and dashboard tables built from event data
- Naming and language standards — shared names for data, fields and tables so new games and features fit in
- Data observability tool (third-party, not named) — custom monitors that alert on data outages
- GitHub — version control and code review for the table models
- Slack — one central request channel that the team lead uses to prioritize work
Google or a one-time email code. No password.