Problem
Thumbtack is a marketplace where pros bid for customer leads. Each bid is stored per service offering in DynamoDB. A pricing change rewrites a few hundred thousand of these settings, several times a year.
Each rollout used to be a custom Go script. The scripts were reviewed, run in staging, dry-run and staged, but they had no runtime machinery. A crashed script could not resume. It did not know how hard it pushed the database. It left no durable record of what it touched, so undoing it needed another script.
Example from the article: a script that computes the increase as it runs lands the price increase twice if it is run twice.
Solution in brief
Run the rollout the way a deploy runs. The input holds target values, not relative changes, and a shared engine handles reading, dry run, paced writes, audit and rollback. Only a small pure transform function is written per change.
Questions this case answers
- How to apply a bulk data change to a live key-value store so that it can be previewed, resumed after failure, paced against capacity and reversed
- How to roll out a pricing change to hundreds of thousands of bid settings without writing a one-off script each time?
- How to make a bulk pricing rollout resumable after a crash?
- How to undo a bulk pricing change without overwriting edits made after it?
Tools and methods
- Declarative target-value input — makes reruns idempotent
- Shared engine with pure transform — per-change logic is about thirty lines of Go
- Dry run on the real pipeline — same code with writes off
- Per-service DynamoDB transaction — bid write, history entry and audit record commit together
- Capacity-based pacing — sleeps between writes, backs off on throttling
- Audit-log rollback — restores a value only if it still matches what was written
- Amazon S3 — dry-run reports and error files
Google or a one-time email code. No password.