How Canva collects 25 billion events per day

Long Nguyen10 Jul 2024

Problem

Canva's product analytics feeds A/B tests, personalization, recommendations and user-facing statistics. It runs at 25 billion events per day with 99.999% uptime. Events are stored indefinitely, so a drifting schema makes years of history unreadable. The first pipeline, built on SQS and SNS, was easy to run, but those two services took about 80% of the cost of the whole service.

Example from the article: a data scientist should be able to read event data created 3 years ago using today's schema versions.

Solution in brief

Every event has a Protobuf schema with full transitive compatibility, two named owners and comments, enforced by a code generator in the repository. Events go through a thin collection server, a Kinesis Data Stream (KDS), an enrichment worker and a separate router. Moving from SQS/SNS to KDS cut cost 20 times. Batched zstd compression and an SQS overflow path cut cost further and keep p99 response time under 20 ms.

Questions this case answers
  • How to keep event data readable across schema versions when events are stored indefinitely and producers update at different speeds
  • How to cut the transport cost of a high-volume event stream while keeping tail latency of the ingestion endpoint low
  • How to make every product analytics event carry a documented schema that stays readable for years?
  • How to choose a message bus for a high-volume event pipeline without overspending?

Tools and methods

  • Protobuf event schemas — the machine-readable contract for every event
  • Datumgen (in-house, built on protoc) — checks compatibility rules and generates code and the event catalog
  • Amazon Kinesis Data Streams — transport between server, ingest-worker and router
  • Amazon SQS — overflow and failover path for KDS
  • zstd batch compression — a few hundred events per record
  • Snowpipe Streaming — delivery of events into Snowflake
Continue reading — it's free

Google or a one-time email code. No password.