Problem
Grab's first feature store, Amphawa, stored each feature as a key-value pair and exposed new values as soon as they were written. As models grew more complex, three problems appeared. Models could not fetch rows by a composite key. Long-running updates showed old and new data side by side, so users wrote compatibility code in their models. Readers competed for read capacity, and in some storage systems writes also ate I/O (the noisy-neighbor effect).
Example from the article: a recommendation model needs all entities matching a key such as consumer_id + restaurant_id to build context before the call.
Solution in brief
Grab replaced single features with feature tables: data-lake tables that are validated, loaded into the serving database and made visible only after the whole load succeeds. The database is Amazon Aurora for PostgreSQL, where writer and readers scale separately. The read library runs inside each model-serving orchestrator, not in a central service.
Questions this case answers
- How to publish a new version of a lookup dataset to online consumers atomically, so readers never see a mix of old and new values
- How to isolate read capacity of a low-latency lookup store from bulk write load when readers compete for shared resources
- How to make a feature update visible to models all at once, not key by key?
- How to stop heavy feature ingestion from slowing down feature reads for online models?
Tools and methods
- Feature table — a table with its own schema and indexes, written as Parquet to a data lake.
- Reverse ETL with atomic switch — loads S3 data into Postgres, visible only after a full load.
- Shadow reading — copies live reads to the new tables before the switch, as a warm-up.
- Amazon Aurora for PostgreSQL — serving store with separate writer and reader nodes.
- Apache Spark and Amazon S3 — where feature tables are built and stored.
Google or a one-time email code. No password.