User foundation models for Grab

Abhinav Rai26 Sep 2025

Problem

Grab's recommender models relied on hundreds to thousands of hand-built user features, such as food types ordered last week, ride frequency or average spend. Each feature served one task, sat inside one team, and took manual effort to build. These features also capture the order of in-app actions poorly. Data that mixes text, numbers, IDs and locations across many services does not fit an off-the-shelf language model.

Example from the article: a user books a ride to a mall, then two hours later searches "Japanese food" from that mall. A siloed model treats the two events as unrelated, though the drop-off location explains the search.

Solution in brief

Grab pre-trains one transformer on tabular attributes and clickstream together, without task labels. Every input becomes a key:value token: tabular tokens form an unordered set, clickstream tokens an ordered sequence. The model yields long-term and short-term user embeddings that downstream models use as input features.

Questions this case answers
  • How to learn one reusable representation of an entity from an unordered attribute table and an ordered event log together, when the inputs mix text, numbers, IDs and coordinates
  • How to train and serve a classifier over a vocabulary of hundreds of millions of IDs without a prohibitively large output layer
  • How to build user embeddings that learn jointly from clickstream sequences and tabular profile data?
  • How to replace hundreds of hand-built, team-specific user features with one reusable representation?

Tools and methods

  • key:value tokenization — one token format for table columns and clickstream events
  • Modality adapters — per-type encoders (text, IDs, numbers, locations) projected into one vector space
  • Masked modeling plus next-action prediction — task-free pre-training objective
  • Hierarchical classification — predicts huge ID vocabularies in two stages
  • Ray and Ray Data — separate CPU and GPU worker pools for training and daily inference
Continue reading — it's free

Google or a one-time email code. No password.