Problem
A bank or marketplace predicts a user outcome (churn, loan default, next purchase type) from the user's own event sequence: an encoder makes one vector, and a classifier such as CatBoost sits on top. This setup has two blind spots. A user with a long silence has no recent events, so the vector is stale. And the encoder sees nothing outside its own dataset, such as a market shift or a new product launch, so it cannot react to them.
Example from the article: when a new iPhone comes out, purchases of it jump suddenly, and a model that sees only a single user's history cannot know this.
Solution in brief
Take the latest embeddings of other users who do have recent events and aggregate them into one external context vector. Feed it to the final model together with the user's own vector. This replaces hand-picked external features such as exchange rates. Among the tested aggregators, kernel attention ranked best; mean and max pooling came next.
Questions this case answers
- How to add external context that shifts over time to a per-entity prediction model built on event-sequence embeddings, without hand-picking external factors
- How to predict outcomes for entities that have had no recent events, when their encoded representation is stale
- How to add external context (market shifts, new product launches) to a model that predicts user outcomes from event sequences
- How to predict churn or default for users who have had no recent events
Tools and methods
- CoLES — contrastive self-supervised pretraining of the sequence encoder.
- Static pooling (mean, max) — aggregation of other users' vectors with no extra parameters.
- Learnable attention variants (learnable, symmetric, kernel) — trainable aggregation of the context vectors.
- Hawkes-style time weighting — decay by time since each context user's last event.
- CatBoost or a neural head — final classifier on the concatenated internal and external vectors.
Google or a one-time email code. No password.
