Problem
A bank needs forecasts tied to places: demand for services in a location, client flows, the best spots for ATMs, rent for real estate. Each of these needs data from several domains at once, from points of interest to the geolocation of client transactions. The speaker's approach is to build multimodal location embeddings once and train a meta-model on top; she says combining domains improves the model's ability to generalize.
Example from the article: forecast the average commercial rent per square meter in a location, using DomClick listings as the target. LightGBM without embeddings reaches 0.71; with embeddings 0.815 to 0.85.
Solution in brief
Aggregate data from three domains by H3 hexagon, train one embedding per hexagon with PyTorch-Lifestream (CoLES, NSP or SOP), and pass the vectors as features to a downstream model. The embeddings are computed on a schedule and stored for any task to reuse.
Questions this case answers
- How to build one learned vector per location from several event data sources aggregated by grid cell, and reuse it across downstream predictions
- How to choose between contrastive, next-event and sequence-order objectives for embeddings of event sequences
- How to build location embeddings from customer-activity and infrastructure data
- How to improve a location-level forecast, such as rent per square meter, with learned embeddings
Tools and methods
- H3 hexagonal geohash — aggregates events to a hexagon instead of a point
- PyTorch-Lifestream — framework for embeddings of event sequences; runs on Spark
- CoLES — contrastive learning on event sequences
- NSP, SOP — next sentence and sentence order prediction, adapted to neighboring locations
- LightGBM — downstream model for the benchmark
Google or a one-time email code. No password.