Problem
A team at Ozon Bank builds batch scoring models for sellers on the Ozon marketplace (revenue drop and default). Features come from many raw tables: accounting entries, sales, reviews, warehouse logs and an external registry service. They used to be computed by one long multi-iteration SQL script that was hard to read and explain to stakeholders and did not scale. If the calculation failed midway, nothing was saved, and there was no monitoring.
Example from the article: total money credited to a seller's account over the last 3 days, calculated for every seller and every scoring date.
Solution in brief
The team wrote Feature Script, a PySpark module of four parts: a sample of (id, date) events, a class per raw table, a class per feature and a class that joins and aggregates. Each event is joined with the entity's strictly earlier history, then grouped and aggregated by the feature definitions. Feature blocks run as separate Airflow tasks.
Questions this case answers
- How to compute aggregate features over each entity's strictly earlier event history for many entity and scoring-date pairs without target leakage and without one long script
- How to split a feature pipeline into independently rerunnable and monitored steps in a small team that cannot run a heavy feature store
- How to compute event-time window aggregate features for a scoring model without a long SQL script
- How to avoid target leakage when joining an events table with a raw history table by date
Tools and methods
- Feature Script (PySpark module) — computes aggregate features, business metrics and even the target.
- Sample / Table / Feature / Feature Script classes — event dataset, source table, aggregate definition, executor.
- Strict-date join — pulls only history before the event date.
- Airflow — runs feature blocks as separate tasks on a schedule.
- Hadoop (HDFS, parquet) and Vertica — Vertica holds the data; raw tables are copied to Hadoop and quality-checked there.
Google or a one-time email code. No password.
