Problem
New features can lift a production model, but a feature store holds hundreds to thousands of candidates, many near-copies of features the model already uses. Testing them one by one is slow and misses interactions. Adding everything at once raises serving cost and latency, makes the model more fragile to feature drift and upstream pipeline outages, and makes SHAP or other importance scores unreliable for pruning, because they depend on one trained model and ignore redundancy.
Example from the article: a business-critical binary classifier at Uber with 75 features had to be improved from a pool of more than 2,000 extra features in the feature store.
Solution in brief
Rank all candidate features with a greedy information-theoretic method, maximum relevance and minimum redundance (MRMR), which does not depend on any trained model. Review the ranking for leakage and compliance, then train models on the top features for several values of and keep the best on validation.
Questions this case answers
- How to select a compact, non-redundant feature subset for a supervised model from a pool of thousands of candidates when trying them one by one is too slow
- How to rank candidate features by new information they add, independent of any one trained model
- How to find which of thousands of candidate features actually improve a production model?
- How to cut the feature count of a model without losing accuracy?
Tools and methods
- MRMR (maximum relevance, minimum redundancy) — greedy ranking of features by mutual information with the target, penalized by overlap with already chosen features
- Mutual information — measures both relevance (feature to target) and redundance (feature to feature)
- Apache Spark — the distributed engine under Uber's X-Ray feature selection tool
- Random forest — the baseline model in the case study
Google or a one-time email code. No password.