Problem
Netflix runs on TVs and set-top boxes, which have much less memory than phones or laptops. Running out of memory is a common reason for the app to be killed. If the app could predict a kill in advance, it could lower its own performance on that device (for example, clear caches) and avoid the crash. The hard part is the dataset: device specifications and runtime memory readings live in different sources, the runtime logs are huge, and kills are very rare, so almost every row is a normal state.
Example from the article: in the sample dataset, more than 99.1% of rows are not related to a kill, and only 0.9% are.
Solution in brief
Build one table of memory readings joined to device capabilities. Label each reading by how close it is to a later kill in the same device and session, using a sliding window, or graded levels for a multi-class model. Then any standard classifier can be trained on it.
Questions this case answers
- How to formulate prediction of a rare failure event from periodic telemetry logs as a labeled classification dataset
- How to join static entity attributes with high-volume runtime logs without exhausting the query engine's memory
- How to turn rare app crashes on memory-limited devices into a supervised prediction task?
- How to label telemetry readings that precede a crash when the crash itself cannot be a model input?
Tools and methods
- Device capability table — static features per (device type ID, SDK version), cleaned by majority vote.
- Sliding-window labeling — marks readings shortly before a kill as errors, with graded levels by time to kill.
- SparkSQL-like engine — joins the logs; filter before the join and tune driver and executor memory.
- Classifiers (ANN, XGBoost, AdaBoost, ElasticNet with softmax) — named as suitable models; no results are given.
Google or a one-time email code. No password.