Formulating ‘Out of Memory Kill’ Prediction on the Netflix App as a Machine Learning Problem

Aryan Mehra, Farnaz Karimdady Sharifabad, Prasanna Vijayanathan, Chaïna Wade, Vishal Sharma, Mike Schassberger22 Jul 2022

Problem

Netflix runs on TVs and set-top boxes, which have much less memory than phones or laptops. Running out of memory is a common reason for the app to be killed. If the app could predict a kill in advance, it could lower its own performance on that device (for example, clear caches) and avoid the crash. The hard part is the dataset: device specifications and runtime memory readings live in different sources, the runtime logs are huge, and kills are very rare, so almost every row is a normal state.

Example from the article: in the sample dataset, more than 99.1% of rows are not related to a kill, and only 0.9% are.

Solution in brief

Build one table of memory readings joined to device capabilities. Label each reading by how close it is to a later kill in the same device and session, using a sliding window, or graded levels for a multi-class model. Then any standard classifier can be trained on it.

Questions this case answers
  • How to formulate prediction of a rare failure event from periodic telemetry logs as a labeled classification dataset
  • How to join static entity attributes with high-volume runtime logs without exhausting the query engine's memory
  • How to turn rare app crashes on memory-limited devices into a supervised prediction task?
  • How to label telemetry readings that precede a crash when the crash itself cannot be a model input?

Tools and methods

  • Device capability table — static features per (device type ID, SDK version), cleaned by majority vote.
  • Sliding-window labeling — marks readings shortly before a kill as errors, with graded levels by time to kill.
  • SparkSQL-like engine — joins the logs; filter before the join and tune driver and executor memory.
  • Classifiers (ANN, XGBoost, AdaBoost, ElasticNet with softmax) — named as suitable models; no results are given.
Continue reading — it's free

Google or a one-time email code. No password.