Using a recommender system to find anomalies in operating system process launches

Igor Pestretsov29 Oct 2022

Problem

A SIEM (Security Information and Event Management) system collects tens of billions of events, and every alert lands on an operator. Only hundredths of a percent of alerts deserve attention, and a couple of dozen false positives per day is the limit. The task is to flag process launches atypical for a user from the bare fact "user X launched process Y", because job titles and software catalogs are rarely kept in order. The model must be light enough for hundreds of thousands of events per second.

Example from the article: an accountant who works in Word, Excel and 1C suddenly launches a tool for viewing the Active Directory domain structure or vladimir; a programmer who moves from VS Code to PyCharm must not be flagged.

Solution in brief

Treat users as consumers and processes as items, and train a collaborative filtering recommender on who launches what. A process that scores as a poor fit for the user is anomalous. The per-user scores form a time series that is smoothed with an aggregation function, and anomalies are searched in the smoothed series to keep false positives low.

Questions this case answers
  • How to detect actions that are atypical for an entity from (entity, action) logs alone, when no entity or action metadata is reliable and normal behavior differs between groups of entities
  • How to detect process launches that are atypical for a user without job titles or software catalogs?
  • How to flag a hijacked account without alerting every time an employee changes tools?
  • How to keep a behavior model light enough for a stream of hundreds of thousands of events per second?

Tools and methods

  • Collaborative filtering — scores how typical a process is for a user.
  • User embeddings — place similar users close together, replacing job titles.
  • Moving average over scores — only sustained bursts raise an alert.
  • Windows event 4688 and Sysmon — sources of the launch log.
Continue reading — it's free

Google or a one-time email code. No password.