Data Generation for Information Security Tasks

Alsu Nurutdinova, Alexey Zhurin28 Oct 2023

Problem

The ML team at Positive Technologies, an information security vendor, needs a lot of data to train detection models. Security data is confidential, the law restricts distributing malicious samples, the illegitimate class is much smaller than the legitimate one, and the data loses representativeness quickly. Collecting more real data is expensive. The team also wants to publish open datasets for competitions and teaching.

Example from the article: binary classification of files run in a sandbox, with vectorized system-call traces of more than 5500 numeric features and very few malicious files.

Solution in brief

Generate synthetic data with off-the-shelf tabular and time-series generators (SDV, Gretel), choosing the algorithm per dataset. Judge quality three ways: a statistical score, the production model's results on synthetic versus real data, and a classifier trained to tell real from synthetic.

Questions this case answers
  • How to generate synthetic tabular and time-series data when real data is confidential and the malicious class is very rare
  • How to validate synthetic data by checking whether a production model trained on it behaves like one trained on real data
  • How to generate synthetic tabular data when real data is confidential and the malicious class is rare
  • How to generate synthetic time series and event sequences and check that they are realistic

Tools and methods

  • CTGAN, CopulaGAN, BayesianModelSampling (SDV) — tabular generation.
  • Gretel synthetic batch — tabular generation for many columns.
  • SDV Probabilistic Auto-Regressive (PAR), Gretel DoppelGANger — time series and sequences.
  • Overall Quality Score, adversarial validation, production-algorithm check — quality assessment.
Continue reading — it's free

Google or a one-time email code. No password.