This paper describes EMBER: a labeled benchmark dataset for training machine
learning models to statically detect malicious Windows portable executable
files. The dataset includes features extracted from 1.1M binary files: 900K
training samples (300K malicious, 300K benign, 300K unlabeled) and 200K test
samples (100K malicious, 100K benign). To accompany the dataset, we also
release open source code for extracting features from additional binaries so
that additional sample features can be appended to the dataset. This dataset
fills a void in the information security machine learning community: a
benign/malicious dataset that is large, open and general enough to cover
several interesting use cases. We enumerate several use cases that we
considered when structuring the dataset. Additionally, we demonstrate one use
case wherein we compare a baseline gradient boosted decision tree model trained
using LightGBM with default settings to MalConv, a recently published
end-to-end (featureless) deep learning model for malware detection. Results
show that even without hyper-parameter optimization, the baseline EMBER model
outperforms MalConv. The authors hope that the dataset, code and baseline model
provided by EMBER will help invigorate machine learning research for malware
detection, in much the same way that benchmark datasets have advanced computer
vision research.