COMPARISON OF MACHINE LEARNING MODELS FOR ACTIVE DIRECTORY ATTACK DETECTION USING WINDOWS SECURITY EVENT LOGS

Abstract

Windows Active Directory (AD) is central to nearly every enterprise domain for authentication/authorization purposes. As such, it is commonly a targeted threat vector as bad actors will use the credentials for AD accounts to perform lateral movement or privilege escalation. Windows Security Event Logs contain large amounts of useful structured information for threat detection but nearly all rules based methodologies produce too many false positives, do not generalize well, and fail to detect complex behaviors across many different features.

We compare five types of supervised machine learning models to detect malicious usage of AD-related events. We tested Logistic Regression, a linear Support Vector Machine (SVM) model, Random Forest, Gradient Boosting, and XGBoost. Each model used the same training methodology with preprocessing of the data to maintain accuracy and consistency between results.

Our results also show that tree ensemble models have better performance than linear models on our collected dataset and achieve the highest scores in ROC-AUC as well as PR-AUC. We also cover topics including result interpretability, model deployment, and security operation center operating constraints.