Hunter in the Dark: Discover Anomalous Network Activity Using Deep Ensemble Network
This addresses alert fatigue in cybersecurity for organizations like financial institutes and government agencies, representing an incremental improvement over existing methods.
The paper tackles the problem of high false alarm rates in machine learning-based intrusion detection systems, proposing DarkHunter, a deep ensemble network that combines supervised and unsupervised learning to detect anomalous network activity and reduce mis-detections, achieving high detection accuracy with a low false positive rate on the UNSW-NB15 dataset.
Machine learning (ML)-based intrusion detection systems (IDSs) play a critical role in discovering unknown threats in a large-scale cyberspace. They have been adopted as a mainstream hunting method in many organizations, such as financial institutes, manufacturing companies and government agencies. However, existing designs achieve a high threat detection performance at the cost of a large number of false alarms, leading to alert fatigue. To tackle this issue, in this paper, we propose a neural-network-based defense mechanism named DarkHunter. DarkHunter incorporates both supervised learning and unsupervised learning in the design. It uses a deep ensemble network (trained through supervised learning) to detect anomalous network activities and exploits an unsupervised learning-based scheme to trim off mis-detection results. For each detected threat, DarkHunter can trace to its source and present the threat in its original traffic format. Our evaluations, based on the UNSW-NB15 dataset, show that DarkHunter outperforms the existing ML-based IDSs and is able to achieve a high detection accuracy while keeping a low false positive rate.