Machine Learning Approaches for Intelligent Anomaly Detection

Abstract

Anomaly detection identifies objects or events that do not behave as expected or correlate with other data points. Anomaly detection has been used to identify and investigate abnormal data components. Anomaly detection in network traffic is a critical task for identifying malicious activities and ensuring the security of computer networks. Machine learning techniques, particularly deep learning has enabled tremendous advancements in the area of anomaly detection. This project aims to design and evaluate an intelligent intrusion detection system (IDS) capable of detecting both known and dynamic cyber-attacks with higher accuracy. The implementation uses a combined KDD and NSL-KDD dataset, which contains network traffic records described by 41 request signature features and labelled as either normal or attack traffic. The dataset includes nearly 30 different attack types, which are further categorized into five major classes: Normal, DoS, Probe, R2L, and U2R. During pre-processing, categorical features and attack labels are converted into numeric values to make the data suitable for machine learning algorithms. Existing models such as Support Vector Machine (SVM) and Random Forest are implemented to evaluate classical detection performance. However, these models show limited accuracy when handling dynamic or previously unseen attack patterns. To overcome this limitation, a Deep Neural Network (DNN) model with an optimized hidden layer structure is proposed. The DNN dynamically learns complex patterns from the data and achieves superior detection accuracy compared to traditional approaches. Additionally, Extreme Learning Machine (ELM) is also evaluated for performance comparison. Experimental results demonstrate that the proposed DNN-based model provides improved accuracy and robustness, making it more effective for modern network intrusion detection systems.