Network Intrusion Detection System Datasets

evaluates standard Network Intrusion Detection Systems (NIDSs) feature sets based on the NetFlow network meta-data collection protocol and system.

Datasets

Where the UNSW-NB15 and CSE-CIC-IDS2018 datasets have very high benign-toattack ratios, whereas the ToN-IoT and BoT-IoT datasets are mainly made up of attack samples, which do not represent a realistic network behaviour. some of the features in the UNSW-NB15, BoT-IoT, and CSE-CIC-IDS2018 datasets are handcrafted features that are not originally found in network packets but are statistically calculated based on other features, such as the total number of bytes transferred over the last 100 seconds.

• UNSW-NB15
• BoT-IoT
• ToN-IoT
• CICIDS2017 dataset


NetFlow features

If a data flow is located in the attack events it would be labelled as an attack (class 1) in the binary label and its respective attack’s type would be recorded in the attack label, otherwise, the sample is labelled as a benign flow (class 0).

Data Preprocessing

As part of the data pre-processing, the flow identifiers such as IDs, source/destination IP and ports, timestamps, and start/end time are dropped to avoid learning bias towards attacking and victim end nodes. For the UNSW-NB15 and NF-UNSW-NB15-v2 datasets, The Time To Live (TTL)-based features are dropped due to their extreme correlation with the labels. Additionally, the minmax normalisation technique has been applied to scale all datasets’ values between 0 and the datasets have been split into 70%-30% for training and testing purposes.

Binary-class classification

Extra Tree classifier to compare the predictive power of our proposed NetFlow based feature set, with the proprietary features sets provided with the original benchmark NIDS datasets. Extremely Randomized Trees Classifier(Extra Trees Classifier)

Multi-class classification

Reference List

1. https://link.springer.com/article/10.1007/s11036-021-01843-0
2. https://staff.itee.uq.edu.au/marius/NIDS_datasets/
3. Sarhan, M., Layeghy, S., & Portmann, M. (2022). Towards a standard feature set for network intrusion detection system datasets. Mobile networks and applications, 1-14.