Abstract
Detecting the anomalous entity in real-time network traffic is a popular area of research in recent times. Very few researches have focused on creating malware that fools the intrusion detection system and this paper focuses on this topic. We are using Deep Convolutional Generative Adversarial Networks (DCGAN) to trick the malware classifier to believe it is a normal entity. In this work, a new dataset is created to fool the Artificial Intelligence (AI) based malware detectors, and it consists of different types of attacks such as Denial of Service (DoS), scan 11, scan 44, botnet, spam, User Datagram Portal (UDP) scan, and ssh scan. The discriminator used in the DCGAN discriminates two different attack classes (anomaly and synthetic) and one normal class. The model collapse, instability, and vanishing gradient issues associated with the DCGAN are overcome using the proposed hybrid Aquila optimizer-based Mine blast harmony search algorithm (AO-MBHS). This algorithm helps the generator to create realistic malware samples to be undetected by the discriminator. The performance of the proposed methodology is evaluated using different performance metrics such as training time, detection rate, F-Score, loss function, Accuracy, False alarm rate, etc. The superiority of the hybrid AO-MBHS based DCGAN model is noticed when the detection rate is changed to 0 after the retraining method to make the defensive technique hard to be noticed by the malware detection system. The support vector machines (SVM) is used as the malicious traffic detection application and its True positive rate (TPR) goes from 80% to 0% after retraining the proposed model which shows the efficiency of the proposed model in hiding the samples.