Leveraging Tabular Transformers for AdvancedDetection of Data Exfiltration in DNS Traffic

Abstract Recent advancements in DNS protocols like DNS over HTTPS (DoH), DNS over TLS (DoT), and DNS over QUIC (DoQ) have enabled secure communications for enterprise networks through encrypted connections. While DoH supports secure communication on multiple platforms, malicious implementations can pose significant security risks, including evasion of monitoring, malware communication, and data exfiltration. This study aims to address the security challenges posed by malicious implementations of DNS over HTTPS (DoH) by developing a robust classification model that can differentiate between benign and malicious DoH traffic. We propose a novel model based on the TabTransformer architecture, utilizing self-attention mechanisms. This model transforms network capture features into latent representations, allowing for the effective categorization of DoH traffic. The model is specifically designed to enhance the detection of DNS data exfiltration attacks, particularly those arising from misconfigurations in DNS servers. The performance of the proposed TabTransformer-based attention model is evaluated using the BCCC-CIC-Bell-DNS-2024 dataset. Results demonstrate a significant improvement in the accuracy of classifying DoH traffic as malicious or benign, highlighting the efficacy of embedding generation and attention techniques in enhancing detection capabilities. Our findings show that using the TabTransformer model can significantly improve the monitoring and classification of malicious DoH traffic, reducing security threats in enterprise networks.