Revisiting Streaming Anomaly Detection: Benchmark and Evaluation

Anomaly detection in streaming data is a crucial task for many real-world applications, such as network security, fraud detection, and system monitoring. However, streaming data often exhibit concept drift, which means that the data distribution changes over time. This poses a significant challenge for anomaly detection algorithms, as they need to adapt to the evolving data to maintain high detection accuracy. Existing streaming anomaly detection algorithms lack a unified evaluation framework that can assess their performance and robustness under different types of concept drift and anomaly. In this paper, we conduct a systematic technical review of the state-of-the-art methods for anomaly detection in streaming data. We propose a new data generator, called SCAR (Streaming data generator with Customizable Anomalies and concept dRifts), that can synthesize streaming data based on synthetic and real-world datasets from different domains. Furthermore, we adapt four static anomaly detection models to the streaming setting using a generic reconstruction strategy as baselines, and then compare them systematically with 9 existing streaming anomaly detection algorithms on 76 synthesized datasets that exhibit various types of anomalies and concepts. The challenges and future research directions for anomaly detection in streaming data are also presented. All the codes and datasets are publicly available at https://github.com/yixiaoma666/ SCAR.