1. bookVolume 12 (2022): Issue 1 (January 2022)
Journal Details
License
Format
Journal
First Published
30 Dec 2014
Publication timeframe
4 times per year
Languages
English
access type Open Access

Anomaly Pattern Detection in Streaming Data Based on the Transformation to Multiple Binary-Valued Data Streams

Published Online: 08 Oct 2021
Page range: 19 - 27
Received: 10 Jul 2020
Accepted: 06 Oct 2020
Journal Details
License
Format
Journal
First Published
30 Dec 2014
Publication timeframe
4 times per year
Languages
English
Abstract

Anomaly pattern detection in a data stream aims to detect a time point where outliers begin to occur abnormally. Recently, a method for anomaly pattern detection has been proposed based on binary classification for outliers and statistical tests in the data stream of binary labels of normal or an outlier. It showed that an anomaly pattern can be detected accurately even when outlier detection performance is relatively low. However, since the anomaly pattern detection method is based on the binary classification for outliers, most well-known outlier detection methods, with the output of real-valued outlier scores, can not be used directly. In this paper, we propose an anomaly pattern detection method in a data stream using the transformation to multiple binary-valued data streams from real-valued outlier scores. By using three outlier detection methods, Isolation Forest(IF), Autoencoder-based outlier detection, and Local outlier factor(LOF), the proposed anomaly pattern detection method is tested using artificial and real data sets. The experimental results show that anomaly pattern detection using Isolation Forest gives the best performance.

Keywords

[1] D. Hawkins. Identification of outliers. Springer Netherlands, 1980. Search in Google Scholar

[2] C.H. Park. Outlier and anomaly pattern detection on data streams. The Journal of Supercomputing, 75:6118–6128, 2019. Search in Google Scholar

[3] T. Kim and C.H. Park. Anomaly pattern detection for streaming data. Expert Systems with Applications, 149, 2020. Search in Google Scholar

[4] F. Liu, K. Ting, and Z. Zhou. Isolation forest. In Proceedings of the 8th International Conference on Data Mining, 2008. Search in Google Scholar

[5] Q. Feng, Y. Zhang, C. Li, Z. Dou, and J. Wang. Anomaly detection of spectrum in wireless communication via deep auto-encoders. The Journal of Supercomputing, 73(7):3161–3178, 2017. Search in Google Scholar

[6] P. Remy. Anomaly detection in time setries using auto encoders. bolg positng from http://philipperemy.github.io/anomaly-detection. Search in Google Scholar

[7] D. Pokrajac, A. Lazarevic, and L.J. Latecki. Incremental local outlier detection for data streams. In Proceedings of the CIDM, 2007. Search in Google Scholar

[8] C. Aggarwal. Outlier analysis. Springer, 2017. Search in Google Scholar

[9] D. Padilla, R. Brinkworth, and M. McDonnell. Performance of a hierarchical temporal memory network in noisy sequence learning. In Proceedings of IEEE international conference on computational intelligence and cybernetics, 2013. Search in Google Scholar

[10] S. Ahmad and S. Purdy. Real-time anomaly detection for streaming analytics, 2016. Retrieved from https://arxiv.org/pdf/1607.02480.pdf. Search in Google Scholar

[11] W. Wong, A. Moore, G. Cooper, and M. Wagner. Rule-based anomaly pattern detection for detecting disease outbreaks. In Proceedings of the 18th International Conference on Artificial Intelligence, 2002. Search in Google Scholar

[12] K. Das, J. Schneider, and D. Neil. Anomaly pattern detection in categorical datasets. In Proceedings of KDD, 2008. Search in Google Scholar

[13] F. et al. Pedregosa. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12:2825–2830, 2011. Search in Google Scholar

[14] M.M. Breunig, H-P. Kriegel, R.T. Ng, and J. Sander. Lof: Identifying density-based local outliers. In Proceedings of the 2000 ACM Sigmod International Conference on Management of Data, 2000. Search in Google Scholar

[15] P. Tan, M. Steinbach, and V. Kumar. Introduction to data mining. Addison Wesley, Boston, 2006. Search in Google Scholar

[16] S. Hawkins, H. Hongxing, G. Williams, and R. Baxter. Outlier detection using replicator neural networks. In Proceedings of the International Conference on Data Warehousing and Knowledge Discovery, 2002. Search in Google Scholar

[17] A. Bife, G. Holmes, R. Kirkby, and B. Pfahringer. Moa: Massive online analysis. Journal of Machine Learning Research, 11:1601–1604, 2010. Search in Google Scholar

[18] Y. Zhao, Z. Nasrullah, and Z. Li. Pyod: A python toolbox for scalable outlier detection. Journal of Machine Learning Research, 20(96):1–7, 2019. Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo