research-article

Generative Evolutionary Anomaly Detection in Dynamic Networks

Authors:

Huaming WuAuthors Info & Claims

IEEE Transactions on Knowledge and Data Engineering, Volume 35, Issue 12

Pages 12234 - 12248

https://doi.org/10.1109/TKDE.2021.3129057

Published: 22 November 2021 Publication History

Abstract

Anomaly detection in dynamic networks aims to find network elements (e.g., nodes, edges, subgraphs, change points) with significantly different behaviors from the vast majority, it can also devote to community detection and evolution and prediction tasks. Most existing methods focus on one specific task, that is, only detect anomalies of one type of element isolated, so they lose the ability to model the correlation and driving mechanism between different abnormal behavior. Considering that the anomaly detection of one type of element is helpful to other types of elements, i.e., the temporal evolution hidden the dynamic networks are driven by indivisible behavior patterns. So in this paper, we propose a unified Generation model to analyze the dynamic network for Exploring the Abnormal Behaviors of different Scales (GEABS). It can model the relation and catch different levels (node, community and network) of anomaly with a joint statistical network model and detect the community structure and its evolution. Specifically, we denote the parameters of node popularity, community membership to generate the dynamic network with stochastic block model (SBM), we also describe the varying of node and community by dynamic process. With a well-designed generative mechanism, it can detect the change point on network level, temporal evolution on community level and abnormal behavior on node level synchronously, besides, it also detects the community structure effectively. We also propose an effective optimization algorithm with variational inference. Experimental results show that the GEABS achieves better performance on abnormal behavior and community structure compared with baselines.

References

[1]

V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: A survey,” ACM Comput. Surv., vol. 41, no. 3, pp. 1–58, 2009.

Digital Library

[2]

H. Zenati, M. Romain, C.-S. Foo, B. Lecouat, and V. Chandrasekhar, “Adversarially learned anomaly detection,” in Proc. IEEE Int. Conf. Data Mining, 2018, pp. 727–736.

[3]

M. Ahmed, A. N. Mahmood, and M. R. Islam, “A survey of anomaly detection techniques in financial domain,” Future Gener. Comput. Syst., vol. 55, pp. 278–288, Feb. 2016.

Digital Library

[4]

S. Amraee, A. Vafaei, K. Jamshidi, and P. Adibi, “Anomaly detection and localization in crowded scenes using connected component analysis,” Multimedia Tools Appl., vol. 77, no. 12, pp. 14 767–14 782, 2018.

[5]

H. Kasai, W. Kellerer, and M. Kleinsteuber, “Network volume anomaly detection and identification in large-scale networks based on online time-structured traffic tensor tracking,” IEEE Trans. Netw. Service Manag., vol. 13, no. 3, pp. 636–650, Sep. 2016.

Digital Library

[6]

Q. Zhou, L. Li, and H. Tong, “Towards real time team optimization,” in Proc. IEEE Int. Conf. Big Data, 2019, pp. 1008–1017.

[7]

Y. Lv, Y. Chen, X. Zhang, Y. Duan, and N. L. Li, “Social media based transportation research: The state of the work and the networking,” IEEE/CAA J. Automatica Sinica, vol. 4, no. 1, pp. 19–26, Jan. 2017.

[8]

M. Ley, “The DBLP computer science bibliography: Evolution, research issues, perspectives,” in Proc. Int. Symp. String Process. Inf. Retrieval, 2002, pp. 1–10.

[9]

K. Ding, J. Li, and H. Liu, “Interactive anomaly detection on attributed networks,” in Proc. 12th ACM Int. Conf. Web Search Data Mining, 2019, pp. 357–365.

[10]

K. Ding, J. Li, N. Agarwal, and H. Liu, “Inductive anomaly detection on attributed networks,” in Proc. 29th Int. Joint Conf. Artif. Intell., 2020, pp. 1288–1294.

[11]

S. Ranshous, S. Shen, D. Koutra, S. Harenberg, C. Faloutsos, and N. F. Samatova, “Anomaly detection in dynamic networks: A survey,” Wiley Interdisciplinary Rev., Comput. Statist., vol. 7, no. 3, pp. 223–247, 2015.

Digital Library

[12]

H. Huang, J. Tang, L. Liu, J. Luo, and X. Fu, “Triadic closure pattern analysis and prediction in social networks,” IEEE Trans. Knowl. Data Eng., vol. 27, no. 12, pp. 3374–3389, Dec. 2015.

Digital Library

[13]

R. Hassanzadeh and R. Nayak, “A semi-supervised graph-based algorithm for detecting outliers in online-social-networks,” in Proc. 28th Annu. ACM Symp. Appl. Comput., 2013, pp. 577–582.

[14]

N. A. Heard, D. J. Weston, K. Platanioti, and D. J. Hand, “Bayesian anomaly detection methods for social networks,” Ann. Appl. Statist., vol. 4, pp. 645–662, 2010.

[15]

L. Xiao, Y. Zhang, Z. Hu, and J. Dai, “Performance benefits of robust nonlinear zeroing neural network for finding accurate solution of Lyapunov equation in presence of various noises,” IEEE Trans. Ind. Informat., vol. 15, no. 9, pp. 5161–5171, Sep. 2019.

[16]

L. Xiao, B. Liao, S. Li, Z. Zhang, L. Ding, and L. Jin, “Design and analysis of FTZNN applied to the real-time solution of a nonstationary Lyapunov equation and tracking control of a wheeled mobile manipulator,” IEEE Trans. Ind. Informat., vol. 14, no. 1, pp. 98–105, Jan. 2018.

[17]

L. Xiao et al., “A new noise-tolerant and predefined-time ZNN model for time-dependent matrix inversion,” Neural Netw., vol. 117, pp. 124–134, 2019.

Digital Library

[18]

Y. Hulovatyy and T. Milenković, “SCOUT: Simultaneous time segmentation and community detection in dynamic networks,” Sci. Rep., vol. 6, 2016, Art. no.

[19]

R. C. Y. Cheung, A. Aue, S. Hwang, and T. C. M. Lee, “Simultaneous detection of multiple change points and community structures in time series of networks,” IEEE Trans. Signal Inf. Process. Netw., vol. 6, pp. 580–591, Aug. 2020.

[20]

H. Wang and C. Qiao, “A nodes’ evolution diversity inspired method to detect anomalies in dynamic social networks,” IEEE Trans. Knowl. Data Eng., vol. 32, no. 10, pp. 1868–1880, Oct. 2020.

[21]

H. Cheng, P.-N. Tan, C. Potter, and S. Klooster, “A robust graph-based algorithm for detection and characterization of anomalies in noisy multivariate time series,” in Proc. IEEE Int. Conf. Data Mining Workshops, 2008, pp. 349–358.

[22]

R. K. Darst, C. Granell, A. Arenas, S. Gómez, J. Saramäki, and S. Fortunato, “Detection of timescales in evolving complex systems,” Sci. Rep., vol. 6, no. 1, pp. 1–8, 2016.

[23]

L. Zheng, Z. Li, J. Li, Z. Li, and J. Gao, “AddGraph: Anomaly detection in dynamic graph using attention-based temporal GCN,” in Proc. 28th Int. Joint Conf. Artif. Intell., 2019, pp. 4419–4425.

[24]

C. C. Aggarwal, Y. Zhao, and P. S. Yu, “Outlier detection in graph streams,” in Proc. IEEE 27th Int. Conf. Data Eng., 2011, pp. 399–409.

[25]

S. Ranshous, S. Harenberg, K. Sharma, and N. F. Samatova, “A scalable approach for outlier detection in edge streams using sketch-based approximations,” in Proc. SIAM Int. Conf. Data Mining, 2016, pp. 189–197.

[26]

J. Sun, Y. Xie, H. Zhang, and C. Faloutsos, “Less is more: Compact matrix decomposition for large sparse graphs,” in Proc. SIAM Int. Conf. Data Mining, 2007, pp. 366–377.

[27]

K. Shin, B. Hooi, J. Kim, and C. Faloutsos, “D-Cube: Dense-block detection in terabyte-scale tensors,” in Proc. 10th ACM Int. Conf. Web Search Data Mining, 2017, pp. 681–689.

[28]

K. Shin, B. Hooi, and C. Faloutsos, “M-Zoom: Fast dense-block detection in tensors with quality guarantees,” in Proc. Joint Eur. Conf. Mach. Learn. Knowl. Discov. Databases, 2016, pp. 264–280.

[29]

M. Mongiovi, P. Bogdanov, and A. K. Singh, “Mining evolving network processes,” in Proc. IEEE 13th Int. Conf. Data Mining, 2013, pp. 537–546.

[30]

B. A. Miller, N. Arcolano, and N. T. Bliss, “Efficient anomaly detection in dynamic, attributed graphs: Emerging phenomena and Big Data,” in Proc. IEEE Int. Conf. Intell. Secur. Informat., 2013, pp. 179–184.

[31]

D. Eswaran, C. Faloutsos, S. Guha, and N. Mishra, “SpotLight: Detecting anomalies in streaming graphs,” in Proc. 24th ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2018, pp. 1378–1386.

[32]

N. Du, X. Jia, J. Gao, V. Gopalakrishnan, and A. Zhang, “Tracking temporal community strength in dynamic networks,” IEEE Trans. Knowl. Data Eng., vol. 27, no. 11, pp. 3125–3137, Nov. 2015.

Digital Library

[33]

A. Karaaslanlı and S. Aviyente, “Community detection in dynamic networks: Equivalence between stochastic blockmodels and evolutionary spectral clustering,” IEEE Trans. Signal Inf. Process. Netw., vol. 7, pp. 130–143, Feb. 2021.

[34]

X. Han, Y. Zhao, and M. Small, “Identification of dynamical behavior of pseudoperiodic time series by network community structure,” IEEE Trans. Circuits Syst., II, Exp. Briefs, vol. 66, no. 11, pp. 1905–1909, Nov. 2019.

[35]

S. Y. Bhat and M. Abulaish, “HOCTracker: Tracking the evolution of hierarchical and overlapping communities in dynamic social networks,” IEEE Trans. Knowl. Data Eng., vol. 27, no. 4, pp. 1019–1013, Apr. 2015.

Digital Library

[36]

P. Bródka, S. Saganowski, and P. Kazienko, “GED: The method for group evolution discovery in social networks,” Soc. Netw. Anal. Mining, vol. 3, no. 1, pp. 1–14, 2013.

[37]

X. Ma and D. Dong, “Evolutionary nonnegative matrix factorization algorithms for community detection in dynamic networks,” IEEE Trans. Knowl. Data Eng., vol. 29, no. 5, pp. 1045–1058, May 2017.

Digital Library

[38]

B. Baingana and G. B. Giannakis, “Joint community and anomaly tracking in dynamic networks,” IEEE Trans. Signal Process., vol. 64, no. 8, pp. 2013–2025, Apr. 2016.

Digital Library

[39]

W. Yu, W. Cheng, C. C. Aggarwal, K. Zhang, H. Chen, and W. Wang, “NetWalk: A flexible deep embedding approach for anomaly detection in dynamic networks,” in Proc. 24th ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2018, pp. 2672–2681.

[40]

L. Peel and A. Clauset, “Detecting change points in the large-scale structure of evolving networks,” in Proc. AAAI Conf. Artif. Intell., 2015, pp. 2914–2920.

[41]

T. Zhu, P. Li, L. Yu, K. Chen, and Y. Chen, “Change point detection in dynamic networks based on community identification,” IEEE Trans. Netw. Sci. Eng., vol. 7, no. 3, pp. 2067–2077, Third Quarter 2020.

[42]

D. Koutra, J. T. Vogelstein, and C. Faloutsos, “DELTACON: A principled massive-graph similarity function,” in Proc. SIAM Int. Conf. Data Mining, 2013, pp. 162–170.

[43]

S. Huang, Y. Hitti, G. Rabusseau, and R. Rabbany, “Laplacian change point detection for dynamic graphs,” in Proc. 26th ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2020, pp. 349–358.

[44]

G. Schwarz et al., “Estimating the dimension of a model,” Ann. Statist., vol. 6, no. 2, pp. 461–464, 1978.

[45]

D. Greene, D. Doyle, and P. Cunningham, “Tracking the evolution of communities in dynamic social networks,” in Proc. Int. Conf. Advances Soc. Netw. Anal. Mining, 2010, pp. 176–183.

[46]

R. Görke, M. Holzer, O. Hopp, J. Theuerkorn, and K. Scheibenberger, “Dynamic network of email communication at the department of informatics at Karlsruhe institute of technology,” 2011. Accessed: Dec. 19, 2021. [Online]. Available: www.iti.kit.edu/projects/spp1307/emaildata

[47]

G. J. Benston and A. L. Hartgraves, “Enron: What happened and what we can learn from it,” J. Accounting Public Policy, vol. 21, no. 2, pp. 105–127, 2002.

[48]

K. S. Gleditsch, “Expanded trade and GDP data,” 2002. Accessed: Dec. 19, 2021. [Online]. Available: http://ksgleditsch.com/exptradegdp.html

[49]

F. Liu, J. Wu, C. Zhou, and J. Yang, “Evolutionary community detection in dynamic social networks,” in Proc. Int. Joint Conf. Neural Netw., 2019, pp. 1–7.

[50]

F. Liu, J. Wu, S. Xue, C. Zhou, J. Yang, and Q. Sheng, “Detecting the evolving community structure in dynamic social networks,” World Wide Web, vol. 23, no. 2, pp. 715–733, 2020.

[51]

F. Liu, D. Choi, L. Xie, and K. Roeder, “Global spectral clustering in dynamic networks,” Proc. Nat. Acad. Sci. USA, vol. 115, no. 5, pp. 927–932, 2018.

[52]

T. Yang, Y. Chi, S. Zhu, Y. Gong, and R. Jin, “Detecting communities and their evolutions in dynamic social networks—A Bayesian approach,” Mach. Learn., vol. 82, no. 2, pp. 157–189, 2011.

Digital Library

[53]

Y.-R. Lin, Y. Chi, S. Zhu, H. Sundaram, and B. L. Tseng, “Analyzing communities and their evolutions in dynamic social networks,” ACM Trans. Knowl. Discov. Data, vol. 3, no. 2, pp. 1–31, 2009.

Digital Library

[54]

T. Hartmann, A. Kappes, and D. Wagner, “Clustering evolving networks,” in Algorithm Engineering. Berlin, Germany: Springer, 2016, pp. 280–329.

[55]

A. Rushton, P. Croucher, and P. Baker, The Handbook of Logistics and Distribution Management: Understanding the Supply Chain. London, U.K.: Kogan Page Publishers, 2014.

[56]

H. H. Karrar, “The resumption of Sino–central asian trade, c. 1983–94: Confidence building and reform along a cold war fault line,” Central Asian Surv., vol. 35, no. 3, pp. 334–350, 2016.

[57]

J. C. Cuaresma and M. Roser, “Borders redrawn: Measuring the statistical creation of international trade,” World Economy, vol. 35, no. 7, pp. 946–952, 2012.

[58]

S. Tuli et al., “HealthFog: An ensemble deep learning based smart healthcare system for automatic diagnosis of heart diseases in integrated IoT and fog computing environments,” Future Gener. Comput. Syst., vol. 104, pp. 187–200, 2020.

Digital Library

[59]

S. S. Gill et al., “Transformative effects of IoT, blockchain and artificial intelligence on cloud computing: Evolution, vision, trends and open challenges,” Internet Things, vol. 8, 2019, Art. no.

[60]

R. I. Ansari et al., “A new dimension to spectrum management in IoT empowered 5G networks,” IEEE Netw., vol. 33, no. 4, pp. 186–193, Jul./Aug. 2019.

Cited By

Zhang XJiao PGao MLi TWu YWu HZhao Z(2024)VGGM: Variational Graph Gaussian Mixture Model for Unsupervised Change Point Detection in Dynamic NetworksIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.337754819(4272-4284)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3377548
Jiao PChen HTang HBao QZhang LZhao ZWu H(2024)Contrastive representation learning on dynamic networksNeural Networks10.1016/j.neunet.2024.106240174:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106240

Index Terms

Generative Evolutionary Anomaly Detection in Dynamic Networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Anomaly detection in dynamic networks: a survey

Anomaly detection is an important problem with multiple applications, and thus has been studied for decades in various research domains. In the past decade there has been a growing interest in anomaly detection in data represented as networks, or graphs,...
Community-based anomaly detection in evolutionary networks

Networks of dynamic systems, including social networks, the World Wide Web, climate networks, and biological networks, can be highly clustered. Detecting clusters, or communities, in such dynamic networks is an emerging area of research; however, less ...
Two-stage anomaly detection algorithm via dynamic community evolution in temporal graph
Abstract
Detecting anomalies from a massive amount of user behavioral data is often liken to finding a needle in a haystack. While tremendous efforts have been devoted to anomaly detection from temporal graphs, existing studies rarely consider community ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Knowledge and Data Engineering

IEEE Transactions on Knowledge and Data Engineering Volume 35, Issue 12

Dec. 2023

1114 pages

ISSN:1041-4347

Issue’s Table of Contents

1041-4347 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Educational Activities Department

United States

Publication History

Published: 22 November 2021

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang XJiao PGao MLi TWu YWu HZhao Z(2024)VGGM: Variational Graph Gaussian Mixture Model for Unsupervised Change Point Detection in Dynamic NetworksIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.337754819(4272-4284)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3377548
Jiao PChen HTang HBao QZhang LZhao ZWu H(2024)Contrastive representation learning on dynamic networksNeural Networks10.1016/j.neunet.2024.106240174:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106240

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents