Relational network-service clustering analysis with set evidences
Authors: 
Li Pu, Boi Faltings, 
Qiang Yang, Derek Hao HuAuthors Info & Claims
Li Pu, Boi Faltings, Qiang Yang, Derek Hao HuAuthors Info & ClaimsPages 35 - 44
Abstract
Network administrators are faced with a large amount of network data that they need to sift through to analyze user behaviors and detect anomalies. Through a network monitoring tool, we obtained TCP and UDP connection records together with additional information of the associated users and software in an enterprise network. Instead of using traditional payload inspection techniques, we propose a method that clusters such network traffic data by using relations between entities so that it can be analyzed for frequent behaviors and anomalies. Relational methods like Markov Logic Networks is able to avoid the feature extraction stage and directly handle multi-relation situations. We extend the common pairwise representation in relational models by adopting set evidence to build a better objective for the network service clustering problem. The automatic clustering process helps the administrator filter out normal traffic in shorter time and get an abstract overview of opening transport layer ports in the whole network, which is beneficial for assessing network security risks. Experimental results on synthetic and real datasets suggest that our method is able to discover underlying services and anomalies (malware or abused ports) with good interpretations.
References
[1]
}}I. Bhattacharya and L. Getoor. Collective entity resolution in relational data. ACM Transactions on Knowledge Discovery from Data, 1(1):5, 2007.
[2]
}}V. Blondel, J. Guillaume, R. Lambiotte, and E. Lefebvre. Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 2008.
[3]
}}U. Brandes, D. Delling, M. Gaertler, et al. On modularity clustering. IEEE Transactions on Knowledge and Data Engineering, 20(2):172--188, 2007.
[4]
}}V. Chandola, A. Banerjee, and V. Kumar. Anomaly detection: A survey. ACM Computing Surveys (CSUR), 41(3):15, 2009.
[5]
}}M. Crotti, M. Dusi, F. Gringoli, and L. Salgarelli. Traffic classification through simple statistical fingerprinting. ACM SIGCOMM Computer Communication Review, 37(1):16, 2007.
[6]
}}N. Duffield, P. Haffner, B. Krishnamurthy, and H. Ringberg. Rule-based anomaly detection on IP flows. In IEEE INFOCOM, 2009.
[7]
}}J. Erman, M. Arlitt, and A. Mahanti. Traffic classification using clustering algorithms. In Proceedings of the 2006 SIGCOMM Workshop on Mining Network Data, 2006.
[8]
}}S. Fortunato. Community detection in graphs. Physics Reports, 2009.
[9]
}}L. Getoor and B. Taskar. Introduction to statistical relational learning. The MIT Press, 2007.
[10]
}}J. Gómez, C. Gil, N. Padilla, R. Baños, and C. Jim&3233;nez. Design of a Snort-Based Hybrid Intrusion Detection System. Distributed Computing, Artificial Intelligence, Bioinformatics, Soft Computing, and Ambient Assisted Living, pages 515--522, 2009.
[11]
}}J. Homer and X. Ou. SAT-solving approaches to context-aware enterprise network security management. IEEE JSAC Special Issue on Network Infrastructure Configuration, 2009.
[12]
}}S. Kandula, R. Mahajan, P. Verkaik, S. Agarwal, J. Padhye, and P. Bahl. Detailed diagnosis in enterprise networks. In Proceedings of the 2009 conference on ACM SIGCOMM 2009 conference, 2009.
[13]
}}T. Karagiannis, K. Papagiannaki, and M. Faloutsos. BLINC: multilevel traffic classification in the dark. ACM SIGCOMM Computer Communication Review, 35(4):240, 2005.
[14]
}}C. Kemp, J. Tenenbaum, T. Griffiths, T. Yamada, and N. Ueda. Learning systems of concepts with an infinite relational model. In Proceedings of the National Conference on Artificial Intelligence, 2006.
[15]
}}H. Kim, K. Claffy, M. Fomenkov, D. Barman, M. Faloutsos, and K. Lee. Internet traffic classification demystified: myths, caveats, and the best practices. In Proceedings of the 2008 ACM CoNEXT conference, 2008.
[16]
}}S. Kok and P. Domingos. Statistical predicate invention. In Proceedings of the 24th international conference on Machine learning, 2007.
[17]
}}S. Kok, P. Singla, M. Richardson, P. Domingos, M. Sumner, H. Poon, and D. Lowd. The Alchemy system for statistical relational AI. Dept. of Computer Science and Engineering, Univ. of Washington, Technical Report. http://www.cs.washington.edu/ai/alchemy, 2007.
[18]
}}B. Long, Z. Zhang, and P. Yu. A probabilistic framework for relational clustering. In Proceedings of the 13th ACM SIGKDD, 2007.
[19]
}}A. Moore and K. Papagiannaki. Toward the accurate identification of network applications. Passive and Active Network Measurement, pages 41--54, 2005.
[20]
}}A. Moore and D. Zuev. Internet traffic classification using bayesian analysis techniques. In Proceedings of the 2005 ACM SIGMETRICS, 2005.
[21]
}}M. Newman. Modularity and community structure in networks. Proceedings of the National Academy of Sciences, 103(23), 2006.
[22]
}}M. Newman and M. Girvan. Finding and evaluating community structure in networks. Physical review E, 69(2), 2004.
[23]
}}H. Poon and P. Domingos. Joint unsupervised coreference resolution with Markov Logic. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2008.
[24]
}}H. Poon, P. Domingos, and M. Sumner. A general method for reducing the complexity of relational inference and its application to MCMC. In Proceedings of the National Conference on Artificial Intelligence, 2008.
[25]
}}M. Richardson and P. Domingos. Markov logic networks. Machine Learning, 62(1):107--136, 2006.
[26]
}}M. Thottan, G. Liu, and C. Ji. Anomaly detection approaches for communication networks. Algorithms for Next Generation Networks, pages 239--261, 2010.
[27]
}}N. Williams, S. Zander, and G. Armitage. A preliminary performance comparison of five machine learning algorithms for practical IP traffic flow classification. ACM SIGCOMM Computer Communication Review, 36(5):16, 2006.
[28]
}}T. Yang, R. Jin, Y. Chi, and S. Zhu. Combining link and content for community detection: a discriminative approach. In Proceedings of the 15th ACM SIGKDD, 2009.
Index Terms
- Relational network-service clustering analysis with set evidences
Recommendations
Comparing relational and non-relational algorithms for clustering propositional data
SAC '13: Proceedings of the 28th Annual ACM Symposium on Applied ComputingCluster detection methods are widely studied in Propositional Data Mining. In this context, data is individually represented as a feature vector. This data has a natural non-relational structure, but can be represented in a relational form through ...
Efficient Disk-Based K-Means Clustering for Relational Databases
K-means is one of the most popular clustering algorithms. This article introduces an efficient disk-based implementation of K-means. The proposed algorithm is designed to work inside a relational database management system. It can cluster large data ...
TCP Congestion Control Approach for Improving Network Services
With the continuous increasing demand of Internet applications, networks are experiencing a serious congestion problem. This affects directly the networks' services and management due to large amounts of data loss and long transmission delays. The ...
Comments
Information & Contributors
Information
Published In
Copyright © 2010 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 October 2010
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
CCS '10
Sponsor:
CCS '10: 17th ACM Conference on Computer and Communications Security 2010
October 8, 2010
Illinois, Chicago, USA
Acceptance Rates
AISec '10 Paper Acceptance Rate 10 of 15 submissions, 67%;
Overall Acceptance Rate 94 of 231 submissions, 41%
Upcoming Conference
CCS '25
- Sponsor:
- sigsac
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 240Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 18 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in