Abstract
Anomaly detection is based on profiles that represent normal behavior of users, hosts or networks and detects attacks as significant deviations from these profiles. Our methodology is based on the application of several data mining methods and returns an adaptive normal daily model of the network traffic as a result of four main steps, which are illustrated in the paper. The original observation units (the network connections) are transformed in symbolic objects and the normal model itself is given by a particular set of symbolic objects. A new symbolic object is considered an anomaly if it is dissimilar from those belonging to the model and it can be added to the model if it is ranked as a changing point, i.e. a new but legal behavior of the network traffic, otherwise it is an outlier, i.e. a new but illegal aspect of the network traffic. The obtained model of network connections can be used by a network administrator to identify deviations in network traffic patterns that may demand for her attention. The methodology is applied to the firewall logs of our Department network.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Ghoting, A., Otey, M.E., Parthasarathy, S.: Loaded: Link-based Outlier and Anomaly detection in Evolving Data Sets. In: Proceeedings of the IEEE International Conference on Data Mining, IEEE Computer Society Press, Los Alamitos (2004)
Takeuchi, J., Yamanashi, K.: A Unifying Framework for Identifying Changing Points and Outliers. IEEE Transactions on Knowledge and Data Engineering 18(4) (2006)
Wang, K., Stolfo, S.: Anomalous Payload-based Network Intrusion Detection. In: RAID (2004)
Knorr, N., Ng, P.: Algorithms for Mining Distance-Based Outliers in Large Datasets. In: VLDB (1998)
Breunig, et al.: LOF: Identifying Density-Based Local Outliers. In: KDD (2000)
Eskin, E., Arnold, A., Prerau, M., Portnoy, L., Stolfo, S.: A Geometric Framework for Unsupervised Anomaly Detection: Detecting Intrusions in Unlabeled Data (2002)
Yamanishi, K.: On-line unsupervised outlier detection using finite mixture with discounting learning algorithms. In: KDD (2000)
Mahoney, M., Chan, P.: Learning Nonstationary Models of Normal Network Traffic for Detecting Novel Attacks. In: 8th ACM KDD (2002)
Hofmeyr, S., et al.: Intrusion Detection using Sequences of System Calls (1997)
Tandon, G., Chan, P.: Learning Rules from System Call Arguments and Sequences for Anomaly Detection. In: Workshop on Data Mining for Computer Security. In: ICDM (2003)
Wang, K., Stolfo, S.: One Class Training for Masquerade Detection. In: Workshop on Data Mining for Computer Security. ICDM (2003)
Jain, A.K., Murty, M.N., Flyn, P.J.: Data Clustering: a Review. ACM Computing Surveys 31(3) (1999)
Witten, I., Frank, E.: Generate Accurate Rule Sets Without Global Optimisation. In: Machine Learning: Proceedings of the 15th International Conference, Morgan Kaufmann Publishers, San Francisco (1998)
Gowda, K.C., Diday, E.: Symbolic Clustering Using a New Dissimilarity Measure. Pattern Recognition 24(6), 567–578 (1991)
Caruso, C., Malerba, D., Papagni, D.: Learning the daily model of network traffic. In: Hacid, M.-S., Murray, N.V., Raś, Z.W., Tsumoto, S. (eds.) ISMIS 2005. LNCS (LNAI), vol. 3488, pp. 131–141. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Caruso, C., Malerba, D. (2007). A Data Mining Methodology for Anomaly Detection in Network Data. In: Apolloni, B., Howlett, R.J., Jain, L. (eds) Knowledge-Based Intelligent Information and Engineering Systems. KES 2007. Lecture Notes in Computer Science(), vol 4693. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74827-4_14
Download citation
DOI: https://doi.org/10.1007/978-3-540-74827-4_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74826-7
Online ISBN: 978-3-540-74827-4
eBook Packages: Computer ScienceComputer Science (R0)