research-article

Classification and exploration of TSM log file based on datamining Algorithms

Authors:

Jamal El abdelkhalki,

Mohamed Ben ahmed,

Boudhir Hakim AnouarAuthors Info & Claims

ICCWCS'17: Proceedings of the 2nd International Conference on Computing and Wireless Communication Systems

Article No.: 64, Pages 1 - 7

https://doi.org/10.1145/3167486.3167553

Published: 14 November 2017 Publication History

Abstract

analyzing the log file for software or device provides a focal point for making incremental improvements; it is the performed step to start the incident analysis. Although, log messages format or contents may not always be fully documented, and described in many different formats. It makes the log analysis task more difficult, affects the correction deadline of incidents and therefore involves a high financial risk. In this paper, we survey the log file analysis and the existing systems elaborated to resolve current issue. Then, we propose a methodology to support the log analysis in the complex environment related to big data issues. Finally, we illustrate our proposal on the file log of the Tivoli Storage Manager (TSM) and provide a discussion of the result clusters.

References

[1]

J. Valdman, "Log file analysis," Dep. Comput. Sci. Eng. FAV UWB Tech Rep DCSETR-2001-04, 2001.

[2]

"Advances and challenges in log analysis."

[3]

M. Charrad, M. B. Ahmed, and Y. Lechevallier, "Extraction des connaissances à partir des fichiers logs," Atelier Fouille Web EGC2006, vol. 768, 2005.

[4]

J.-S. Vayre, "Les big data et la relation client," in 12ème Journées Normandes de Recherches sur la Consommation: Société et Consommation, 2013, pp. 1--20.

[5]

"What is Log Analysis? - Definition from Techopedia," Techopedia.com. {Online}. Available: https://www.techopedia.com/definition/31756/log-analysis. {Accessed: 20-Aug-2017}.

[6]

K. E. Nawyn, "A security analysis of system event logging with syslog," Inst. No Part Inf. Secur. Read. Room, 2003.

[7]

D. Casey, "Turning log files into a security asset," Netw. Secur., vol. 2008, no. 2, pp. 4--7, 2008.

Digital Library

[8]

S. Narkhede and T. Baraskar, "HMR Log Analyzer: Analyze Web Application Logs Over Hadoop MapReduce," Int. J. UbiComp, vol. 4, no. 3, pp. 41--51, Jul. 2013.

[9]

H. Yu and D. Wang, "Mass log data processing and mining based on Hadoop and cloud computing," in 2012 7th International Conference on Computer Science Education (ICCSE), 2012, pp. 197--202.

[10]

"SAFAL: A MapReduce Spatio-temporal Analyzer for UNAVCO FTP Logs." {Online}. Available: https://www.computer.org/csdl/proceedings/cse/2013/5096/00/5096b083-abs.html. {Accessed: 28-Jul-2017}.

[11]

R. S. J. de Baker and P. S. Inventado, "Chapter X: Educational Data Mining and Learning Analytics."

[12]

L. Wang et al., "G-Hadoop: MapReduce across distributed data centers for data-intensive computing," Future Gener. Comput. Syst., vol. 29, no. 3, pp. 739--750, Mar. 2013.

Digital Library

[13]

K. Nagaraj, C. Killian, and J. Neville, "Structured comparative analysis of systems logs to diagnose performance problems," in Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation, 2012, pp. 26--26.

Digital Library

[14]

P. J. Rousseeuw and L. Kaufman, Finding Groups in Data. Wiley Online Library, 1990.

[15]

K. Alsabti, S. Ranka, and V. Singh, "An efficient k-means clustering algorithm," 1997.

[16]

G. Nathiya, S. C. Punitha, and M. Punithavalli, "An analytical study on behavior of clusters using k means, em and k* means algorithm," ArXiv Prepr. ArXiv10041743, 2010.

[17]

"IBM Knowledgecenter - Tivoli Storage Manager server and client messages format." {Online}. Available: https://www.ibm.com/support/knowledgecenter/en/SSGSG7_7.1.4/msgs.tsm/msgs_msgsformat.html. {Accessed: 28-Jul-2017}.

[18]

"Weka 3 - Data Mining with Open Source Machine Learning Software in Java." {Online}. Available: http://www.cs.waikato.ac.nz/ml/weka/. {Accessed: 20-Aug-2017}.

Cited By

Korzeniowski LGoczyla K(2022)Landscape of Automated Log Analysis: A Systematic Literature Review and Mapping StudyIEEE Access10.1109/ACCESS.2022.315254910(21892-21913)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3152549
El Abdelkhalki JBen Ahmed M(2020)Smart Incident Management, Prediction Engine and Performance EnhancementInnovations in Smart Cities Applications Edition 310.1007/978-3-030-37629-1_73(1011-1025)Online publication date: 1-Feb-2020
https://doi.org/10.1007/978-3-030-37629-1_73
Abdelkhalki JAhmed MSlimani A(2019)Incident prediction through logging management and machine learningProceedings of the 4th International Conference on Smart City Applications10.1145/3368756.3369069(1-8)Online publication date: 2-Oct-2019
https://dl.acm.org/doi/10.1145/3368756.3369069
Show More Cited By

Recommendations

Incident prediction through logging management and machine learning
SCA '19: Proceedings of the 4th International Conference on Smart City Applications

Analyzing the log file for software or device provides a focal point for making incremental improvements; it is the performed step to start the incident analysis. Although, log messages format or contents may not always be fully documented, and ...
A log file service exploiting write-once storage
The design and implementation of a log-structured file system

This paper presents a new technique for disk storage management called a log-structured file system. A log-structured file system writes all modifications to disk sequentially in a log-like structure, thereby speeding up both file writing and crash ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICCWCS'17: Proceedings of the 2nd International Conference on Computing and Wireless Communication Systems

November 2017

512 pages

ISBN:9781450353069

DOI:10.1145/3167486

Conference Chairs:
Jamal Zbitou
FSTS, Hassan 1st University, Morocco
,
Setti Larbi
UAE University Morocco
,
En-naimi El Mokhtar
FST of Tangier, UAE University Morocco

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICCWCS'17

ICCWCS'17: International Conference on Computing and Wireless Communication Systems

November 14 - 16, 2017

Larache, Morocco

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
71
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Korzeniowski LGoczyla K(2022)Landscape of Automated Log Analysis: A Systematic Literature Review and Mapping StudyIEEE Access10.1109/ACCESS.2022.315254910(21892-21913)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3152549
El Abdelkhalki JBen Ahmed M(2020)Smart Incident Management, Prediction Engine and Performance EnhancementInnovations in Smart Cities Applications Edition 310.1007/978-3-030-37629-1_73(1011-1025)Online publication date: 1-Feb-2020
https://doi.org/10.1007/978-3-030-37629-1_73
Abdelkhalki JAhmed MSlimani A(2019)Incident prediction through logging management and machine learningProceedings of the 4th International Conference on Smart City Applications10.1145/3368756.3369069(1-8)Online publication date: 2-Oct-2019
https://dl.acm.org/doi/10.1145/3368756.3369069
Zong CHuang SLiu EYao YTang S(2019)Nowhere to Hide Methodology: Application of Clustering Fault Diagnosis in the Nuclear Power IndustryIEEE Access10.1109/ACCESS.2019.29578077(179864-179879)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2957807

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents