Abstract
Social media platforms let users share their opinions through textual or multimedia content. In many settings, this becomes a valuable source of knowledge that can be exploited for specific business objectives. In this work, we report on an implementation aiming at cleaning the data collected from social content, within specific domains or related to given topics of interest. Indeed, topic-based collection of social media content is performed through keyword-based search, which typically entails very noisy results. Therefore we propose a method for data cleaning and removal of off-topic content based on supervised machine learning techniques, i.e. classification, over data collected from social media platforms based on keywords regarding a specific topic. We define a general method for this and then we validate it through an experiment of data extraction from Twitter, with respect to a set of famous cultural institutions in Italy, including theaters, museums, and other venues. For this case, we collaborated with domain experts to label the dataset, and then we evaluated and compared the performance of classifiers that are trained with different feature extraction strategies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Statista, on social media usage. https://www.statista.com/statistics/433871/daily-social-media-usage-worldwide/, Last accessed 4 Apr 2018.
- 2.
Omnicore Agency, https://www.omnicoreagency.com/twitter-statistics/, Last accessed 4 Apr 2018.
References
Salloum, S.A., Al-Emran, M., Monem, A.A., Shaalan, K.: A survey of text mining in social media Facebook and Twitter perspectives. Adv. Sci. Technol. Eng. Syst. J. (2017)
Sakaki, T., Okazaki, M., Matsuo, Y.: Earthquake shakes Twitter users: real-time event detection by social sensors. In: Proceedings of the 19th International Conference on World Wide Web, New York, USA, pp. 851–860 (2010)
Culotta A: Towards detecting influenza epidemics by analyzing Twitter messages. In: Proceedings of the First Workshop on Social Media Analytics, Washington, D.C., pp. 115–122 (2010)
Paul, M.J., Dredze, M.: Discovering health topics in social media using models. PLoS ONE 9, e103408 (2014)
Sarker, A., et al.: Social media mining for toxicovigilance: automatic monitoring of prescription medication abuse from Twitter. Drug Saf. 39(3), 231–240 (2016)
Lin, J., Snow, R., Morgan, W.: Smoothing techniques for adaptive online language models: topic tracking in tweet streams. In: KDD, pp. 422–429 (2011)
Khan, M.A.H., Iwai, M., Sezaki, K.: An improved classification strategy for filtering relevant tweets using bag-of-word classifiers. J. Inf. Process. 21(3), 507–516 (2013)
Kunneman, F., Bosch, A.: Event detection in Twitter: a machine-learning approach based on term pivoting. In: BNAIC, pp. 65–72 (2014)
Castillo, C., Mendoza, M., Poblete, B.: Information credibility on Twitter. In: Proceedings of the 20th International Conference on World Wide Web, Hyderabad, pp. 675–684. ACM (2011)
Bollegala, D., Matsuo, Y., Ishizuka, M.: Measuring semantic similarity between words using web search engines. In: Proceedings of the 16th International Conference on World Wide Web (WWW2007), pp. 757–766. ACM Press, New York (2007)
Yang, S., Kolcz, A., Schlaikjer, A., Gupta, P.: Large-scale high-precision topic modeling on Twitter. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2014), pp. 1907–1916. ACM, New York (2014)
Banerjee, S., Ramanathan, K., Gupta, A.: Clustering short texts using Wikipedia. In: SIGIR, pp. 787–788 (2007)
Li, Q., Liu, X., Shah, S., Nourbakhsh, A.: Tweet topic classification using distributed language representations. In: Proceedings of the 2016 IEEE/WIC/ACM International Conference on Web Intelligence, Nebraska, USA (2016)
Hajjem, M., Latiri, C.: Combining IR and LDA topic modeling for filtering microblogs. Procedia Comput. Sci. 112, 761–770 (2017)
Joachims, T.: Text categorization with support vector machines: learning with many relevant features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 137–142. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0026683
Lewis, J.P.: Tutorial on SVM. In: CGIT Lab, USC (2004)
Sun, A.: Short text classification using very few words. In: SIGIR, pp. 1145–1146. ACM (2012)
Sriram, B., Fuhry, D., Demir, E., Ferhatosmanoglu, H., Demirbas, M.: Short text classification in Twitter to improve information filtering. In: Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (2010)
Damashek, M.: Gauging similarity with n-grams: language independent categorization of text. Science 267(5199), 843–848 (1995)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of Workshop at ICLR (2013)
Acknowledgements
We gratefully acknowledge Fluxedo S.r.l. (http://www.fluxedo.com/) for sharing the data used for our experiments and the Osservatorio MIP Innovazione Digitale nei Beni e Attività Culturali (https://www.osservatori.net/it_it/osservatori/innovazione-digitale-nei-beni-e-attivita-culturali.) for the useful insights and discussion on the matter.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Calisir, E., Brambilla, M. (2018). The Problem of Data Cleaning for Knowledge Extraction from Social Media. In: Pautasso, C., Sánchez-Figueroa, F., Systä, K., Murillo RodrÃguez, J. (eds) Current Trends in Web Engineering. ICWE 2018. Lecture Notes in Computer Science(), vol 11153. Springer, Cham. https://doi.org/10.1007/978-3-030-03056-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-03056-8_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03055-1
Online ISBN: 978-3-030-03056-8
eBook Packages: Computer ScienceComputer Science (R0)