research-article

Damage Assessment from Social Media Imagery Data During Disasters

Authors:

Muhammad Imran,

Prasenjit MitraAuthors Info & Claims

ASONAM '17: Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017

Pages 569 - 576

https://doi.org/10.1145/3110025.3110109

Published: 31 July 2017 Publication History

Abstract

Rapid access to situation-sensitive data through social media networks creates new opportunities to address a number of real-world problems. Damage assessment during disasters is a core situational awareness task for many humanitarian organizations that traditionally takes weeks and months. In this work, we analyze images posted on social media platforms during natural disasters to determine the level of damage caused by the disasters. We employ state-of-the-art machine learning techniques to perform an extensive experimentation of damage assessment using images from four major natural disasters. We show that the domain-specific fine-tuning of deep Convolutional Neural Networks (CNN) outperforms other state-of-the-art techniques such as Bag-of-Visual-Words (BoVW). High classification accuracy under both event-specific and cross-event test settings demonstrate that the proposed approach can effectively adapt deep-CNN features to identify the severity of destruction from social media images taken after a disaster strikes.

References

[1]

M. Imran, C. Castillo, F. Diaz, and S. Vieweg, "Processing social media messages in mass emergency: A survey," ACM Computing Surveys (CSUR), vol. 47, no. 4, p. 67, 2015.

Digital Library

[2]

C. Castillo, Big Crisis Data. Cambridge University Press, 2016.

[3]

K. Starbird, L. Palen, A. L. Hughes, and S. Vieweg, "Chatter on the red: what hazards threat reveals about the social life of microblogged information," in 2010 ACM conference on Computer supported cooperative work, 2010, pp. 241--250.

[4]

D. T. Nguyen, K. Al-Mannai, S. R. Joty, H. Sajjad, M. Imran, and P. Mitra, "Robust classification of crisis-related data on social networks using convolutional neural networks." in ICWSM, 2017, pp. 632--635.

[5]

M. Imran, S. M. Elbassuoni, C. Castillo, F. Diaz, and P. Meier, "Extracting information nuggets from disaster-related messages in social media," ISCRAM, 2013.

[6]

M. Imran, C. Castillo, J. Lucas, P. Meier, and S. Vieweg, "AIDR: Artificial intelligence for disaster response," in Proceedings of the 23rd International Conference on World Wide Web. ACM, 2014, pp. 159--162.

Digital Library

[7]

S. Cresci, M. Tesconi, A. Cimino, and F. Dell'Orletta, "A linguistically-driven approach to cross-event damage assessment of natural disasters from social media messages," in Proceedings of the 24th International Conference on World Wide Web. ACM, 2015, pp. 1195--1200.

Digital Library

[8]

D. T. Nguyen, F. Alam, F. Ofli, and M. Imran, "Automatic image filtering on social networks using deep learning and perceptual hashing during crises," in ISCRAM, May 2017.

[9]

M. Imran, P. Mitra, and J. Srivastava, "Cross-language domain adaptation for classifying crisis-related short messages," ISCRAM, 2016.

[10]

M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, "The pascal visual object classes (voc) challenge," International Journal of Computer Vision, vol. 88, no. 2, pp. 303--338, Jun. 2010.

Digital Library

[11]

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, "ImageNet Large Scale Visual Recognition Challenge," International Journal of Computer Vision (IJCV), vol. 115, no. 3, pp. 211--252, 2015.

Digital Library

[12]

A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Advances in Neural Information Processing Systems 25, 2012, pp. 1097--1105.

Digital Library

[13]

P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, "Overfeat: Integrated recognition, localization and detection using convolutional networks," in International Conference on Learning Representations (ICLR 2014).

[14]

K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," CoRR, vol. abs/1409.1556, 2014. [Online]. Available: http://arxiv.org/abs/1409.1556

[15]

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015, pp. 1--9.

[16]

K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.

[17]

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," CoRR, vol. abs/1512.00567, 2015.

[18]

S. Zagoruyko and N. Komodakis, "Wide residual networks," CoRR, vol. abs/1605.07146, 2016. [Online]. Available: http://arxiv.org/abs/1605.07146

[19]

M. D. Zeiler and R. Fergus, Visualizing and Understanding Convolutional Networks. Cham: Springer International Publishing, 2014, pp. 818--833.

[20]

I. Goodfellow, Y. Bengio, and A. Courville, "Deep learning," 2016, book in preparation for MIT Press. [Online]. Available: http://www.deeplearningbook.org

[21]

D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vision, vol. 60, no. 2, pp. 91--110, Nov. 2004.

Digital Library

[22]

N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in IEEE Conference on Computer Vision and Pattern Recognition, 2005, pp. 886--893.

Digital Library

[23]

G. Csurka, C. R. Dance, L. Fan, J. Willamowski, and C. Bray, "Visual categorization with bags of keypoints," in Workshop on Statistical Learning in Computer Vision, ECCV, 2004, pp. 59--74.

[24]

S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, 2006, pp. 2169--2178.

Digital Library

[25]

P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, "Object Detection with Discriminatively Trained Part-Based Models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1627--1645, Sep. 2010.

Digital Library

[26]

R. B. Girshick, P. F. Felzenszwalb, and D. McAllester, "Discriminatively trained deformable part models, rel 5," http://people.cs.uchicago.edu/rbg/latent-release5/.

[27]

S. Fidler, R. Mottaghi, A. Yuille, and R. Urtasun, "Bottom-up segmentation for top-down detection," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2013, pp. 3294--3301.

Digital Library

[28]

M. Pesaresi, A. Gerhardinger, and F. Haag, "Rapid damage assessment of builtup structures using vhr satellite data in tsunamiaffected areas," International Journal of Remote Sensing, vol. 28, no. 13-14, pp. 3013--3036, 2007.

Digital Library

[29]

T. Feng, Z. Hong, Q. Fu, S. Ma, X. Jie, H. Wu, C. Jiang, and X. Tong, "Application and prospect of a high-resolution remote sensing and geo-information system in estimating earthquake casualties," Natural Hazards and Earth System Sciences, vol. 14, no. 8, pp. 2165--2178, 2014.

[30]

M. Turker and B. T. San, "Detection of collapsed buildings caused by the 1999 izmit, turkey earthquake through digital analysis of post-event aerial photographs," International Journal of Remote Sensing, vol. 25, no. 21, pp. 4701--4714, 2004.

[31]

J. Fernandez Galarreta, N. Kerle, and M. Gerke, "UAV-based urban structural damage assessment using object-based image analysis and semantic reasoning," Natural Hazards and Earth System Science, vol. 15, no. 6, pp. 1087--1101, 2015.

[32]

F. Ofli, P. Meier, M. Imran, C. Castillo, D. Tuia, N. Rey, J. Briant, P. Millet, F. Reinhard, M. Parkan, and S. Joost, "Combining human computing and machine learning to make sense of big (aerial) data for disaster response," Big Data, vol. 4, no. 1, pp. 47--59, Mar 2016.

[33]

R. Lagerstrom, Y. Arzhaeva, P. Szul, O. Obst, R. Power, B. Robinson, and T. Bednarz, "Image classification to support emergency situation awareness," Frontiers in Robotics and AI, vol. 3, p. 54, 2016.

[34]

S. Daly and J. A. Thom, "Mining and classifying image posts on social media to analyse fires," in ISCRAM, May 2016, pp. 1--14.

[35]

A. Bosch, A. Zisserman, and X. Munoz, "Image classification using random forests and ferns," in IEEE International Conference on Computer Vision, 2007.

Cited By

Gao SYang TXu YMou NWang XHuang H(2025)Enhancing Disaster Situation Awareness Through Multimodal Social Media Data: Evidence from Typhoon HaikuiApplied Sciences10.3390/app1501046515:1(465)Online publication date: 6-Jan-2025
https://doi.org/10.3390/app15010465
Gupta TRoy S(2025)DisasterRes-Net: A framework for analyzing social media images in disaster responseInternational Journal of Disaster Risk Reduction10.1016/j.ijdrr.2024.105119116(105119)Online publication date: Jan-2025
https://doi.org/10.1016/j.ijdrr.2024.105119
Dar SRehman MBais KHaseeb MKumar N(2025)A social context-aware graph-based multimodal attentive learning framework for disaster content classification during emergenciesExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.125337259:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.eswa.2024.125337
Show More Cited By

Damage Assessment from Social Media Imagery Data During Disasters
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms

Recommendations

Characterization of social media response to natural disasters
WWW '12 Companion: Proceedings of the 21st International Conference on World Wide Web

Online social networking websites such as Twitter and Facebook often serve a breaking-news role for natural disasters: these websites are among the first ones to mention the news, and because they are visited by millions of users regularly the websites ...
Social Roles and Consequences in Using Social Media in Disasters: a Structurational Perspective

Disaster management officials, as well as the general public, are increasingly using social media to communicate. Such usage has resulted in new and emergent social consequences for disaster management and has reformed the roles of its relevant ...
Social media crowdsourcing for rapid damage assessment following a sudden-onset natural hazard event
Highlights
- Develop a crowdsourcing approach using Twitter data for rapid earthquake damage assessment.
Abstract
Rapid appraisal of damages related to hazard events is of importance to first responders, government agencies, insurance industries, and other private and public organizations. While satellite monitoring, ground-based sensor systems, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASONAM '17: Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017

July 2017

698 pages

ISBN:9781450349932

DOI:10.1145/3110025

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 July 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

ASONAM '17

Sponsor:

SIGKDD

ASONAM '17: Advances in Social Networks Analysis and Mining 2017

July 31 - August 3, 2017

Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 116 of 549 submissions, 21%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

127
Total Citations
View Citations
720
Total Downloads

Downloads (Last 12 months)88
Downloads (Last 6 weeks)9

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gao SYang TXu YMou NWang XHuang H(2025)Enhancing Disaster Situation Awareness Through Multimodal Social Media Data: Evidence from Typhoon HaikuiApplied Sciences10.3390/app1501046515:1(465)Online publication date: 6-Jan-2025
https://doi.org/10.3390/app15010465
Gupta TRoy S(2025)DisasterRes-Net: A framework for analyzing social media images in disaster responseInternational Journal of Disaster Risk Reduction10.1016/j.ijdrr.2024.105119116(105119)Online publication date: Jan-2025
https://doi.org/10.1016/j.ijdrr.2024.105119
Dar SRehman MBais KHaseeb MKumar N(2025)A social context-aware graph-based multimodal attentive learning framework for disaster content classification during emergenciesExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.125337259:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.eswa.2024.125337
Thota JPadala A(2024)Human Remains Detection in Natural Disasters using YOLO: A Deep Learning ApproachEngineering, Technology & Applied Science Research10.48084/etasr.848314:6(17678-17682)Online publication date: 2-Dec-2024
https://doi.org/10.48084/etasr.8483
Venkat Narayana Rao TJakkam PMedipally S(2024)Future Trends and Innovations in Natural Disaster Detection Using AI and MLPredicting Natural Disasters With AI and Machine Learning10.4018/979-8-3693-2280-2.ch005(110-134)Online publication date: 31-May-2024
https://doi.org/10.4018/979-8-3693-2280-2.ch005
Luo KLian I(2024)Building a Vision Transformer-Based Damage Severity Classifier with Ground-Level Imagery of Homes Affected by California WildfiresFire10.3390/fire70401337:4(133)Online publication date: 11-Apr-2024
https://doi.org/10.3390/fire7040133
SÜNBÜL GSOYLUK A(2024)A Review of Using Deep Learning Technology in the Built Environment of Disaster Management PhasesYapılı Çevrede Afet Yönetimi Aşamalarında Derin Öğrenme Teknolojisinin Kullanımına İlişkin Bir İncelemeMimarlık Bilimleri ve Uygulamaları Dergisi (MBUD)10.30785/mbud.1333736(201-218)Online publication date: 23-Jan-2024
https://doi.org/10.30785/mbud.1333736
WANG XYANG TXU YMING MWANG HLI LXU JZHANG JZHANG PWANG H(2024)A dataset of semantic segmentation in earthquake disaster detection based on social media imagesChina Scientific Data10.11922/11-6035.noda.2023.0005.zh9:3(1-10)Online publication date: 27-Sep-2024
https://doi.org/10.11922/11-6035.noda.2023.0005.zh
Firmansyah HLorini VMulayim MGomes JFernandez-Marquez J(2024)Improving Social Media Geolocation for Disaster Response by Using Text From Images and ChatGPTProceedings of the 2024 11th Multidisciplinary International Social Networks Conference10.1145/3675669.3675696(67-72)Online publication date: 21-Aug-2024
https://dl.acm.org/doi/10.1145/3675669.3675696
Żarski MMiszczak J(2024)Multi-Step Feature Fusion for Natural Disaster Damage Assessment on Satellite ImagesIEEE Access10.1109/ACCESS.2024.345942412(140072-140081)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3459424
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents