research-article

Social Haystack: Dynamic Quality Assessment of Citizen-Generated Content during Emergencies

Authors:

Christian Reuter,

Volkmar PipekAuthors Info & Claims

ACM Transactions on Computer-Human Interaction (TOCHI), Volume 22, Issue 4

Article No.: 17, Pages 1 - 27

https://doi.org/10.1145/2749461

Published: 29 June 2015 Publication History

Abstract

People all over the world are regularly affected by disasters and emergencies. Besides official emergency services, ordinary citizens are getting increasingly involved in crisis response work. They are usually present on-site at the place of incident and use social media to share information about the event. For emergency services, the large amount of citizen-generated content in social media, however, means that finding high-quality information is similar to “finding a needle in a haystack”. This article presents an approach to how a dynamic and subjective quality assessment of citizen-generated content could support the work of emergency services. First, we present results of our empirical study concerning the usage of citizen-generated content by emergency services. Based on our literature review and empirical study, we derive design guidelines and describe a concept for dynamic quality measurement that is implemented as a service-oriented web-application “Social Haystack.” Finally, we outline findings of its evaluation and implications thereof.

References

[1]

F. Abel, C. Hauff, and R. Stronkman. 2012a. Semantics + Filtering + Search = Twitcident exploring information in social web streams categories and subject descriptors. In Proc. Hypertext. Milwaukee, USA, 5--8.

Digital Library

[2]

F. Abel, C. Hauff, and R. Stronkman. 2012b. Twitcident: Fighting fire with information from social web streams. In Proc. WWW. New York, 5--8.

Digital Library

[3]

E. Agichtein, C. Castillo, and D. Donato. 2008. Finding high-quality content in social media. In Proc. WSDM. Atlanta, USA, 183--193.

Digital Library

[4]

M. Baroni, S. Bernardini, A. Ferraresi, and E. Zanchetta. 2009. The WaCky wide web: A collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation 43, 3, 209--226.

[5]

N. Bharosa, M. Janssen, and J. Lee. 2008. Adaptive information orchestration: architectural principles improving information quality. In Proc. ISCRAM. Washington, 556--565.

[6]

N. Bharosa, J. Lee, and M. Janssen. 2009. Challenges and obstacles in sharing and coordinating information during multi-agency disaster response: Propositions from field exercises. Information Systems Frontiers 12, 1, 49--65.

Digital Library

[7]

S. K. Card, G. G. Robertson, and J. D. Mackinlay. 1991. The information visualizer: An information workspace. In Proc. CHI. ACM New York, NY, USA, 181--186.

Digital Library

[8]

R. Chen, R. Sharman, H. R. Rao, and S. J. Upadhyaya. 2008. Coordination in emergency response management. Communications of the ACM 51, 5, 66--73.

Digital Library

[9]

D. Christofzik and C. Reuter. 2013. The aggregation of information qualities in collaborative software. International Journal of Entrepreneurial Venturing (IJEV) 5, 3, 257--271.

[10]

G. Convertino, H. M. Mentis, A. Slavkovic, M. B. Rosson, and J. M. Carroll. 2011. Supporting common ground and awareness in emergency management planning. ToCHI 18, 4, 1--34.

Digital Library

[11]

M. J. Eppler. 2006. Managing Information Quality. Springer, Berlin, Heidelberg, New York.

Digital Library

[12]

M. Faruqui and S. Pad. 2010. Training and evaluating a German named entity recognizer with semantic generalization at Stuttgart. In Proc. KONVENS. Saarbrücken, Germany, 129--133.

[13]

C. Fisher, E. Lauria, S. Chengalur-Smith, and R. Wang. 2011. Introduction to information quality. AuthorHouse, Bloomington, Indiana, USA.

Digital Library

[14]

C. W. Fisher and B. R. Kingma. 2001. Criticality of data quality as exemplified in two disasters. Information & Management 39, 2, 109--116.

Digital Library

[15]

T. Friberg, S. Prödel, and R. Koch. 2010. Analysis of information quality criteria in a crisis situation as a characteristic of complex situations. In Proc. IQ. Little Rock, AR.

[16]

T. Friberg, S. Prödel, and R. Koch. 2011. Information quality criteria and their importance for experts in crisis situations. In Proc. ISCRAM. Lisbon, Portugal.

[17]

H. Gao, G. Barbier, and R. Goolsby. 2011. Harnessing the crowdsourcing power of social media for disaster relief. IEEE Intelligent Systems 26, 3, 10--14.

Digital Library

[18]

R. Gaspar, S. Gorjão, B. Seibt, L. Lima, J. Barnett, A. Moss, and J. Wills. 2014. Tweeting during food crises: A psychosocial analysis of threat coping expressions in Spain, during the 2011 European EHEC outbreak. IJHCS 72, 2, 239--254.

Digital Library

[19]

S. Ha and J. Ahn. 2011. Why are you sharing others’ Tweets?: The impact of argument quality and source credibility on information sharing behavior. In Proc. ICIS. Shanghai, China, 1--11.

[20]

G. R. Hayes. 2011. The relationship of action research to human-computer interaction. ToCHI 18, 3, 1--20.

Digital Library

[21]

G. N. Hearn, J. A. Tacchi, M. Foth, and J. Lennie. 2008. Action research and new media: Concepts, Methods and Cases. Hampton Press, New York, USA.

[22]

A. Hevner and S. Chatterjee. 2010. Design Research in Information Systems: Theory and Practice. Springer, New York, USA.

Digital Library

[23]

A. Hevner, S. T. March, J. Park, and S. Ram. 2004. Design science in information systems research. MIS Quarterly 28, 1, 75--105.

[24]

S. R. Hiltz, P. Diaz, and G. Mark. 2011. Introduction: Social media and collaborative systems for crisis management. ToCHI 18, 4, 1--6.

Digital Library

[25]

Homeland Security. 2013. Lessons learned: Social media and Hurricane Sandy - virtual social media working group and group DHS first responders. Retrieved June 7, 2015 from http://www.ghinternational.com/docs/DHS&lowbar;VSMWG&lowbar;Lessons&lowbar;Learned&lowbar;Social&lowbar;Media&lowbar;and&lowbar;Hurricane&lowbar;Sandy&lowbar;Formatted&lowbar;June&lowbar;2013&lowbar;FINAL.pdf.

[26]

M. E. Jennex. 2012. Social media -- truly viable for crisis response? In Proc. ISCRAM. Vancouver, Canada, 1--5.

[27]

G. Kane and S. Ransbotham. 2012. Codification and collaboration: Information quality in social media. In Proc. ICIS. Orlando, Florida, USA, 1--16.

[28]

A. Kao and S. R. Poteet. 2007. Natural Language Processing and Text Mining. Springer, London.

Digital Library

[29]

A. M. Kaplan and M. Haenlein. 2010. Users of the world, unite&excl; The challenges and opportunities of Social Media. Business Horizons 53, 1, 59--68.

[30]

M. Kyng, E. T. Nielsen, and M. Kristensen. 2006. Challenges in designing interactive systems for emergency response. In Proc. DISp. 301. ACM Press, New York, USA, p. 301.

Digital Library

[31]

K. Lewin. 1958. Group Decision and Social Change. Holt, Rinehart and Winston, New York, USA.

[32]

B. Ley, T. Ludwig, V. Pipek, D. Randall, C. Reuter, and T. Wiedenhoefer. 2014. Information and expertise sharing in inter-organizational crisis management. JCSCW 23, 4--6, 347--387.

Digital Library

[33]

S. B. Liu and L. Palen. 2010. The new cartographers: crisis map mashups and the emergence of neogeographic practice. Cartography and Geographic Information Science 37, 1, 69--90.

[34]

P. H. Longstaff. 2005. Security, Resilience and Communication in Unpredictable Environments such as Terrorism, Natural Disasters and Complex Technology. Harvard University, Cambridge, UK.

[35]

T. Ludwig, C. Reuter, and V. Pipek. 2013. What you see is what I need: Mobile reporting practices in emergencies. In Proc. ECSCW (181--206). Springer, Paphos, Cyprus.

[36]

T. Ludwig, C. Reuter, T. Siebigteroth, and V. Pipek. 2015. CrowdMonitor: Mobile crowd sensing for assessing physical and digital activities of citizens during emergencies. In Proc. CHI. ACM Press, Seoul, Korea.

Digital Library

[37]

N. Luhmann. 1996. Social Systems (1st. ed.). Stanford University Press, Stanford, California, USA.

[38]

A. Marcus, M. Bernstein, and O. Badar. 2011. Twitinfo: Aggregating and visualizing microblogs for event exploration. In Proc. CHI. Vancouver, Canada.

Digital Library

[39]

T. B. Marino, B. S. do Nascimento, and M. R. S. Borges. 2012. GIS supporting data gathering and fast decision making in emergencies situations. In Proc. ISCRAM. Vancouver, Canada.

[40]

I. Mulder, H. de Poot, C. Verwij, R. Janssen, and M. Bijlsma. 2006. An information overload study: Using design methods for understanding. In Proc. OZCHI. 245--252.

Digital Library

[41]

J. Nielsen. 1993. Usability engineering. Morgan Kaufmann, San Francisco, California, USA.

Digital Library

[42]

M. Normark and D. Randall. 2005. Local expertise at an emergency call centre. In H. Gellersen, K. Schmidt, M. Beaudouin-Lafon, and W. Mackay (Eds.), Proc. ECSCW. Springer, Paris, France, 347--366.

Digital Library

[43]

C. A. O’Reilly. 1980. Individuals and information overload in organizations: Is more necessarily better? Academy of Management Journal 23, 4, 684--696.

[44]

O. Okolloh. 2008. Ushahidi, or “testimony”: Web 2.0 tools for crowdsourcing crisis information. Participatory Learning and Action 59 (January), 65--70.

[45]

L. Palen and S. B. Liu. 2007. Citizen communications in crisis: Anticipating a future of ICT-supported public participation. In Proc. CHI. San Jose, ACM Press, USA.

Digital Library

[46]

L. Palen, S. Vieweg, and K. M. Anderson. 2011. Supporting “Everyday Analysts” in safety- and time-critical situations. The Information Society 27, 1, 52--62.

Digital Library

[47]

H. Purohit, A. Hampton, S. Bhatt, V. L. Shalin, A. P. Sheth, and J. M. Flach. 2014. Identifying seekers and suppliers in social media communities to support crisis coordination. JCSCW 23, 4--6, 513--545.

Digital Library

[48]

Y. Qu, P. F. Wu, and X. Wang. 2009. Online community response to major disaster: A study of Tianya forum in the 2008 Sichuan earthquake. In Proc. HICSS. Waikoloa, USA.

[49]

C. Reuter, O. Heger, and V. Pipek. 2013. Combining real and virtual volunteers through social media. In Proc. ISCRAM. Baden-Baden, Germany, 780--790.

[50]

C. Reuter, T. Ludwig, M.-A. Kaufhold, and V. Pipek. 2015. XHELP: Design of a cross-platform social-media application to support volunteer moderators in disasters. In Proc. CHI. ACM Press, Seoul, Korea.

Digital Library

[51]

C. Reuter, T. Ludwig, and V. Pipek. 2014. Ad hoc participation in situation assessment: Supporting mobile collaboration in emergencies. ToCHI 21, 5, 26:1--26:26.

Digital Library

[52]

C. Reuter, A. Marx, and V. Pipek. 2012. Crisis management 2.0: Towards a systematization of social software use in crisis situations. IJISCRAM 4, 1, 1--16.

[53]

A. Ritter, S. Clark, and O. Etzioni. 2011. Named entity recognition in tweets: An experimental study. In Conference on Empirical Methods in Natural Language Processing. Edinburgh, UK, 1524--1534.

Digital Library

[54]

A. Robinson, A. Savelyev, S. Pezanowski, and A. M. MacEachren. 2013. Understanding the utility of geospatial information in social media. In Proc. ISCRAM. Baden-Baden, Germany, 918--922.

[55]

K. Schmidt. 2014. The concept of “Practice”: What's the point? In Proc. COOP. 427--444. Nice, France.

[56]

A. Shamoug, R. Juric, and S. Paurobally. 2012. Ontological reasoning as a tool for humanitarian decision making. In Proc. ISCRAM. Vancouver, Canada.

[57]

G. Shankaranarayanan, B. Iyer, and D. Stoddard. 2012. Quality of social media data and implications of social media for data quality. In Proc. ICIQ. Paris, France, 311--325.

[58]

C. Shirky. 2008. Here Comes Everybody: The Power of Organizing Without Organizations. Penguin Books, London, UK.

[59]

R. A. Stallings and E. L. Quarantelli. 1985. Emergent citizen groups and emergency management. Public administration review 45 (Special Issue), 93--100.

[60]

K. Starbird, G. Muzny, and L. Palen. 2012. Learning from the crowd: Collaborative filtering techniques for identifying on-the-ground twitterers during mass disruptions. In Proc. ISCRAM. Vancouver, Canada.

[61]

K. Starbird, L. Palen, A. L. Hughes, and S. Vieweg. 2010. Chatter on the red: What hazards threat reveals about the social life of microblogged information. In Proc. CSCW. 241--250.

Digital Library

[62]

K. Starbird and J. Stamberger. 2010. Tweak the Tweet: Leveraging microblogging proliferation with a prescriptive syntax to support citizen reporting. In Proc. ISCRAM. Seattle, USA.

[63]

A. L. Strauss. 1987. Qualitative Analysis for Social Scientists. Cambridge Press, Cambridge, UK.

[64]

A. H. Tapia and K. Moore. 2014. Good enough is good enough: Overcoming disaster response organizations’ slow social media data adoption. JCSCW 23, 4--6, 483--512.

Digital Library

[65]

T. Terpstra, A. de Vries, R. Stronkman, and G. L. Paradies. 2012. Towards a realtime Twitter analysis during crises for operational crisis management. In Proc. ISCRAM. Vancouver, Canada, 1--9.

[66]

M. Turoff, M. Chumer, B. van de Walle, and X. Yao. 2004. The design of a dynamic emergency response management information system (DERMIS). JITTA 5, 4, 1--35.

[67]

M. Turoff, S. R. Hiltz, C. White, L. Plotnick, A. Hendela, and X. Yao. 2009. The past as the future for emergency planning and response. IJISCRAM 1, 1, 12--28.

[68]

M. Twidale, D. Randall, and R. Bentley. 1994. Situated evaluation for cooperative systems. In Proc. CSCW. Chapel Hill, North Carolina, USA, 441--452.

Digital Library

[69]

Twitter. 2014. TweetDeck. Retrieved July 23, 2014 from https://about.twitter.com/products/tweetdeck.

[70]

I. Uysal and W. B. Croft. 2011. User oriented tweet ranking: A filtering approach to microblogs. In Proc. CIKM. Glasgow, Great Britain, 2261--2264.

Digital Library

[71]

T. Wachtendorf and J. M. Kendra. 2006. Improvising disaster in the city of Jazz: Organizational response to Hurricane Katrina. Retrieved March 25, 2015 from http://understandingkatrina.ssrc.org/Wachtendorf&lowbar;Kendra/.

[72]

R. Wang and D. Strong. 1996. Beyond accuracy: What data quality means to data consumers. JMIS 12, 4, 5--33.

Digital Library

[73]

B. van de Walle and M. Turoff. 2008. Decision support for emergency situations. ISeB 6, 3, 295--316.

[74]

A. Wexelblat and P. Maes. 1999. Footprints: History-rich tools for information foraging. In Proc. CHI. 270--277.

Digital Library

[75]

A. Wu, G. Convertino, C. H. Ganoe, J. M. Carroll, and X. L. Zhang. 2013. Supporting collaborative sense-making in emergency management through geo-visualization. IJHCS 71, 1, 4--23.

Digital Library

[76]

V. Wulf, M. Rohde, V. Pipek, and G. Stevens. 2011. Engaging with practices: Design case studies as a research framework in CSCW. In Proc. CSCW. ACM Press, Hangzhou, China.

Digital Library

[77]

D. Yates and S. Paquette. 2011. Emergency knowledge management and social media technologies: A case study of the 2010 Haitian earthquake. IJIM 31, 1, 6--13.

Digital Library

[78]

J. Yin, S. Karimi, B. Robinson, and M. Cameron. 2012. ESA: Emergency situation awareness via microbloggers. In Proc. CIKM. 1--3.

Digital Library

Cited By

Zang YZhao JJiang WZhao T(2024)Advanced Linguistic Complex T-Spherical Fuzzy Dombi-Weighted Power-Partitioned Heronian Mean Operator and Its Application for Emergency Information Quality AssessmentSustainability10.3390/su1607306916:7(3069)Online publication date: 7-Apr-2024
https://doi.org/10.3390/su16073069
Dash PAxtell BGeiskkovitch DNeustaedter CStuerzlinger W(2024)Multimedia-Enabled 911: Exploring 911 Callers’ Experience of Call Taker Controlled Video Calling in Simulated EmergenciesProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3643055(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3643055
Rizza C(2023)Social Media Contribution to the Crisis Management Processes: Towards a More Accurate Response Integrating Citizen-Generated Content and Citizen-Led ActivitiesHandbook of Computational Social Science for Policy10.1007/978-3-031-16624-2_22(421-436)Online publication date: 24-Jan-2023
https://doi.org/10.1007/978-3-031-16624-2_22
Show More Cited By

Index Terms

Social Haystack: Dynamic Quality Assessment of Citizen-Generated Content during Emergencies
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

XHELP: Design of a Cross-Platform Social-Media Application to Support Volunteer Moderators in Disasters
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Recent disasters have shown an increase in the significance of social media for both affected citizens and volunteers alike in the coordination of information and organization of relief activities, often independently of and in addition to the official ...
Uses and gratifications of social networking sites for bridging and bonding social capital

Applying uses and gratifications theory (UGT) and social capital theory, our study examined users of four social networking sites (SNSs) (Facebook, Twitter, Instagram, and Snapchat), and their influence on online bridging and bonding social capital. ...
How Do People Participate in Social Network Sites After Crises? A Self-Determination Perspective

People increasingly rely on social network sites (SNSs) to find out timely information about crises. Thus, emergency managers are interested in how people participate and how to promote their participation in SNSs after crises. Based on self-...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 22, Issue 4

July 2015

147 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/2798442

Editor:
Shumin Zhai
Google, Inc.

Issue’s Table of Contents

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 June 2015

Accepted: 01 March 2015

Revised: 01 March 2015

Received: 01 September 2014

Published in TOCHI Volume 22, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

grant of the European Union

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

40
Total Citations
View Citations
720
Total Downloads

Downloads (Last 12 months)43
Downloads (Last 6 weeks)3

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zang YZhao JJiang WZhao T(2024)Advanced Linguistic Complex T-Spherical Fuzzy Dombi-Weighted Power-Partitioned Heronian Mean Operator and Its Application for Emergency Information Quality AssessmentSustainability10.3390/su1607306916:7(3069)Online publication date: 7-Apr-2024
https://doi.org/10.3390/su16073069
Dash PAxtell BGeiskkovitch DNeustaedter CStuerzlinger W(2024)Multimedia-Enabled 911: Exploring 911 Callers’ Experience of Call Taker Controlled Video Calling in Simulated EmergenciesProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3643055(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3643055
Rizza C(2023)Social Media Contribution to the Crisis Management Processes: Towards a More Accurate Response Integrating Citizen-Generated Content and Citizen-Led ActivitiesHandbook of Computational Social Science for Policy10.1007/978-3-031-16624-2_22(421-436)Online publication date: 24-Jan-2023
https://doi.org/10.1007/978-3-031-16624-2_22
Dash PNeustaedter CJones BYip C(2022)The Design and Evaluation of Emergency Call Taking User Interfaces for Next Generation 9-1-1Frontiers in Human Dynamics10.3389/fhumd.2022.6706474Online publication date: 16-Feb-2022
https://doi.org/10.3389/fhumd.2022.670647
Gao GZheng JChoe EYamashita N(2022)Taking a Language Detour: How International Migrants Speaking a Minority Language Seek COVID-Related Information in Their Host CountriesProceedings of the ACM on Human-Computer Interaction10.1145/35556006:CSCW2(1-32)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555600
Norris WVoida AVoida S(2022)People Talk in Stories. Responders Talk in Data: A Framework for Temporal Sensemaking in Time- and Safety-critical WorkProceedings of the ACM on Human-Computer Interaction10.1145/35129556:CSCW1(1-23)Online publication date: 7-Apr-2022
https://dl.acm.org/doi/10.1145/3512955
Yu XXie ZMashhadi AHong LBaeza-Yates RWeller KPortela MSeneviratne OWeber IYasseri TBon ASrinivas SIbáñez L(2022)Multi-task Models for Multi-faceted Classification of Pandemic Information on Social MediaProceedings of the 14th ACM Web Science Conference 202210.1145/3501247.3531552(327-335)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1145/3501247.3531552
Eismann KPosegga OFischbach K(2022)Opening organizational learning in crisis managementThe Journal of Strategic Information Systems10.1016/j.jsis.2021.10169230:4Online publication date: 9-Apr-2022
https://dl.acm.org/doi/10.1016/j.jsis.2021.101692
Khaokaew YHolcombe-James IRahaman MLiono JTrippas JSpina DBailey PBelkin NBennett PRen YSanderson MScholer FWhite RSalim F(2022)Imagining future digital assistants at work: A study of task management needsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2022.102905168(102905)Online publication date: Dec-2022
https://doi.org/10.1016/j.ijhcs.2022.102905
Dwarakanath LKamsin ARasheed RAnandhan AShuib L(2021)Automated Machine Learning Approaches for Emergency Response and Coordination via Social Media in the Aftermath of a Disaster: A ReviewIEEE Access10.1109/ACCESS.2021.30748199(68917-68931)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3074819
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents