article

Extreme event detection and assimilation from multimedia sources

Authors:

Ashit Talukder,

Anand PanangadanAuthors Info & Claims

Multimedia Tools and Applications, Volume 70, Issue 1

Pages 237 - 261

https://doi.org/10.1007/s11042-012-1088-y

Published: 01 May 2014 Publication History

Abstract

A new event-based multimedia processing framework for detection, retrieval, and cross-media content assimilation of geo-spatiotemporal phenomena is described. Multimedia information relevant to geo-spatiotemporal events are available from sources such as remote satellites, in-situ sensors as image streams, and other outlets such as news articles, weather bulletins as text documents and in various formats, each with widely varying properties. We pose an event-based framework to automatically detect geo-spatiotemporal phenomena from raw untagged remote-sensing satellite image streams, extract attributes for such events, match the spatiotemporal properties of the detected phenomenon with events in a database to automatically derive a media-independent event description, and subsequently use the media-independent event descriptions to assimilate relevant information of the same event across other media sources from the web using a mashup. A virtual globe interface enables simultaneous visualization of the assimilated spatiotemporal information annotated with Internet sources. This framework is demonstrated for the automatic detection of tropical cyclones from satellite imagery followed by the retrieval and assimilation of related information from government-run weather sites and commercial news portals.

References

[1]

Bobick AF, Ivanov YA (1998) Action recognition using probabilistic parsing. In: Conference on computer vision and pattern recognition (CVPR), Santa Barbara, CA, USA. IEEE Comput Society, pp 196---202

[2]

Callahan PS, Lungu T (2006) QuikSCAT Science Data Product User's Manual (v3.0). Jet Propulsion Laboratory, Pasadena, CA

[3]

Castaño R, Mazzoni D, Tang N, Greeley R, Doggett T, Cichy B, Chien SA, Davies A (August 20---23, 2006) Onboard classifiers for science event detection on a remote sensing spacecraft. In: Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Philadelphia, PA, USA, pp 845---851

[4]

DeMenthon D, Doermann D (2003) Video retrieval using spatio-temporal descriptors. Paper presented at the Proceedings of the eleventh ACM international conference on Multimedia, Berkeley, CA, USA

[5]

Dvorak VF (1984) Tropical cyclone intensity analysis using satellite data. vol 11. NOAA Tech. Rep. NESDIS

[6]

Fowler C, Qasemizadeh B (2009) Towards a Common Event Model for an Integrated Sensor Information System. Paper presented at the 1st International Workshop on the Semantic Sensor Web (SemSensWeb)

[7]

Francois ARJ, Nevatia R, Hobbs J, Bolles RC, Smith JR (2005) VERL: an ontology framework for representing and annotating video events. IEEE Multimed 12(4):76---86

Digital Library

[8]

Franklin J (2011) National Hurricane Center Forecast Verification. National Hurricane Center. http://www.nhc.noaa.gov/verification/index.shtml

[9]

Ho S-S, Talukder A (2008) Automated cyclone discovery and tracking using knowledge sharing in multiple heterogeneous satellite data. In: International Conference on Knowledge Discovery and Data Mining, Las Vegas, Nevada, USA, pp 928---936

[10]

Ho S-s, Talukder A (2009) Utilizing spatio-temporal text information for cyclone eye annotation in satellite data, IJCAI, Pasadena, CA, 13 July, 2009. Paper presented at the Workshop on Cross-Media Information Access and Mining, International Joint Conferences on Artificial Intelligence, Pasadena, CA, USA, July 13, 2009

[11]

Hongeng S, Bremond F, Nevatia R (June 13---15 2000) Representation and Optimal Recognition of Human Activities. In: Conference on Computer Vision and Pattern Recognition (CVPR), Hilton Head, SC, USA. pp 1818---1825

[12]

Hwang T-H, Choi K-H, Joo I-H, Lee J-H (21---25 July 2003) MPEG-7 metadata for video-based GIS applications. In: Geoscience and Remote Sensing Symposium, 2003. IGARSS `03. Proceedings. 2003 IEEE International, pp 3641---3643 vol.3646

[13]

Khalid S, Naftel A (2005) Evaluation of Matching Metrics for Trajectory-based Indexing and Retrieval of Video Clips. Paper presented at the Seventh IEEE Workshop on Applications of Computer Vision (WACV/MOTION)

[14]

Kim S-S, Park J-H (2004) Linking geographic hypermedia using the remotely sensed data. In: Multimedia and Expo, 2004. ICME `04. 2004 IEEE International Conference on, 30---30 June 2004. pp 811---814 Vol.812

[15]

Koprinska I, Carrato S (2001) Temporal video segmentation: a survey. Signal Process Image Comm 16(5):477---500

[16]

Li Z, Tan Y-P (2005) Event detection using multimodal feature analysis. In: Circuits and Systems, 2005. ISCAS 2005. IEEE International Symposium on, 23---26 May 2005, pp 3845---3848 Vol. 3844

[17]

Lippman A (1980) Movie-maps: an application of the optical videodisc to computer graphics. SIGGRAPH Comput Graph 14(3):32---42

Digital Library

[18]

Mazzoni D, Garay MJ, Davies R, Nelson D (2007) An operational MISR pixel classifier using support vector machines. Remote Sens Environ 107:149---158

[19]

Medioni G, Cohen I, Bremond F, Hongeng S, Nevatia R (2001) Event detection and analysis from video streams. IEEE Trans Pattern Anal Mach Intell 23(8):873---889

Digital Library

[20]

Moncrieff S, Dorai C, Venkatesch S (2001) Detecting indexical signs in film audio for scene interpretation. In: Multimedia and Expo, 2001. ICME 2001. IEEE International Conference on, 22---25 Aug. 2001, pp 989---992

[21]

Navarrete T, Blat J (2006) A semantic approach for the indexing and retrieval of geo-referenced video. Paper presented at the International Semantic Web Conference (ISWC)

[22]

Nguyen NT, Phung DQ, Venkatesh S, Bui H (20---25 June 2005) Learning and detecting activities from movement trajectories using the hierarchical hidden Markov model. In: Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, pp 955---960 vol. 952

[23]

Nobre EMN, Camara AS (2002) SPATIAL VIDEO: exploring space using multiple digital videos. Paper presented at the Proceedings of the sixth Eurographics workshop on Multimedia 2001, Manchester, UK

[24]

Oliver N, Horvitz E, Garg A (October 2002) Layered representations for recognizing office activity. In: Fourth IEEE International Conference on Multimodal Interaction, Pittsburgh, PA, USA, pp 3---8

[25]

Panangadan A, Ho S-S, Talukder A (November 4---6 2009) Cyclone tracking using multiple satellite image sources. In: 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Seattle

[26]

Pasch RJ, Stewart SR, Brown DP (2003) Comments on "early detection of tropical cyclones using seawinds-derived vorticity". Bull Am Meteorol Soc 85(10):1415---1416

[27]

Peuquet DJ, Duan N (1995) An event-based spatiotemporal data model (ESTDM) for temporal analysis of geographical data. Int J Geogr Inform Sci 9(1):7---24

[28]

Rath TM, Manmatha R (2003) Word image matching using dynamic time warping. In: Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on, 18---20 June 2003, pp II-521-II-527 vol.522

[29]

Rhome JR (2009) Technical Summary of the National Hurricane Center Track and Intensity Models. National Hurricane Center. http://www.nhc.noaa.gov/modelsummary.shtml

[30]

Rui Y, Gupta A, Acero A (2000) Automatically extracting highlights for TV Baseball programs. Paper presented at the Proceedings of the eighth ACM international conference on Multimedia, Marina del Rey, California, United States

[31]

Sakaki T, Okazaki M, Matsuo Y (2010) Earthquake shakes Twitter users: real-time event detection by social sensors. Proceedings of the 19th international conference on World wide web. ACM, Raleigh, North Carolina, USA

[32]

Stefanidis A, Eickhorst K, Agouris P, Partsinevelos P (2003) Modeling and comparing change using spatiotemporal helixes. In: 11th ACM international symposium on Advances in geographic information systems (GIS `03). ACM, New York, NY, USA, pp 86---93

[33]

Wei H, Xiong J, Yan F (2006) Management of Remote Sensing Data and Multimedia Data for Disaster Monitoring. In: Geoscience and Remote Sensing Symposium, 2006. IGARSS 2006. IEEE International Conference on, July 31 2006-Aug. 4 2006. pp 835---837

[34]

Westerman U, Jain R (2007) Toward a common event model for multimedia applications. IEEE Multimed 4(1):19---29

[35]

Wimmers A, Velden CS (2004) Satellite-based center-fixing of tropical cyclones: new automated approaches. In: 26th Conference on Hurricanes and Tropical Meteorology

[36]

Wulder M, White J, McDonald S (2005) Truth in the air: geolocated video validates satellite land cover. GEO World. November

[37]

Zeinalipour-Yazti D, Lin S, Gunopulos D (2006) Distributed Spatio-Temporal Similarity Search. Paper presented at the 15th ACM Conference on Information and Knowledge Management, Arlington, VA, USA, November 6---11

[38]

Zhai Y, Tian Y-L, Hampapur A (2006) Composite spatio-temporal event detection in multi-camera surveillance networks. In: Semantic Learning Applications in Multimedia

[39]

Zotkin D, Duraiswami R, Davis LS (2001) Multimodal 3-D tracking and event detection via the particle filter. In: Detection and Recognition of Events in Video, 2001. Proceedings. IEEE Workshop on, 2001, pp 20---27

Cited By

Li YLiu YYu RZong HXie W(2023)Dual attention based spatial-temporal inference network for volleyball group activity recognitionMultimedia Tools and Applications10.1007/s11042-022-13867-z82:10(15515-15533)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s11042-022-13867-z
Singh VSharma GKumar M(2017)Compressed sensing based acoustic event detection in protected area networks with wireless multimedia sensorsMultimedia Tools and Applications10.1007/s11042-016-4241-176:18(18531-18555)Online publication date: 1-Sep-2017
https://dl.acm.org/doi/10.1007/s11042-016-4241-1
Chen XZhan YKe JChen X(2016)Complex video event detection via pairwise fusion of trajectory and multi-label hypergraphsMultimedia Tools and Applications10.1007/s11042-015-2514-875:22(15079-15100)Online publication date: 1-Nov-2016
https://dl.acm.org/doi/10.1007/s11042-015-2514-8

Index Terms

Extreme event detection and assimilation from multimedia sources
1. Applied computing
  1. Physical sciences and engineering
    1. Earth and atmospheric sciences
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Scene understanding

Index terms have been assigned to the content through auto-classification.

Recommendations

Event-centric multisource stream processing and multimedia assimilation for geospatiotemporal phenomena
EiMM '10: Proceedings of the 2nd ACM international workshop on Events in multimedia

A new event-based multimedia stream data processing and representation solution is presented for large scale geographically distributed phenomena. Wide area geospatiotemporal patterns, such as cyclones or oil spills, have unique attributes and are ...
Research on Content-Based Multimedia Information Retrieval
ICCIS '11: Proceedings of the 2011 International Conference on Computational and Information Sciences

This paper firstly studies the features of content-based multimedia information retrieval, then analyzes the system structure of content-based multimedia information retrieval and retrieval procedures, lastly the author discusses the exist limitations ...
Information assimilation framework for event detection in multimedia surveillance systems

Most multimedia surveillance and monitoring systems nowadays utilize multiple types of sensors to detect events of interest as and when they occur in the environment. However, due to the asynchrony among and diversity of sensors, information ...

Comments

Information & Contributors

Information

Published In

cover image Multimedia Tools and Applications

Multimedia Tools and Applications Volume 70, Issue 1

May 2014

586 pages

ISSN:1380-7501

Issue’s Table of Contents

Copyright © Copyright © 2014 Springer Science+Business Media New York.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 May 2014

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li YLiu YYu RZong HXie W(2023)Dual attention based spatial-temporal inference network for volleyball group activity recognitionMultimedia Tools and Applications10.1007/s11042-022-13867-z82:10(15515-15533)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s11042-022-13867-z
Singh VSharma GKumar M(2017)Compressed sensing based acoustic event detection in protected area networks with wireless multimedia sensorsMultimedia Tools and Applications10.1007/s11042-016-4241-176:18(18531-18555)Online publication date: 1-Sep-2017
https://dl.acm.org/doi/10.1007/s11042-016-4241-1
Chen XZhan YKe JChen X(2016)Complex video event detection via pairwise fusion of trajectory and multi-label hypergraphsMultimedia Tools and Applications10.1007/s11042-015-2514-875:22(15079-15100)Online publication date: 1-Nov-2016
https://dl.acm.org/doi/10.1007/s11042-015-2514-8

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents