Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Research on remote sensing image storage management and a fast visualization system based on cloud computing technology

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the continuous development of remote sensing technology, the data volume of remote sensing images has increased exponentially, resulting in many difficulties in the storage, management, transmission, calculation, and other processes of remote sensing images. In order to solve the above problems, this paper studies the use of the Hadoop Distributed File System (HDFS) and related technologies to design and implement a browser/server (B/S) architecture for a massive, multisource, remote sensing images distributed storage management system. The image data are stored in the HDFS, and the image metadata are stored in a MySQL database. The distributed parallel construction of the image pyramid is completed based on the Spark computing engine, and the Akka framework is used to construct WMTS (Web Map Tile Service) to realize the release of remote sensing images. Finally, the rapid visual display of remote sensing images is carried out using Leaflet. The system also supports image data management, image target detection, user management, and other functions. After testing, this system can support the storage and management of multisource remote sensing image data, and can solve perfectly the problems of insufficient storage space and insufficient computing power of a single server. It is found that the upload and download speeds of a large amount of remote sensing images can be close to the maximum speed of a gigabit local area network (LAN). In the gigabit LAN environment, the average upload speed of a single remote sensing image is 97.74 MB/s, and the average download speed is 87.62 MB/s. In terms of image pyramid construction, the speed of a multi-node parallel construction based on Spark is two times higher than that of a single-node construction. Additionally, compared to similar systems, this system has better data transmission and retrieval speed, better data computing ability, and higher concurrency processing ability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Due to the nature of this research, the participants of this study do not agree for their data to be shared publicly. Hence, supporting data are not available.

References

  1. Blaschke T (2010) Object based image analysis for remote sensing. ISPRS J Photogramm Remote Sens 65(1):2–16

    Article  Google Scholar 

  2. Deren LI, Liangpei Z, Guisong X (2014) Automatic analysis and mining of remote sensing big data. Acta Geodaetica Cartogr Sin 43(12):1211

    Google Scholar 

  3. Chi M, Plaza A, Benediktsson JA et al (2016) Big data for remote sensing: challenges and opportunities. Proc IEEE 104(11):2207–2219

    Article  Google Scholar 

  4. Huang YQ (2019) The concept and development trend of spatial database management system. China Manage Informationization 22(08):165–166

    Google Scholar 

  5. Lü XF, Cheng C, Gong J et al (2011) Review of data storage and management technologies for massive remote sensing data. Sci China Technol Sci 54:3220–3232

    Article  Google Scholar 

  6. Yan J, Liu Y, Wang L et al (2021) An efficient organization method for large-scale and long time-series remote sensing data in a cloud computing environment. IEEE J Sel Top Appl Earth Observ Remote Sens 14:9350–9363

    Article  Google Scholar 

  7. Wang L, Ma Y, Yan J et al (2018) PipsCloud: high performance cloud computing for remote sensing big data management and processing. Futur Gener Comput Syst 78:353–368

    Article  Google Scholar 

  8. Cheng Y, Zhou K, Wang J, Yan J (2020) Big earth observation data integration in remote sensing based on a distributed spatial framework. Remote Sens 12(6):972. https://doi.org/10.3390/rs12060972

  9. Jing W, Tian D (2018) An improved distributed storage and query for remote sensing data. Procedia Comput Sci 129:238–247

    Article  Google Scholar 

  10. Li J, Zhang P, Li Y et al (2017) A data-check based distributed storage model for storing hot temporary data. Futur Gener Comput Syst 73:13–21

    Article  Google Scholar 

  11. Zheng K, Fu Y (2013) Research on vector spatial data storage schema based on Hadoop platform. Int J Database Theory Appl 6(5):85–94

    Article  Google Scholar 

  12. Zhong Y, Sun S, Liao H, Zhao Y, Fang J (2011) A novel method to manage very large raster data on distributed key-value storage system. In: 2011 19th International Conference on Geoinformatics. Shanghai, China, pp 1–6. https://doi.org/10.1109/GeoInformatics.2011.5980711

  13. Rajak R, Raveendran D, Bh MC, Medasani SS (2015) High resolution satellite image processing using Hadoop framework. In: 2015 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM). Bangalore, India, pp 16–21. https://doi.org/10.1109/CCEM.2015.16

  14. Zhu J, Zhang Z, Zhao F et al (2023) Efficient management and scheduling of massive remote sensing image datasets. ISPRS Int J Geo-Information 12(5):199

    Article  Google Scholar 

  15. Zhou X, Wang X, Zhou Y et al (2021) Rsims: large-scale heterogeneous remote sensing images management system[J]. Remote Sens 13(9):1815

    Article  Google Scholar 

  16. Wang C, Hu F, Hu X et al (2015) A Hadoop-based distributed framework for efficient managing and processing big remote sensing images. ISPRS Ann Photogramm Remote Sens Spat Inf Sci 2:63–66

    Article  Google Scholar 

  17. Kong W, Wang T, Liu L et al (2023) A novel design and application of spatial data management platform for natural resources. J Clean Prod 411:137183

    Article  Google Scholar 

  18. Wei H, Yuheng Z (2017) The massive remote sensing data organization and management strategies//MATEC Web of Conferences. EDP Sciences 128:02011

    Google Scholar 

  19. Wang S, Li G, Yao X et al (2019) A distributed storage and access approach for massive remote sensing data in MongoDB. ISPRS Int J Geo-Information 8(12):533

    Article  Google Scholar 

  20. Rathore MM, Ahmad A, Paul A et al (2016) Urban planning and building smart cities based on the internet of things using big data analytics. Comput Netw 101:63–80

    Article  Google Scholar 

  21. Shan TJ, Zhong HW et al (2019) Building of remote sensing images tile pyramid based on Spark. Intell Comput Appl 9(04):226–229

    Google Scholar 

  22. Zaharia M, Xin RS, Wendell P et al (2016) Apache spark a unified engine for big data processing. Commun ACM 59(11):56–65

    Article  Google Scholar 

  23. Kini A, Emanuele R, Geotrellis (2014) Adding geospatial capabilities to Spark. In: Spark Summit 2014, from https://docs.huihoo.com/apache/spark/summit/2014/Geotrellis-Adding-Geospatial-Capabilities-to-Spark-Ameet-Kini-Rob-Emanuele.pdf

  24. Chen X, Zhang C, Ge B et al (2016) Efficient historical query in HBase for spatio-temporal decision support. Int J Comput Commun Control 11(5):613–630

    Article  Google Scholar 

  25. Jonasson M (2014) The Akka-board–performing mobility, disability and innovation. Disabil Soc 29(3):477–490

    Article  Google Scholar 

  26. Farkas G (2017) Applicability of open-source web mapping libraries for building massive web GIS clients. J Geogr Syst 19(3):273–295

    Article  Google Scholar 

  27. Huang F, Chen S, Wang Q et al (2023) Using deep learning in an embedded system for real-time target detection based on images from an unmanned aerial vehicle: vehicle detection as a case study[J]. Int J Digit Earth 16(1):910–936

    Article  Google Scholar 

  28. Zhu Q, Huang F, Lu J et al (2017) Research on the implementation of multi-source remote sensing image management system based on B/S architecture. 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). IEEE, pp 5233–5236

Download references

Acknowledgements

This study was mainly supported by the National Science Foundation of China (Grant No. 42271390) and the Technological Innovation R&D Project of Chengdu Science and Technology Bureau (Grant No. 2022-YF05-00967-SN). This work was partial funded by the Fundamental Research Funds for the Central Universities (Grant Nos. ZYGX2019J069 and ZYGX2019J072) and Hubei Provincial Key Laboratory of Intelligent Geo-information Processing (China University of Geosciences; Grant Nos. KLIGIP-2018A08).

Author information

Authors and Affiliations

  1. School of Transportation Science and Engineering, Beihang University, Beijing, 100191, China

    Lichun Yang

  2. Jiangsu Automation Research Institute, Lianyungang, Jiangsu Province, 222061, China

    Lichun Yang

  3. School of Recourses and Environment, University of Electronic Science and Technology of China (UESTC), Chengdu, Sichuan Province, 611731, China

    Weibing He, Xiaoyong Qiang, Jinjun Zheng & Fang Huang

Authors
  1. Lichun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weibing He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyong Qiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinjun Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lichun Yang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, L., He, W., Qiang, X. et al. Research on remote sensing image storage management and a fast visualization system based on cloud computing technology. Multimed Tools Appl 83, 59861–59886 (2024). https://doi.org/10.1007/s11042-023-17858-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17858-6

Keywords

Search

Navigation