research-article

The data partition strategy based on hybrid range consistent hash in NoSQL database

Authors:

Huiyu YangAuthors Info & Claims

ICIMCS '13: Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service

Pages 358 - 363

https://doi.org/10.1145/2499788.2499812

Published: 17 August 2013 Publication History

Abstract

With the development of Internet technology and Cloud Computing, more and more applications have to be confronted with the challenges of big data. NoSQL Database is fit to the management of big data because of the characteristics of high scalability, high availability and high fault-tolerance. The data partitioning strategy plays an important role in the NoSQL database. The existing data partitioning strategies will cause some problems such as low scalability, hot spot and low performance and so on. In this paper we proposed a new data partitioning strategy---HRCH, which can partitioning the data in a reasonable way. At last we use some experiments to verify the effectiveness of HRCH. It shows that the HRCH can improve the scalability of the system. It also can avoid the hot spot problem as far as possible. And it also can improve the parallel degree of processing to improve the system's performance in some processing.

References

[1]

Big data. http://en.wikipedia.org/wiki/Big_data

[2]

Grobelnik M. Big-data computing: Creating revolutionary breakthroughs in commerce, science, and society. http://videolectures.net/eswc2013_gobelnik_big_data/

[3]

Barwick H. The "four Vs" of Big Data. Implementing Information Infrastructure Symposium. http://www.computerworld.com.au/article/396198/iiis_four_vs_big_data/

[4]

What is big data?. http://www-01.ibm/software/data/bigdata/

[5]

NoSQL Database. http://en.wikipedia.org/wiki/NoSQL

[6]

Chang F, Dean J, Ghemawat S, et al. Bigtable: A distributed storage system for structured data {J}. ACM Transactions on Computer Systems (TOCS), 2008, 26(2): 4.

Digital Library

[7]

HBase Development Team. HBase: BigTable-like structured storage for Hadoop HDFS. http://wiki.apache.org/hadoop/Hbase/.

[8]

Lakshman A, Malik P. Cassandra: A structured storage system on a P2P network{C}//Proceedings of the twenty-first annual symposium on Parallelism in algorithms and architectures. ACM, 2009: 47--47.

Digital Library

[9]

DeCandia G, Hastorun D, Jampani M, et al. Dynamo: amazon's highly available key-value store{C}//ACM Symposium on Operating Systems Principles: Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles. 2007, 14(17): 205--220.

Digital Library

[10]

Cooper B F, Ramakrishnan R, Srivastava U, et al. PNUTS: Yahoo!'s hosted data serving platform {J}. Proceedings of the VLDB Endowment, 2008, 1(2): 1277--1288.

Digital Library

[11]

Ghandeharizadeh S, DeWitt D J. Hybrid-range partitioning strategy: A new declustering strategy for multiprocessor database machines{C}//Proceedings of the 16th International Conference on Very Large Data Bases. Morgan Kaufmann Publishers Inc., 1990: 481--492.

Digital Library

[12]

Nishimura S, Das S, Agrawal D, et al. MD-HBase: a scalable multi-dimensional data infrastructure for location aware services{C}//Mobile Data Management (MDM), 2011 12th IEEE International Conference on. IEEE, 2011, 1: 7--16.

Digital Library

[13]

Scholl T, Kuntschke R, Reiser A, et al. Community training: Partitioning schemes in good shape for federated data grids{C}//e-Science and Grid Computing, IEEE International Conference on. IEEE, 2007: 195--203.

Digital Library

[14]

Curino C, Jones E P C, Popa R A, et al. Relational cloud: A database-as-a-service for the cloud {J}. 2011.

[15]

Zhou J, Bruno N, Lin W. Advanced partitioning techniques for massively distributed computation{C}//Proceedings of the 2012 international conference on Management of Data. ACM, 2012: 13--24.

Digital Library

[16]

Liroz-Gistau M, Akbarinia R, Pacitti E, et al. Dynamic Workload-Based Partitioning for Large-Scale Databases{C}//Database and Expert Systems Applications. Springer Berlin Heidelberg, 2012: 183--190.

[17]

Karger D, Lehman E, Leighton T, et al. Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the World Wide Web{C}//Proceedings of the twenty-ninth annual ACM symposium on Theory of computing. ACM, 1997: 654--663.

Digital Library

[18]

Common Address Redundancy Protocol. http://en.wikipedia.org/wiki/Common_Address_Redundancy_Protocol.

[19]

Cooper B F, Silberstein A, Tam E, et al. Benchmarking cloud serving systems with YCSB{C}//Proceedings of the 1st ACM symposium on Cloud computing. ACM, 2010: 143--154.

Digital Library

Cited By

Liao XLuo CLu PXiao PZhu ZLei M(2023)Design and Research of the Whole Process Non-blocking Technology in High Concurrency Scenario2023 8th International Conference on Computer and Communication Systems (ICCCS)10.1109/ICCCS57501.2023.10150980(49-55)Online publication date: 21-Apr-2023
https://doi.org/10.1109/ICCCS57501.2023.10150980
Castillo-García ARodríguez-Mazahua LCastro-Medina FArrioja-Rodríguez MSánchez-Cervantes JRodríguez-Mazahua N(2022)Design of a Dynamic Horizontal Fragmentation Method for Multimedia DatabasesHandbook on Decision Making10.1007/978-3-031-08246-7_4(71-91)Online publication date: 27-Sep-2022
https://doi.org/10.1007/978-3-031-08246-7_4
Castillo-García ARodríguez-Mazahua LCastro-Medina FOlivares-Zepahua BAbud-Figueroa M(2021)A Review of Horizontal Fragmentation Methods Considering Multimedia Data and Dynamic Access PatternsNew Perspectives in Software Engineering10.1007/978-3-030-89909-7_6(69-82)Online publication date: 17-Oct-2021
https://doi.org/10.1007/978-3-030-89909-7_6

Index Terms

The data partition strategy based on hybrid range consistent hash in NoSQL database
1. Information systems
  1. Data management systems
    1. Database design and models
      1. Data model extensions
      2. Relational database model

Recommendations

Hybrid Range Consistent Hash Partitioning Strategy -- A New Data Partition Strategy for NoSQL Database
TRUSTCOM '13: Proceedings of the 2013 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications

With the development of Internet technology and Cloud Computing, more and more applications have to be confronted with the challenges of big data. NoSQL Database is fit to the management of big data because of the characteristics of high scalability, ...
Model Transformation From Object Relational Database to NoSQL Document Database
NISS '19: Proceedings of the 2nd International Conference on Networking, Information Systems & Security

With the high increase of data growing, NoSQL databases play a key role in storing large amount of data, to gain flexibility and scalability many industries are now replacing their relational and object relational databases by adopting NoSQL database ...
Big Data Management in Digital Forensics
CSE '14: Proceedings of the 2014 IEEE 17th International Conference on Computational Science and Engineering

The past few years have witnessed an exponential growth in the volume of data on digital forensics leading to big data issues. Digital forensics data is complex and heterogeneous in that it can be structured, unstructured and semi-structured. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICIMCS '13: Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service

August 2013

419 pages

ISBN:9781450322522

DOI:10.1145/2499788

Conference Chair:
Tat-Seng Chua
National University of Singapore, Singapore
,
General Chairs:
Ke Lu
University of Chinese Academy of Sciences, China
,
Tao Mei
Microsoft Research Asia, China
,
Xindong Wu
Hefei University of Technology, China

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

NSF of China: National Natural Science Foundation of China
University of Sciences & Technology, Hefei: University of Sciences & Technology, Hefei
Beijing ACM SIGMM Chapter

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 August 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
Ministry of Science and Technology of the People's Republic of China
National 242 Information Security Program of China

Conference

ICIMCS '13

Sponsor:

NSF of China
University of Sciences & Technology, Hefei

ICIMCS '13: International Conference on Internet Multimedia Computing and Service

August 17 - 19, 2013

Huangshan, China

Acceptance Rates

ICIMCS '13 Paper Acceptance Rate 20 of 94 submissions, 21%;

Overall Acceptance Rate 163 of 456 submissions, 36%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
461
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liao XLuo CLu PXiao PZhu ZLei M(2023)Design and Research of the Whole Process Non-blocking Technology in High Concurrency Scenario2023 8th International Conference on Computer and Communication Systems (ICCCS)10.1109/ICCCS57501.2023.10150980(49-55)Online publication date: 21-Apr-2023
https://doi.org/10.1109/ICCCS57501.2023.10150980
Castillo-García ARodríguez-Mazahua LCastro-Medina FArrioja-Rodríguez MSánchez-Cervantes JRodríguez-Mazahua N(2022)Design of a Dynamic Horizontal Fragmentation Method for Multimedia DatabasesHandbook on Decision Making10.1007/978-3-031-08246-7_4(71-91)Online publication date: 27-Sep-2022
https://doi.org/10.1007/978-3-031-08246-7_4
Castillo-García ARodríguez-Mazahua LCastro-Medina FOlivares-Zepahua BAbud-Figueroa M(2021)A Review of Horizontal Fragmentation Methods Considering Multimedia Data and Dynamic Access PatternsNew Perspectives in Software Engineering10.1007/978-3-030-89909-7_6(69-82)Online publication date: 17-Oct-2021
https://doi.org/10.1007/978-3-030-89909-7_6

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents