Abstract
Migration is a complex process involving many challenges while migrating from an existing system to a new one. Database migration involves schema transformation, migration of data, complex query support, and indexing. This paper presents a) Journey on existing migration techniques from RDBMS (SQL) to NoSQL databases. Schema migration and Data migration are two main aspects while migrating from relational to NoSQL database. b) The various existing techniques related to schema migration and data migration. The techniques presented for schema migration include graph-based algorithms, denormalization techniques such as column level denormalization and table level denormalization, link list-based, and ETL tools. The paper also addresses different business methods for data migration, commonly known as basic tools and ETL tools. c) The authors highlight some open challenges like need and cost of denormalization and selection of columns for denormalization. The assessment of various techniques (based on space or time costs) is presented. The key challenge is to pick a particular datastore on which to apply a specific technique. d) The paper also describes current market-driven migration tools based on each particular data store. e) It also throws light on different organizations that successfully migrated to a particular NoSQL data store. Thus, the purpose of this study is to contribute to the state-of-the-art in the field of database migration and to serve as a foundation for selecting and developing RDBMS-to-NoSQL data migration techniques or tools .
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Floratou, A., Teletia, N., DeWitt, D.J., Patel, J.M., Zhang, D.: Can the elephants handle the NoSQL onslaught? Proc. VLDB Endow. 5(12) (2012). https://doi.org/10.14778/2367502.2367511
Özcan, F., et al.: Are we experiencing a big data bubble? In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, pp. 1407–1408 (2014). https://doi.org/10.1145/2588555.2618215
Zhao, G., Lin, Q., Li, L., Li, Z.: Schema conversion model of SQL database to NoSQL. In: Ninth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing, IEEE, pp. 355–362 (2014). https://doi.org/10.1109/3PGCIC.2014.137
Rith, J., Lehmayr, P.S., Meyer-Wegener, K.: Speaking in tongues: SQL access to NoSQL systems. In: Proceedings of the 29th Annual ACM Symposium on Applied Computing, ACM (2014). https://doi.org/10.1145/2554850.2555099
Yoo, J., Lee, K., Jeon, Y.-H.: Migration from RDBMS to NoSQL using column-level denormalization and atomic aggregates, J. Inf. Sci. Eng. 34(1) (2018). https://doi.org/10.1007/978-981-10-6520-0_3
Kim, H.-J., Ko, E.-J., Jeon, Y.-H., Lee, K.-H.: Techniques and guidelines for effective migration from RDBMS to NoSQL. J. Supercomput. 76(10), 7936–7950 (2018). https://doi.org/10.1007/s11227-018-2361-2
Davoudian, A., Chen, L., Liu, M.: A survey on NoSQL stores, ACM Comput. Surv. 51(2), Article 40, 43 p (2018). https://doi.org/10.1145/3158661
Kim, H.-J., Ko, E.-J., Jeon, Y.-H., Lee, K.-H.: Migration from RDBMS to column-oriented NoSQL: lessons learned and open problems. In: Lee, W., Choi, W., Jung, S., Song, M. (eds.) Proceedings of the 7th International Conference on Emerging Databases. LNEE, vol. 461, pp. 25–33. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-6520-0_3
Sun, W., Fokoue, A., Srinivas, K., Kementsietsidis, A., Hu, G., Xie, G.: Sqlgraph: an efficient relational-based property graph store, In: Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, ACM, May 2015, pp. 1887–1901. https://doi.org/10.1145/2723372.2723732
RDBMS to MongoDB migration guide, MongoDB white paper (2018). https://www.mongodb.com/collateral/rdbms-mongodb-migration-guide. Accessed 10 May 2020
Kjellman, M.: MySQL to cassandra migrations/ https://academy.datastax.com/planet-cassandra/mysql-to-cassandra-migration. Accessed 10 May 2020
Neo4j: Import: RDBMS to Graph. https://neo4j.com/developer/relational-to-graph-import. Accessed 10 May 2020
Redisco 0.1.4, Python Containers and Simple Models for Redis. https://pypi.org/project/redisco/. Accessed 10 May 2020
Chasseur, C., Li, Y., Patel, J.M.: Enabling JSON document stores in relational systems, In: WebDB, vol. 13, pp. 14–15 (2013). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.363.7814
Lee, C.-H., Zheng, Y.-L.: Automatic SQL-to-NoSQL schema transformation over the MySQL and HBase databases. In: International Conference on Consumer Electronics-Taiwan, IEEE (2015). https://doi.org/10.1109/ICCE-TW.2015.7216979
Hanine, M., Bendarag, A., Boutkhoum, O.: Data migration methodology from relational to NoSQL databases, world academy of science, engineering, and technology. Int. J. Comput. Elect. Autom. Control Inf. Eng. 9, 12 (2016). https://doi.org/10.5281/zenodo.1339211
Oliveira, F., Oliveira, A., Alturas, B.: Migration of relational databases to NoSQL-methods of analysis, Mediterr. J. Soc. Sci. 9(2) (2018). https://doi.org/10.2478/mjss-2018-0042
Sullivan, D.: NoSQL for Mere Mortals, 1st edn. Pearson Education, Boston (2015)
Redislabs, Redis Documentation. https://redis.io/documentation. Accessed 11 May 2020
Memcached, Documentation. https://memcached.org. Accessed 12 May 2020
Riak, Riak Products. https://riak.com/products/. Accessed 15 May 2020
Apache Cassandra: Apache Cassandra Documentation v4.0-alpha3. http://cassandra.apache.org/doc/. Accessed 15 Mar 2020
Apache HBase: Welcome to Apache HBase. https://hbase.apache.org/. Accessed 15 Mar 2020
Scylla: The World's Fastest NoSQL Database. https://www.scylladb.com/product/. Accessed 17 Mar 2020
Apache CouchDB: Apache CouchDB® 2.3.1 Documentation, http://docs.couchdb.org/en/stable/. Accessed 17 Mar 2020
MongoDB, Introduction to MongoDB. https://www.mongodb.com/what-is-mongodb. Accessed 17 Mar 2020
Amazon DynamoDB, Amazon DynamoDB Fast and flexible NoSQL database service for any scale. https://aws.amazon.com/dynamodb/. Accessed 17 Mar 2020.
Neo4j, The Internet-Scale Graph Platform. https://neo4j.com/product/. Accessed 17 Mar 2020
AllegroGraph: AllegroGraph - The Enterprise Knowledge Graph. https://allegrograph.com/. Accessed 10 Mar 2020
Couchbase: Data Model – Mapping from MySQL to CouchbaseServer. https://docs.couchbase.com/server/4.1/install/migrate-mysql.html. Accessed 10 Mar 2020
Lee, Y.S.: Near zero downtime migration from MySQL to DynamoDB, 18 April 2018. https://aws.amazon.com/blogs/big-data/near-zero-downtime-migration-from-mysql-to-dynamodb/. Accessed 8 Mar 2020
Riak: Moving from MySQL to Riak, 23 April 2014. https://riak.com/posts/technical/moving-from-mysql-to-riak/. Accessed 07 Mar 2020
MongoDB: Flexible enough to fit any industry. https://www.mongodb.com/who-uses-mongodb
Neo4j: The World of Graphs Powered by Neo4j. https://neo4j.com/customers/. Accessed 10 Feb 2020
Apache Cassandra: MySQL to Cassandra Migration. https://academy.datastax.com/planet-cassandra/mysql-to-cassandra-migration. Accessed 10 Feb 2020
DB engines: DB-Engine Ranking. https://db-engines.com/en/ranking. Accessed 1 Feb 2020
Schreiner, G.A., Duarte, D., Mello, R.d.S.: When relational-based applications go to nosql databases: a survey. Information 10(7), 241 (2019). https://doi.org/10.3390/info10070241
Abadi, D., et al.: The Seattle report on database research. ACM SIGMOD Record. 48(4), 44–53 (2020). https://doi.org/10.1145/3385658.3385668
Jia, T., Zhao, X., Wang, Z., Gong, D., Ding, G.: Model transformation and data migration from relational database to MongoDB. In: IEEE International Congress on Big Data (BigData Congress) 2016 June 27, pp. 60–67. IEEE (2016). https://doi.org/10.1109/BigDataCongress.2016.16
Karnitis, G., Arnicans, G.: Migration of relational database to document-oriented database: structure denormalization and data transformation. In: 7th International Conference on Computational Intelligence, Communication Systems and Networks 2015 June 3, pp. 113–118. IEEE (2015). https://doi.org/10.1109/CICSyN.2015.30
Wang, Y., Shah, R., Criswell, A., Pan, R., Dillig, I.: Data migration using Datalog program synthesis. Proc. VLDB Endow. 13(7) (2020). https://doi.org/10.14778/3384345.3384350
Serrano, D., Stroulia, E.: From relations to multidimensional maps: a SQL-to-hbase transformation methodology. In: Proceedings of the 26th Annual International Conference on Computer Science and Software Engineering, pp. 156–165, October 2016. https://doi.org/10.5555/3049877.3049893
Unal, Y., Oguztuzun, H.: Migration of data from relational database to graph database. In: Proceedings of the 8th International Conference on Information Systems and Technologies, pp. 1–5 (2018). https://doi.org/10.1145/3200842.3200852
Fouad, T., Mohamed, B.: Model transformation from object-relational database to NoSQL column based database. In: Proceedings of the 3rd International Conference on Networking, Information Systems & Security, pp. 1–5, March 2020. https://doi.org/10.1145/3386723.3387881
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Bansal, N., Soni, K., Sachdeva, S. (2022). Journey of Database Migration from RDBMS to NoSQL Data Stores. In: Sachdeva, S., Watanobe, Y., Bhalla, S. (eds) Big-Data-Analytics in Astronomy, Science, and Engineering. BDA 2021. Lecture Notes in Computer Science(), vol 13167. Springer, Cham. https://doi.org/10.1007/978-3-030-96600-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-96600-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-96599-0
Online ISBN: 978-3-030-96600-3
eBook Packages: Computer ScienceComputer Science (R0)