International Journal of Computer Applications (0975 – 8887)

International Conference on Recent Trends & Advancements in Engineering Technology (ICRTAET 2015)

Database Migration from Structured Database to non-

Structured Database

Mital Potey Megha Digrase Gaurav Deshmukh Minal Nerkar

Department of Department of Department of Department of
Computer Engg., Computer Engg., Computer Engg., Computer Engg.,
AISSMS’s IOIT, AISSMS’s IOIT, AISSMS’s IOIT, AISSMS’s IOIT,
Pune – 411001 Pune – 411001 Pune – 411001 Pune – 411001
India India India India

ABSTRACT To get a gist of working of the database migration, the

Database are growing very fast and becoming more complex migration program will be written in Python [1] [2] language
in the volume (terabyte to petabyte), variety (structured, un- which will be deployed on Amazon Web Service EC2. Web
structured and hybrid), and velocity (high speed in growth). interface will be developed using Django Framework for
Management of database (BigData, so nomenclature) has uploading and downloading the converted database. Parallel
become the global challenge. The data collection is currently computing is used to enhance the speed of computing.
managed and exploited mostly by using conventional data his paper will give a brief introduction to all the tools and
management tools such as classic relational database programming language used for database migration. It also
management systems (RDBMS) or conventional search talks about an optimization method to increase the speed of
engines. These systems and tools have been observed to be migrating huge data from one format to another.
inefficient and incapable to handle the data in most of the
cases e.g.: social networking sites. Efforts are being made to
come up with some utility support in order to overcome the
observed limitations. The research study was conducted to
develop a Cloud-based Utility. It was aimed to convert
structural database, SQL to un-structural database, NoSQL
[11] systematically and sufficiently. The outcome of the study
revealed that the database management becomes considerably
scalable, flexible and efficient when traditional RDBMS are
complimented by specifically designed rich set of alternative
DBMS such as NoSQL, NewSQL based systems. The
database conversion shall help in Database analysis and Fig 1: Problem Statement: Database Conversion from
migration from traditional database to today’s BigData. SQL to NoSQL
General Terms Problem Definition
SQl, NoSQL, Database, Python, Django, BigData. There's nothing wrong with the traditional RDBMS. It simply
isn’t enough for the application dealing with huge databases.
1. INTRODUCTION Also, NoSQL databases need cheap hardware. Hence, some of
Many organizations collect vast amounts of customer, the MySQL databases need to be converted to NoSQL
scientific, sales, and other data for future analysis. databases. Many institutions and big companies need to
Traditionally, most of these organizations have stored convert their big databases (MySQL).Our utility will help
structured data in Relational Databases for subsequent access them to convert from MySQL to NoSQL [9] [10] [12].
and analysis. However, a growing number of developers and
users have begun turning to various types of Non-Relational
now frequently called NoSQL databases.
2. RELATED WORK
Last decade records a huge amount of data transition taking
Relational databases such as MySQL has predefined schemas place on the internet. Today’s world is connected by social
fixed table names and types of columns and Relational networking sites. Such sites have huge sizes of databases. Our
databases don’t work easily in a distributed manner because review in this area shows that not many approaches that
joining their tables across a distributed system is difficult. provide tools or utilities for database conversion.
NoSQL databases have dynamic schemas. NoSQL generally Ton Blankers in [3] proposed discusses trends, challenges and
process data faster than relational databases. In most typical solutions of data conversion in application modernization
situations, SQL databases are vertically scalable. You can projects. An utility for converting one SQL database into
manage increasing load by increasing the CPU, RAM, SSD, another SQL database was proposed in this paper.
etc. on a single server. On the other hand, NoSQL databases
The white paper published by Oracle [4] mentions techniques
are horizontally scalable. You can just add few more servers
and utilities for migrating non oracle databases to oracle
easily in your NoSQL database infrastructure to handle the
databases.
large traffic.

1
 International Journal of Computer Applications (0975 – 8887)
International Conference on Recent Trends & Advancements in Engineering Technology (ICRTAET 2015)

Abdelsalam Maatuk in [5] describes an investigation into once the database has been uploaded, the server remains free
approaches and techniques used for database conversion. for fetching of another database. Also, once the database has
Constructing object views on top of a Relational DataBase been uploaded, the speed of the conversion increase
(RDB), simple database integration and database migration dramatically as the database is accessed by the server from the
are among these approaches. server and not over the internet.
Andreas Meier in [6] reviews and describes data and code
conversion to assist present applications in running entirely on Database Conversion
new generation database technology. This module comprises of the main logic of the Database
A B M Moniruzzaman in [7] aims at providing - conversion.
classification, characteristics and evaluation of NoSQL It is mainly divided in following step:
databases in Big Data Analytics. i)Database Validation : The first step of the conversion is to
Though there isn’t any available utility converting NoSQL validate the entered SQL database. This will be done by
databases into SQL databases, Sanobar Khan in [8] attempts checking the schema of the database and the relevant criteria
to use NoSQL database to replace the relational database. It of the selected SQL database.
mainly focuses on one of the boosting technology of NoSQL ii)Database Schema Extraction : The second step is to extract
database i.e. MongoDB, and makes a comparison with the schema of the database entered. This includes extracting
MySQL and thus justifies why MongoDB is preferred over the names of the tables, the respective columns, the datatypes
MySQL. of the columns, etc. This schema extraction will be provided
as an input for the formation of the NoSQL database.
Currently there is no utility present for converting SQL iii)Database Schema Creation : The third step is to create the
databases into NoSQL databases. As NoSQL is a new trend, schema for the new NoSQL database. The schema extracted
there are no software or utilities that are designed for the in the previous step is used for this purpose. The new NoSQL
desired conversion. Most of the utilities developed were database is formed according to the schema of the NoSQL
developed earlier for converting one form of SQL database to database selected by the user. This varies for every database.
another SQL database. For e.g. The conversion to MongoDB consists of considering
The significance of our methodology is that we provide an every row of the SQL database as a document and pushing it
online utility that helps to convert MySQL database to types in the MongoDB schema which is made by the extraction of
of NoSQL databases such as MongoDB. Not only is our the column names from the SQL database.
methodology according to today’s professional requirements iv)Database Deployment : The last step of the conversion
but is also launched using cloud services which offer other process is to deploy the converted database. The user will be
services like scalability, elasticity and high availability of able to download the converted database from the server once
data. the conversion has been completed. This allows for user to be
less dependent of continuous internet access. The user only
3. PROPOSED SYSTEM needs to have a continuous internet access for uploading the
In the previous section, we have reviewed the software database. He can download the converted database anytime
interface required for our utility to function. We have needed.
proposed a utility that will be working on Amazon cloud
utilizing the Amazon Cloud Services. It provides for an end 4. COMPARITIVE ANALYSIS AND
user to register; purchasing the subscription of the utility in RESULT
which the user can later upload his SQL database and get a Designing the above mentioned system, we did some
downloadable copy of the converted NoSQL version of his performance evaluation tests. Though the database sizes used
database. Proposed System is divided into 3 main for the analysis is comparatively smaller compared, we do get
steps/modules as follows. a clear difference in the various factors of comparison.
Module 1 is User Registration, We converted the some databases by manually extracting data
Module 2 is Database Uploading/Downloading, from SQL databases and then creating NoSQL databases. We
Module 3 is Database Conversion wherein lies the also converted the same databases using some of the methods
main logic of database conversion. using heuristic methods and coding. And later we tested our
utility’s performance under similar circumstances. The
User Registration following table describes the observed and analyzed results.
In this module the user is supposed to register for accessing Table 1. Performance evaluation between manual
the database conversion utility on Amazon Web service and approach, heuristic solutions and suggested approach
pay the relevant purchase amount as decided by the
organization. We will be providing a simple GUI for User Sr. Factors Manual Heuristic Suggested
Registration which will be designed in Django framework. No. Approach Solutions Approach
There will also be an administrator login for managing all the 1 Efficiency Less Good Very Good
registrations done on the server. 2 Human More Moderate Less
Effort
Database Uploading/Downloading 3 Resource Less Moderate High
In this module the user will be provided with a simple and Utilization
easy to understand GUI for uploading his SQL database and 4 Error More Less Less
downloading his converted NoSQL database. Scope
User has to enter in the type of database he will be uploading
and the database he wants his SQL database to be converted
into.
The uploading of the database facilitates the server freedom
and enhances the efficiency. This is due to the reason that

2
 International Journal of Computer Applications (0975 – 8887)
International Conference on Recent Trends & Advancements in Engineering Technology (ICRTAET 2015)

5. CONCLUSION [5] Relational Database Migration: A Perspective, School of

If computational and storage requirements of applications Computing, Engineering & Information Sciences,
such as for Big Data Analytics, Business Intelligence and Northumbria University, Newcastle upon Tyne, UK
social networking over peta-byte datasets have pushed sql-like [6] Providing Database Migration Tools A Practitioner’s
centralized databases to their limits. This led to the View
development of horizontally scalable, distributed non-
relational No-SQL databases. NoSQL databases are now [7] A B M Moniruzzaman and Syed Akhter Hossain, NoSQL
getting attention of application developers due to the Database: New Era of Databases for Big data Analytics -
following reasons: Classification, Characteristics and Comparison,
NoSQL databases provides schema-less dynamic flexible data International Journal of Database Theory and
model, that is most suitable for the big users and big data. Application Vol. 6, No. 4. 2013
NoSQL databases have an ability to scale dramatically to [8] Sanobar Khan and Prof.Vanita Mane, SQL Support over
support global users and big data. MongoDB using Metadata International Journal of
NoSQL databases provide an improved performance to satisfy Scientific and Research Publications, Volume 3, Issue
big user’s expectation without compromising scalability. 10, October 2013
Thus in our project NoSQL database approaches to supporting
applications that process huge volumes of data as well as to [9] R. Lawrence. Integration and virtualization of relational
provide a global overview of this non-relational NoSQL sql and nosql systems including mysql and mongodb. In
databases. Computational Science and Computational Intelligence
(CSCI), 2014 International Conference on, volume 1,
6. REFERENCES pages 285{290, March 2014.
[1] Multi-paradigm programming language. [10] N. Leavitt. Will nosql databases live up to their promise?
developer.mozilla.org, 21 October 2013. Computer, 43(2):12{14, Feb 2010.
[2] Python 3.4.1. Python Software Foundation, 21 May 2014. [11] Yishan Li and S. Manoharan. A performance comparison
[3] Tom Blankers, Uniface Client Manager, Data Conversion of sql and nosql databases. In Communications,
for Modern Applications, 2014 Computers and Signal Processing (PACRIM), 2013
IEEE Pacific Rim Conference on, pages 15-19, Aug
[4] Migrating Applications and Databases with Oracle 2013
Database 12c, An Oracle White Paper, June 2013

IJCATM : www.ijcaonline.org 3