Cs9152 Unit I

CS9152 - DATABASE TECHNOLOGY
UNIT I
CS9152 DATABASE
TECHNOLOGY
UNIT I
DISTRIBUTED DATABASES
TEXT BOOK
1. Elisa Bertino, Barbara Catania, Gian Piero Zarri, Intelligent Database Systems,
Addison-Wesley, 2001.
REFERENCES
1. Carlo Zaniolo, Stefano Ceri, Christos Faloustsos, R.T.Snodgrass, V.S.Subrahmanian,
Advanced Database Systems, Morgan Kaufman, 1997.
2. N.Tamer Ozsu, Patrick Valduriez, Principles of Distributed Database Systems,
Prentice Hal International Inc. , 1999.
3. C.S.R Prabhu, Object-Oriented Database Systems, Prentice Hall Of India, 1998.
4. Abdullah Uz Tansel Et Al, Temporal Databases: Theory, Design And
Principles,Benjamin Cummings Publishers , 1993.
5. Raghu Ramakrishnan, Johannes Gehrke, Database Management Systems, Mcgraw
Hill, Third Edition, 2004.
6. Henry F Korth, Abraham Silberschatz, S. Sudharshan, Database System Concepts,
Fourth Ediion, McGraw Hill , 2002.
7. R. Elmasri, S.B. Navathe, Fundamentals of Database Systems, Pearson Education,
2004.
UNIT I
Syllabus:
UNIT I
5
Distributed Databases Vs Conventional Databases Architecture
Fragmentation Query Processing Transaction Processing Concurrency
Control Recovery.
Table of Contents
SL No. Topic
1
Introduction to Distributed Databases
2
Distributed Databases Vs Conventional
Databases
3
Architecture
4
Fragmentation
5
Query Processing
6
Transaction Processing
7
Concurrency Control
8
Recovery.
9
Sample Questions
10
University Questions
Page
2
8
9
15
29
35
38
43
49
51
UNIT I
Topic 1: Introduction to Distributed Databases

Distributed Databases- Definition
What is a distributed database?
A
logically interrelated collection of shared data (and a description of this data),

physically distributed over a computer network
(DDBMS) is the software that manages the DDB and provides an access
mechanism that makes this distribution transparent to the users.
A database that consists of two or more data files located at different sites on a
computer network. Because the database is distributed, different users can access it
without interfering with one another. However, the DBMS must periodically
synchronize the scattered databases to make sure that they all have consistent data.
DDBMS to Avoid `islands of information problem
A Distributed Database: is a logically interrelated collection of shared data (and
a description of this data), physically distributed over a computer network.
A Distributed DBMS (DDBMS): is a Software system that permits the
management of the distributed database and makes the distribution transparent to
users.
Distributed Database System
DBMS
DBMS
DBMS
DBMS
data
data
data
data
Fundamental Principle: make distribution transparent to user.
UNIT I
In distributed databases, data spread over multiple machines (also referred to

as sites or nodes.
Network interconnects the machines
Data shared by users on multiple machines
DDBMS has following characteristics:
Collection of logically-related shared data.

Data split into fragments.
Fragments may be replicated.
Fragments/replicas allocated to sites.
Sites linked by a communication network.
Data at each site is under control of a DBMS.
DBMSs handle local applications autonomously.
Each DBMS participates in at least one global application.
Important difference between DDBMS and distributed processing
UNIT I
Distributed processing of centralised DBMS
Distributed Processing
Much more tightly coupled than a DDBMS.

Database design is same as for standard DBMS
No attempt to reflect organizational structure
Much simpler than DDBMS
More secure than DDBMS
No local autonomy
Functions of a DDBMS
Expect DDBMS to have at least the functionality of a DBMS.

UNIT I
Also to have following functionality:
Extended communication services.
Extended Data Dictionary.
Distributed query processing.
Extended concurrency control.
Extended recovery services

Additional Functions Provided:
Ability to access remote sites and transmit queries and data among various
sites via a communication network.
Ability to track of the data distribution and replication in DDBMS catalog.
Ability to devise execution strategies for queries and transaction that access
data from more than one site.
Ability to decide on which copy of a replicated data item to access.
Ability to maintain the consistency of copies of a replicated data item.
Ability to recover from individual site crashes and from new types of
failures such as the failure of a communication link.
Hardware Level:
Multiple computers, called sites or nodes.
Communication network.
Local Area Network
Long-haul Network
Network Topologies
Advantages of DDBSs:
Distributed nature of some database applications

Local users
Global users
Increased reliability and availability
Reliability - Probability that a system is up at a particular moment.
Availability Probability that the system is continuously available
during a time interval.
Improvement is achieved by replicating data and software at more than
one site.
Allowing data sharing while maintaining some measure of
local control
Controlled sharing of data throughout the distributed system.
Improved Performance
Smaller database exists at a single site.
Accesses to more than one site proceed in parallel.
Reduced response time.
UNIT I
Advantages of distributed databases

Capacity and incremental growth
Increase reliability and availability
Modularity
Reduced communication overhead
Protection of valuable data
Efficiency and Flexibility
Reflects organizational structure
Improved shareability and local autonomy
Improved availability
Improved reliability
Improved performance
Economics
Modular growth
Disadvantages of distributed databases
Complexity
Cost
Security
Integrity control more difficult
Lack of standards
Lack of experience
Database design more complex or Increased complexity in system design
and implementation.
Applications of DDBMS:
Manufacturing - especially multi-plant manufacturing
Military command and control
Electronic fund transfers and electronic trading
Corporate MIS
Airline restrictions
Hotel chains
Any organization which has a decentralized organization structure
User access the distributed data via applications.
Two types of applications:
Logical Applications: Applications that do not required data from other
sites.
Physical Applications: Applications that required data from other sites.
UNIT I
Types of DDBMS
In a homogeneous distributed database:

All sites have identical software.
Are aware of each other and agree to cooperate in processing user
requests.
Each site surrenders part of its autonomy in terms of right to change
schemas or software.
Appears to user as a single system.
In a heterogeneous distributed database:

Different sites may use different schemas and software.
Difference in schema is a major problem for query processing.
Difference in software is a major problem for transaction
processing.
Sites may not be aware of each other and may provide only limited
facilities for cooperation in transaction processing.
UNIT I
Two main issues in DDBMS

Making query from one site to the same or remote site.
Logical database is partitioned in to different data streams and located at
different sites.
Topic 2: Distributed Databases Vs Conventional Databases
mimics organisational structure with data

local access and autonomy without exclusion
cheaper to create and easier to expand
improved availability/reliability/performance by removing reliance on
a central site
Reduced communication overhead
Most data access is local, less expensive and performs better
Improved processing power
Many machines handling the database rather than a single server
more complex to implement
UNIT I
more costly to maintain

security and integrity control
standards and experience are lacking
Design issues are more complex
Centralized Database System:

All system components (data. DBMS software, secondary storage devices
and tapes for backup) reside at a single site.
Remote access via terminals connected to the site is possible.
Distributed Database:
Physically spread over the sites of a computer network.

Communication network
Distributed Database Systems (DDBSs)
Distributed Database Management System (DDBMS)
Topic 3: Distributed Databases Architecture

Overview of Client-Server Architecture
3 levels of DDBMS software modules:
Client - Applicatrion Processor (AP) or front-end machine
Server Database Processor (DP) or back-end machine
Communications software
Reference to DDBMS catalog by client
Query processing:
Client parses a query and decomposes it into a number of independent
site queries. Each site query is sent to the appropriate server site.
Each server processes the local query and sends the resulting relation to
the client site.
Client site combines the results of the subqueries to produce the result
of the originally submitted query.
Overview of Distributed Database Architecture
Location Transparency
User does not have to know the location of the data.
Data requests automatically forwarded to appropriate sites
Local Autonomy
UNIT I
Local site can operate with its database when network connections fail
Each site controls its own data, security, logging, recovery
Synchronous Distributed Database
All copies of the same data are always identical
Data updates are immediately applied to all copies throughout network
Good for data integrity
High overhead slow response times
Advantages
Increased reliability & availability
Local control
Modular growth
Lower communication costs
Faster response
Disadvantages
Software cost & complexity
Processing overhead
Data integrity
Slow response
Asynchronous Distributed Database

Some data inconsistency is tolerated
Data update propagation is delayed
Lower data integrity
Less overhead faster response time
Defines the structure of the system
o components identified
o functions of each component defined
o interrelationships and interactions between components defined
DDBS = DB + Communication
non-centralised
DDBMS
Motivated by need to integrate operational data and to provide controlled
access
manages the Distributed database
makes the distribution transparent to the user
10
UNIT I
Centralized DBMS on a Network
Site 1
Site 2
Site 5
Site 3
Site 4
Distributed DBMS Environment
Site 1
Site 5
Site 2
Site 3
Site 4
11
UNIT I
Implicit Assumptions
Data stored at a number of sites each site logically consists of a single
processor.
Processors at different sites are interconnected by a computer network no
multiprocessors
o parallel database systems
Distributed database is a database, not a collection of files data logically
related as exhibited in the users access patterns
o relational data model
D-DBMS is a full-fledged DBMS
o not remote file system, not a TP system
Dimensions of the Problem
Distribution
o Whether the components of the system are located on the same
machine or not
Heterogeneity
o Various levels (hardware, communications, operating system)
o DBMS important one
data model, query language,transaction management algorithms
Autonomy
o Not well understood and most troublesome
o Various versions
Design autonomy: Ability of a component DBMS to decide on
issues related to its own design.
Communication autonomy: Ability of a component DBMS to
decide whether and how to communicate with other DBMSs.
Execution autonomy: Ability of a component DBMS to execute
local operations in any manner it wants to.
Issues of a DDBMS
Allocation
Where to locate data and whether to replicate?
Data Fragmentation
Partition the database
Distributed catalog management
Distributed transactions
Distributed Queries
Making all of the above transparent to the user is the key of DDBMSs
Data
12
UNIT I
Replication or Data Replications in DDBMS

What Is Replication?
Replication is the process of copying and maintaining database objects in multiple
databases that make up a distributed database system.
Changes applied at one site are captured and stored locally before being forwarded
and applied at each of the remote locations.
Replication provides user with fast, local access to shared data, and protects
availability of applications because alternate data access options exist. Even if one
site becomes unavailable, users can continue to query or even update the remaining
locations.
Replication Objects, Groups, and Sites
The following sections explain the basic components of a replication system,
including replication sites, replication groups, and replication objects.
Replication Objects
A replication object is a database object existing on multiple servers in a distributed
database system.
Oracle's replication facility enables you to replicate tables and supporting objects
such as views, database triggers, packages, indexes, and synonyms. SCOTT.EMP
and SCOTT.BONUS are examples of replication objects.
Replication Groups
In a replication environment, Oracle manages replication objects using
replication groups.
By organizing related database objects within a replication group, it is easier
to administer many objects together
create and use a replication group to organize the schema objects necessary to
support a particular database application. That is not to say that replication
groups and schemas must correspond with one another.
Objects in a replication group can originate from several database schemas
and a schema can contain objects that are members of different replication
groups.
The restriction is that a replication object can be a member of only one group.
13
UNIT I
Database
replication is the frequent electronic copying data from a database in

one computer or server to a database in another so that all users share the same
level of information.
The result is a distributed database in which users can access data relevant to
their tasks without interfering with the work of others.
If a site (or network path) fails, the data held there is unavailable
Consider replication (duplication) of data to improve availability
No replication:
Disjoint fragments
Partial replication:
Site dependent
Full replication:
Every site has copy of all data
slows down update for consistency
expensive
Fragments of relations are placed across the sites multiple times
increases reliability - if some sites fail, the data is still available
increases locality - the data can be retrieved from the closest or local site
increases performance - a certain fragment may be accessed by less users
but the question
of mutual consistency
concurrency control
transparency must be addressed
A DB can be
partitioned (no replication)
replicated
fully replicated - the whole DB is copied to each site
partially replicated
What typical units of data are replicated in the process of data replication in
DDBMS?
Data may be replicated row by row, table by table, or database by database,
depending on what you need.
If you need to be able to operate independently, then you have a number of
replication models to choose from, depending on how much latency is
acceptable and how much autonomy is required.
Microsoft servers have a replication model that allows complete autonomy... but
could theoretically break atomicity.
14
UNIT I
o A distributed database is not stored in its entirety at a single physical

location. Instead, it is spread across a network of computers that are
geographically dispersed and connected via communications links.
o A distributed database allows faster local queries and can reduce network
traffic. With these benefits comes the issue of maintaining data integrity.
o A key objective for a distributed system is that it looks like a centralized
system to the user. The user should not need to know where a piece of
data is stored physically.
Comparison of Replication Alternatives:
Full Replication Partial
Replication
Partitioning
Query
Processing
Easy
Same
Difficulty
Directory
Management
Easy or
nonexistent
Same
Difficulty
Concurrency
Control
Moderate
Difficult
Easy
Reliability
Very High
High
Low
Reality
Possible
Application
Realistic
Possible
application
Topic 4: Fragmentation
Concept
Division of relation r into fragments r1, r2, , rn which contain sufficient

information to reconstruct relation r.
Horizontal fragmentation: each tuple of r is assigned to one or more
fragments.
Vertical fragmentation: the schema for relation r is split into several smaller
schemas.
All schemas must contain a common candidate key (or superkey) to
ensure lossless join property.
15
UNIT I
A special attribute, the tuple-id attribute may be added to each schema

to serve as a candidate key.
Example : relation account with following schema.
Account-schema = (branch-name, account-number, balance).
Distribution Design Issues

Why fragment at all?
How to fragment?
How much to fragment?
How to test correctness?
How to allocate?
Information requirements?
Can't we just distribute relations?
What is a reasonable unit of distribution?
o relation
views are subsets of relations locality
extra communication
o fragments of relations (sub-relations)
concurrent execution of a number of transactions that access
different portions of a relation
views that cannot be defined on a single fragment will
require extra processing
semantic data control (especially integrity enforcement)
more difficult
Why fragment?
Usage:
- Apps work with views rather than entire relations.
Efficiency:
- Data stored close to where most frequently used.
- Data not needed by local applications is not stored.
Security:
- and so not available to unauthorized users.
Parallelism:
- With fragments as unit of distribution, T can be divided into several
subqueries that operate on fragments.
Disadvantages: Performance & Integrity.

Types of Fragmentation
a) Horizontal Fragmentation (HF)
splitting the database by rows
e.g. A-J in site 1, K-S in site 2 and T-Z in site 3
16
UNIT I
o Primary Horizontal Fragmentation (PHF)

o Derived Horizontal Fragmentation (DHF)
b) Vertical Fragmentation (VF)
Splitting database by columns/fields
e.g. columns/fields 1-3 in site A, 4-6 in site B
Take the primary key to all sites
c) Hybrid Fragmentation (HF)
-Horizontal and vertical could even be combined
Four types of fragmentation:

1. Horizontal: Consists of a subset of the tuples of a relation.
- Defined using Selection operation
- Determined by looking at predicates used by Ts.
- Involves finding set of minimal (complete and relevant)
predicates.
- Set of predicates is complete, iff, any two tuples in same fragment
are referenced with same probability by any application.
- Predicate is relevant if there is at least one application that accesses
fragments differently.
2. Vertical: subset of atts of a relation.
- Defined using Projection operation
- Determined by establishing affinity of one attribute to another.
3. Mixed: horizontal fragment that is vertically fragmented, or a vertical fragment
that is horizontally fragmented.
- Defined using Selection and Projection operations
4. Derived: horizontal fragment that is based on horizontal fragmentation of a
parent relation.
- Ensures fragments frequently joined together are at same site.
- Defined using Semijoin operation
a) Horizontal Fragmentation (HF)
17
UNIT I
HORIZONTAL FRAGMENTATION
Original relation
A1
A2
An
T1
A1
A2
An
T1
T2
T3
.T60
T61
Tn
T2
T3
.
.T60
Site 1
A1
A2
An
T61
.
.
Tn
Site 2
-Fragments contain subsets of complete tuples (all attributes at all

sites)
How to reconstruct
R= Rs1 Rs2
Rsn
18
UNIT I
19
UNIT I
PHF Example
20
UNIT I
Fragmentation of relation PROJ

o Applications:
Find the name and budget of projects given their no.
Issued at three sites
Access project information according to budget
one site accesses 200000 other accesses >200000
o Simple predicates
o For application (1)
p 1 : LOC = Montreal
p 2 : LOC = New York
p 3 : LOC = Paris
o For application (2)
p 4 : BUDGET 200000
p 5 : BUDGET > 200000
o Pr = Pr' = {p 1 ,p 2 ,p 3 ,p 4 ,p 5 }
o Minterm fragments left after elimination
m 1 : (LOC = Montreal) (BUDGET 200000)
m 2 : (LOC = Montreal) (BUDGET > 200000)
m 3 : (LOC = New York) (BUDGET 200000)
m 4 : (LOC = New York) (BUDGET > 200000)
m 5 : (LOC = Paris) (BUDGET 200000)
m 6 : (LOC = Paris) (BUDGET > 200000)
PHF Correctness
Completeness
o Since Pr' is complete and minimal, the selection predicates are
complete
Reconstruction
o If relation R is fragmented into F R = {R 1 ,R 2 ,,R r }
R = "Ri FR R i
Disjointness
o Minterm predicates that form the basis of fragmentation should be
mutually exclusive.
21
UNIT I
Given a link L where owner(L)=S and member(L)=R, the derived horizontal

fragments of R are defined as
R i = R F S i , 1iw
where w is the maximum number of fragments that will be defined on R and
S i = s Fi (S)
where F i is the formula according to which the primary horizontal fragment S i is
defined.
22
UNIT I
More Examples:
Derived Horizontal Fragmentation

Defined on a member relation of a link
according to a selection operation specified
on its owner.
Example
Consider two tables
Emp
Link between the owner and the member

relations is defined as equiequi-join
An equiequi-join can be implemented by means
of semijoins.
semijoins.
Given a link L where owner (L) = S and
member (L) = R, the derived horizontal
fragments of R are defined as
PAY
Id
Name
Dept
Dept
Sal
100
D1
D1
10K
200
D2
D2
20K
300
D3
D3
30K
PAY1 = EMP1 PAY

PAY2 = EMP2 PAY
Emp1 = Sal <= 20K (Emp)
Emp)
Emp2 = Sal > 20K (Emp)
Emp)
Ri = R Si, 1 <= I <= w

Where,
Si = Fi (S)
w is the max number of fragments that will be
defined on
Fi is the formula using which the primary horizontal
fragment Si is defined
PAY1
Id
Name
Dept
100
D1
200
D2
PAY2
Id
Name
Dept
300
D3
DHF Correctness
Completeness
o Referential integrity
o Let R be the member relation of a link whose owner is relation S
which is fragmented as F S = {S 1 , S 2 , ..., S n }. Furthermore, let A be the
join attribute between R and S. Then, for each tuple t of R, there
should be a tuple t' of S such that
t[A]=t'[A]
Reconstruction
o Same as primary horizontal fragmentation.
Disjointness
o Simple join graphs between the owner and the member fragments.
b) Vertical Fragmentation (VF)
23
UNIT I
VERTICAL FRAGMENTATION
A1
Original
Relation
(R)
TID Tuple ID
Hidden Attribute to
ensure account
and simple join
reconstruction
A3
A4
t1
How to Reconstruct:
t2
R=Rs1
Rs2
Rsn
tn
A1
RS1
A2
A2
t1
t2
TID
TID
1
2
1
2
A3
RS2
A4
t1
t2
RS1.TID=RS2.TID
tn
tn
SITE1
Join condition
SITE2
Has been studied within the centralized context

o design methodology
o physical clustering
More difficult than horizontal, because more alternatives exist.
o Two approaches :
o grouping
attributes to fragments
o splitting
relation to fragments
Overlapping fragments
o grouping
Non-overlapping fragments
o splitting
We do not consider the replicated key attributes to be overlapping.
Advantage:
Easier to enforce functional dependencies
(for integrity checking etc.)
VF Information Requirements
Application Information
o Attribute affinities
a measure that indicates how closely related the attributes are
This is obtained from more primitive usage data
o Attribute usage values
Given a set of queries Q = {q 1 , q 2 ,, q q } that will run on the
relation R[A 1 , A 2 ,, A n ],
24
use(qi,Aj) =
UNIT I
1 if attribute Aj is referenced by query qi

0 otherwise
use(q i ,) can be defined accordingly

Two problems :
Cluster forming in the middle of the CA matrix
o Shift a row up and a column left and apply the algorithm to find the
best partitioning point
o Do this for all possible shifts
o Cost O(m2)
More than two clusters
o m-way partitioning
o try 1, 2, , m1 split points along diagonal and try to find the best
point for each of these
o Cost O(2m)
VF Correctness
A relation R, defined over attribute set A and key K, generates the vertical
partitioning F R = {R 1 , R 2 , , R r }.
n Completeness
The following should be true for A:
A = A Ri
n Reconstruction
Reconstruction can be achieved by
R = K
R i "R i F R
n Disjointness
TID's are not considered to be overlapping since they are maintained
by the system
Duplicated keys are not considered to be overlapping
More Examples:
Vertical Fragmentation of employee-info Relation
25
UNIT I
Vertical Fragmentation of employeeemployee-info

Relation
branch-name
tuple-id
customer-name
Lowman
1
Hillside
Camp
2
Hillside
Camp
3
Valleyview
Kahn
4
Valleyview
Kahn
5
Hillside
Kahn
6
Valleyview
Green
7
Valleyview
deposit1= branch-name, customer-name, tuple-id(employee-info)
account number
tuple-id
balance
500
A-305
1
336
A-226
2
205
A-177
3
10000
A-402
4
62
A-155
5
1123
A-408
6
750
A-639
7
deposit2= account-number, balance, tuple-id(employee-info)
Vertical Fragmentation & Horizontal Fragmentation of Employee Relation
Id
Name
Sal
Dept
100
10K
D1
200
20K
D2
300
30K
D3
Horizontal Fragmentation
Rows split : Sal > 20K

Id
Name
Sal
Dept
100
10K
D1
200
20K
D2
Id
Name
Sal
Dept
300
30K
D3
Vertical Fragmentation
Columns split : Primary

Key retained
Id
Name
Id
Sal
Dept
100
100
10K
D1
200
200
20K
D2
300
300
30K
D3
26
UNIT I
c) Hybrid Fragmentation (HF)

Horizontal or Vertical fragmentation of a Database schema will not be
sufficient to satisfy the requirements of user applications.
In certain cases, a vertical fragmentation may be followed by a horizontal
one, or viceversa.
In case of horizontal fragmentation, one has to stop when each fragment
consists of only one tuple, where as the termination part for vertical
fragmentation is one attribute per fragment.
Since two two types of partitioning strategies are applied one after the other,
this alternative is called hybrid fragmentation.
MIXED FRAGMENTATION
A1
Original
Relation
(R)
TID Tuple ID
Hidden Attribute to
ensure account
and simple join
reconstruction
A3
A4
How to Reconstruct:
t2
R=Rs1
Rs2
Rsn
tn
A1
RS1
A2
t1
A2
t1
t2
TID
TID
1
2
1
2
A3
RS2
A4
t1
t2
RS1.TID=RS2.TID
tn
tn
SITE1
Join condition
SITE2
27
UNIT I
Advantages of Fragmentation
Horizontal:
allows parallel processing on fragments of a relation
allows a relation to be split so that tuples are located where they are
most frequently accessed
Vertical:
allows tuples to be split so that each part of the tuple is stored where it
is most frequently accessed
tuple-id attribute allows efficient joining of vertical fragments
allows parallel processing on a relation
Vertical and horizontal fragmentation can be mixed.
Fragments may be successively fragmented to an arbitrary depth.
Advantages
1. Permits a number of transactions to executed concurrently
2. Results in parallel execution of a single query
3. Increases level of concurrency, also referred to as, intra query concurrency
4. Increased System throughput
Disadvantages
1. Applications whose views are defined on more than one fragment may suffer
performance degradation, if applications have conflicting requirements.
28
UNIT I
2. Simple asks like checking for dependencies, would result in chasing after
data in a number of sites
PHF Vs VF :
Primary Horizontal
Fragmentation
Vertical Fragmentation
Grouping
Starts by assigning each attribute to
one fragment
Primary horizontal fragmentation is

defined by a selection operation on the
owner relation of a database schema.
Given relation Ri, its horizontal fragments
are given by
Ri = Fi(R),
1<= i <= w
Fi selection formula used to obtain fragment
Ri
At each step, joins some of the

fragments until some criteria is
satisfied.
Results in overlapping fragments
Splitting
The example mentioned in slide 20, can be

represented by using the above formula
as
Emp1 = Sal <= 20K
(Emp)
Emp)
Emp2 = Sal > 20K (Emp)
Emp)
Starts with a relation and decides on

beneficial partitioning based on the
access behavior of applications to the
attributes
Fits more naturally within the toptop- down
design
Generates nonnon- overlapping fragments.
Topic 5: Query Processing

What is a Query ?
A database query is the vehicle for instructing a DBMS to update or retrieve specific
data to/from the physically stored medium.
The actual updating and retrieval of data is performed through various low-level
operations.
Examples of such operations for a relational DBMS can be relational algebra
operations such as project, join, select, Cartesian product, etc.
While the DBMS is designed to process these low-level operations efficiently, it can
be quite the burden to a user to submit requests to the DBMS in these formats.
Consider the following request:
Give me the vehicle ids of all Chevrolet Camaros built in the year 1977.
While this is easily understandable by a human, a DBMS must be presented with a
format it can understand, such as this SQL statement:
29
UNIT I
select vehicle_id
from vehicles
where year = 1977
Note that this SQL statement will still need to be translated further by the DBMS so
that the functions/methods within the DBMS program can not only process the
request, but do it in a timely manner.
Basic Steps in Query Processing
1.
2.
3.
Parsing and translation

Optimization
Evaluation
n Parsing and translation

translate the query into its internal form. This is then translated into
relational algebra.
Parser checks syntax, verifies relations
n Evaluation
The query-execution engine takes a query-evaluation plan, executes
that plan, and returns the answers to the query.
Optimization
n A relational algebra expression may have many equivalent expressions
30
UNIT I
E.g., balance<2500(balance(account)) is equivalent to

balance(balance<2500(account))
n Each relational algebra operation can be evaluated using one of several
different algorithms
Correspondingly, a relational-algebra expression can be evaluated in
many ways.
n Annotated expression specifying detailed evaluation strategy is called an
evaluation-plan.
E.g., can use an index on balance to find accounts with balance <
2500,
or can perform complete relation scan and discard accounts with
balance 2500
n Query Optimization: Amongst all equivalent evaluation plans choose the one
with lowest cost.
Cost is estimated using statistical information from the
database catalog
e.g. number of tuples in each relation, size of tuples, etc.
using client-server architecture

user creates query
client parses and sends to server(s) (SQL?)
servers return appropriate Tables
client combines into one Table
Issue of data transfer cost over a network
o optimise the query to transfer the least amount
31
UNIT I
Query Processing Components

Query language that is used
o SQL: intergalactic dataspeak
Query execution methodology
o The steps that one goes through in executing high-level (declarative)
user queries.
Query optimization
o How do we determine the best execution plan?
Query Optimization Objectives

Minimize a cost function
I/O cost + CPU cost + communication cost
These might have different weights in different distributed environments
Wide area networks
o communication cost will dominate
low bandwidth
low speed
high protocol overhead
o most algorithms ignore all other cost components
Local area networks
o communication cost not that dominant
o total cost function should be considered
Can also maximize throughput
Query Optimization Issues Types of Optimizers

Exhaustive search
o cost-based
o optimal
o combinatorial complexity in the number of relations
Heuristics
o not optimal
o regroup common sub-expressions
o perform selection, projection first
o replace a join by a series of semijoins
o reorder operations to reduce intermediate relation size
o optimize individual operations
Optimization Granularity
32
UNIT I
Single query at a time

o cannot use common intermediate results
Multiple queries at a time
o efficient if many similar queries
o decision space is much larger
Optimization Timing
Static
o compilation optimize prior to the execution
o difficult to estimate the size of the intermediate results error
propagation
o can amortize over many executions
o R*
Dynamic
o run time optimization
o exact information on the intermediate relation sizes
o have to reoptimize for multiple executions
o Distributed INGRES
Hybrid
o compile using a static algorithm
o if the error in estimate sizes > threshold, reoptimize at run time
o MERMAID
Statistics
Relation
o cardinality
o size of a tuple
o fraction of tuples participating in a join with another relation
Attribute
o cardinality of domain
o actual number of distinct values
Common assumptions
o independence between different attribute values
o uniform distribution of attribute values within their domain
Decision Sites
Centralized
o single site determines the best schedule
o simple
o need knowledge about the entire distributed database
Distributed
o cooperation among sites to determine the schedule
o need only local information
o cost of cooperation
Hybrid
o one site determines the global schedule
each site optimizes the local subqueries
33
UNIT I
Network Topology
Wide area networks (WAN) point-to-point
o characteristics
low bandwidth
low speed
high protocol overhead
o communication cost will dominate; ignore all other cost factors
o global schedule to minimize communication cost
o local schedules according to centralized query optimization
Local area networks (LAN)
o communication cost not that dominant
o total cost function should be considered
o broadcasting can be exploited (joins)
o special algorithms exist for star networks
Step 1 Query Decomposition

Input : Calculus query on global relations
Normalization
o manipulate query quantifiers and qualification
Analysis
o detect and reject incorrect queries
o possible for only a subset of relational calculus
Simplification
o eliminate redundant predicates
Restructuring
o calculus query algebraic query
o more than one translation is possible
34
UNIT I
o use transformation rules

Step 2 Data Localization
Input: Algebraic query on distributed relations
Determine which fragments are involved
Localization program
o substitute for each global query its materialization program
o optimize
Step 3 Global Query Optimization
Input: Fragment query
Find the best (not necessarily optimal) global schedule
o Minimize a cost function
o Distributed join processing
Bushy vs. linear trees
Which relation to ship where?
Ship-whole vs ship-as-needed
o Decide on the use of semijoins
Semijoin saves on communication at the expense of more local
processing.
o Join methods
nested loop vs ordered joins (merge join or hash join)
Centralized Query Optimization
INGRES
o dynamic
o interpretive
System R
o static
o exhaustive search
Topic 6: Transaction Processing

Transaction
A transaction is a collection of actions that make consistent transformations of
system states while preserving system consistency.
concurrency transparency
failure transparency
Transaction may access data at several sites.
Each site has a local transaction manager responsible for:

Maintaining a log for recovery purposes
Participating in coordinating the concurrent execution of the transactions
executing at that site.
35
UNIT I
Each site has a transaction coordinator, which is responsible for:

Starting the execution of transactions that originate at the site.
Distributing subtransactions at appropriate sites for execution.
Coordinating the termination of each transaction that originates at the site, which
may result in the transaction being committed at all sites or aborted at all sites.
Transaction system Architecture
Database in a consistent state ------------------------

Database may be temporarily in an inconsistent state during execution
--------------------------------
------------
Database in a consistent state

Transaction Structure
Flat transaction
- Consists of a sequence of primitive operations embraced between a begin
and end marks.
Begin_transaction Reservation
.
.
End.
Nested Transaction
-The operations of a transaction may themselves be transactions.
Begin_transaction Reservation
36
Begin_ transaction Airline

.
End. Airline
.
Begin_ transaction Hotel .
.
End. Hotel
UNIT I
End. Reservation
Properties of Transactions
ACID (Atomicity, Consistency, Isolation, Durability) Property
Atomicity All or Nothing
Consistency No violation of integrity constraints
Isolation Concurrent changes invisible & serialisable
Durability Committed update persist
Transaction Processing Issues

Transaction structure (usually called transaction model)
Flat(simple), nested
Internal database consistency
- Semantic data control (integrity enforcement) algorithms
37
UNIT I
Reliability Protocols
Atomicity & Durability
Local recovery protocols
Global commit protocols
Concurrency control algorithms

- How to synchronize concurrent transaction executions (correctness
criterian)
- Intra-transaction consistency, isolation
Replica control protocols
- How to control the mutual consistency of replicated data
- One copy of equivalence and ROWA
Topic 7: Concurrency Control

Whats concurrency control?
Concurrency control deals with preventing concurrently running processes from
improperly inserting, deleting, or updating the same data. Concurrency control is
maintained through two mechanisms: Transactions and Locks.
Whats transactions?
A transaction is a logical unit of work. It is both the unit of work and the unit of
recovery. The statements nested within a transaction must either all happen or none
happen.
Transactions are a mandatory facility for maintaining the integrity of a database
while running multiple concurrent operations.
Transactions are atomic: there is no such thing as a partial transaction.
A set of transactions is said to be serializable if and only if it produces the same
result as some arbi trary serial execution of those same transactions for arbitrary
input. A set of transactions can be correct only if it is serializable.
38
UNIT I
Transactions are a mandatory facility for maintaining the integrity of a database

while running multiple concurrent operations. A transaction is a logical unit of
work. It is both the unit of work and the unit of recovery.
The statements nested within a transaction must either all happen or none happen.
Transactions are atomic: there is no such thing as a partial transaction.
Concurrency control deals with preventing concurrently running processes from
improperly inserting, deleting, or updating the same data.
Two Concurrency control mechanisms:
Concurrency control is maintained through two mechanisms: Transactions and
Locks.
Whats lock?
A lock is a means of claiming usage rights on some resource.
There can be several different types of resources that can be locked and several
different ways of locking those resources.
Most locks used on Teradata resources are locked automatically by default. The
Teradata lock manager implicitly locks the following objects:
Database, Table, View and Row hash.
User can apply four different levels of locking on Teradata resources:
Exclusive, Write, Read and Access.
The Teradata R DBMS applies most of its locks automatically.
Modify concurrency control schemes for use in distributed environment.
We assume that each site participates in the execution of a commit protocol to

ensure global transaction automicity.
We assume all replicas of any item are updated
The problem of synchronizing concurrent transactions such that the
consistency of the database is maintained while, at the same time, maximum
degree of concurrency is achieved.
Anomalies:
o Lost updates
The effects of some transactions are not reflected on the
database.
o Inconsistent retrievals
39
UNIT I
A transaction, if it reads the same data item more than once,

should always read the same value.
Extends centralised concurrency mechanisms
Multiple copies of data items
o maintain consistency
failures in individual sites/network
o continue operations, update and rejoin
distributed commit
o 2-phase protocol (local and global)
distributed deadlock
Global serialisation must occur
o i.e. serialise local serialisations!
o Locks and timestamping apply
If database not replicated and transactions all local or performable at one
remote site then:
o Use centralised concurrency mechanisms
Otherwise mechanisms need to be extended
o To deal with replication or transactions involving multiple sites
Need to consider deadlocks at local and global levels
Distributed Locks
Just like centralised mechanisms. But we need to consider locks that
manage replication and sub-transactions
Four modes of management possible:
o Centralised 2PL
Read any copy, update all for updates
Single site, bottleneck, failure?
o Primary Copy 2PL
Distributes locks, one copy designated primary, others slaves
Only primary copy locked for updates, slaves updated later
o Distributed 2PL
Each site manages its own data locks
All copies locked for an update, high cost of comms
o Majority Locking
Diagrammatic representation
40
UNIT I
D=Data item (PC=Primary Copy, only for Primary copy 2PL)
Site 1
D1
D2
Site 2
Site 3
D1 (PC)
D2
D3 (PC)
D2 (PC)
D3
Centralised: e.g. Site 1 is the only Lock Manager

Primary Copy: e.g. Site 1 handles locks on D1/D3
o Site 3 handles locks on D2
o remember the site does NOT have to hold the PC
Distributed: All sites lock own data
(lock all copies for writing)
Majority Locking
Extension of distributed 2PL
Doesnt lock all copies before update
Needs more than half of locks on a copy to proceed
If so, it informs other sites
Otherwise it cancels request
Only one transaction with an exclusive lock
Many transactions can hold a majority lock on a shared lock
Deadlock
T3 waiting for T1
T1
41
UNIT I
T1 waiting for T2
T2 waiting for T3
Example
Locally:
Site 1: T3 waiting for T1

Site 3: T2 waiting for T3
Text T3 T1 Text
Text T1 T2 Text
Text T2 T3 Text
Maybe
Deadlock?
Site 1 sends WFG to site 2, site 2 combines WFG to
Text T3 T1 T2 Text
Site 2 sends WFG to site 3, site 3 combines WFG to
Text T3 T1 T2 T3 Text
Definitely Deadlock!
Distributed Reliability Protocols

Commit protocols
o How to execute commit command for distributed transactions.
o Issue: how to ensure atomicity and durability?
Termination protocols
o If a failure occurs, how can the remaining operational sites deal with
it.
o Non-blocking : the occurrence of failures should not force the sites to
wait until the failure is repaired to terminate the transaction.
Recovery protocols
42
UNIT I
o When a failure occurs, how do the sites where the failure occurred
deal with it.
o Independent : a failed site can determine the outcome of a transaction
without having to obtain remote information.
Independent recovery non-blocking termination
Topic 8: Recovery
Purpose of Database Recovery
To bring the database into the last consistent state, which
existed prior to the failure.
To preserve transaction properties (Atomicity, Consistency,
Isolation and Durability).
Example: If the system crashes before a fund transfer transaction completes its
execution, then either one or both accounts may have incorrect value. Thus, the
database must be restored to the state before the transaction modified any of the
accounts.
Ensures database is fault tolerant, and not corrupted by software, system or media
failure
7x24 access to mission critical data.
Failure can occur through
Loss of message
By network protocol
DDBMS deals with it transparently
Loss of a communication link

Network partitioning (see diagram)
Site failure
Types of Failure
The database may become unavailable for use due to
Transaction failure: Transactions may fail because of incorrect input,
deadlock, incorrect synchronization.
System failure: System may fail because of addressing error,
application error, operating system fault, RAM failure, etc.
Media failure: Disk head crash, power disruption, etc.
43
UNIT I
Database Recovery Techniques

Transaction Log
For recovery from any type of failure data values prior to modification (BFIM BeFore Image) and the new value after modification (AFIM AFter Image) are
required. These values and other information is stored in a sequential file called
Transaction log.
Data Update
Immediate Update: As soon as a data item is modified in cache, the
disk copy is updated.
Deferred Update: All modified data items in the cache is written
either after a transaction ends its execution or after a fixed number of
transactions have completed their execution.
Shadow update: The modified version of a data item does not
overwrite its disk copy but is written at a separate disk location.
In-place update: The disk version of the data item is overwritten by
the cache version.
Data Caching
Data items to be modified are first stored into database cache by the Cache Manager
(CM) and after modification they are flushed (written) to the disk. The flushing is
controlled by Modified and Pin-Unpin bits.
Pin-Unpin: Instructs the operating system not to flush the data item.
Modified: Indicates the AFIM of the data item
Roll-back: One execution of T1, T2 and T3 as recorded in the log.
Write-Ahead Logging
When in-place update (immediate or deferred) is used then log is necessary for
recovery and it must be available to recovery manager. This is achieved by WriteAhead Logging (WAL) protocol.
Checkpointing
Time to time (randomly or under some criteria) the database flushes its buffer to
database disk to minimize the task of recovery.
Steal/No-Steal and Force/No-Force
Possible ways for flushing database cache to database disk:
Steal: Cache can be flushed before transaction commits.
No-Steal: Cache cannot be flushed before transaction commit.
Force: Cache is immediately flushed (forced) to disk.
44
UNIT I
No-Force: Cache is deferred until transaction commits.

Recovery Scheme
Deferred Update (No Undo/Redo)
Recovery in multidatabase system
The multiple nodes agree to commit individually the part of the transaction
they were executing. This commit scheme is referred to as two-phase
commit (2PC).
If any one of these nodes fails or cannot commit the part of the transaction,
then the transaction is aborted. Each node recovers the transaction under its
own recovery protocol.
E.g. S4 out of contact with S1

S4 crashed?
Link down?
Partitioned?
S4 busy?
S1
S4
S2
S3
S5
Recovery after failure?

Distributed recovery maintains atomicity and durability
What happens then?
Abort transactions affected by the failure
Including all subtransactions
Flag the site as failed
Check for recovery or wait for message to confirm
On restart, abort partial transactions which were active at the time of
the failure
Perform local recovery
Update copy of database to be consistent with remainder of the system
45
UNIT I
Rcovery Protocol
Protocols at failed site to complete all transactions outstanding at the time of
failures.
Classes of failures
1. Site failure
2. Lost messages
3. Network partitioning
4. Byzantine failures
Effects of failures
1. Inconsistent database
2. Transaction processing is blocked
3. Failed component unavailable
Independent Recovery
A recovering site makes a transition directly to a final state without
communicating with other sites.
Lemma
For a protocol, if a local states concurrency set contains both an abort and
commit, it is not resilient to an arbitrary failure of a single site.
S i commit because other sites may be in abort
S i abort because other sites may be in commit
Rule 1: S: Intermediate state
If C(s) contains a commit failure transition from S to commit
Otherwise failure transition from S to abort
Unscheduled restarts occur for one of the following reasons:
AMP or disk failure
Software failure
Parity error
Transaction recovery describes how the Teradata RDBMS restarts itself after a
system or media failure.
Two types of automatic recovery of transactions can occur when an unscheduled
restart occurs:
Single transaction recovery
RDBMS recovery
The following table details when these two automatic recovery mechanisms take
place:
This Recovery Type
Happens When
46
Single transaction
RDBMS
UNIT I
The RDBMS aborted a single

transaction because of:
Transaction deadlock timeout
User error
User-initiated abort command
An inconsistent data table
Unavailable resources for
parsing
Single transaction recovery uses
the transient journal to effect its
data restoration
A RDBMS restart is caused by:
Hardware failure
Software failure
User command
Two-Phase Commit Protocol
47
UNIT I
Two-phase commit (2PC) is a protocol for assuring concurrency of data in multiple

databases in which each participant database manager votes to either commit or
abort the changes.
The participants wait before committing the change until it is known that all
participants can commit. By voting to commit, the participant guarantees that it can
either commit or rollback its part of the transaction, even if it crashes before
receiving the result of the vote.
The 2PC protocol allows C ICS and IMS applications to be developed that can
update one or more Teradata RDBMS databases and/or databases under some other
DBMS in a synchronized manner. The result is that all updates requested in a
defined unit of work will either succeed or fail.
2PC Recovery Protocols Additional Cases
Arise due to non-atomicity of log and message send actions.
48
UNIT I
Sample Questions
Topic 1:
1. What is a distributed database? (2M)
2. Define Distributed DBMS (DDBMS) (2M)
3. What are the characteristics of DDBMS ? (2M)
4. What is important difference between DDBMS and distributed processing? (2M)
5. What are the Functions of a DDBMS ? (2M)
6. What are the Advantages and Disadvantages of DDBSs? (2M)
7. What are the Applications of DDBMS? (2M)

8. Explain in detail about the Types of DDBMS. (16M)
9. List the two main issues in DDBMS (2M)
Topic 2:
1. Explain Distributed Database in detail. ( 8M)
2. Explain Centralized Database System in detail. (8M).
3. List and differentiate between Distributed Databases Vs Conventional
Databases
Topic 3:
1. Explain the Client-Server Architecture with a neat diagram. (8M)
2. Explain the Distributed Database Architecture with a neat diagram. (8M)
3. What is Synchronous Distributed Database?
4. What is Asynchronous Distributed Database?
5. Explain the major issues a DDBMS in detail. (8M)
6. Explain how Replication or Data Replications used in DDBMS. (8M)
Topic 4:
1. What is the Concept behind fragmentation ? Give examples. (8M)
2. Why we need fragmentation? (2M)
3. Explain in detail the Types of Fragmentation and give examples for
each. (16M)
4. Explain in detail on Horizontal Fragmentation (HF). (8M)
5. Explain in detail on Vertical Fragmentation (VF). (8M)
6. Explain in detail on Hybrid Fragmentation (HF). (8M)
7. Two problems :
8. What are the Advantages and Disadvantages of Fragmentation (2M)
9. Compare and contrast the PHF and VF. (8M)
49
UNIT I
Topic 5:
1.
2.
3.
4.
5.
6.
7.
8.
9.
What is a Query ? (2M)

What are Query Processing? (2M)
List and describe the basic Steps in Query Processing. (8M)
Explain the concept of Query Optimization. (8M)
List and describe the Query Processing Components. (8M)
List the Query Optimization Objectives. ( 4M)
Explain the major Query Optimization Issues. (8M)
Explain in details on Query processing (8M)
What are Centralized Query Optimization? Explain briefly. (8M)
Topic 6:
1. What is transaction ? Give examples. (2M)
2. Give the local transaction manager responsibilities. (2M)
3. Explain in detail on Transaction system Architecture. Illustrate with a
neat diagram (8M)
4. Explain Transaction Structure in detail. (8M)
5. What are the three major Properties of Transactions ? (2M)
6. What is ACID?
7. List and describe the Transaction Processing Issues. (8M).
Topic 7:
1. What is currency control ? (2M)
2. Explain Concurrency control mechanisms. (2M)
3. Whats lock? (2M)
4. What is Distributed Locks ? (2M)
5. Describe the Majority Locking. (2M)
6. Explain in detail on currency control in handled in DDBMS. (8M)
Topic 8:
1. What is Failure? (2M)
2. What is Recovery? (2M)
3. Is Recovery after failure? Explain. (2M)
4. Explain about Recovery Protocol. (2M)
5. What are the major Effects of failures? (8M)
6. Explain in detail about two automatic recovery mechanisms (8M)
7. Explain in detail on Two-Phase Commit Protocol. (8M)
8. Explain in detail on Recovery in handled in DDBMS. (8M)
50
UNIT I
University Questions
1. Differentiate homogenous and hetrogenous databases with reference to
distributed databases. (2M)
2. Name the fragmentations supported in a distributed system and write
examples for each. (2M)
3. Explain how concurrency control and recovery techniques are handled in
DDBMS. (16M)
4. Draw simplified physical client Architecture for distributed database systems
and discuss in detail (8M)
5. Discuss the techniques of fragmentation, data replication used in distributed
database design. (8M)
*************************** End of Unit I ************************
51

Cs9152 Unit I

Uploaded by

Copyright:

Available Formats

Cs9152 Unit I

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cs9152 Unit I

Uploaded by

Copyright:

Available Formats

CS9152 - DATABASE TECHNOLOGY

CS9152 - DATABASE TECHNOLOGY

CS9152 - DATABASE TECHNOLOGY

Topic 1: Introduction to Distributed Databases

logically interrelated collection of shared data (and a description of this data),

Distributed Database System

Fundamental Principle: make distribution transparent to user.

CS9152 - DATABASE TECHNOLOGY

In distributed databases, data spread over multiple machines (also referred to

DDBMS has following characteristics:

Collection of logically-related shared data.

Important difference between DDBMS and distributed processing

CS9152 - DATABASE TECHNOLOGY

Distributed processing of centralised DBMS

Much more tightly coupled than a DDBMS.

Expect DDBMS to have at least the functionality of a DBMS.

CS9152 - DATABASE TECHNOLOGY

Also to have following functionality:

Extended communication services.

Extended Data Dictionary.

Distributed query processing.

Extended concurrency control.

Extended recovery services

Distributed nature of some database applications

CS9152 - DATABASE TECHNOLOGY

Advantages of distributed databases

CS9152 - DATABASE TECHNOLOGY

In a homogeneous distributed database:

In a heterogeneous distributed database:

CS9152 - DATABASE TECHNOLOGY

Two main issues in DDBMS

Topic 2: Distributed Databases Vs Conventional Databases

mimics organisational structure with data

CS9152 - DATABASE TECHNOLOGY

more costly to maintain

Centralized Database System:

Physically spread over the sites of a computer network.

Topic 3: Distributed Databases Architecture

CS9152 - DATABASE TECHNOLOGY

Asynchronous Distributed Database

CS9152 - DATABASE TECHNOLOGY

Centralized DBMS on a Network

Distributed DBMS Environment

CS9152 - DATABASE TECHNOLOGY

CS9152 - DATABASE TECHNOLOGY

Replication or Data Replications in DDBMS

CS9152 - DATABASE TECHNOLOGY

replication is the frequent electronic copying data from a database in

CS9152 - DATABASE TECHNOLOGY

o A distributed database is not stored in its entirety at a single physical

Division of relation r into fragments r1, r2, , rn which contain sufficient

CS9152 - DATABASE TECHNOLOGY

A special attribute, the tuple-id attribute may be added to each schema

Distribution Design Issues

Disadvantages: Performance & Integrity.

CS9152 - DATABASE TECHNOLOGY

o Primary Horizontal Fragmentation (PHF)

Four types of fragmentation: