Topic 11 Data Warehouses Database Design and Development

V1.0 Visuals Handout Page 1

Database Design and
NCC Education Limited
V1.0
Database Design and
Development
Topic 11:
Data Warehouses
Data Warehouses Topic 11 - 11.2
Scope and Coverage
This topic will cover:
The need for business intelligence and the concept
of the data warehouse
The difference between Online Transaction
NCC Education Limited V1.0
The difference between Online Transaction
Processing (OLTP) systems and data warehousing
The architecture and main components of a data
warehouse
Data Warehouses Topic 11 - 11.3
Learning Outcomes
By the end of this topic students will be able to:
Understand the potential need for a data warehouse
Differentiate between on-line transaction processing
systems and data-warehouse system
NCC Education Limited V1.0
systems and data warehouse system
Identify the main components of a data warehouse
Topic 11 Data Warehouses Database Design and Development
V1.0 Visuals Handout Page 2
Data Warehouses Topic 11 - 11.4
Why need a Data Warehouse? - 1
Two types of database processing
OLTP - On-line transaction processing.
- It is a class of program that facilitates and manages
transaction-oriented applications.
NCC Education Limited V1.0
- It is used for supporting daily busyness.
OLAP - On-line analytical processing
- It is a way of viewing data in a multidimensional
format.
- It is used for supporting decision making.
Data Warehouses Topic 11 - 11.5
Why need a Data Warehouse? - 2
The need for business intelligence
- competitive environment
- strategic planning
- decision making
NCC Education Limited V1.0
Proliferation of different systems
Data Warehouses Topic 11 - 11.6
Content
Accessibility
Form
Databases Designed for OLTP are
not Suitable for OLAP
NCC Education Limited V1.0
Performance
Availability
Data Warehouse is a solution
Topic 11 Data Warehouses Database Design and Development
V1.0 Visuals Handout Page 3
Data Warehouses Topic 11 - 11.7
Internet and
VPN or WAN
On-line shopping
Webserver and database for
On line shopping
Supermarket Systems
NCC Education Limited V1.0
Stock taking and reordering database
Customer Records database
LAN
OLTP for point of sales
Point of Sale
Customer with loyalty card
Data Warehouses Topic 11 - 11.8
Activity Identify the Types of Data
been Collected and Used here?
NCC Education Limited V1.0
Data Warehouses Topic 11 - 11.9
And What Benefits from Bringing
this Data Together? - 1
NCC Education Limited V1.0
Topic 11 Data Warehouses Database Design and Development
V1.0 Visuals Handout Page 4
Data Warehouses Topic 11 - 11.10
And What Benefits from Bringing
this Data Together?
Sales Trends
Customer Buying habits
R i l i ti
NCC Education Limited V1.0
Regional variations
Variations by time
Goods generating
profit
Data Warehouses Topic 11 - 11.11
Transform Data into Information
Data Warehouse provides a multidimensional view
of an organizations operational (OLTP) data to help
user make more informed, fast decisions.
NCC Education Limited V1.0
Data Warehouses Topic 11 - 11.12
Subject-oriented
Integrated
Time-variant
Non-volatile
What is a Data Warehouse?
NCC Education Limited V1.0
Data Warehouse
Combining data in support
of managements decisions
Topic 11 Data Warehouses Database Design and Development
V1.0 Visuals Handout Page 5
Data Warehouses Topic 11 - 11.13
Subject Orientation
Operational System
sales
Data warehouse
customer
NCC Education Limited V1.0
sales
warehouse
Loyalty
card
Online
sales
An application orientation
supplier
customer
product
A subject orientation
buying
Data Warehouses Topic 11 - 11.14
Integration
OLTP System Data warehouse
NCC Education Limited V1.0
App1-m,f
App2-1,0
App3-male,female
Integration
Date(ddmmyy)
App1-date(yymmdd)
App2-date(mmddyy)
App3-date(ddmmyy)
m,f
Integration
Data Warehouses Topic 11 - 11.15
Time Variant
OLTP System Data warehouse
time horizon 60-90 days
depending on business
keywill not usuallyhave an
time horizon long term
5-10 years
keywill contain an
NCC Education Limited V1.0
key will not usually have an
element of time
data can be changed
key will contain an
element of time
data cannot be changed
Topic 11 Data Warehouses Database Design and Development
V1.0 Visuals Handout Page 6
Data Warehouses Topic 11 - 11.16
Non-Volatile
create
update
load
access

NCC Education Limited V1.0

Operational System Data warehouse
retrieve
delete
access
Data Warehouses Topic 11 - 11.17
The Data Warehouse Functional
Model
Query
Acquisition
Storage
Access
Users
NCC Education Limited V1.0
Date
Extraction
&Prep
Data base
Or Other
Storage
Query
OLAP
Statistics
Discovery
Mining
Others
Users
Data Warehouses Topic 11 - 11.18
Identifying the necessary data from legacy system
(and other data sources).
Validating that the data is accurate, appropriate,
and usable.
E i h d f h i i l
Acquisition
NCC Education Limited V1.0
Extracting the data from the original source
Preparing the data for inclusion into the new
environment.
Staging the information making the data ready for
loading into the warehouse itself
Topic 11 Data Warehouses Database Design and Development
V1.0 Visuals Handout Page 7
Data Warehouses Topic 11 - 11.19
Storage
Storage is the heart of a data warehouse
An environment (the data warehouse) is
NCC Education Limited V1.0
constructed to provide a place from which the data
from the source systems can be accessed
Data Warehouses Topic 11 - 11.20
Access Tools
Query and Reporting Tools
OLAP Tools
Statistical Analysis Tools
NCC Education Limited V1.0
Statistical Analysis Tools
Data Discovery / Data mining Tools
Graphical and Geographic Information Systems
Data Warehouses Topic 11 - 11.21
Seven Steps to Building a Data
Warehouse
Determine the needs of the end users
Identify the necessary data sources
Analyse the data sources in depth
Use the information to work out how the data will need to be
NCC Education Limited V1.0
transformed
Create the meta data which describes the transformation
and integration that to occur
Create the physical data warehouse and populate from
various sources
Create the end use applications
Topic 11 Data Warehouses Database Design and Development
V1.0 Visuals Handout Page 8
Data Warehouses Topic 11 - 11.22
An Example of A Data Warehouse
Purchasing
System
Transformation/Integration
Order Processing
System
Inventory
System
D t W h
Meta Data
NCC Education Limited V1.0
Transformation/Integration
Process
Applications
Data Warehouse
Production Planning
Distribution
Customer Service
Data Warehouses Topic 11 - 11.23
Data Warehouse Schemas
Star Schemas
Snowflake schemas
NCC Education Limited V1.0
Starflake schemas
Data Warehouses Topic 11 - 11.24
Fact Table
On-line sales
Customer
loyalty data
Central table surrounded by reference tables
Star Schema
NCC Education Limited V1.0
e.g. Sales
trends
Store sales
Topic 11 Data Warehouses Database Design and Development
V1.0 Visuals Handout Page 9
Data Warehouses Topic 11 - 11.25
On line sales
Item Type
sales by
customer
Snowflake Schema
Each dimension can have a number of its own
dimensions
NCC Education Limited V1.0
Fact Table
e.g Sales
Trends
On-line sales
Customer
loyalty
Store sales
Region
information
Store sales
by Item type
Data Warehouses Topic 11 - 11.26
Fact Table
e.g Sales
On-line sales
Customer
loyalty
Starflake Schema
Some de-normalisation
NCC Education Limited V1.0
Trends
Store sales
Region
information
Store sales
by Item type
Data Warehouses Topic 11 - 11.27
OLAP On-line Analytical
Processing
Consolidation
Drilling-down
NCC Education Limited V1.0
Pivoting
Multi-dimensional data
Topic 11 Data Warehouses Database Design and Development
V1.0 Visuals Handout Page 10
Data Warehouses Topic 11 - 11.28
A M J J A
Month
North
South
Midlands
Region
NCC Education Limited V1.0
34 36
55 38 34
34 54
58
60
56
2009
2010
Year
Multi-dimensional data sales of Ice cream in thousands
Data Warehouses Topic 11 - 11.29
Codds Rules for OLAP Tools - 1
Multi-dimensional conceptual view
Transparency
Accessibility
NCC Education Limited V1.0
Consistent reporting performance
Client-server architecture
Generic dimensionality
Dynamic sparse matrix handling
j1
Data Warehouses Topic 11 - 11.30
Codds Rules for OLAP Tools - 2
Multi-user support
Unrestricted cross-dimensional operations
NCC Education Limited V1.0
Intuitive data manipulation
Flexible reporting
Unlimited dimensions
Slide 29
j1 split
jamie.bazley, 01/09/2011
Topic 11 Data Warehouses Database Design and Development
V1.0 Visuals Handout Page 11
Data Warehouses Topic 11 - 11.31
Learning Outcomes
By the end of this unit students will be able to:
Understand the potential need for a data warehouse
Differentiate between on-line transaction processing
systems and data-warehouse system
NCC Education Limited V1.0
systems and data warehouse system
Identify the main components of a data warehouse
Did we meet them?
Data Warehouses Topic 11 - 11.32
References
Benyon-Davies, Paul. Database Systems Palgrave
Third Edition 2004 Chapters 40 and 41
Connolly, Thomas M., and Begg, Carolyn E.,
Database Systems: A Practical Approach to Design
and Implementation Addision-Wesley, Fourth
NCC Education Limited V1.0
and Implementation Addision Wesley, Fourth
Edition 2005 Chapter 31, 32 and 33
Inmon, W.H., Building the data warehouse
http://inmoncif.com/inmoncif-
old/www/library/whiteprs/ttbuild.pdf retrieved 15th
August 2011
Data Warehouses Topic 11 - 11.33
Topic 11 Data Warehouses
NCC Education Limited V1.0
Any Questions?