Application of data warehouse technology in power analysis

Xiaoguang Hong, Juan Chu

Computer Department of Shandong University, Ji Nan 250061, Shandong province, China
hxg@sdu.edu.cn, chj@dareway.com.cn
Abstract: Data warehouse technology transforms the

and the decision-making using the preceding

operational data store to general and compositive

computer technologies and the basic application

information. It also provides effective way for analysis

systems. It can help the electric power corporations

and statistic to the mass data, and helps to do the

enhance the management level and the circulation

decision-making. The paper introduces the data

efficiency, improve service quality and finally

warehouse and the online analysis process. According to

upgrade the enterprise competition. The method to

the power management information system, the paper

solve the problem is building the data warehouse.

analyzes the power-oriented multidimensional database

The decision-making system based on OLAP

modeling and the presentation of multidimensional

has been applied in the areas such as banks, hospitals

views. The system also builds power-oriented data

and chain sales. In [8], the OLAP is applied in the

warehouse using DB2 database, Warehouse Manager,

telecom management system. In [9], the paper gives

Cube View and Alphablox of IBM. Then, this paper

an instance using OLAP in insurance operation. In

discusses the concrete applications of data warehouse in

the setting of power sales analysis model, the paper

completing the extract, transform and load (ETL) of

builds a power sales-oriented data warehouse using

power data. The discussion includes the optimization of

Warehouse Manager, Cube View and Alphablox of

the system after creating the MQT tables using Cube

IBM according to the power management system [1].

View and the presentation of the analysis system. The

The rest of the paper is organized as follows.

system can help power corporations to do the

Section

introduces

the

data

warehouse

decision-making.

technology. Section presents the data warehouse

Key words: data warehouse; power management; olap;

design and implement in power analysis system.

multidimensional views; power system

Finally, section discusses related work and draws

some conclusions.

I.

INTRODUCTION

Along with the prevalence of the computer

II.

DATA WAREHOUSE TECHNOLOGY

application technology, the foundation of the power

A data warehouse is, primarily, a data

management information system advances rapidly.

collection which is to favor decision-making of

The basic application systems have been founded and

enterprises

are running steadily, such as the power management

compositive, cannot be renewable and can be

information system (MIS), Automatic Meter Reading

changed at any time.[3] Because data warehouse

system, the Distribution Management System, the

doesnt have strict mathematic theory base, it leans to

Automatic

information

be an engineering project. In technology, it can be

System, the Substation Automation and the office

divided into key technologies such as data extraction,

automation (OA) system. It is an important resort to

data storage and management, data presentation

build higher level application making for the analysis

according to its work process. Data warehouse is a

Mapping

Management

or

corporations,

subject-oriented,

location of data for storage and analysis. OLAP is a

technology allowing client applications to access
these

data

builds

logic model
After seeing into the Sales Atomization System,

analysis-oriented multidimensional data model based

the Automatic Meter Reading system and so on, the

on the integrated data of the data warehouse. It also

paper picks up the power sales as the subject and

analyses and compares the subjects from several

designs the snowflake type model according to the

different angles of views using the multidimensional

related factors affecting the subject. The factors that

analysis method. It provides a validated analyzing

affect the power sales subject are: users (including

method through its powerful analysis function and a

the big users), time (year, half year, quarter, and

high performance decision-making for users who

month), industry classes, used electric levels, power

have definite analysis scope and analysis requests [6].

prices grades. As a result, the paper forms the data

OLAP

The foundation of the snowflake type data

the

III.

effectively.

set needed by the power query into three dimensions

DATA WAREHOUSE DESIGN AND IMPLEMENT

(time, power price, and user) and a measurement

Research background

attribute (power sales). Time dimension makes up of

Through the data presentation of power supply

month, quarter, half year, year; user dimension

and power sales, it can provide the power basic

makes up of units, industry and other attributes;

guideline data. The presentation may be about the

Power price dimension makes up of power price;

whole corporation, subsections, depart transformer

big user dimension makes up of big users and the

substations, depart voltage levels, depart time

attributes concerned by the power companies.

segments, and depart lines. After the data mining of

Moreover,

the basic data, it can carry out a lot of analysis, such

characters, such as units levels and industry classes.

as the power sales analysis in different time

Because of the quantity and the various levels of

segments, the power used in different areas, the

users, it is not possible to record every basic unit

power sales analysis in different electric prices,

exhaustively. Then, the paper selects several

power usage analysis in different industries, and

representative objects from the thin granularity

power usage analysis in different units and the big

level. For example, when analyzing the historical

users analysis. Data source is the power supply

data of the user granularity, the paper merely records

automotive system.

the big user. This is useful in practical applications.

Power analysis system contains modeling, ETL,

cube-building and final presentation. To achieve these
aims, the paper uses DB2 ESE, DB2 Warehouse
The whole frame is shown in figure 1.

Figure 1 system frame

dimension

distinguishes

user

According to the upwards analysis, the paper builds

System frame

Manager, DB2 Cube Views and DB2 Alphablox

user

[7]

the snowflake type logic model which is shown in

figure 2. The center of the snowflake model is
analysis content, corresponding to the fact table.
Around it is the access angles, corresponding to the
dimension table. Each dimension also can be
divided into different granularity [4].

extract of an object whose function is comparatively

absolute. For example, an extract process completes
the data extract of a dimension or the fact table. A
data extract process may come down to several
tables, and need to apply several data extract rules,
and do some data exchanges. Besides, there are
restricting relations in the data exchanges and a data
exchange needs to be done after one or several
exchanges. For example, user dimensions data
extract process needs to extract data from users basic
information table and clear the unneeded user
information. At the same time, it also appends the
Figure 2: snowflake type logic model
D

Building the data warehouse

The implement of data warehouse is completed

by several supporting tools. IBM DB2 provides the
supporting tools to implement the data warehouse,
such as Warehouse Manager and Cube View,
Alphablox and so on.

the final user dimension. In the extract process, we

also encounter a lot of problems. For instance, in the
process of extracting the fact table data using the
warehouse manager, because of the excessive data,
DB2 reports problem: transaction log is already full
[2]

. The paper solves it by input the command below

in db2 command center to augment the transaction

1) Data extract
Data extract is to clear and transform the data
sources from different places and the data of different
forms. This is complied with the definition of the
subject table, the data source definition, and the data
extract rules definition in the Meta database. The data
extract also reconstructs and processes the data to
load it into the target data warehouse. When
organizing the data from different sources, the paper
turns the data into a middle model and takes it into
the casual workspace. Processing the data is to
guarantee the integrality and the consistency of the
data in the database [2]. When extracting the data, the
paper cooperates intimately with the end users to
achieve the unification of the data in deed. Data
extract in early stage is implemented by handwork
programming and the program builder. But now, we
carry it out through high-effected tools, such as
Infomoter of Ardent Company, SAS/WA (Warehouse
Administrator) of SAS and the complete data
warehouse solutions provided by several data
warehouse companies

units levels and the industry classes of the users to

[5]

. In this project designing

and implement, the paper adopts the DB2 Warehouse

Manager of IBM to complete the data extract.
Each data extract process completes a data

log:
Update database configuration for SDL using logfilsiz 5120;
Update database configuration for SDL using logprimary 15;
Update database configuration for SDL using logsecond 100;

SDL is the name of the database

After the data extract, the system completes the

original load of the power sales analysis data
warehouse. Thereafter, the system periodically adds
to the data warehouse according to the update rules
and the load frequency.
2) Building multidimensional database
Up to now, the founded data warehouse only
contains current detail level data. In order to get the
comprehensive data set, the system selects OLAP
presentation as the interface between data warehouse
and users. This is in allusion to the data warehouse
model power system that will be built. Then the paper
can also satisfy the query and the report forms
requirement

of

the

decision-making

and

the

multidimensional environment. OLAP presentation

can reveal the data of the same subject in different
angles. Users can also make up the angles and the
mode of the presentation based on their requirement.
For instance, the user wants to analyze the power
sales subject. Accordingly, the paper presents the

power sales from different angles such as user

This is the ultimate aim of building data warehouse.

(including the big users), time (year, half year,

After we built the cube at the background of the

quarter, month), industry classes, unit level, and

server, we can connect with the Alphabloxs front

power price. When analyzing, the users can select all

presentation through the Alphabloxs Cube Server of

of the above angles and analyze in the composite

Alphablox tools. Query is the base of data

effects of the angles. They can also select their

presentation. So the Alphablox Cube Server Adapter

interested angles to analyze. The cube model built by

provided by DB2 Alphablox is mostly used for

IBM DB2 olap center is shown in figure 3.

carrying out the data query function of cube. Query

system

includes

MDX

(Multidimensional

Expression) query, visual interface query and so

forth.
I.

MDX query
DB2 Alphablox queries the DB2 Cube with the

MDX multidimensional query language. MDX is

component of the special query language OLE DB
which is created by Microsoft Company. DB2
Alphablox cube support MDX syntax and its
functions. We write peak quantity query program of
big user power sales using the MDX language as
follows:
Figure 3: cube model of power sales
According to the metadata and the input of

Query=" SELECT
DISTINCT( {[SDL-DKH].[price dimension]} )

users, DB2 Cube View use the optimize adviser to

DISTINCT(

make suggestions of corresponding MQT. The

AXIS(1)

optimization is based on the different levels and the

FROM [SDL-DKH]

requirement of every dimension. MQT can improve

WHERE

the

query

performance

of

the

GROUP BY,

ON AXIS(0),

{[SDL-DKH].[big client dimension]} )

GROUPING, RANK and ROLLUP OLAP functions.

[SDL-DKH]. [Measures].[peak price],

For example, after building the cube of power sales

[SDL-DKH]. [Time dimension]

as figure 3, the system transfers the model optimize

)"

adviser. The paper appoints the optimize slice like

II. Visual interface query

ON

this: time dimensions any; user dimensions any,

Alphablox designs the database visually through

power price dimensions power prices. Then, in the

the presentation template of IBM which includes

appointed time scope and sampling data, the system

three levels: edit.jsp, report.jsp, and template.xml.

gets suggested MQT tables sql sentences. If we are

Data subject is power sales analysis, and the data

content to the MQT, we can execute the sql sentences

dimensions are carried out through two dimensions or

and create the materialized table. In the process of

three dimensions views. The two dimensions view

analyzing mass data, db2 can automatically transfer

adopts straight square figure, histogram, graph, cake

the MQT. It is to get the optimization of time at the

figure and so on; and the three dimensions view

expense of space. So the query response time is

adopts solid straight square figure, solid cake figure,

shortened and the system is more efficient.

and radar figure and so on to build charts. Besides, on

3) Data presentation

the basis of dimension level design, we can drill up or

The data in data warehouse must be interactive

drill down on the graphical interfaces to achieve

with users and then become useful information. It can

multidimensional presentation. Figure 4 shows the

be final user-oriented and business subject-oriented.

power sales analysis system presentation.

information system is necessary. Based on the power

analysis system, we can improve the data warehouse
system response speed furthermore and develop the
front-end presentation more satisfactory for clients.
And the system also can be optimized to be base for
data mining.
ACKNOWLEDGEMENTS
The work is partially supported by Jiang Hua of
IBM China Company and the Dareway Company of
Shandong University.
Figure 4: power sales analysis system presentation
REFERENCE
IV.

CONCLUSION

[1] W. H. InmonBuilding the data warehouse (the 2nd edition)

The paper develops on the base of IBM DB2

[M] New YorkWiley Computer Pub1996

and introduces the design and the implement of

[2] Yang Qichang, Li Yulong etc. IBM DB2 high-level

power data warehouse. The data warehouse consists

management guide [M]. Beijing: publishing house of

of a lot of main functions. For example, it provides

electronics industry, 2004

powerful integrate analysis statistic function on the

[3] Wang Shan. Data warehouse technology and online analysis

mass power data and presenting exhaustive analysis

process [M]. Beijing: Science Press. 1998.

result. It also offers various report forms and charts to

[4] Lou Weijin, Kong Fansheng, Lou Weizhong. Data warehouse

do decision-making analysis. And the system

and knowledge discover [J]. Computer Engineering and

provides corresponding analysis result according to

Applications, 2000, 36(10): 111-113

different

operation

requirements

departments

through

each

application

authorization

[5] Zhang Wenying, Shu Hongchun, Zhang Ye. Application of

and

data warehouse technology in power consumption analysis

authentication platform. Contemporarily, it supports

of Kun Ming power network [A]. Power System Technology,

several databases like DB2, Oracle, Sybase and

2005.

several data interface like OLE DB, ODBC. The data

[6]

Ralph

Kimball

Laura

Reeves,

Margy

Ross

Warren

warehouse also supports multidimensional analysis

Thornthwaite. The data warehouse lifecycle toolkit: Expert

and

and

methods for designing, developing and deploying data

predigesting data analysis transaction logic. The data

warehouse. Beijing: Publishing home of Electronics Industry,

in data warehouse always keep consistent with the up

2004.

users

multi

angles

gathering

data

to date operation data, therefore it guarantees the data

currency. DB2 offers a complete disaster recover
method

and

establishes

the

base

for

the

decision-making and the data mining in later stage.

Data warehouse is suitable for the enterprises
with complex constructions and client environment.
Therefore, building data warehouse in power

[7] Michael. Gonzales. IBM Data Warehousing With IBM

Business Intelligence Tools [M]. John Wiley & Sons. 2003.
[8] Zhang Weimin, Zhang Xiaoying, Ju Di. The application of data
warehouse technology in telecom management system.
Electron engineer. 2000.2.
[9] Zhan Min, Sun Yufang. The implement of OLAP in insurance
system. Journal of Software. 2000.11.