STRATEGY EXECUTION

Data Quality: You Don't Just

Need a Dashboard!
By Peter Graham

Peter would liiie to thank Brian Swarbrick of Palladium Figure 1: Gartner's Magic Quadrant for Data
Group for iois contribution to tiois column.
Quality Tools, 2007

T
hroughout this series, I have discussed how or- Challengers Leaders
ganizations can become information empowered,
Trillium Software
with an inherent assumption that it's accurate • • Business Objects
information. When this assumption is incorrect, • DataFlux
the results can be devastating: decisions made Group 1 Software • IBM
with poor or incomplete information can often
, FUZZY! Informatik • Informática
impact a company's bottom line more negatively than if the O
II I ^ •Human Inference
information were not presented at all. This month, I'll discuss % uniserv ^ Innovative Systems
how data quality issues occur and how your organization •DataMentors ,
Netrics • DataLever
can begin to identify and address them. Datanomic*
Data quality issues arise for a variety of reasons, includ- Datactics •
ing acquisitions of new data sources, poorly designed and
developed applications, and ironically business intelligence Niche Players Visionaries
(BI) implementations, where there can be focus on data pres- HCompleteness of Vision f
entation to the detriment of understanding information accuracy.
As of June 2007
The popular adage "you just need a dashboard" doesn't apply
anymore. A well-architected BI solution that enforces data in- Source: Gartner Inc., June 2007
tegrity and implements robust data quality procedures is a create a roadmap for enterprise data quality improvement.
requirement for any enterprise solution but should not be seen Depending on organization size, the data steward(s)
as a complete solution for the organization's data quality issues. should have complete responsibility for data quality gover-
Data quality is a business issue that transcends the entire organ- nance in his or her area. These gatekeepers must understand
ization, including operational systems. their respective systems, the quality of information across those
Gartner predicted that more than 50 percent of BI proj- systems and the implications of poor data quality. They
ects through 2007 would fail or receive lack of acceptance must be subject matter experts that provide direction anywhere
because of data quality issues.' TDWI estimated that customer their information is used and be the sole point of contact for
data quality issues alone cost U.S. businesses over $600 bil- all decisions related to the data they own. This data gover-
lion dollars a year.^ With the enactment of the Sarbanes-Oxley nance role should fall within the BICC if one exists.
(SOX) Act, quality of information is an ever-increasing pri- The rr organization should support the data quality improve-
ority. A clear initiative that focuses on the data quality ment initiative, ensuring effective implementation of processes
challenge is required to address this situation. and procedures and making supporting technologies available.
Data quality is a business issue, and business managers To assist IT in this endeavor, many vendors now offer soft-
should drive improvement. The first step in identifying data ware solutions that support enterprise data quality initiatives.
quality issues is to interview business users across the enterprise, Gartner now categorizes these data quality products as a sepa-
assessing their use of data and potential opportunities to improve rate commodity, although vendors offer several of the underlying
data quality. This assessment can be done using external re- components as standalone offerings (see Figure 1).
sources or leveraging roles from the business intelligence These tools help to identify, discover and improve data
competency center. The key to this assessment is to understand quality anomalies. Typically required for implementing qual-
business uses of data, therefore this requires an individual with ity BI applications, these products should also be used to
business expertise. Once this assessment is complete, the data should support data quality initiatives beyond a BI engagement.
be traced to its source systems to investigate the cause of poten- Typical functionalities available in these products include:
tial issues. From a BICC perspective, the data architect is often ' Data discovery: Quickly analyzes large volumes of data
most qualified to lead this investigation. While this investigation across different applications and provides statistics regard-
could be done through human evaluation of data and code, ing the data content (unique values, missing content, data
there are now many data discovery tools that evaluate data. Once formats, etc.). Analyses can be used to drive change back
documented, data quality opportunities should be prioritized based through the operational systems and BI applications.
on the perceived financial and operational impact, the resources ' Data cleansing: Integrates and cleanses customer data
required to address them and any project interdependencies to GRAHAM continued on page 37

36 August 2008 I DM Review www.dmreview.com

 SCEALES and MORRIS continued from page 21 GRAHAM continued from page 36
flexibility to continue migrating while this is done. (primarily name and address data) from multiple sources and geo-
graphic locations for individuals and organizations.
Rule four: If you can't count it, it doesn't • Matching or householding: Groups individuals or organizations
count. Another challenge is how to meas- together for deduplication or to understand relationships, a task
ure data quality in order to assess the impossible to achieve without a robust data cleansing strategy.
state of your legacy data and determine ' Ongoing data quality monitoring: Tracks data quality improvements
the level of quality your business users over time.
require. To make matters worse, data quality • Master data management: Defines and maintains the data critical
is not static; it erodes and improves over time. It is to an organization's business processes.
important that the measures you use make sense to ' Integration with Bl and operational systems: Defines rules for improv-
business users and not just to technologists. This ing data quality that existing applications can integrate. Changes to
enables you to measure deliverabies, perform gap rules are automatically available to subscribers of that information.
analyses and monitor and improve ongoing data qual- To be truly successful, a data quality initiative must be adopted
ity. It also ensures that you are concentrating your across the organization, and there must be no barriers. With IT and
efforts on where business users see value and can business management working together, improvements in the qual-
quantify the benefits. ity of data can be adopted across the operational systems. Bl
Reconciliation of the data migrated from applications will also benefit from improvements in quality. Business
source(s) to target(s) is always a critical activity - decisions will become reliable, and in turn, impact on the bottom line
how do you know when you're done? When dealing will be positive. O
with dynamic environments where you can't freeze
the data, this becomes even more challenging; References:
1. "Gartner Identifies the 'Fatal Flaws' of Business Intelligence and Advises Organisations
you need to be able to handle a shifting scope. A on How to Avoid Them." Press Release from Gartnei; Inc., February 3,2005.
flexible data model and a closely coupled 2. "The Data Warehousing Institute's Recent Study Finds High Quality Data Is Critical To
reporting capability are key to understanding and The Success Of Businesses Worldwide." Press Release from TDWI, February 1,2002.
driving this process.
Peter Graham is the vice president ofthe tnformation Strategy and Architecture practice
at Palladium Group, Inc. He can be reached at pgraham@thepalladiumgroup.com.
Achieving a Business-Driven Migration
Having put a business-driven migration project in place,
the trick is to select methodologies and technologies that
can deliver against these requirements. A business-driven
migration involves decoupling the technical problem of
moving the data from the business processes that use it.
This requires a migration solution that enables you to
easily encapsulate the business problems you face while
being flexible enough to cope when those requirements
change. This ensures that ROI from new applications is AddressObject API
maximized and operations are enhanced rather than
adversely affected. Prevent errors before they occur to
The critical importance of getting the migration ensure the integrity of your database.
right from a business perspective was articulated at a
Use AddressObject API to verify,
recent British Computer Society meeting by BT's Phil
Dance: "Increasingly, our business case is going to cieanse and format customer data at
depend on how good we are at getting our data the point of entry or in batch. Address
across. A bad data migration ultimately means a bad Object easiiy integrates with Perl, PHP
customer migration, and in a competitive market, Java and .NET. - offering a better way
that's very bad news." O
to build-in data vaiidation.
References:
1. Phil Howard and Carl Potter. "Data Migration in the Global 2000 - The ability to maintain clean, reliable
Research, Forcasts and Survey Results." Bloor Research, September marketing data across multiple capture
2007. mediums is essential for everything we do
2. Phil Dance. "BT PIC's CIO for Technology." Data Migration - Kevin Parker, Digital Evergreen
Matters, November 2007.

Tony Sceales is CTO of third-generation migration technology vendor,

Celona Technologies. He may be reached at tony.sceates@celona.com.

John Morris is director of iergo ltd and author of Practicat Data

Migration (British Computer Society, 2006). He may be reached at
imorris@iergo.co.uk.

www.dmreview.com DM Review I August 2008 37