Information Management

IBM Software Group

Discovering the Value of

IBM InfoSphere Information Analyzer

Steven Green
[email protected]
1 Discovering the Value of IBM InfoSphere Information Server © 2013 IBM Corporation
 Information Management

InfoSphere Information Analyzer

Requirements
 Perform data quality
Assess data quality and assessment
facilitate ongoing data quality
 Define business rules to
monitoring and exception
monitor data quality
management
 Establish stewards for
governance of data
quality

Benefits
 Identify data quality
issues early to reduce
project risks
 Monitor quality metrics
over time for compliance
 Create business
confidence with trusted
information
 Results promotable
across IBM Information
Server
© 2013 IBM Corporation
Information Management

Common Data Problems

© 2013 IBM Corporation

 Information Management

Pain – The Cost of Dirty Data

Scrap and rework
83% of data integration Increased costs
projects either overrun or fail

Lack of consumer
confidence

Lost
Inaccurate or incomplete data opportunities
is a leading cause of failure in
business-intelligence and Low data quality costs
CRM projects companies $611 billion
annually
25% of time is
spent clarifying Undetected defects will cost 10 to
bad data 100 times as much to fix upstream

© 2013 IBM Corporation

Information Management

Business Drivers For Information Quality

 Poor data quality costs U.S. businesses over $600 billion each year

 Data deteriorates up to 3% every month

 What is the key to integrating corporate data? Having the right data before
you start

Ensuring adequate data quality

Understanding source data
Creating complex transformations
Creating complex mappings
Ensuring adequate performance
Collecting and maintaining meta data
Finding skilled programmers
Providing access to meta data
Ensuring adequate scalability
Integrating 3rd party tools
Ensuring adequate reliability

0 10 20 30 40 50 60 70 80 90 100
© 2013 IBM Corporation
Information Management

InfoSphere Information Server – Profiling and Quality Features

Column Analysis

Quality Monitoring Key Analysis

Cross Domain Analysis

Quality Rule Validation

© 2013 IBM Corporation

Information Management

InfoSphere Information Server – Profiling and Quality Features

Column Analysis

Understanding Source Data

1. Assess data content

2. Assess data Structure
3. Quality within and across
heterogenous systems

© 2013 IBM Corporation

Information Management

Column Analysis
 Domain values and validation
 Data classification
 Data properties
 Formats

8 Discovering the Value of IBM InfoSphere Information Server © 2013 IBM Corporation
Information Management

InfoSphere Information Server – Profiling and Quality Features

© 2013 IBM Corporation

Information Management

InfoSphere Information Server – Profiling and Quality Features

Column Analysis

Gain Insight into your data

© 2013 IBM Corporation

Information Management

InfoSphere Information Server – Profiling and Quality Features

Column Analysis

Validation of keys
Key Analysis
1. Primary/Foreign Key
2. Data Preview
3. Data Relationships

© 2013 IBM Corporation

Information Management

Primary Foreign Key Discovery

 Automated Primary-foreign key discovery
 Full statistics on discovered keys
 Data preview
 Missed records, orphan foreign keys

12 Discovering the Value of IBM InfoSphere Information Server © 2013 IBM Corporation
Information Management

InfoSphere Information Server – Profiling and Quality Features

© 2013 IBM Corporation

Information Management

InfoSphere Information Server – Profiling and Quality Features

Column Analysis

Gain Insight into your data

Improve Time to Value Key Analysis

© 2013 IBM Corporation

Information Management

InfoSphere Information Analyzer – Profiling Features

Column Analysis

Assess Data Integrity

1. Redundant Information Key Analysis

2. Referential Integrity
3. Unknown business rules

Cross Domain Analysis

© 2013 IBM Corporation

Information Management

Cross Domain Analysis

 Cross-domain relationships
 Data redundancy
 Data preview
 Missed records, orphan foreign keys

16 Discovering the Value of IBM InfoSphere Information Server © 2013 IBM Corporation
Information Management

InfoSphere Information Server – Profiling and Quality Features

© 2013 IBM Corporation

Information Management

InfoSphere Information Server – Profiling and Quality Features

Column Analysis

Gain Insight into your data

Improve Time to Value Key Analysis

Increase Productivity

Cross Domain Analysis

© 2013 IBM Corporation

Information Management

InfoSphere Information Server – Profiling and Quality Features

Column Analysis

Uncover Data Issues

1. Missing Values Key Analysis

2. Inconsistencies
3. Type alignment

Cross Domain Analysis

Quality Rule Validation

© 2013 IBM Corporation

Information Management

Rule Based Analysis

 Rules Analysis: Enables ongoing measurement and baseline reporting of information
quality
– Validation of data rules individually or within a broader set
– Establishment of benchmark thresholds

20 Discovering the Value of IBM InfoSphere Information Server © 2013 IBM Corporation
Information Management

Over 130 Data Rules Available for Everyone!

 Predefined rule definitions available out-of-the-box to reduce effort
– Populated for all projects when Information Analyzer is installed
– ~200 rules cover a broad array of common data validation conditions
• Common domains: keys, national identifiers, dates, country codes, email addresses, etc.
• Basic conditions: completeness checks, valid values, range checks, aggregated totals,
equations, etc.
– Serve as models, templates, and examples for additional rule design
– Copy within a project and make changes to establish your own models

Increased Productivity and

rule examples –
out of the box!
© 2013 IBM Corporation
Information Management

InfoSphere Information Server – Profiling and Quality Features

© 2013 IBM Corporation

Information Management

InfoSphere Information Server – Profiling and Quality Features

Column Analysis

Gain Insight into your data

Improve Time to Value Key Analysis

Increase Productivity

Ensure Trusted Information

Cross Domain Analysis

Quality Rule Validation

© 2013 IBM Corporation

Information Management

InfoSphere Information Server – Profiling and Quality Features

Column Analysis

Monitor Over Time

Quality Monitoring Key Analysis
1. Baseline Reporting
2. Validation of Rules
3. Benchmark Thresholds

Cross Domain Analysis

Quality Rule Validation

© 2013 IBM Corporation

Information Management

Continuously Manage and Monitor Data Quality

Establish baselines of information

 Where do we need to implement more stringent controls?
 How do we ensure that critical data meets our standards?
Identify and mitigate risk.
View differences
between different
executions including
baseline Analyze trends

25 Discovering the Value of IBM InfoSphere Information Server © 2013 IBM Corporation
Information Management

InfoSphere Information Server – Profiling and Quality Features

© 2013 IBM Corporation

Information Management

InfoSphere Information Server – Profiling and Quality Features

Column Analysis

Gain Insight into your data

Quality Monitoring Improve Time to Value Key Analysis

Increase Productivity

Ensure Trusted Information

Eliminate Risk

Quality Rule Validation Cross Domain Analysis

© 2013 IBM Corporation

Information Management

InfoSphere Information Server – Profiling and Quality Features

“Just having an ETL engine is not enough,” says Janes. “With Information
Analyzer, we can see what the data actually looks like, quickly adjust
project requirements and refine development code early in a project’s
lifecycle.”
– Kevin Janes, Senior Solutions Architect, Shared Health
“We now can deliver services that previously only larger companies could
handle,” says Janes. “IBM Information Server is an integral part of our
ability to be who we are.”
– Kevin Janes, Senior Solutions Architect, Shared Health

InfoSphere Information Analyzer's data analysis capabilites are multi-faceted to suit any
organization's data analysis and profiling needs. The addition of Rules analysis is a key
aspect for organizations looking to be able to track data quality over time. Our clients tell us
that trusted information based on solid data quality is critical for them to take sound
business decisions in order to be successful and Information Analyzer helps customers do
just that.

- Timothy Moon, Managing Director – Zenith Solutions

© 2013 IBM Corporation

Information Management

ITALIAN HINDI FRENCH JAPANESE BRAZILIAN PORTUGUESE SIMPLIFIED CHINESE

TRADITIONAL CHINESE SPANISH RUSSIAN TAMIL THAI GERMAN ARABIC

29 Discovering the Value of IBM InfoSphere Information Server © 2013 IBM Corporation