Agile Data Governance Tutorial

Agile Data Governance
Tami Flowers
Director, Governance Solutions
MetaGovernance Solutions, LLC
DGIQ June 2016

Who am I?
 Tami Flowers
 Director, Governance Solutions at MetaGovernance
 PMI-PMP, PMI-ACP, and Certified Scrum Master
 Agile Coach
Developer Project Manager
Enterprise
Architect
Data Governance
Consultant
MetaGovernance Solutions LLC. All Rights Reserved. 2016

Today
 Data Governance
 Agile
 How to meld them together
 A few minutes on the topic
 An “experience”
 Debrief/Reflection

Experience: Ice Breaker
1. Write your answer to the following questions on sticky
notes. One answer per note. As many answers as you wish.
 What are you hoping to learn today?
 What specific questions are you hoping get answered today?
2. Discuss amongst your team. Group like answers together.
3. Have each person mark their level of experience with the
following:
No experience Some
experience
Quite a bit of
experience
Agile
Data Governance

Data Governance
 Two viewpoints
 Establishing a Data Governance Organization
Framework
 Implementing Data Governance  actionable Data
Governance

Data Management Functions
Data
Governance
Data
Architecture
Data
Development
Data
Operations
Data Security
Reference
and Master
Data
EDW and BI
Document &
Content
Metadata
Data Quality
Data Governance is at the heart of all Data Management activities.
As defined by the Data Management Association – Data Management Body of Knowledge (DAMA-DMBOK)

Data Governance Framework
Data
Governance
Organization
Data Management
Functions
Projects
Agile Data
Governance

Data Governance Organization Framework
Executives
Business & Technical
Members
Stewards/analysts/architects/
administrators/engineers
Consumers
Data Governance Council (DGC)
Data Governance
Steering Committee
(DGSC)
Data Stewardship
Teams (virtual)
• Establish & train DGC members
• Agree on Data Governance roadmap
• Determine members & empower DGSC
• Define Data Governance policy
• Meet on recurring basis
• Receive training and education
• Execute DG roadmap
• Define DG communication plan
• Select & empower stewardship teams
• Define roles and responsibilities
• Define charter
• Meet on recurring basis
• Technical and Business Members
• Receive training and education
• Execute DG roadmap
• Business Members
• Receive training and
education
End Users

Data Governance Organization Framework
 Actionable Data Governance
Executives
Business & Technical
Members
Stewards/analysts/architects/
administrators/engineers
Consumers
Data Governance Council (DGC)
Data Governance
Steering Committee
(DGSC)
Data Stewardship
Teams (virtual)
• Lead and promote program
• Empower DGSC
• Arbitrate data related issues, as needed
• Execute DG communication plan
• Define data management policies, procedures and
standards
• Define stakeholders, and sources and use of data
per subject area
• Approve and oversee corrective actions
• Technical and Business Members
• Implement policies, procedures,
standards and controls
• Identify and report data issues
• Corrective action input & execution
• Corrective action status reporting
• Business Members that
consume and use data
• Identify and report data
issues
End
Users

Registered Data Governance Stakeholders
Data Subject
Area ID
Major Subject
Group 1
st
Sub Grouping 2
nd
Sub Grouping
Accounting
Operations
Collateral
CreditRisk
Finance
FinancialReporting
InternalAudit
MarketRisk
Sales
LoanOperations
Compliance
Information
Technology
1 Trade Loan Adjustable-Rate
Loan
2 Trade Loan Fixed-Rate Loan
3 Trade Loan Structured Loan
4 Trade Loan Amortizing Loan
5 Trade Loan Letters of Credit
6 Trade Loan Prepayment Fee
11 Financial Accounting Accrued Interest
12 Financial Accounting Outstanding
Principle
LEGEND
CUSTODIAN
CONSUMER
STEWARD
DELEGATE

The Agile Manifesto
We are uncovering better ways of developing software by doing it and helping
others do it. Through this work we have come to value:
Individuals and interactions over Processes and tools
Working software over Comprehensive documentation
Customer collaboration over Contract negotiation
Responding to change over Following a plan
That is, while there is value in the items on the right, we value the items on the
left more.
In February 2001, 17 software developers met at the Snowbird resort in Utah to discuss lightweight development methods.
They published the Manifesto for Agile Software Development.

Agile Principles
1. Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.
2. Welcome changing requirements, even late in
development. Agile processes harness change for
the customer's competitive advantage.
3. Deliver working software frequently, from a
couple of weeks to a couple of months, with a
preference to the shorter timescale.
4. Business people and developers must work
together daily throughout the project.
5. Build projects around motivated individuals.
Give them the environment and support they need,
and trust them to get the job done.

Agile Principles
6. The most efficient and effective method of
conveying information to and within a development
team is face-to-face conversation.
7. Working software is the primary measure of progress.
8. Agile processes promote sustainable development.
The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.
9. Continuous attention to technical excellence
and good design enhances agility.
10. Simplicity--the art of maximizing the amount
of work not done--is essential.
11. The best architectures, requirements, and designs
emerge from self-organizing teams.
12. At regular intervals, the team reflects on how
to become more effective, then tunes and adjusts
its behavior accordingly.

Experience: Agile Principles
 How can we apply these principles to establishing a data
governance framework
 How can we apply these principles to actionable data
governance (day-to-day governance)?

The many flavors

Scrum

Lean Software Development
Eliminate
Waste
Empower
the Team
Deliver
Fast
Optimize
the Whole
Build
Quality In
Defer
Decisions
Amplify
Learning

Experience: Agile Practices to
Lean Principles
Agile Practice/Lean
Principle
Eliminate
Waste
Empower
theTeam
Deliver
Fast
Optimize
theWhole
Build
Qualityin
Defer
Decisions
Amplify
Learning
Teams make own
decisions
Just in time planning
Team retrospectives
Two week iterations
Unit test as we go
Shadow the business to
learn what they do

Experience: Data Governance to
Lean Principles
Agile Practice/Lean
Principle
Eliminate
Waste
Empower
theTeam
Deliver
Fast
Optimize
theWhole
Build
Qualityin
Defer
Decisions
Amplify
Learning
Teams make decisions
Just in time planning
Retrospectives
Frequent iterations
Validate/approve as we go
Involve the business
stakeholders

Kanban
 Capture the work
 Make it visible
 Each story is about the same size
 Limit the work in progress (WIP)

Experience: WIP
 8-10 people per team
 4 workers – will do the work
 4 managers – will watch over their worker
 1-2 timers – will time how long each round takes
 This is called the Penny Game and is available on
www.tastycupcakes.org

Commonalities of all
Agile/Lean flavors
 Capture the work
 Make it visible
 Minimally Viable Product
(MVP)
 Engage the owner/customer/
stakeholders
 Have owner prioritize the
work
 Work on the highest priority
work first
 Iterate
 Continually retrospect and
improve

Roles
 Stakeholders
 Product owner is one
 All have a stake in the project
 DGSC members
 Product Owner
 Chairperson of DGSC
 Has the vision and can convey it to the team and stakeholders
 Prioritizes backlog
 Provides direction to and is engaged with the team
 Scrum Master – facilitator
 Bulldozer: Remove roadblocks
 Servant leader: Do anything needed to help team perform at their highest level
 Stewardship Team Roles
 Stewards
 Developers/Engineers/Architects/Administrators
 Business Analysts/Quality Assurance Analysts

Ceremonies
Ceremonies Traditional
Agile/Lean Project
Data
Governance
Framework
Actionable Data
Governance
Planning Meetings Start of every
sprint/iteration
DGSC or
subgroup
meetings
Start of every
sprint/iteration
Scrums/Stand Ups Daily, 15 minutes 1-3 times a
week
Daily, 15 minutes
Review meetings End of every
scrum/iteration
DGSC or
subgroup
meetings
End of every
sprint/iteration
Retrospectives End of every
sprint/iteration
End of every
sprint/iteration
End of every
sprint/ iteration
Backlog Grooming In the week
between
sprints/iterations
In between
sprints/
iterations
In the week
between
sprints/iterations

4 quadrants to help with
backlog grooming
Broccoli Organic Beef
Twinkies Éclair
Effort
Value
One of our classmates shared this quadrant with us during a discussion. They use this
in their organization to help discuss the value of a story in the backlog and it helps
them select stories for the sprint/iteration.

Artifacts
Artifacts Traditional
Agile/Lean
Data Governance
Framework
Actionable Data
Governance
Product Backlog Created from
conversations
with stakeholders
Created from
Data Governance
Roadmap
Created from DG
policies and
procedures
Sprint/Iteration
Backlog
Prioritized by
product owner
during planning
meeting
Prioritized by
DGSC
chairperson
during planning
meeting
Prioritized by
product owner
during planning
meeting
Stories Created from
conversations
with stakeholders
Created from
Data Governance
Roadmap
Created from DG
policies and
procedures

Data Governance artifacts
within a project
 Stories
 As a <role> I want <what> so that <business value>
 If not a story, include data governance as acceptance
criteria in stories
 As an accountant, I want current balances for loans, so that I
can complete daily reporting
 All data elements are defined in the business dictionary
 Data profiling of the balances occurs when loading into the EDW
 An automated control is added so the current balance is
automatically reconciled between the G/L and Loan System

Reality Check
 Agile isn’t a silver bullet that will magically fix
everything
 New, unexpected challenges will occur
 Teamwork
 Soft skills
 Growing pains

Reality Check
 Data Governance isn’t a silver bullet that will be easy to
quickly implement
 Must include the business
 Must be added to existing processes to be successful
 Lots of training and education is needed
 Executive support is critical
 Will uncover data issues that some don’t want uncovered

Experience: Adding Governance
to a Project
 Identify what is needed to add governance to a project.
 What 2-3 items are the highest priority?
 Project: bring new data source in
 Needs: Ensure all registered stakeholders and their data
governance roles are known. Identify and define business
attributes. Profile data for potential issues.

Group experience: What stories are
needed if we are bringing in a new
data source?
 Identify data quality dimensions and thresholds
 Identify archiving and data retention needs
 Classify the data for visibility and security
 Identify data stewards
 Establish profiling and cleansing rules
 Establish SLAs with the provider
 Identify legal aspects, regulatory impacts
 Identify restrictions on the data, permissible use
 Evaluate risk involved with data
 Determine requirements to match to master or reference data; identity attributes
 Determine upwards and downward impacts to business processes
 Configuration management; hardware/software needed, documentation
 Define metadata, definitions, associations.

Registered Data Governance Stakeholders (we
also want to identify and record this metadata
each iteration/sprint)
Data Subject
Area ID
Major Subject
Group 1
st
Sub Grouping 2
nd
Sub Grouping
Accounting
Operations
Collateral
CreditRisk
Finance
FinancialReporting
InternalAudit
MarketRisk
Sales
LoanOperations
Compliance
Information
Technology
1 Trade Loan Adjustable-Rate
Loan
2 Trade Loan Fixed-Rate Loan
3 Trade Loan Structured Loan
4 Trade Loan Amortizing Loan
5 Trade Loan Letters of Credit
6 Trade Loan Prepayment Fee
11 Financial Accounting Accrued Interest
12 Financial Accounting Outstanding
Principle
LEGEND
CUSTODIAN
CONSUMER
STEWARD
DELEGATE

Data Governance (Overall Framework)
Provide data governance common
vocabulary
Define key governance roles
Define roles & responsibilities of key gov
roles
Define data governance policy
Define concept of data governance
controls
Project Specific Data Governance –Risk Project
Confirm data subject areas within project
scope
Confirm system of record for each subject
area
Define data quality issues and standards
Define data quality testing & remediation
strategy
Confirm data gov roles for each subject area
Determine existing data controls for each sub
area
Determine recommended data gov controls
Define recommended data gov control &
quality implementation priority
Obtain DGWG approval
FEB MAR APR MAY JUN JULJAN AUG SEP
Risk Project Timeline (sample timeline)

Experience: Setting up a
Governance Framework
 Identify what is needed to setup a governance framework.
Generally this is done by looking at the data governance
roadmap (see next slide for example)
 Have the product owner prioritize the first 3 items.

Data Governance (DG) Framework
Establish Data Governance Council (DGC)
- Identify and appoint members
- Train and educate DGC
- Agree on common data governance vocabulary
- Agree on common DG stakeholder roles
- Agree on roles and responsibilities for stakeholders
- Agree on scope, authority and structure of DGC
- Agree on initial DGSC members
- Define DGC charter and guiding principles
- Agree on key metrics to evaluate governance
- Participate in recurring DGC meetings
- Champion and promote DG
Establish Data Governance Steering Committee
- Identify and appoint members
- Train and educate DGSC
- Define common data governance vocabulary
- Define key DG stakeholder roles
- Define roles and responsibilities for DG roles
- Define scope, authority and structure of DGSC
- Identify and appoint data stewards
- Agree on DGSC charter and guiding principles
- Define objectives of DGSC
- Identify Governance Coordinator for DG activities
- Participate in recurring DGSC meetings
- Champion, communicate and promote DG
Data Governance Framework Sample Timeline
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8

Ways Agile can help
 Increased communication and visibility
 Increased teamwork
 More frequent delivery of business value
 Implement governance on high priority items first, show
success, move on…continuous improvement

Continuous improvement
 Use retrospectives to make your process better
 It’s ok to add just 1 or 2 new things at a time
 Get help if you need it, lots of resources and experts are out
there
 Mentoring and coaching will be critical

Experience: Temperature
Reading
 Appreciations
 Puzzles
 Complaints with recommendations
 Hopes and wishes
 New Information

Retrospectives
 At end of each sprint/iteration
 What went well – keep doing
 What didn’t go well – stop doing
 What could go better – change it
 Entire team attends

Experience: Retrospective
 What went well
 What didn’t go well
 What could go better

Lessons learned
 There isn’t one definition of Agile; all companies tend to do it
differently. That’s ok. Make it what works best for your
organization.
 If you are setting up governance for the first time, get the
framework in place first. Then determine low hanging fruit, such
as data quality issues, and implement governance starting
there.
 Data Governance MUST involve the business; it can’t just be an
IT focus.
 A lot of training and education is needed for governance.
 Data Governance must be added to existing processes and
procedures in order to give it teeth.
 SDLC
 Architecture reviews

Good resources for Agile and
Team Building
 Mike Cohn – MountainGoatSoftware.com
 PMI-ACP Exam Prep – Mike Griffiths
 Johanna Rothman books: http://www.jrothman.com/books/
 The Human Side of Agile, Gil Broza
 Agile Retrospectives: Making Good Teams Great, Esther Derby and Diana Larsen
 Crucial Conversations, Kerry Patterson
 Crucial Confrontations, Kerry Patterson
 Influencer
 The Deadline: A Novel about Project Management, Tom DeMarco
 Teamwork is an Individual Skill, Christopher Avery
 Behind Closed Doors: Secrets of Great Management, Johanna Rothman and Esther
Derby
 DGIQ presentations: Roberta, Maggie from GM

Me
 Twitter: TamiLFlowers
 LinkedIn: Tami Flowers
 Email: tflowers@metagovernance.com
 Slideshare: www.slideshare.nettamiflowers
 www.metagovernance.com (blog)
 Blog for DataVersity
 Thanks!

Agile Data Governance Tutorial

More Related Content

What's hot(20)

Viewers also liked(20)

Similar to Agile Data Governance Tutorial(20)

More from Tami Flowers(7)

Recently uploaded(20)

Agile Data Governance Tutorial

Editor's Notes