Learning to Drive a CAR at the

U.S. Census Bureau

Capability, Agility & Resilience

in Software/Systems Solution Development

March 22, 2017

Harry A. Lee, Assistant Director for

Information Technology and Deputy CIO
1
 Road Trip
 Why a CAR?
 The Census Enterprise Data Collection and
Processing (CEDCaP) Journey
 Planning for the Trip
 Making the Trip
 Did We Arrive Safely?
 Ready to Go Again?

2
 Why a CAR?
 A means to transport you from a current place
(“As Is”) to a desired place (“To Be”)
 Capability - Know where you want to go
 Agility - Understand that there may be multiple
ways to get there, and that you want to have
checkpoints along the way to make sure you are
on the right course
 Resilience - Realize that you may hit road bumps
or road blocks - things may change or go wrong
along the way
3
 Why a CAR?
 Capability – Define what is needed
 Technical Integration Framework
 Agility – Demonstrate progress
 Lifecycle Management Frameworks
 Scaled Agile Framework (SAFe)
 Resilience – Prepare for change and risks
 Systems Readiness Framework
 Risk Management Framework

4
 The CEDCaP Journey
 The Census Bureau’s mission is to serve as the
leading source of quality data about the
nation's people and economy.
 The Census Bureau is committed to shared
services as a means to harness the benefits of
enterprise-wide solutions over duplicative,
single-use solutions.

5
 The CEDCaP Journey
 The Census Enterprise Data Collection and Processing
(CEDCaP) Program is creating an integrated and
standardized enterprise suite of systems that offers shared
data collection and processing services for all censuses and
surveys.
 Constitutionally mandated census of population and
housing every 10 years (Decennial Census)
 Census of all business establishments and of all
governmental units every 5 years (Economic Census and
Census of Governments)
 Several ongoing business and household surveys that
provide the information in several of the Nation’s key
economic indicators
6
 A BPS
1

A CPRS
2
Censuses & Surveys
R MHS
3
1) Construction Indicators

R RHFS
4

AR SOC
5

R SOMA
6

A VIP
7

R PC
8

R CJRP
9

R
10
2) Criminal Justice

CJEE R
11

JRFC R
12

NPS8 R
13

SSV A
14

ACES A
15
3) Cross Sector

ASE A
16

SQ-
Construction Manufacture Rental Value of Census of Adult Census Criminal Juvenile Survey
Building Survey Survey of Market
& State Federal
National Annual Annual CLASS
Permits Progress d Housing of Absorption Construction of Juveniles Justice Residential Prisoner of Sexual Capital Survey of
A Survey Reporting Homes Finance Construction of New Multifamily Put in Place Correctional Facilities in Expenditure Facility Statistics – 8 Victimization Expenditures Entrepreneur
Business &
Professional
A
Survey Survey Survey Units Survey Residential & Census Cap. Punish. Survey s
Classificatio
6/18 13 5/16 13 9/17 12,13,42,O 12/17 12,13 5/17 13 3/17 12 5/16 13 NAEvery 5 or 6 Years
15,42 5/16 Placement 42 NA Employment15,42 11/16 42 11/16 42,O 6/17 O 2/17 13 NA NA 9/16 13,26
Monthly/Annually Monthly Biennially Monthly Mnthly/Qrtly/Annlly Monthly/Annually NA Annually Annually n
$1,937,000 $4,700,000 Monthly $3,450,000 NA $1,750,000 $4,700,000 NA/NA Biennially Biennially $326,800 $689,800 Annually Annually Survey
2,000/500 252,000/49,000 $720,000 45,207/9,447 900/225 12,000/6,000 252,000/49,000 $508,844 Annually $483,000 3,298/1,636 1,538/1,353 $4,600,000 NA
3) Cross Sector 4) Education
Provide Economy-Wide Business Related Statistics &

4,800/1,600 2,550/9,243 NA 2,545/5,090 75,000/146,57 5) Employment

NA 6) Financial Indicators
Quarterly
0 $750,000
NA
A BDS A COS A CBP & A ESIA R SBLS A ICTS A NES A NCE A SBO R CCD R EDGE R NPEFS R PLS R SABS A QWI A QFR
52,000/11,267
Business Company FDIC Information Non- Nonresidenti Survey Common Education National Public School Quarterly Quarterly
Dynamics
ZBP Economic
Small & al Core of Public Libraries Attendance Financial
Organization Census Employer of Demographi Workforce
B Statistics Survey
County & Zip
Code
of Business Communicat Statistics Coverage Business Data c and Education Survey Boundary Indicators Report B
Island Areas Lending ion Evaluation Owners (Non-Fiscal) Geographic Financial Survey
Business
NA NA 5/18 13 NA 13,26 12/13 13 8/16 Survey 13 7/17 Technology 13 NA 13,26 NA O 2/15 13 NA 13 NA Estimates NA 1/16 Survey 20,O 12/16 O 6/16 20 NA 13 12/15 13,O
Annually Annually Patterns Quinquenniall Annually Annually Biennially Quarterly Quarterly
Survey
NA $11,300,000 y Annually Monthly Quinquennially NA Annually Annually $1,055,663 $2,000,000 NA $5,800,000
Economic Indicators

NA/NA 47,000/143,60 Annually NA NA/NA 55/4,417 13,600/13,600 NA/NA 50,296/115,11

Periodic Annually NA NA NA NA $1,446,503
7) Health Statistics 8 NA 8) International Trade Indicators
59,100/56,825 NA/NA 9) Manufacturing Sector 56/5,264 10) Public Sector 1
NA $1,700,000 NA/NA 875,000/149,1 NA/NA
NA/NA
16,500 45,000/79,286 67
R MEPS A AR A MITR A FT-900a A PIE A RP A FT-900 A ASM A M3 A M3UFO R MECS R QPC A ASLGF A ASPEP A ASPP A ASFIN
Medical Advance Manufacturi Preliminary Profile of Related U.S. Annual Manufacturers’ Manufacture Manufacturi Quarterly Annual Annual Annual Annual
Expenditure Report ng & U.S. Imports Importing & Parties International Survey Shipments, rs’ ng Survey of Survey of Survey Survey of Survey of
C Panel Survey International for Exporting Trade in of Inventories, & Unfilled Energy Plant Local of Public Public State C
Trade Report Consumptio Goods & Manufacture Orders Survey Orders Consumptio Capacity Government Employmen Pensions Government
12/15 13,42 NA NA NA 13 NA n of Steel 13 NA 13 NA NA NA Services 13 11/17 s 13 11/17 13 4/16 Survey 13 1/18 n 15 8/15 Utilization 13 NA Finances 13 9/15 t and Payroll 13 NA 13 NA Finances 13
Annually Monthly Products Annually Annually Monthly Survey
$10,400,000 NA Annually NA NA Monthly $3,600,000 Quarterly Annually Annually Annually Annually
124,404/25,15 NA/NA NA NA/NA NA/NA NA 51,600/17,200 Annually $1,227,700 NA $2,100,000 NA NA
Monthly Annually Periodic
1 NA/NA 10) Public Sector NA NA/NA 11) Research & Development
$6,509,000 $388,323 $1,331,723 12) Retail & Wholesale Indicators
30,000/62,500 NA/NA 5,872/47,903 NA/NA 13) Service
NA/NA Sector
NA/NA 68,000/172540 6,000/3,000 5,167/47,603
A STC A CoG R FAC A GUS A QTAX A QSPP R BRDIS R R&D R SGRD A MARTS A ARTS A AWTS A BES A MRTS A MWTS A QSS
Annual Census Federal Government Quarterly Summary Quarterly Business Research, Research & State Advance Annual Annual Business Monthly Monthly Quarterly
Survey of Audit Units of State & Local Survey of Development, & Developmen Government Monthly Retail Trade Wholesale Expenses Retail Trade Wholesale Services
D of State Government Clearinghous Survey Government Tax Public Innovation t Research & Retail Trade Survey Trade Supplement Survey Trade Survey Survey D
Government s e Revenue Pensions Survey Link Developmen Survey Survey
NA Tax 13 12/08 13 NA O 12/12 13 9/16 13 9/15 13 12/15 13 NA NA 5/16 t 13,15,O 8/15 13 3/18 13 3/18 13 12/08 13 7/17 13 7/17 13 8/15 13
Collections Quinquenniall Quarterly Annually Annually Annually Supplement Depend. Monthly Monthly Quarterly
Quinquenniall Continuously y $5,400,000 $877,000 Quarterly $5,000,000 Annually Biennially Monthly $2,800,000 $1,200,000 NA $3,400,000 $1,500,000 $7,770,000
Annually y NA 76,000/57,375 29,536/8,011 $220,000 133,000/775,990 NA $62,500 $1,595,838 20,557/11,426 7,210/3,663 28,363/76,850 123,660/14,42 50,400/5,880 94,000/20,900
13) Service Sector 14) Transportation
NA 15) Miscellaneous
NA/NA 4 & 10 1,9,12,13 & Mining 400/300 NA/NA 276/542 60,000/5,000 7
NA Table 1 Table 2
NA/NA NA/NA AR Funding Type: (Appropriated (A) or Reimbursable (R) X EX Directorate Responsible for the Census/Survey (See Table
A SAS R CFS A SUSB F-33 A EC Economic Directorate 1)
12 = Title 12 of the United States Code outlines the role of Banks and Banking
13 = Title 13 of the United States Code outlines the role of the United States Census
Service Commodity Statistics Annual Economic 15 = Title 15 of the United States Code outlines the role of the Commerce and Trade
Example “Star” Indicates a Mission Critical Census/Survey
Annual Flow of Survey Census Decennial Directorate Census or Survey Name 16 = Title 16 of the United States Code outlines the role of Conservation
E Survey Survey U.S. of School
Census or
Survey 20 = Title 20 of the United States Code outlines the role of Education E
Businesses District Demographic Directorate Information Collection Expiration NA NA Information Collection Authorizing Statues (See Table 26 = Title 26 of the United States Code outlines the Internal Revenue Code
12/15 13 9/13 13 NA 13,26 8/15 Finances 13 NA 13,26 Date 2) 29 = Title 29 of the United States Code is a code that outlines the role of Labor
Quinquenniall Field Directorate Periodicit Annual Cost to Federal Government 42 = Title 42 of the United States Code deals with Public Health, Social Welfare, & Civil
Annually Quinquenniall Annually
y Rights
$14,770,000 y Annually Annually y # of Approved Annual Responses $XXX Approved Annual Time Burden (Hours)
NA Research & Methodology Directorate A = Article 1, Section 2, Clause 3 of the U.S. Constitution
83,648/290,36 $23,000,000 NA $5,800,000 XX / XX
8 A) Criminal Justice
400,000/800,0 NA/NA 3,232/3,625 NA B) Economic Characteristics C) Education
O = Other
00
R ITS R NCVS R PPCS A CPS- R CES R CED R CEQ A SIPP R TPOPS R BTLS R NITES R NHES R NSCG R NTPS R PFS R SCS
Identity National Police-Public ASEC Consumer Consumer Consumer Survey Telephone Beginning National National National National Principal School
Theft Crime Contact Expenditure Expenditure Expenditures of Income & Point of Teacher Adult Household Survey of Teacher & Follow-Up Crime
F Supplement Victimization Survey
Annual
Survey Diary Survey Quarterly Program Purchase Longitudinal Training and Education College Principal Study Supplement F
Social &
Survey Participation Survey Study Education Survey Graduates Survey
Economic
3/15 42 8/15 13,42 12/16 13,42 7/15 13,29 NA 13,29 12/17 13,29 NA 13,29 12/16 13 3/18 13,29 10/12 O 9/17 Survey O 9/17 20 8/15 13,42,O 1/18 13,O NA NA 9/17 13,42
Supplement
Publish Household Counts

Biennially Monthly Approx. 3 Annually Quarterly Quadrennially Periodically Biennially Biennially Biennially Biennially Biennially
& Demographic Statistics

Annually Annually
$1,020,104 NA Years of the CPS NA $41,000,000 NA $51,000,000 $4,400,000 $1,575,000 Annually $6,500,000 $562,500 $3,100,00 NA $768,120
83,300/7,656 240,000/80,45 $94,450 NA/NA 68,185/60,052 NA/NA 73,500/73,500 55,571/11,818 3,744/899 NA 198,736/32,24 121,300/50,54 5,620/1,368 NA/NA 14,461/2,444
C) Education 0 91,663/6,722 D) Estimates & Projections
Annually E) Health F) Housing
13,500/27,000 0 G) International
2 H) Miscellaneous
$2,000,000
R SSOCS R TFS A PE & PJ A SAHIE
78,000/32,500 A SAIPE R NAMCS R NHIS R NHAMCS R NSCH R AHS A HVS R NYCHVS A IDB R ATUS A BAS AR MOPS
School Teacher Population Small Area Small Area National National National Hospital National American Housing New York International American Boundary & Managemen
Survey Follow-Up Estimates & Health Income & Ambulatory Health Ambulatory Medical Survey of Housing Vacancy City Programs/Studies Time Use Annexation t&
G on 3
Survey Projections Insurance Poverty Medical Care Interview Care Children’s Survey Survey Housing & Survey Survey Organization G
Crime & Estimates Estimates Survey Survey Survey Health Vacancy al Practices
10/12 Safety 13,O 3/15 13,O NA NA NA NA NA NA 2/18 13,42 12/17 13,42 2/18 13,42 5/18 NA 4/18 12,13 8/17 13,29 6/15 Survey 13,O NA NA 12/16 13,29 1/16 13 2/14 Survey 13
Biennially Annually/Biennially Annually Annually Monthly Monthly Annually Biennially Quarterly Annually Monthly Annually
$5,000,000 NA NA NA $6,289,206 Monthly $6,289,206 $2,776,000 $30,000,000 $900,000 Every 3 Years NA $5,000,000 $8,000,000
Biennially 12,870/2,145 NA/NA NA/NA NA/NA 105,768/4,412 $42,000,000 105,768/4,412 16,000/2,262 129,582/64,62 84,000/4,318 NA NA 13,200/3,520 84,464/169,36 Periodic
H) Miscellaneous
$974,631
I) Population J)163,400/47,63
Recreation I & Employment 2 20,645/9,391 8 $1,500,000
A 3,919/1,795
SPD A ACS A CPRIA A 2020 A PRCS A SCP R FHWAR
3
A CPS U.S. CENSUS BUREAU 50,000/25,000

Survey American Census of Decennial Census Puerto Rico Special National Survey of Current
of Program Community Puerto Rico of Population & Community Census Fishing, Hunting, & Population Business Architecture
H Dynamics Survey & Island Housing Survey Program Wildlife Survey H
Areas * - Data from 2010 Census Assoc. Recreation Chief Enterprise Architect – Necarsia McKinnon
2/03 42 6/18 13 NA NA 13,A NA NA 8/15 13 9/12 13,16 7/15 13,29 The Office of Management & Budget (OMB), Office of Information and Regulatory Affairs (OIRA), Office of Information Collection Review (OICR) is the source for the Information Collection Expiration Date,
Annually Annually Decennially Annually Dependent Quinquennially Monthly/Annually # of Approved Annual Responses, Approved Annual Time Burden (Hours), Annual Cost to Federal Government, and Information Collection Authorizing Statues
NA $225,000,000 $13,000,000,000 * NA NA $2,637,913 $88,000,000
Decennially (http://www.reginfo.gov/public/do/PRAMain)
41,990/25,138 3,805,200/2,455,868 308,745,538*/NA NA 248,430/53,52 117,677/19,990 708,000/19,347
NA
NA 7
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

7
Stages of the Survey LifeCycle
Lifecycle stages

1 Survey design

2 Frame development

Instrument
3 development

Sample design &

4 implementation

5 Data collection
1 2 3 4 6 7 8
Data editing,
6 imputation &
estimation

Data review, analysis

7 & correction
5
Data products &
8 dissemination

In Scope
Out of Scope

8
 Planning for the Trip
 Guidance & Governance
 Program Lifecycle Framework
 Systems Development Lifecycle
 Technical Integration Framework
 Systems & Software Engineering Discipline
 Architecture & Engineering Standards
 Scaled Agile Framework (SAFe)
 DevOps & Continuous Integration Framework
 Technical Solutions Assessment Framework
 Risk Management Framework

9
 Technical Integration Framework
High level, formal reviews (similar to those held at the project level) are held to ensure:


Initial high level approach is agreed to at Decennial Program level and CM controlled thereafter
Initial direction is clearly conveyed to projects
!
 System is appropriately tested and ready for operations to begin testing

Systems Readiness Per

Reviews 1 2 3 each 4
Release

PgLC Concept 1 Definition 2 Development 3 Execution 4 Operations & Disposal 5

CBR PBR SRR CDR TRR PRR ORR
Program SEIT Teams execute the technical PgLC Enterprise Organizations and Governance Bodies
Program

processes and conduct reviews of Projects’ conduct Systems Readiness Reviews during
technical activities, ESDLC artifacts, and outputs each PgLC phase and prior to each release
throughout the Project Life Cycle (ESDLC)

Critical Business Review of initial high level architecture to ensure Test Readiness Ensures appropriate test objectives,
CBR Proposal Review inclusion of appropriate systems to implement the TRR Review
methods procedures, scope and
desired subset of the 34 operations. environments.
Project Baseline Review of program baselines. Schedule, Production
PBR Review Assessment of test results to ensure
organizational structure, etc. PRR Readiness system is ready for operations testing.
Review
System Joint review of PLBR/CAP requirements by
SRR Requirements engineering and operations. Operational
Review Assessment of Operational testing results
ORR Readiness to ensure system is ready for production
Critical Design Review
CDR Review operations to begin.

10
 Systems Readiness Framework

Technical Review
Acronyms:
Critical Business Proposal
CBR Review

Project Baseline
PBR Review

SRR System Requirements

Review

CDR Critical Design Review

DDR Detailed Design Review

CR Code Review

TRR Test Readiness Review

Vulnerability
VAR Assessment

PRR Production Readiness

Review

PIR Post Implementation Review

AGILE Meeting Acronyms:
System Disposition
SDR Review PKO
Project Kick-
SPM
Sprint Planning
PBG
Product Backlog
SDM
Solution Design
DSM
Daily Scrum
SRM
Sprint Review
SR
Sprint
Off Meeting Grooming Meetings Meetings Meeting Retrospective

11
 Making the Trip
 Setting up the Program Management Office
 Defining Business Capability Requirements
 Defining the Business Solutions Architecture
 Determining a Segmented Technical Solutions
Architecture
 Engaging IT Security & Privacy
 Defining Functional and non-Functional Requirements
 Planning for Agile Development
 Involving the Business Experts
 Conducting the Release Trains & Readiness
Assessments

12
Defining Requirements

13
 Business Solutions Architecture
CEDCaP capabilities delivered across a series of transition states called product releases.

Current State End State

Survey Design Instrument Sample Design & Data Collection Data Editing,
(SLC 10) Development Implementation (SLC 50) Imputation, &
(SLC 30) (SLC 40) Estimation
(SLC 60)

ECaSE – QDM CaRDS ECaSE – OCS

ECaSE – ALM

ECaSE – ISR ECaSE – ISR

ECaSE – ENUM ECaSE – ENUM

iCADE

Post-Response Data
eCorrespondence
Processing TBD*

CAES

Enterprise IT Enabling Services

(MES 60)

Census Data Access Layer (C-DAL)

Application Program Interface (API) Infrastructure

Enterprise Development, Integration, Test Environment (EDITE)

*The system name is TBD but the functionality will be similar to StEPS II

14
Scaled Agile Framework™ Big
Picture

1
5
 Release Train Agile Process
Requirement Identification
Program Backlog Program Roadmap

Program …………………… Program

Increment 1 Increment N
Prioritized By
Business Value
Product
Release
PgMO

Office of
Innovation and
Implementation
(OII)

Technical Teams

Planning Interim Demo Interim Demo PI Demo & Retro

Wk. 1 Wk. 2 Wk. 3 Wk. 4 Wk. 5 Wk. 6 Wk. 7 Wk. 8

16
Release Train Agile Governance

17
DevOps & Continuous Integration

BCC

AWS AWS
Gov Cloud E/W

IaaS CI
Tools System Application Packages
Tools
NSX Blueprints
vRealize Automation (Software Defined Nexus Maven
Networking)

EDITE Customer Configuration Subversion Serena

vRealize Census Baselined Scripts Business
Orchestrator OS Templates Manager
vRealize Business

Code Stream Jenkins AppDyanamics

18
 Did We Arrive Safely?
 Releases delivered on time & within budget
 Enterprise Requirements Management Team & Change
Control Board established
 Integrated Master Schedule & Integrated Risk Registers
developed
 Transition Plans & Onboarding Plans developed
 Technical Integration & Systems Readiness Frameworks
matured
 COTS Capability Assessment & Analysis
 Agile & Waterfall methodologies integrated
 Integrated Test Planning & Execution
 Did not fully leverage enterprise DevOps

19
 COTS Capability Assessment &
Analysis (CCAA)

Prepare Physical
Assessment Plan
Perform Physical
Assessment

Yes
Conduct Analysis Perform Solution Finalize COTS
Custom Create Executive
Preparation Data Gathering Physical Capability
Solution No Assess No Summary &
Exist Analysis &
Conduct Recommendation
Assessment
Analysis
Yes

Perform AoA

Evaluation Criteria
• Business Functional Need
• System Design
• Schedule
• Cost
• Vendor/Development Team Viability

20
 Ready to Go Again?
 Requirements, Requirements, Requirements
 Effective Guidance as well as strong Governance
 Get support from senior executive leadership
 Effective Program Delivery requires effective
Program Management
 Strong Systems Engineering discipline
 Keep Agile agile
 Build security into every aspect of the lifecycle
 Test, Test, Test
 Involve the Business along the way
21