Simplifying ICF Catalog Management

With Tivoli Advanced Catalog

Management for z/OS

Janet Sun
Rocket Software
[email protected]
Session 13771
 Agenda

• Why Are ICF Catalogs Important?

• Catalog Management Activities
• Backup
• Diagnostics
• Recovery
• Split/Merge
• Reorganization
• Disaster recovery synchronization
• Monitoring Catalog Health

2
 ICF Catalogs Are Critical

Why Are Catalogs Important?

• All current and migrated data is cataloged
• Cataloged data is not accessible if the catalog is not
available
• If a catalog becomes damaged and is out of service, large
amounts of business data will be unavailable until the
catalog is recovered
• Compare the loss of a catalog to the
loss of access to your company phone
and email list – how would you contact
someone when you needed them?

3
Catalog Management
 Catalog Management

Catalog Management Activities Are Batch Oriented

• Many catalog management tasks are performed with
IDCAMS
• Backup
• Diagnostics
• Split/Merge
• Reorganization
• Some catalog management tasks are performed with
multiple utilities
• Recovery
• Catalog synchronization at disaster recovery

5
Catalog Backup
 Catalog Backup

Backup with IDCAMS EXPORT

• IDCAMS EXPORT reads through the index of the KSDS to
back up the data in the catalog
• EXPORT may not be able to back up the BCS if it is
damaged
• If there is a broken sequence set chain pointer, EXPORT
will back up only what can be accessed by the index
 Many records may not be backed up, but a return code of 0 will
be received
• Only one BCS can be backed up per invocation of EXPORT

7
 Catalog Backup

Using IDCAMS EXPORT to Back Up Catalogs

//STEP1 EXEC PGM=IDCAMS

//OUTCAT DD DSN=BACKUP.CATALOG.TEST,DISP=(NEW,CATLG),
// SPACE=(CYL,(20,5)),UNIT=SYSALLDA
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
EXPORT CATALOG.UCATTEST OUTFILE(OUTCAT) TEMPORARY
/*

8
 Catalog Backup

Backup with Advanced Catalog Management

• Can back up a BCS with a broken index or a damaged self-
describing record
• Directly accesses the data component of the KSDS to read
data
• Ensures all data is backed up even if the index is damaged
• Backs up aliases from the master catalog
• Backs up BCS definition parameters
• Much faster run time than IDCAMS EXPORT
• Can back up many or all BCSs in one command invocation
using name masking

9
 Catalog Backup

Using Advanced Catalog Management for z/OS

//STEP1 EXEC PGM=CKM00010

. . .
//OUTCAT DD DSN=BACKUP.MANY.CATS(+1),DISP=(NEW,CATLG),
// SPACE=(CYL,(100,50)),UNIT=SYSALLDA
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
BACKUP BCS(CATALOG.**) OUTFILE(OUTCAT)
/*

10
 Catalog Backup

Simplifying and Improving Backup Processing

• Ensure all BCSs are being backed up by using name
masking
• Back up the master catalog and all connected user catalogs with
one command
• Rapid backup capability makes it practical to take backups
more frequently
• Easy to create multiple backup copies
• The detailed return code summary makes it easy to identify
where a problem occurred
• Multi-tasking option for backups can further reduce
execution time

11
Catalog Diagnostics
 Catalog Diagnostics

Diagnostics with IDCAMS

• EXAMINE INDEXTEST ensures that sequential and key
direct access is accurate
• EXAMINE DATATEST reads all data CIs to ensure structural
integrity
• DIAGNOSE ICFCATALOG (without compare) checks
information integrity within each BCS record (inside-the-BCS
only)
• DIAGNOSE VVDS (without compare) checks information
integrity within each VVDS record (inside-the-VVDS only)

13
 Catalog Diagnostics

Diagnostics with IDCAMS (continued)

• DIAGNOSE ICFCATALOG (with compare) checks the
catalog entries to confirm that they point to the VVDS being
referenced
• A maximum of 99 VVDS names can be specified
• DIAGNOSE VVDS (with compare) checks the data sets to
confirm that they point to the BCS being referenced

14
 Catalog Diagnostics

EXAMINE and DIAGNOSE Error Messages

• EXAMINE
• IDC0xxxx – Supportive informational messages
• IDC1xxxx – Individual data set structural error messages
• IDC2xxxx – Summary error messages
• IDC3xxxx – Function not performed error messages
• DIAGNOSE
• IDC21364I – ERROR DETECTED BY DIAGNOSE:
{VVDS|ICFCAT} ENTRY: entry RECORD: rec OFFSET: offset
REASON: reason-code

15
 Catalog Diagnostics

EXAMINE and DIAGNOSE Error Messages (continued)

• EXAMINE and DIAGNOSE produce error messages but do
not correct the problems identified
• Sample EXAMINE error message:
• IDC11734I SEQUENCE SET AND DATA CI KEY SEQUENCE
MISMATCH
• Sample DIAGNOSE reason codes:
• 11 – Incomplete delete detected
• 20 – Association not found
• 23 – Truename loop failure
• 33 – Incomplete update detected
• 45 – Volume cell not found

16
 Catalog Diagnostics

Diagnostics with Advanced Catalog Management

• Two unique options for diagnostics
• Invoke IDCAMS EXAMINE and DIAGNOSE to verify structural
integrity of catalogs
• Data set name masking eliminates requirement to manually
code each catalog in a separate invocation of EXAMINE or
DIAGNOSE
• Return code summary table facilitates identifying catalogs
with errors
• Customized diagnostic commands to analyze between BCSs
and volumes (VVDSs and VTOCs), and master catalog aliases
• Provides detailed report of problems identified
• Generates fixes to correct problems found

17
 Catalog Diagnostics

Customized Diagnostics
• Diagnostic commands verify data set entries on catalogs
exist on DASD and that data sets on DASD are cataloged
• Extensive diagnostics between VTOC and VVDS to identify
any orphan components
• Thorough multi-volume data set analysis
• Analysis of master catalog aliases to determine if they are in
sync with each other
• Identification of empty aliases and user catalogs without any
aliases associated to them
• Check tape management database pointers towards BCSs
to identify uncataloged tapes and catalog entries for missing
18
tape data sets
Catalog Recovery
 ICFRU

Used for BCS Forward Recovery

• IBM field developed product
• Incorporated into DFSMS 1.7
• Takes IDCAMS EXPORT copy of the BCS and SMF
records from all sharing systems as input
• Creates a new EXPORT format backup which is used
as input to IDCAMS IMPORT to rebuild the catalog

20
 Catalog Recovery

Forward Recovery with ICFRU and IDCAMS IMPORT

• Gather the SMF records from all systems sharing access to
the catalog
• Execute the CRURRSV component of ICFRU with the SMF
data as input to extract the appropriate SMF records
• Execute the CRURRAP component of ICFRU with the
extracted SMF records from CRURRSV along with an
IDCAMS EXPORT format backup of the catalog to create a
new, updated EXPORT format backup
• Execute IDCAMS DELETE and DEFINE for the catalog
• Execute IDCAMS IMPORT to load the catalog from the
EXPORT format backup created by CRURRAP
21
 Catalog Recovery

Forward Recovery with Advanced Catalog Management

• Gather the SMF records from all systems sharing access to
the catalog
• Execute the RECOVER command providing an Advanced
Catalog Management format backup or an IDCAMS EXPORT
format backup and the SMF data as input to create a new,
updated catalog ready for use

Note: Advanced Catalog Management provides a simulation

option to allow advanced testing and error correction of all
recovery commands

• New: Advanced Catalog Management release 2.5 supports

recovery of z/OS 2.1 RLS catalogs
22
Splitting and Merging Catalogs
 Catalog Split/Merge

Why Should Entries in a Catalog Be Split or Merged?

• When the data sets for multiple critical business applications
are all cataloged in a single catalog
• When the size of the BCS becomes very large
• When the number of entries in a BCS is very high
• When related applications are spread across several
catalogs
• When mergers and acquisitions occur and data must be
combined
• When business divestitures occur and data must be divided

24
 Catalog Split/Merge

Using IDCAMS REPRO

• IDCAMS REPRO MERGECAT is used to move data set
entries either individually, by a high level qualifier group, or all
entries from one catalog to another
• The output catalog may already have other data set entries in it
• Entries are deleted from the input catalog after they are
successfully added to the output catalog
• The VVRs for the data set entries moved are updated to point to
the new output catalog

25
 Catalog Split/Merge

Using IDCAMS REPRO (continued)

• IDCAMS REPRO NOMERGECAT is used to copy the entire
input catalog into a new, empty output catalog
• The output catalog must be empty
• The VVRs for the data set entries copied are updated to point to
the new output catalog following REPRO NOMERGECAT
processing
• All subsequent processing must be done to the new output
catalog

26
 Catalog Split/Merge

Using Advanced Catalog Management for z/OS

• MERGECAT is used to move data set entries either
individually, by one or more high level qualifier groups, or all
entries from one catalog to another
• The output catalog may already have other data set entries in it
• Entries are deleted from the input catalog after they are
successfully added to the output catalog
• The VVRs for the data set entries moved are updated to point to
the new output catalog
• Alias pointers for high level groups moved can be updated in any
number of connected master catalogs
• Execution time much faster than IDCAMS
• Simulation option allows advanced testing

27
 Catalog Split/Merge

Using Advanced Catalog Management for z/OS (continued)

• MERGECAT COPYONLY is used to copy the entire input
catalog into a new, empty output catalog
• The output catalog must be empty
• The VVRs for the data set entries copied are not
updated to point to the new output catalog following MERGECAT
processing
• The original catalog is used for subsequent processing

28
Catalog Reorganization
 Catalog Reorganization

When Should A Catalog Be Reorganized?

• When it is approaching maximum extents
• An attribute of the catalog needs to be changed
• When there has been a large amount of deletion activity
against the catalog
• Mass deletions
• Removal of many entries to another catalog as a result of
splitting the catalog
• If the catalog resides on a volume where there is not enough
room to take additional extents and there is no other data
sets that can be moved off of the volume to provide
additional space and reclamation of space through
reorganization will help
30
 Catalog Reorganization

Reorganization with IDCAMS

• Execute IDCAMS EXPORT to create a backup copy of the
BCS
• Execute IDCAMS IMPORT to delete, reallocate, and load
the BCS
• If allocation attribute changes are required, manually delete
and redefine the BCS after EXPORT and add the
INTOEMPTY keyword to the IMPORT step

31
 Catalog Reorganization

Reorganization with Advanced Catalog Management

• Execute the BACKUP command to create a backup copy of
the BCS
• Execute the RECOVER command to delete, reallocate, and
load the BCS
• If allocation attribute changes are required, specify the
appropriate keywords on the RECOVER command to
automatically reallocate the BCS with the desired parameter
changes
• Space allocation • SMS classes
• CI sizes • Free space
• Volume serial • Buffers
• Data set name • String number
32
 Catalog Reorganization

Another Option with Advanced Catalog Management!

• The REORG command will allow a BCS to be reorganized
while it is open and in use
• No need to schedule an application or system outage
• Recommended to run at a low-activity time for the BCS being
reorganized
• Attribute changes are supported during REORG processing
• Space allocation • Free space
• CI sizes • Buffers
• Volume serial • String number
• SMS classes • EATTR value

33
 Catalog Reorganization

Another Option with Advanced Catalog Management! (continued)

• The REORG command will allow a BCS to be reorganized
while it is open and in use
• Structural problems can be repaired
• Simulation option allows advanced testing
• Automatically removes IMBED and REPLICATE if found to be
present

34
Catalog Synchronization at
Disaster Recovery
 Catalogs and Disaster Recovery

What is the Problem?

• When data is recovered at the disaster recovery site, the
entries in the catalog are not synchronized with the actual
data sets that are recovered
• Catalog entries may indicate a data set resides on a different
volume than where it was actually recovered
• Not all data sets will be recovered at the disaster recovery site
• Catalog entries that exist for data sets that do not exist could
cause job failures or allocation failures
• Occurs most frequently when catalogs are in a fully populated
state, usually as the result of full volume restore or mirroring

36
 Catalogs and Disaster Recovery

Using IDCAMS
• When catalog entries exist for data sets that do not, IDCAMS
can be used to remove those entries
• The entries to be removed must be identified first
• There is no easy way to accomplish this
• A user-written program could identify the data sets and build the
IDCAMS DELETE … NOSCRATCH commands
• Once the commands are created, run DELETE NOSCRATCH
to remove the unwanted entries
• The DELETE NOSCRATCH commands may take a long time
to run

37
 Catalogs and Disaster Recovery

Using Advanced Catalog Management

• Use the CATSCRUB command to compare each record in
the selected BCSs against the actual data sets on the online
DASD
• If the catalog record doesn’t match up with a DASD data set, the
catalog record is deleted
• The records are deleted directly from the BCS by CATSCRUB
without invoking IDCAMS
• Results in extremely fast processing
• A report is provided detailing the actions
taken
• A simulation capability is available for
advance testing and error correction

38
 Catalogs and Disaster Recovery

Using Advanced Catalog Management (continued)

• Other types of entries exist in the BCS and their dispositions
can also be controlled by CATSCRUB:
• Migrated data sets
• On DASD or on tape
• Tape data sets
• GDG bases without active generations
• Generation data sets not found on volumes
• Non-VSAM aliases where the real data set does not exist
• Multi-volume data sets where one or more components are
missing from the volume
• Specific data set names
• Specific volume serial numbers
39
 Catalogs and Disaster Recovery

Advanced Catalog Management CATSCRUB

• CATSCRUB performance
• Actual customer benchmark:
• 118 BCSs scrubbed
• 1,814,956 records deleted
• IDCAMS required 15+ hours to complete
• CATSCRUB required 20 minutes

40
Monitoring Catalog Health
 Tivoli Enterprise Portal (TEP)

TEP Provides the Central User Interface for IBM Monitoring

and Management Solutions
• Links key facets of System z Storage Management together
• Provides built-in tools to capture expert knowledge
• Reduces reliance on technical “gurus”
• Makes it much easier to detect, diagnose and correct
problems

42
 Advanced Catalog Management and TEP

Advanced Catalog Management is TEP Enabled

• Centralized, real-time viewing of catalog health:
• Number of BCS extents taken
• Amount of available space on the volume for extents
• Alias count nearing the maximum
• Display CAS statistics
• Display return codes from catalog backups taken
• View catalog data set attributes
• Situations provided that monitor thresholds for out of space
conditions
• Can link into OMEGAMON XE for Storage, if available, to
obtain additional information and take actions

43
 Advanced Catalog Management and TEP

Provides an Overview of Key Catalog Information

44
 Advanced Catalog Management and TEP

Drill Down to Get Details When a Problem Occurs

45
 Summary

Tivoli Advanced Catalog Management for z/OS offers:

• Rapid ICF catalog backup
• One-step forward recovery
• Extensive diagnosis and repair facilities
• Reorganize and repair BCSs while open
• High speed split/merge/copy utility
• Comprehensive disaster recovery support
• Audit and synchronization of master catalog
aliases

46
 Conclusion

Simplification of ICF Catalog Management Tasks

• Tivoli Advanced Catalog Management for z/OS simplifies
catalog management tasks:
• Fewer steps to execute for many tasks
• Faster execution time than other options
• Automated error correction for problems identified
• Reduces outage time required for catalog maintenance

47
 For More Information

• z/OS DFSMS Access Method Services for Catalogs –

SC26-7394
• z/OS DFSMS: Managing Catalogs – SC26-7409
• IBM Tivoli Advanced Catalog Management for z/OS, V2.5:
User’s Guide – SC23-9816-03
• ICF Catalog Backup and Recovery: A Practical Guide –
IBM Redbook SG24-5644

48
 Trademarks

The following terms are trademarks of Rocket Software, Incorporated in the

United States and/or other countries:
Rocket®
The following terms are trademarks of the International Business Machines
Corporation in the United States and/or other countries:
IBM®, Tivoli®, z/OS®, DFSMSdfp™
Other company, product, and service names may be trademarks or service
marks of others.

Copyright ©2013 Rocket Software, Inc. All Rights Reserved.

49