Variables used in this document:

This should be hidden, and not in view, make the window small enough to drop text out of view.

1 – in this window, Use Ctrl-A and Shift-F9 to show fields.

2 – Modify the fields’ value.
3 – Use Ctrl-A and F9 to record updated fields
4 – In main doc, Use Ctrl-A and F9 to update document. Do this twice!
5 – Review document, insert customer specific actions where needed.

Hitachi
To insert field in text, use menu Insert->Field->Ref-Field

Switch Framework and procedure names:

GST DISK Disk
Name of the MSF
Suffix of this MSF
Suffix of reverse MSF

iSeries Managed Availability Runbook

Proc for planned switch (*BCKUP)
Proc for unplanned switch
Proc for sync (*SYNC)
Proc for return switch (Like *PROD)
Proc for system recovery

Customer name:
Prepared by: Argentina SA team
Customer name iSeries SA
IBM Argentina
System nodes: Mike Lowen
System name of the production node consultant
Vision
System name of the backup node
Node suffix (system or partition)
Owner: Hitachi GST DISK
Productive names:
Switched Production
Created:IP 05/29/2007
Switched alternate Production03/27/2014
Last revision: IP
Switched Host/System name
Description of the prod environment

MIMIX Automation:
MIMIX program library
MIMIX custom program library
MIMIX switch source file
MIMIX IPL source file
MIMIX other source file

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 1 of 47

<Refresh the table of contents upon completion of the document>

Table of Contents
Purpose and Audience ......................................................................................................... 3
Ownership ............................................................................................................................ 3
Maintaining this procedure.................................................................................................... 3
Revision Changes ....................................................................................................................... 4
Server Switching ......................................................................................................................... 5
Concepts and Strategy.............................................................................................................. 5
Switch Cycle ............................................................................................................................. 5
Graphical full switch cycle overview .......................................................................................... 6
Switch Overview .......................................................................................................................... 7
Planned Switch Overview ......................................................................................................... 7
Unplanned Switch Overview ..................................................................................................... 8
Switch readiness validation ....................................................................................................... 9
Goal ..................................................................................................................................... 9
Switch readiness validation tasks ......................................................................................... 9
SJDISKMFG-SWITCH from NSNJ400B to NSNJ400A ............................................................. 11
Procedure SWITCHOVER-SWITCH – Switch to Backup ........................................................ 11
SWITCHOVER-SWITCH Pre-Switch Tasks........................................................................ 12
SWITCHOVER-SWITCH Planned Switch Tasks ................................................................ 13
SWITCHOVER-SWITCH Post-Switch Tasks ...................................................................... 22
Procedure SYNCHRONIZE-SWITCH – Resynchronize .......................................................... 22
SYNCHRONIZE-SWITCH Pre-Synchronization Tasks ....................................................... 23
SYNCHRONIZE-SWITCH Synchronization Tasks .............................................................. 23
SYNCHRONIZE-SWITCH Post-Synchronization Tasks...................................................... 27
Procedure FAILOVER-SWITCH – Fail over to Backup............................................................ 28
FAILOVER-SWITCH Pre-Switch Tasks .............................................................................. 28
FAILOVER-SWITCH Unplanned Switch Tasks................................................................... 29
FAILOVER-SWITCH Post-Switch Tasks ............................................................................ 30
SJDISKMFG-RETURN from NSNJ400A to NSNJ400B ............................................................. 32
Procedure SWITCHOVER-RETURN – Switch to Backup ....................................................... 32
SWITCHOVER-RETURN Pre-Switch Tasks ....................................................................... 33
SWITCHOVER-RETURN Planned Switch Tasks ............................................................... 34
SWITCHOVER-RETURN Post-Switch Tasks ..................................................................... 35
Procedure SYNCHRONIZE-RETURN – Resynchronize ......................................................... 36
SYNCHRONIZE-RETURN Pre-Synchronization Tasks ...................................................... 36
SYNCHRONIZE-RETURN Synchronization Tasks ............................................................. 37
SYNCHRONIZE-RETURN Post-Synchronization Tasks ..................................................... 38
Procedure FAILOVER-RETURN – Fail over to Backup ........................................................... 38
FAILOVER Pre-Switch Tasks ............................................................................................. 39
FAILOVER-RETURN Unplanned Switch Tasks .................................................................. 39
FAILOVER-RETURN Post-Switch Tasks ............................................................................ 40
Appendix A: Runbook Hyperlinks ........................................................................................... 43
Appendix B: The Runbook Data Capture Tool ....................................................................... 44
Using and Updating the Links Used in this Document ............................................................. 44
Updating The Links............................................................................................................. 45
Method 1: Automatically ............................................................................................. 45
Method 2: Manually ..................................................................................................... 46
About

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 2 of 47

Purpose and Audience
This Runbook, describes the operational actions to switch the SNJ400B production role from the
NSNJ400B Disk to the NSNJ400A Disk and the actions needed to return the SNJ400B production
role from the NSNJ400A Disk to the NSNJ400B Disk. The intention of the document is to guide
the MIMIX administrator through the switch process.

Ownership
The owner of this document named on the cover page is responsible for maintaining the
procedures and schedules presented to comply with your availability goals and objectives. This
document must be revised when changes, ranging from a simple fix update to major software or
hardware changes, occur in your managed availability environment.

Maintaining this procedure

Whenever the system setup changes, it may be needed to change both the Runbook and this
switch procedure because many changes can occur in your managed availability environment that
can impact the effectiveness of your solution. Some of the more common changes that can occur
are:

 New Availability Solution Administrator.

 New Operating system technology (i.e. remote journaling, new protocols) will impact
performance and the configuration of MIMIX and automation code.
 Network changes or additional - such as new hardware or communication components - can
impact the switching of users to a remote system.
 Introduction of a new application on the systems that needs to be included in the managed
availability environment.
 Introduction of new database features such as triggers, null fields or referential integrity
constraints.
 Addition or changes to application change management can result in files on hold and failed
requests.
 Other: __________________________________________

When changes are needed to this Runbook, contact the document owner (listed on the cover) and
notify them of discrepancies and enhancements.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 3 of 47

Revision Changes
Indicate the date and type of changes made to this document.

Date: 05/29/2007
Document creation

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 4 of 47

Server Switching
Concepts and Strategy
Server switching consists of moving users from one server to another in a controlled way. At
Hitachi GST DISK, server switching means moving users from the NSNJ400B server to the
NSNJ400A server, and, when appropriate, moving the users back again to the NSNJ400B server.

The criteria for performing a switch will be different for a planned switch than for an unplanned
switch.

A planned switch is done at a time when it is generally convenient and when the readiness to
switch can be carefully assessed.

An unplanned switch is done when a failure of the current production system has been detected.
In this case, the readiness to switch is difficult to assess. However, that readiness can be
assumed with some confidence if a regimen of auditing, monitoring, and testing has been
followed.

NOTE: A switch is unplanned, if the original production Disk is no longer accessible from the
backup Disk. If the original production Disk is reachable, it is a planned switch, even if it was not
scheduled or intended.

Switch Cycle
Switching systems, when done properly, includes a complete cycle, since it requires not only
switching users to a second system, but also provides for returning the users safely to the original
system when appropriate.

The full cycle involves a -SWITCH and a -RETURN. Each of which has two identical phases. The
first phase is called Switchover/failover If planned it is referred to has a switchover if unplanned it
is considered a failover. The second phase is called Resynchronize. After the -SWITCH, Disk
NSNJ400B now plays the role of backup to Disk NSNJ400A. This allows for the repeating of the
two phases, Switchover/failover and Resynchronize, to return production to NSNJ400B.

When moving the current production from NSNJ400B to NSNJ400A this is considered the -
SWITCH. When you are ready to return the production back to NSNJ400B from NSNJ400A this is
considered the -RETURN.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 5 of 47

Graphical full switch cycle overview

MSFname - Switch Cycle

PRODUCTION BACKUP

PHASE #4 MIMIX s PHASE #1

MSFname-SWITCH
RESYNCHRONIZE SWITCHOVER/

Type = *UNPLANNED
Type = *PLANNED
FAILOVER
System 1 System 2

UNAVAILABLE PRODUCTION
PRODUCTION UNAVAILABLE

System 1 System 1 System 2

System 2
MSFname-RETURN

Type = *UNPLANNED
Type = *PLANNED

BACKUP PRODUCTION
PHASE #3
SWITCHOVER/ MIMIX PHASE #2
FAILOVER RESYNCRONIZE

System 1 System 2

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 6 of 47

Switch Overview
Procedure SJDISKMFG-SWITCH is switching production role from the NSNJ400B Disk to the
NSNJ400A Disk. For a planned switch to NSNJ400A use procedure SWITCHOVER-SWITCH on
page 11, for unplanned switching use procedure FAILOVER-SWITCH on page 28.

Procedure SJDISKMFG-RETURN is switching production role from the NSNJ400A Disk to the
NSNJ400B Disk. For a planned switch to NSNJ400B use procedure SWITCHOVER-RETURN on
page 32, for unplanned switching use procedure FAILOVER-RETURN on page 38.

Planned Switch Overview

The Planned switch scenario includes 2 major steps, all of which are begun by interactively
issuing a command on the system serving as the backup.

For moving production from NSNJ400B to NSNJ400A:

 Switching production from the production system NSNJ400B to the backup system
NSNJ400A. This step, called “switch to backup”, carefully disengages the production system
from the network, connects the backup system to the network, and makes this the new
production system.
See Procedure SWITCHOVER-SWITCH below.
 Starting MIMIX replication from the new production system back to the old production system.
Effectively, the means the old production system now becomes a backup system. This step
does not affect any user or connections. This step is called “catch-up” or “resync” because it
allows the new backup system to catch up on all the changes that have been taking place on
the new production system since Step 1 was performed.
See Procedure SYNCHRONIZE-SWITCH below.

For moving production from NSNJ400A to NSNJ400B:

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 7 of 47
 Switching production from the backup system NSNJ400A to the old production system
NSNJ400B. This step, called “switch to backup”, carefully disengages the production system
from the network, connects the backup system to the network, and makes this the new
production system.
See Procedure SWITCHOVER-RETURN below.
 Starting MIMIX replication from the new production system back to the old production system.
Effectively, the means you have switched full circle and are back to the initial roles of
Production and Backup for the systems. This step does not affect any user or connections.
This step is called “catch-up” or “resync” because it allows the new backup system to catch
up on all the changes that have been taking place on the new production system since Step 1
was performed.
See Procedure SYNCHRONIZE-RETURN below.

Unplanned Switch Overview

The Unplanned scenario includes 3 major steps, all of which are begun by interactively issuing a
command on the NSNJ400A Disk.

For failover production from NSNJ400B to NSNJ400A:

 Failover production to the backup system NSNJ400A. This step, called “fail-over to backup”,
quickly establishes this system as the new production system. The original production system
NSNJ400B cannot be reached so it cannot be changed to no longer hold Production Role.
See Procedure FAILOVER-SWITCH below.
 Repairing and preparing the old production system NSNJ400B to no longer hold Production
Role. This means taking down connections to the network.
 Starting MIMIX replication from the new production system to back to the old production
system. Effectively, the means the old production system now becomes a backup system.
This step does not affect any user or ATM connections. This step is called “catch-up” or
“resync” because it allows the backup system to catch up on all the changes that have been
taking place on the new production system since Step 1 was performed.
See Procedure SYNCHRONIZE-SWITCH below.

For failover production from NSNJ400A to NSNJ400B:

 Failover production to the backup system NSNJ400B. This step, called “fail-over to backup”,
quickly establishes this system as the new production system. The original production system
NSNJ400A cannot be reached so it cannot be changed to no longer hold Production Role.
See Procedure FAILOVER-RETURN below.
 Repairing and preparing the old production system NSNJ400A to no longer hold Production
Role. This means taking down connections to the network.
 Starting MIMIX replication from the new production system to back to the old production
system. Effectively, the means the old production system now becomes a backup system.
This step does not affect any user or ATM connections. This step is called “catch-up” or
“resync” because it allows the backup system to catch up on all the changes that have been
taking place on the new production system since Step 1 was performed.
See Procedure SYNCHRONIZE-RETURN below.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 8 of 47

Switch readiness validation
Note
These steps are not part of any switch. If you are executing an unplanned switch, please proceed:

For failover production from NSNJ400B to NSNJ400A:

To Procedure FAILOVER-SWITCH – Fail over to Backup on page 28 or

For moving production from NSNJ400A to NSNJ400B:

To Procedure FAILOVER-RETURN – Fail over to Backup on page 38

Goal
It is very important to maintain a switch ready environment. Constantly monitoring the MIMIX
replication and regularly running the audit procedure can achieve this.
Below are the steps that will confirm the backup system is healthy and ready to be switched to,
either for a planned or unplanned scenario.

Switch readiness validation tasks

We advise to perform below tasks at least once a week. Also, they must be performed a few days
to a week prior to any planned switch:

Step Action time

done
Check active replication and resolve problems discovered. The MIMIX
1 active replication can be checked from the following command:
MIMIX/WRKDG
Perform and review the audits and resolve problems discovered. The MIMIX
2 audit screen can be reached with the following command:
MIMIX/WRKAUD
Review Object Activity entries for status of *ACTIVE and *FAILED. Resolve
3 any entries that are lingering for more than a few minutes. On both systems:
MIMIX/WRKDGACTE STATUS(*ACTIVE *FAILED)
Review File & Tracking Entries to determine that no files are in any non-
active status. Resolve any entries that are lingering for more than a few
minutes. On both systems
4
MIMIX/WRKDGFE STSVAL(*INACTIVE)
MIMIX/WRKDGOBJTE STSVAL(*INACTIVE)
MIMIX/WRKDGIFSTE STSVAL(*INACTIVE)
On the SNJ400A partition, enable switch availability and check the status of
the last switch:
MIMIX/CHGSWTFWK SWTFWK(SJDISKMFG) STATUS(*ENABLED)
5
MIMIX/CHKSWTFWK SWTFWK(SJDISKMFG)
The program should return with a message “Switch Framework
SJDISKMFG has current status of....”

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 9 of 47

Step Action time
done
On the NSNJ400A partition, the above resulting status has to be one of
‘SCMP’ or ‘PCMP’.
6 If not, then a previous switch did not complete normally, and you have to
correct the status to ‘PCMP’. An FAQ on the www.mimix.com website will
assist you to reset this status.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 10 of 47

 SJDISKMFG-SWITCH
from
NSNJ400B to NSNJ400A

Procedure SJDISKMFG-SWITCH is switching production role from the NSNJ400B Disk to the
NSNJ400A Disk. For a planned switch to backup use procedure SWITCHOVER-SWITCH on
page 11, for unplanned switching use procedure FAILOVER-SWITCH on page 28.

A reminder: A switch is unplanned, if the original production Disk is no longer accessible from the
backup Disk. If the original production Disk is reachable, it is a planned switch, even if it was not
scheduled or intended.

For switching back from the NSNJ400A to the NSNJ400B, please use the procedures for
SJDISKMFG-RETURN procedure.

Procedure SWITCHOVER-SWITCH – Switch to Backup

Goal
Switch users from the NSNJ400B Disk to the NSNJ400A Disk. Almost all the actions are initiated
on the NSNJ400A Disk, which is the controlling system for this procedure.

The next pages hold the following groups of steps:

 SWITCHOVER-SWITCH Pre-Switch Tasks

To be executed immediately before the planned switch

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 11 of 47

  SWITCHOVER-SWITCH Planned Switch Tasks
The actual switch.

 SWITCHOVER-SWITCH Post-Switch Tasks

Checking and cleaning up the switch.

SWITCHOVER-SWITCH Pre-Switch Tasks

These tasks should be performed immediately prior to the switch:

Step Action Time

done
In the next step, connections to the system will be closed. To ensure you
keep your connection, use a session that is not using the switched IP
address 158.58.43.12, but one of the administrative addresses.
Create a 5250 telnet session to both systems. Then, on both systems,
1 transfer your job to QCTL instead of QINTER:
TFRJOB JOBQ(QCTL)

You can also use an HMC 5250 shared session for both
systems

On the NSNJ400A, add the MIMIX and ROLESWAP to the library list.
2 ADDLIBLE ROLESWAP
ADDLIBLE MIMIX
Check active replication and resolve problems discovered. The MIMIX
3 active replication can be checked from the following command:
WRKDG
Review Object Activity entries for status of *ACTIVE and *FAILED. Resolve
4 any entries that are lingering for more than a few minutes. On both systems:
WRKDGACTE STATUS(*ACTIVE *FAILED)
Review File & Tracking Entries to determine that no files are in any non-
active status. Resolve any entries that are lingering for more than a few
minutes. On both systems
5
WRKDGFE STSVAL(*INACTIVE)
WRKDGOBJTE STSVAL(*INACTIVE)
WRKDGIFSTE STSVAL(*INACTIVE)
When you are ready for the actual switch, continue with step 6 below

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 12 of 47

 SWITCHOVER-SWITCH Planned Switch Tasks

These tasks are the actual switch from NSNJ400B to NSNJ400A, and are to be
considered “downtime”.

Step Action Time

done
Shut down the production environment.
Give the users adequate time to end their work on the production system.
6 Check no interactive jobs are active, no batch jobs are active and all
scheduled batch jobs in job queues are held. Also all productive subsystems
should be ended.
Wait for until all above ended subsystems and environments really ended. It
7 may take a few minutes for the MESA subsystems to end. Do *not* end the
MIMIXSBS subsystem.
On the NSNJ400A Disk perform the following command:
8 MIMIX/RUNSWTFWK SWTFWK(SJDISKMFG) PRC(*BCKUP)

This last command will check if the system is in the correct status. Next, it will confirm that
the user wishes to switch by issuing messages to QSYSOPR message queue.

On the NSNJ400A Disk answer the message in QSYSOPR message queue

9 to confirm the switchover. Use SysRq 6 to go to the QSYSOPR message
queue

The RUNSWTFWK process will end the production infrastructure, end MIMIX replication
controlled; switch MIMIX data group direction, switch the network connections, and start
the production infrastructure on the new production Disk NSNJ400A.

This is done by internally calling the following programs: (This is shown here as
documentation, these are not steps you need to perform)

 SWTFWKCFM (running on Disk NSNJ400A)

This program will ask confirmation from the system operator, it also internally calls
programs DGSELECT to determine which data groups to switch. This program
performs no SJDISKMFG specific actions.

 ENDBCKUP (running on Disk NSNJ400A)

PGM PARM(&PRDLIB &SYSDFN &SWTFWK &SWTTYPE)

/**/
DCL VAR(&PRDLIB) TYPE(*CHAR) LEN(10)
DCL VAR(&SYSDFN) TYPE(*CHAR) LEN(8)
DCL VAR(&SWTFWK) TYPE(*CHAR) LEN(10)
DCL VAR(&SWTTYPE) TYPE(*CHAR) LEN(1)
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 13 of 47
 /**/
/* Executes DISK specific shutdown functions in backup environment */
/**/
CALL PGM(ROLESWAP/ENDBCKUP_1)
/**/
CHGDTAARA DTAARA(ROLESWAP/MXSYSROLE) VALUE('SWT')
MONMSG MSGID(CPF0000 CPF9999)
CHGDTAARA DTAARA(*LDA (1 10)) VALUE('SUCCESSFUL')
ENDPGM:
ENDPGM

ENDBCKUP_1

*************** Beginning of data ***************************************************

PGM

DCL VAR(&SYSNAME) TYPE(*CHAR)

RTVNETA SYSNAME(&SYSNAME)
CHGJOB LOG(4 0 *SECLVL) LOGCLPGM(*YES)

MPLUS:
ENDMP
MONMSG MSGID(CPF0000)
DLYJOB DLY(5)
ENDSBS SBS(ZMPLUS) OPTION(*IMMED)
MONMSG MSGID(CPF0000)
QINTER:
ENDSBS SBS(QINTER) OPTION(*IMMED) +
ENDSBSOPT(*NOJOBLOG)
MONMSG MSGID(CPF0000)
DLYJOB DLY(120)
ENDWTR WTR(*ALL) OPTION(*CNTRLD)
MONMSG MSGID(CPF0000)
QCMN:
ENDSBS SBS(QCMN) OPTION(*IMMED)
MONMSG MSGID(CPF0000)
ENDHOSTSVR SERVER(*DATABASE *FILE)
MONMSG MSGID(CPF0000)
STRTCPSVR SERVER(*TELNET)
MONMSG MSGID(CPF0000)
DSPJOBLOG OUTPUT(*PRINT)
MONMSG MSGID(CPF0000)
ENDPGM:
ENDPGM
****************** End of data *****************************************

 ENDPROD (running on Disk NSNJ400B)

PGM PARM(&PRDLIB &SYSDFN &SWTFWK &SWTTYPE)

/**/
DCL VAR(&PRDLIB) TYPE(*CHAR) LEN(10)
DCL VAR(&SYSDFN) TYPE(*CHAR) LEN(8)
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 14 of 47
 DCL VAR(&SWTFWK) TYPE(*CHAR) LEN(10)
DCL VAR(&SWTTYPE) TYPE(*CHAR) LEN(1)
DCL VAR(&SBS) TYPE(*CHAR) LEN(10)
DCL VAR(&INLR) TYPE(*CHAR) LEN(1)
DCL VAR(&LIB) TYPE(*CHAR) LEN(10)
DCL VAR(&SWITCH) TYPE(*CHAR) LEN(4)
/**/
WRKJOBSCDE JOB(*ALL) OUTPUT(*PRINT) PRTFMT(*BASIC)
WRKJOBSCDE JOB(*ALL) OUTPUT(*PRINT) PRTFMT(*FULL)
/**/
/* Executes DISK specific shutdown activities for PROD environment */
/**/
CALL PGM(ROLESWAP/ENDPROD_1)
/**/
CHGDTAARA DTAARA(ROLESWAP/MXSYSROLE) VALUE('SWT')
MONMSG MSGID(CPF0000 CPF9999)
CHGDTAARA DTAARA(*LDA (1 10)) VALUE('SUCCESSFUL')
ENDPGM:
ENDPGM
****************** End of data ****************************************

ENDPROD_1

PGM

DCL VAR(&SYSNAME) TYPE(*CHAR)

DCL VAR(&ASYSNAME) TYPE(*CHAR)
DCL VAR(&BSYSNAME) TYPE(*CHAR)

RTVNETA SYSNAME(&SYSNAME)
RTVDTAARA DTAARA(ROLESWAP/ASYSNAME)
RTNVAR(&ASYSNAME)
RTVDTAARA DTAARA(ROLESWAP/BSYSNAME)
RTNVAR(&BSYSNAME)

CHGJOB LOG(4 0 *SECLVL) LOGCLPGM(*YES)

SNDBRKMSG MSG('MFG DISK maintenance window is +

starting in 10 minutes. Please close your +
session. Thanks') TOMSGQ(*ALLWS)

DLYJOB DLY(300)

SNDBRKMSG MSG('MFG DISK maintenance window is +

starting in 5 minutes. Please close your +
session. Thanks') TOMSGQ(*ALLWS)

DLYJOB DLY(300)

MPLUS:
ENDMP
MONMSG MSGID(CPF0000)
DLYJOB DLY(5)
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 15 of 47
 ENDSBS SBS(ZMPLUS) OPTION(*IMMED)
MONMSG MSGID(CPF0000)

QINTER:
ENDSBS SBS(QINTER) OPTION(*IMMED)
ENDSBSOPT(*NOJOBLOG)
MONMSG MSGID(CPF0000)

/*********************************************************************/
/**/
/* DISABLES END USERS MESA LOGIN BY TURNING DTAARA TO 0 */
/**/
CHGDTAARA DTAARA(MXUSROBJ/MXSYSSTS) VALUE(0)
/**/
/*********************************************************************/
ENDCAPT:
CALL PGM(QGPL/ENDMESA)
MONMSG MSGID(CPF0000)

ENDWTR WTR(*ALL) OPTION(*CNTRLD)

MONMSG MSGID(CPF0000)

DLYJOB DLY(30)
QCMN:
ENDSBS SBS(QCMN) OPTION(*IMMED) ENDSBSOPT(*NOJOBLOG)
MONMSG MSGID(CPF0000)

QBATCH:
ENDSBS SBS(QBATCH) OPTION(*IMMED)
ENDSBSOPT(*NOJOBLOG)
MONMSG MSGID(CPF0000)

CLRJOBQ JOBQ(QBATCH)
MONMSG MSGID(CPF0000)

MESARPTS:
ENDSBS SBS(MESARPTS) OPTION(*IMMED) +
ENDSBSOPT(*NOJOBLOG)
MONMSG MSGID(CPF0000)

SNDUSRSP:

IF COND(&SYSNAME *EQ 'SNJ400B') THEN(DO)

MIMIX/SNDNETOBJ OBJ(BLDSUMRY) LIB(BLD_R00026) +
OBJTYPE(*USRSPC) RMTSYS(NSNJ400A)
ENDDO
IF COND(&SYSNAME *EQ 'SNJ400A') THEN(DO)
MIMIX/SNDNETOBJ OBJ(BLDSUMRY) LIB(BLD_R00026) +
OBJTYPE(*USRSPC) RMTSYS(NSNJ400B)
ENDDO

QMQM:
/* Queue manager name is the same for both systems (SNJ400B) */
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 16 of 47
 RCDMQMIMG OBJ(*ALL) OBJTYPE(*ALL) MQMNAME(SNJ400B)
MONMSG MSGID(CPF0000)
ENDMQM MQMNAME(SNJ400B) OPTION(*CNTRLD)
MONMSG MSGID(CPF0000)
ENDSBS SBS(QMQM)
MONMSG MSGID(CPF0000)

ENDTCPB:

ENDTCPIFC INTNETADR('158.58.43.12')
MONMSG MSGID(TCP1B72)
CHGTCPIFC INTNETADR('158.58.43.12') AUTOSTART(*NO)

ENDHOSTSVR SERVER(*DATABASE *FILE)

MONMSG MSGID(CPF0000)
STRTCPSVR SERVER(*TELNET)
MONMSG MSGID(CPF0000)

SNDMSG MSG('Production Shutdown completed') +

TOUSR(QSYSOPR)
MONMSG MSGID(CPF0000)
DSPJOBLOG OUTPUT(*PRINT)
MONMSG MSGID(CPF0000)
ENDPGM:
ENDPGM
****************** End of data *******************************************

The ENDPROD functions end the production environment on NSNJ400B. This program
performs the following SJDISKMFG specific actions:
 Sets the local system role indicator data area MXSYSROLE to “SWT” (in switch)
 Sets MIMIX auto-start indicator to “*SWITCH” to prevent auto-start of replication
 Ends the TCP/IP interfaces 158.58.43.12
 Kills all production subsystems. These subsystems should already have been
ended nicely, but to prevent accidental activity after the switch, they are killed
from this program.
 Disables the TCP/IP interface 158.58.43.12 associated to DNS host name
SNJ400B
 The replication to the NSNJ400A Disk is ended controlled and the data groups
are switched in their direction of replication. Reverse replication is not started yet.

 STRPROD (running on Disk NSNJ400A)

*************** Beginning of data *************************************

PGM PARM(&PRDLIB &SYSDFN &SWTFWK &SWTTYPE)
/**/
DCL VAR(&PRDLIB) TYPE(*CHAR) LEN(10)
DCL VAR(&SYSDFN) TYPE(*CHAR) LEN(8)
DCL VAR(&SWTFWK) TYPE(*CHAR) LEN(10)
DCL VAR(&SWTTYPE) TYPE(*CHAR) LEN(1)
DCL VAR(&SWITCH) TYPE(*CHAR) LEN(4)
/**/
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 17 of 47
 /* Executes DISK specific start-up functions for PROD. environment */
/**/
CALL PGM(ROLESWAP/STRPROD_1)
/**/
CHGDTAARA DTAARA(ROLESWAP/MXSYSROLE) VALUE('SRC')
MONMSG MSGID(CPF0000 CPF9999)
CHGDTAARA DTAARA(*LDA (1 10)) VALUE('SUCCESSFUL')
ENDPGM:
ENDPGM
****************** End of data ****************************************
STRPROD_1

*************** Beginning of data *************************************

PGM

DCL VAR(&SYSNAME) TYPE(*CHAR)

DCL VAR(&RPY) TYPE(*CHAR) LEN(1)
DCL VAR(&ASYSNAME) TYPE(*CHAR)
DCL VAR(&BSYSNAME) TYPE(*CHAR)

RTVNETA SYSNAME(&SYSNAME)
RTVDTAARA DTAARA(ROLESWAP/ASYSNAME) +
RTNVAR(&ASYSNAME)
RTVDTAARA DTAARA(ROLESWAP/BSYSNAME) +
RTNVAR(&BSYSNAME)
/* */
CHGJOB LOG(4 0 *SECLVL) LOGCLPGM(*YES)
/* */
/* START subsystem QCMN */
/* */
SNDMSG MSG('Starting QCMN') TOUSR(QSYSOPR)
MONMSG MSGID(CPF0000)

QCMN:
QSYS/STRSBS SBSD(QCMN)
MONMSG MSGID(CPF0000)

/* DLYJOB RSMTIME(113800) */

/* START MESA and other Subsystems */

SNDMSG MSG('Starting production') TOUSR(QSYSOPR)

MONMSG MSGID(CPF0000)

STRSBS SBSD(QMQM/QMQM)

STRMQM MQMNAME(SNJ400B) /* same name for both systems */

/*********************************************************************/
/**/
/* ENABLES END USERS MESA LOGIN BY TURNING DTAARA TO 1 */
/**/
CHGDTAARA DTAARA(MXUSROBJ/MXSYSSTS) VALUE(1)
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 18 of 47
 /**/
/*********************************************************************/

CALL PGM(QGPL/STARTMESAW) /* Does not start CAPTURE */

MONMSG MSGID(CPF0000)

DLYJOB DLY(15)

STRSBS SBSD(MESA20U/MESARPTS)
MONMSG MSGID(CPF0000)

/* Start IP Interfases */

STRTCPB:

STRTCPIFC INTNETADR('158.58.43.12')
MONMSG MSGID(TCP1B12)
DLYJOB DLY(15)
CHGTCPIFC INTNETADR('158.58.43.12') AUTOSTART(*YES)
MONMSG MSGID(TCP2658)

STRHOSTSVR SERVER(*ALL)
MONMSG MSGID(CPF0000)

STRTCPSVR SERVER(*HTTP) HTTPSVR(MIMIX)

MONMSG MSGID(CPF0000)

DLYJOB DLY(45)

/* DLYJOB RSMTIME(115000) */

/**/
QINTER: QSYS/STRSBS SBSD(QINTER)
MONMSG MSGID(CPF0000)

STRPRTWTR DEV(*ALL)
MONMSG MSGID(CPF0000)

STRRMTWTR OUTQ(*ALL)
MONMSG MSGID(CPF0000)

SWBKUP:
RLSJOBJS JOB(TAPDLYOP)
MONMSG MSGID(CPF0000)

RLSJOBJS JOB(TAPWKYOP)
MONMSG MSGID(CPF0000)

HLDJOBJS JOB(TAPDLYST)
MONMSG MSGID(CPF0000)

HLDJOBJS JOB(TAPWKYST)
MONMSG MSGID(CPF0000)
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 19 of 47
 RLSJOBJS JOB(BRMDLY1)
MONMSG MSGID(CPF0000)

HLDJOBJS JOB(BRMDLYSBY1)
MONMSG MSGID(CPF0000)

RLSJOBJS JOB(BRMWEEKLY)
MONMSG MSGID(CPF0000)

HLDJOBJS JOB(BRMWKYSTBY)
MONMSG MSGID(CPF0000)

RLSJOBJS JOB(MESAPSR)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(CASSCLNUP)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(DGRMNT)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(MESAMERPT)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(MESAPRD1)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(MESAPRD2)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(MESAPRD3)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(MESAPRD4)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(MESAWERPT)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(MESA1PSR)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(RELSODEV)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(RELSODEV2)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(MQJRNMNT)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(MQRCDMQIMG)
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 20 of 47
 MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(MQRCDSWT)
MONMSG MSGID(CPF0000)

RLSJOBSCDE JOB(RUNGHQRY)
MONMSG MSGID(CPF0000)

MPLUS/HLDMON MON(MMXJOBS)
MONMSG MSGID(CPF0000)

/* MPLUS/RLSMON MON(CAPTURE) */
/* MONMSG MSGID(CPF0000) */

MPLUS/RLSEVTMON MON(QHST) SEQNBR(1060)

MONMSG MSGID(CPF0000)

/* */
MPLUS:
STRSBS SBSD(MPLUS/ZMPLUS)
MONMSG MSGID(CPF0000)

DSPJOBLOG OUTPUT(*PRINT)

ENDPGM
***************** End of data *************

This program starts the production environment on NSNJ400A Disk. This program
performs the following SJDISKMFG specific actions:
 Sets the local system role indicator data area MXSYSROLE to “P” (in
Production)
 Starts the TCP/IP interfaces 158.58.43.12
 Start all application, with the only exception of job CAPTURE, which needs to
be parametrized by the application suppport

Note: The source for the above programs is located in file QCLSRC in library ROLESWAP
on both systems.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 21 of 47

SWITCHOVER-SWITCH Post-Switch Tasks

These tasks should be performed immediately after to the switch:

Step Action Time

done
On the NSNJ400A Disk, check that:
 The 158.58.43.12 TCP/IP interface is active
10 (use command NETSTAT for this)
 The NETSERVER job is started
Check for a job named QZLSSERVER
Record all current journal receiver names and first sequence numbers of all
journals on the new production Disk NSNJ400A, by using the following
11 command:
WRKDG OUTPUT(*OUTFILE)
OUTFILE(ROLESWAP/WRKDG) OUTMBR(SWITCH)
Start the other subsystems on the new production Disk NSNJ400A that you
want to have active during this switch.
Have Applications Support checking and:
12 - starting job CAPTURE (once thee job is running, release the MPLUS
Capture monitor
- releasing MESA application usage by setting data area
MXUSROBJ/MXSYSSTS to ‘0’
Signal to the test users group they can start working. Ensure they
13 understand they should connect to the same production host SNJ400B. It
should automatically connect to new production Disk NSNJ400A

The system is now switched and the production environment and host SNJ400B is now
active on the NSNJ400A partition.
However, there is no replication yet to the original production system, so when
appropriate, start reverse synchronization by following instruction 14 on the next page.
Procedure SYNCHRONIZE-SWITCH – Resynchronize
Goal
After the users were switched to the NSNJ400A Disk by the SJDISKMFG-SWITCH procedure,
this procedure will start MIMIX replication back to the NSNJ400B Disk, which then effectively
becomes the backup system.

The next pages hold the following groups of steps:

 SYNCHRONIZE-SWITCH Pre- Synchronization Tasks.

To be executed before the synchronization

 SYNCHRONIZE-SWITCH Synchronization Tasks.

The actual synchronization.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 22 of 47

 SYNCHRONIZE-SWITCH Post- Synchronization Tasks
Checking and cleaning up the synchronization.

SYNCHRONIZE-SWITCH Pre-Synchronization Tasks

These tasks should be performed some time prior to the synchronization:

Step Action Time

done
Only perform this sync procedure if you are certain that the current
14 production system is the NSNJ400A Disk.
Check that the NSNJ400B Disk is now again available for use (after being
15 repair or maintained).
Add the MIMIX and ROLESWAP libraries to the library list on the
16 NSNJ400A
On both systems, ensure the MIMIX journal and system managers are
running and that the MIMIX data groups are not running. The MIMIX active
17 replication can be checked from the following command:
WRKDG
Check the status of the switch framework (last switch results):
MIMIX/CHKSWTFWK SWTFWK(SJDISKMFG)
18 The command should return with a message “Switch Framework
SJDISKMFG has current status of:

On the NSNJ400A partition, if the above resulting status has to be either

‘BCMP’ or ‘FCMP’.
19
If not, then a previous switch did not complete normally, and you have to
correct the status to ‘BCMP’.

SYNCHRONIZE-SWITCH Synchronization Tasks

These tasks are the actual synchronization from NSNJ400A to NSNJ400B. The NSNJ400A is and
remains the production system after these steps:

Step Action Time

done
On the NSNJ400A, check if you still have MIMIX and ROLESWAP in the
20 library list.
On the NSNJ400A Disk perform the following command:
21
MIMIX/RUNSWTFWK SWTFWK(SJDISKMFG) PRC(*SYNC)

This command will start MIMIX replication from the NSNJ400A Disk to the NSNJ400B Disk
after user confirmation by issuing messages to QSYSOPR message queue.
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 23 of 47
 On the NSNJ400A Disk answer the message in QSYSOPR message queue
22 to confirm the switchover. Use SysRq 6 to go to the QSYSOPR message
queue.

The RUNSWTFWK process will check the system and MIMIX stratus, and start replication
from the NSNJ400A Disk to the NSNJ400B Disk

This is done by internally calling the following programs: (This is shown here as
documentation, these are not steps you need to perform)

1. SWTFWKCFM (on Disk NSNJ400A)

This program will ask confirmation from the system operator, it also internally calls
programs DGSELECT to determine which data groups to start. This program performs
no SJDISKMFG specific actions.

2. STRBCKUP (on Disk NSNJ400B)

PGM PARM(&PRDLIB &SYSDFN &SWTFWK &SWTTYPE)

/**/
DCL VAR(&PRDLIB) TYPE(*CHAR) LEN(10)
DCL VAR(&SYSDFN) TYPE(*CHAR) LEN(8)
DCL VAR(&SWTFWK) TYPE(*CHAR) LEN(10)
DCL VAR(&SWTTYPE) TYPE(*CHAR) LEN(1)
/**/
/* Executes DISK specific start-up functions for backup environment */
/**/
CALL PGM(ROLESWAP/STRBCKUP_1)
/**/
CHGDTAARA DTAARA(ROLESWAP/MXSYSROLE) VALUE('TGT')
MONMSG MSGID(CPF0000 CPF9999)
/**/
/* Line to Fake out MSF for 2 phase switch */
/**/
CHGDTAARA DTAARA(MIMIX/SJDISKMFG (32 4)) VALUE('PCMP')
MONMSG MSGID(CPF0000 CPF9999)
CHGDTAARA DTAARA(*LDA (1 10)) VALUE('SUCCESSFUL')
ENDPGM:
ENDPGM
****************** End of data ********************************************

STRBCKUP_1

*************** Beginning of data *********************************

PGM

DCL VAR(&SYSNAME) TYPE(*CHAR)

DCL VAR(&ASYSNAME) TYPE(*CHAR)
DCL VAR(&BSYSNAME) TYPE(*CHAR)

RTVNETA SYSNAME(&SYSNAME)
RTVDTAARA DTAARA(ROLESWAP/ASYSNAME)
RTNVAR(&ASYSNAME)
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 24 of 47
 RTVDTAARA DTAARA(ROLESWAP/BSYSNAME)
RTNVAR(&BSYSNAME)

CHGJOB LOG(4 0 *SECLVL) LOGCLPGM(*YES)

QCMN:
QSYS/STRSBS SBSD(QCMN)
MONMSG MSGID(CPF0000)
QBATCH:
QSYS/STRSBS SBSD(QBATCH)
MONMSG MSGID(CPF0000)
QINTER:
QSYS/STRSBS SBSD(QINTER)
MONMSG MSGID(CPF0000)

STRPRTWTR DEV(*ALL)
MONMSG MSGID(CPF0000)
STRRMTWTR OUTQ(*ALL)
MONMSG MSGID(CPF0000)
SWBKUP:
HLDJOBJS JOB(TAPDLYOP)
MONMSG MSGID(CPF0000)

HLDJOBJS JOB(TAPWKYOP)
MONMSG MSGID(CPF0000)

RLSJOBJS JOB(TAPDLYST)
MONMSG MSGID(CPF0000)

RLSJOBJS JOB(TAPWKYST)
MONMSG MSGID(CPF0000)

HLDJOBJS JOB(BRMDLY1)
MONMSG MSGID(CPF0000)

RLSJOBJS JOB(BRMDLYSBY1)
MONMSG MSGID(CPF0000)

HLDJOBJS JOB(BRMWEEKLY)
MONMSG MSGID(CPF0000)

RLSJOBJS JOB(BRMWKYSTBY)
MONMSG MSGID(CPF0000)

HLDJOBJS JOB(MESAPSR)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(CASSCLNUP)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(DGRMNT)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(MESAMERPT)
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 25 of 47
 MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(MESAPRD1)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(MESAPRD2)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(MESAPRD3)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(MESAPRD4)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(MESAWERPT)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(MESA1PSR)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(RELSODEV)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(RELSODEV2)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(MQJRNMNT)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(MQRCDMQIMG)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(MQRCDSWT)
MONMSG MSGID(CPF0000)

HLDJOBSCDE JOB(RUNGHQRY)
MONMSG MSGID(CPF0000)

MPLUS/RLSMON MON(MMXJOBS)
MONMSG MSGID(CPF0000)

MPLUS/HLDMON MON(CAPTURE)
MONMSG MSGID(CPF0000)

MPLUS/HLDEVTMON MON(QHST) SEQNBR(1060)

MONMSG MSGID(CPF0000)
MPLUS:
STRSBS SBSD(MPLUS/ZMPLUS)
MONMSG MSGID(CPF0000)
ENDPGM
****************** End of data **********************

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 26 of 47

 This program starts the backup environment on the former production system NSNJ400B.
This program performs no SJDISKMFG specific actions.

3. Starts the Data Groups to replicate from the NSNJ400A production Disk to the
NSNJ400B Disk, which is now a backup system.

Note: The source for the above programs is located in file QCLSRC in library MIMIX on
both systems.

SYNCHRONIZE-SWITCH Post-Synchronization Tasks

These tasks should be performed immediately after the synchronization:

Step Action Time

done
Verify that the MIMIX data groups’ source jobs are active, and they are
replicating from the NSNJ400A to the NSNJ400B partition. The target jobs
23 should not yet be active. Use the following command:
WRKDG
Check that the NSNJ400B Disk still has no production users. Check that
24 there are no production subsystems, and that the TCP/IP 158.58.43.12
interface is not active
When you feel the switch and sync are in order, you can now actually start
writing the updates into the NSNJ400B Disk by starting the apply sessions.

25 (If you are unsure of the status of the switch or replicated data, postpone
this step until advise from database and system administrators and testers
has been obtained)
STRDG DGDFN(*ALL) PRC(*ALLTGT)
After checking all functions are working as expected, disable switch
availability by issuing (in both systems):
26
MIMIX/ CHGSWTFWK SWTFWK(SJDISKMFG) STATUS(*DISABLED)

The switchover of the production environment and host SNJ400B from

NSNJ400B to NSNJ400A is now fully complete, and the NSNJ400B is now
the backup system. If you need to switch back, please use document and
procedure SJDISKMFG-RETURN
No more actions from this document are needed.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 27 of 47

Procedure FAILOVER-SWITCH – Fail over to Backup
Goal
Following the unplanned deactivation (or system crash) of the NSNJ400B production Disk, the
NSNJ400A is activated to take on production role and allow user connections.

Only use this procedure if the NSNJ400B production Disk is not reachable from the NSNJ400A
backup system. Use the SJDISKMFG-SWITCHOVER procedure on page 11 if the NSNJ400B
Disk can still be reached by you as an administrator.

The next pages hold the following groups of steps:

 FAILOVER Pre-Switch Tasks

To be executed immediately before the unplanned switch

 FAILOVER Unplanned Switch Tasks

The actual switch.

 FAILOVER Post-Switch Tasks

Checking and cleaning up the switch.

FAILOVER-SWITCH Pre-Switch Tasks

This procedure applies only to an unplanned switch. These tasks should be performed
immediately prior to the switch:

Step Action Time

done
1 Check if the production NSNJ400B Disk is really no longer reachable.
Check active replication and resolve problems discovered for as far as still
2 possible. The operations procedures can be found in the operations section
of the run book.
3 Add the MIMIX and ROLESWAP libraries to the library list.
Review the latest audits results, which probably ran in the last 7 days. See if
there are any replication/audit issues that make it impossible for you to
switch over to the NSNJ400A partition.
If there are any major issues in the audit results, you should have dealt with
4 them when they were first reported. In any case, you may need to make a
judgment call whether to switch or not.
The MIMIX audit screen can be reached with the following command:
WRKAUD
On NSNJ400A partition, check the status of the last switch:
5 DSPDTAARA DTAARA(MIMIX/SJDISKMFG)
The status is in position 32 – 35.
6 On the NSNJ400A partition, the above resulting status has to be one of
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 28 of 47
Step Action Time
done
‘SCMP’ or ‘PCMP’.
If not, then a previous switch did not complete normally, and you have to
correct the status to ‘PCMP’.

FAILOVER-SWITCH Unplanned Switch Tasks

These tasks are the actual switch from NSNJ400B to NSNJ400A:

Step Action Time

done
On the NSNJ400A, check if you still have MIMIX and ROLESWAP in the
7 library list.
On the NSNJ400A Disk perform the following command:
8 MIMIX/RUNSWTFWK SWTFWK(SJDISKMFG) PRC(*BCKUP)
TYPE(*UNPLANNED)

This last command will check if the system is in the correct status. Next, it will confirm that
the user wishes to switch by issuing messages to QSYSOPR message queue.

On the NSNJ400A Disk answer the message in QSYSOPR message queue

9 to confirm the switchover. Use SysRq 6 to go to the QSYSOPR message
queue.
The RUNSWTFWK process will end the production infrastructure; end MIMIX replication
controlled, switch MIMIX data group direction, switch the network connections, and start
the production infrastructure on the new production system NSNJ400A.

This is done by internally calling the following programs: (This is shown here as documentation,
these are not steps you need to perform)

 SWTFWKCFM (running on Disk NSNJ400A)

This program will ask confirmation from the system operator, it also internally calls
programs DGSELECT to determine which data groups to switch. This program
performs no SJDISKMFG specific actions.

 ENDBCKUP (running on Disk NSNJ400A)

This program ends the backup environment on NSNJ400A. This program performs
no SJDISKMFG specific actions.

 Program ENDPROD is not called, as this is an unplanned switch, and the

production system is assumed crashed and unreachable.

 The replication to the NSNJ400A system is ended controlled and the data groups
are switched in their direction of replication. Reverse replication is not started yet.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 29 of 47

  STRPROD (running on Disk NSNJ400A)
This program starts the production environment on NSNJ400A Disk. This program
performs the following SJDISKMFG specific actions:
 Sets the TCP/IP host name to SNJ400B
 Sets the local system role indicator data area MXSYSROLE to “SRC” (in
Production)
 Starts the TCP/IP interfaces 158.58.43.12
 Starts the required subsystem and controllers

Note: The source for the above programs is located in file QCLSRC in library ROLESWAP
on both systems.

FAILOVER-SWITCH Post-Switch Tasks

These tasks should be performed immediately after to the switch:

Step Action Time

done
On the NSNJ400A Disk, check that:
 The 158.58.43.12 TCP/IP interface is active
use command NETSTAT for this
10
 The TCP host name is set to SNJ400B
use command CFGTCP option 12 for this
 The NETSERVER job is started
Check for a job named QZLSSERVER

Record all current journal receiver names and first sequence numbers of all
journals on the new production Disk NSNJ400A, by using the following
11
command:
WRKDG OUTPUT(*OUTFILE)
OUTFILE(ROLESWAP/WRKDG) OUTMBR(SWITCH)

Start the other subsystems on the new production Disk NSNJ400A that you
want to have active during this switch.
Have Applications Support checking and:
12
- starting job CAPTURE (once thee job is running, release the
MPLUS Capture monitor
- releasing MESA application usage by setting data area
MXUSROBJ/MXSYSSTS to ‘0’

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 30 of 47

Step Action Time
done
Signal to the test users group they can start working. Ensure they
13 understand they should connect to the same production host. It should
automatically connect to new production Disk NSNJ400A

Step Action Time

done
Also notify the users that an unplanned switch has occurred and ask them
14 to check for the most recent transactions they entered just before the
system crashed. (Most likely they will have noticed the unplanned switch
already)
On the NSNJ400B Disk, physically disconnect the Ethernet cable, or at
least ensure when it starts, the 158.58.43.12 production IP addresses is not
15 started during IPL. Also ensure no DISK production environment or other
productive subsystem will not be started during IPL
The system is now switched and the production environment and host
SNJ400B is now active on the NSNJ400A partition.
16 However, there is no replication yet to the original production system, so
when appropriate, start reverse synchronization by following instruction 14
on the synchronize procedure on page 23

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 31 of 47

 SJDISKMFG-RETURN
from
NSNJ400A to NSNJ400B
Procedure SJDISKMFG-RETURN is switching production role from the NSNJ400B Disk to the
NSNJ400A Disk. For a planned switch to backup use procedure SWITCHOVER-RETURN on
page 32, for unplanned switching use procedure FAILOVER-RETURN on page 38.

A reminder: A switch is unplanned, if the original production Disk is no longer accessible from the
backup Disk. If the original production Disk is reachable, it is a planned switch, even if it was not
scheduled or intended.

For switching back from the NSNJ400A to the NSNJ400B, please use the separate document
describing the SJDISKMFG-SWITCH procedure.

Procedure SWITCHOVER-RETURN – Switch to Backup

Goal
Switch users from the NSNJ400A Disk to the NSNJ400B Disk. Almost all the actions are initiated
on the NSNJ400B Disk, which is the controlling system for this procedure.

The next pages hold the following groups of steps:

 SWITCHOVER-RETURN Pre-Switch Tasks

To be executed immediately before the planned switch

 SWITCHOVER-RETURN Planned Switch Tasks

The actual switch.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 32 of 47

  SWITCHOVER-RETURN Post-Switch Tasks
Checking and cleaning up the switch.

SWITCHOVER-RETURN Pre-Switch Tasks

These tasks should be performed immediately prior to the switch:

Step Action Time

done
In the next step, connections to the system will be closed. To ensure you
keep your connection, use a session that is not using the switched IP
address 158.58.43.12, but one of the administrative addresses.
1
Create a 5250 telnet session to both systems. Then, on both systems,
transfer your job to QCTL instead of QINTER:
TFRJOB JOBQ(QCTL)
On the NSNJ400B, add the MIMIX and ROLESWAP to the library list.
2 ADDLIBLE ROLESWAP
ADDLIBLE MIMIX
Check active replication and resolve problems discovered. The MIMIX
3 active replication can be checked from the following command:
WRKDG
Review Object Activity entries for status of *ACTIVE and *FAILED. Resolve
4 any entries that are lingering for more than a few minutes. On both systems:
WRKDGACTE STATUS(*ACTIVE *FAILED)
Review File & Tracking Entries to determine that no files are in any non-
active status. Resolve any entries that are lingering for more than a few
minutes. On both systems
5
WRKDGFE STSVAL(*INACTIVE)
WRKDGOBJTE STSVAL(*INACTIVE)
WRKDGIFSTE STSVAL(*INACTIVE)
When you are ready for the actual switch, continue with step 6 below

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 33 of 47

 SWITCHOVER-RETURN Planned Switch Tasks

These tasks are the actual switch from NSNJ400A to NSNJ400B, and are to be
considered “downtime”.

Step Action Time

done
Shut down the production environment.
Give the users adequate time to end their work on the production system.
6 Check no interactive jobs are active, no batch jobs are active and all
scheduled batch jobs in job queues are held. Also all productive subsystems
should be ended.
Wait for until all above ended subsystems and environments really ended..
7 Do *not* end the MIMIXSBS subsystem.
On the NSNJ400B Disk perform the following command:
8 MIMIX/RUNSWTFWK SWTFWK(SJDISKMFG) PRC(*BCKUP)

This last command will check if the system is in the correct status. Next, it will confirm that
the user wishes to switch by issuing messages to QSYSOPR message queue.

On the NSNJ400B Disk answer the message in QSYSOPR message queue

9 to confirm the switchover. Use SysRq 6 to go to the QSYSOPR message
queue

The RUNSWTFWK process will end the production infrastructure, end MIMIX replication
controlled; switch MIMIX data group direction, switch the network connections, and start
the production infrastructure on the new production Disk NSNJ400B.

This is done by internally calling the following programs: (This is shown here as
documentation, these are not steps you need to perform)

NOTE: Codes are the same as program for SWITCHOVER above.

 SWTFWKCFM (running on Disk NSNJ400A)

This program will ask confirmation from the system operator, it also internally calls
programs DGSELECT to determine which data groups to switch. This program
performs no SJDISKMFG specific actions.

 ENDBCKUP (running on Disk NSNJ400A)

This program ends the backup environment on NSNJ400A. This program
performs no SJDISKMFG specific actions.

 ENDPROD (running on Disk NSNJ400B)

This program ends the production environment on NSNJ400B. This program
performs the following SJDISKMFG specific actions:
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 34 of 47
  Sets the local system role indicator data area MXSYSROLE to “S” (in switch)
 Sets MIMIX auto-start indicator to “*SWITCH” to prevent auto-start of replication
 Ends the TCP/IP interfaces 158.58.43.12
 Kills all production subsystems. These subsystems should already have been
ended nicely, but to prevent accidental activity after the switch, they are killed
from this program.
 Removes the TCP/IP host name to SNJ400B
 The replication to the NSNJ400A Disk is ended controlled and the data groups
are switched in their direction of replication. Reverse replication is not started yet.

 STRPROD (running on Disk NSNJ400A)

This program starts the production environment on NSNJ400A Disk. This program
performs the following SJDISKMFG specific actions:
 Sets the TCP/IP host name to SNJ400B
 Sets the local system role indicator data area MXSYSROLE to “P” (in
Production)
 Starts the TCP/IP interfaces 158.58.43.12

Note: The source for the above programs is located in file QCLSRC in library ROLESWAP
on both systems.

SWITCHOVER-RETURN Post-Switch Tasks

These tasks should be performed immediately after to the switch:

Step Action Time

done
On the NSNJ400B Disk, check that:
 The 1558.58.43.12 TCP/IP interface is active
use command NETSTAT for this
10  The TCP host name is set to SNJ400B
use command CFGTCP option 12 for this
 The NETSERVER job is started
Check for a job named QZLSSERVER
Record all current journal receiver names and first sequence numbers of all
journals on the new production Disk NSNJ400B, by using the following
11 command:
WRKDG OUTPUT(*OUTFILE)
OUTFILE(ROLESWAP/WRKDG) OUTMBR(SWITCH)
Start the other subsystems on the new production Disk NSNJ400B that you
12 want to have active during this switch.
Signal to the test users group they can start working. Ensure they
13 understand they should connect to the same production host SNJ400B. It
should automatically connect to new production Disk NSNJ400B

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 35 of 47

 The system is now switched and the production environment and host SNJ400B is now
active on the NSNJ400B partition.
However, there is no replication yet to the original production system, so when
appropriate, start reverse synchronization by following instruction 14 on the next page.
Procedure SYNCHRONIZE-RETURN – Resynchronize
Goal
After the users were switched to the NSNJ400B Disk by the SJDISKMFG-RETURN procedure,
this procedure will start MIMIX replication back to the NSNJ400A Disk, which then effectively
becomes the backup system.

The next pages hold the following groups of steps:

 SYNCHRONIZE Pre- Synchronization Tasks.

To be executed before the synchronization

 SYNCHRONIZE Synchronization Tasks.

The actual synchronization.

 SYNCHRONIZE Post- Synchronization Tasks

Checking and cleaning up the synchronization.

SYNCHRONIZE-RETURN Pre-Synchronization Tasks

These tasks should be performed some time prior to the synchronization:

Step Action Time

done
Only perform this sync procedure if you are certain that the current
14 production system is the NSNJ400B Disk.
Check that the NSNJ400A Disk is now again available for use (after being
15 repair or maintained).
Add the MIMIX and ROLESWAP libraries to the library list on the
16 NSNJ400B
On both systems, ensure the MIMIX journal and system managers are
running and that the MIMIX data groups are not running. The MIMIX active
17 replication can be checked from the following command:
WRKDG
Check the status of the switch framework (last switch results):
MIMIX/CHKSWTFWK SWTFWK(SJDISKMFG)
18 The command should return with a message “Switch Framework
SJDISKMFG has current status of:

On the NSNJ400B partition, if the above resulting status has to be either

19 ‘BCMP’ or ‘FCMP’.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 36 of 47

 If not, then a previous switch did not complete normally, and you have to
correct the status to ‘BCMP’.

SYNCHRONIZE-RETURN Synchronization Tasks

These tasks are the actual synchronization from NSNJ400B to NSNJ400A. The NSNJ400B is and
remains the production system after these steps:

Step Action Time

done
On the NSNJ400B, check if you still have MIMIX and ROLESWAP in the
20
library list.
On the NSNJ400B Disk perform the following command:
21
MIMIX/RUNSWTFWK SWTFWK(SJDISKMFG) PRC(*SYNC)

This command will start MIMIX replication from the NSNJ400A Disk to the NSNJ400B Disk
after user confirmation by issuing messages to QSYSOPR message queue.

On the NSNJ400B Disk answer the message in QSYSOPR message queue

22 to confirm the switchover. Use SysRq 6 to go to the QSYSOPR message
queue.

The RUNSWTFWK process will check the system and MIMIX status, and start replication
from the NSNJ400B Disk to the NSNJ400A Disk

This is done by internally calling the following programs: (This is shown here as
documentation, these are not steps you need to perform)

4. SWTFWKCFM (on Disk NSNJ400A)

This program will ask confirmation from the system operator, it also internally calls
programs DGSELECT to determine which data groups to start. This program performs
no SJDISKMFG specific actions.

5. STRBCKUP (on Disk NSNJ400B)

This program starts the backup environment on the former production system
NSNJ400B. This program performs no SJDISKMFG specific actions.

6. Starts the Data Groups to replicate from the NSNJ400A production Disk to the
NSNJ400B Disk, which is now a backup system.

Note: The source for the above programs is located in file QCLSRC in library MIMIX on
both systems.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 37 of 47

SYNCHRONIZE-RETURN Post-Synchronization Tasks

These tasks should be performed immediately after the synchronization:

Step Action Time

done
Verify that the MIMIX data groups’ source jobs are active, and they are
replicating from the NSNJ400B to the NSNJ400A partition. The target jobs
23 should not yet be active. Use the following command:
WRKDG
Check that the NSNJ400A Disk still has no production users. Check that
24 there are no production subsystems, and that the TCP/IP 158.58.43.12
interface is not active
When you feel the switch and sync are in order, you can now actually start
writing the updates into the NSNJ400A Disk by starting the apply sessions.

25 (If you are unsure of the status of the switch or replicated data, postpone
this step until advise from database and system administrators and testers
has been obtained)
STRDG DGDFN(*ALL) PRC(*ALLTGT)
After checking all functions are working as expected, disable switch
availability by issuing (in both systems):
26
MIMIX/ CHGSWTFWK SWTFWK(SJDISKMFG) STATUS(*DISABLED)

The switchover of the production environment and host SNJ400B from

NSNJ400A to NSNJ400B is now fully complete, and the NSNJ400A is now
the backup system. If you need to switch back, please use document and
procedure SJDISKMFG-RETURN
No more actions from this document are needed.

Procedure FAILOVER-RETURN – Fail over to Backup

Goal
Following the unplanned deactivation (or system crash) of the NSNJ400A production Disk, the
NSNJ400B Disk is activated to take on production role and allow user connections.

Only use this procedure if the NSNJ400A production Disk is not reachable from the NSNJ400B
backup system. Use the SJDISKMFG SWITCHOVER-RETURN procedure on page 32 if the
NSNJ400A Disk can still be reached by you as an administrator.

The next pages hold the following groups of steps:

 FAILOVER Pre-Switch Tasks

To be executed immediately before the unplanned switch
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 38 of 47
  FAILOVER Unplanned Switch Tasks
The actual switch.

 FAILOVER Post-Switch Tasks

Checking and cleaning up the switch.

FAILOVER Pre-Switch Tasks

This procedure applies only to an unplanned switch. These tasks should be performed
immediately prior to the switch:

Step Action Time

done
1 Check if the production NSNJ400A Disk is really no longer reachable.
Check active replication and resolve problems discovered for as far as still
2 possible. The operations procedures can be found in the operations section
of the run book.
3 Add the MIMIX and ROLESWAP libraries to the library list.
Review the latest audits results, which probably ran in the last 7 days. See if
there are any replication/audit issues that make it impossible for you to
switch over to the NSNJ400B partition.
If there are any major issues in the audit results, you should have dealt with
4 them when they were first reported. In any case, you may need to make a
judgment call whether to switch or not.
The MIMIX audit screen can be reached with the following command:
WRKAUD
On NSNJ400B partition, check the status of the last switch:
5 DSPDTAARA DTAARA(MIMIX/SJDISKMFG)
The status is in position 32 – 35.
On the NSNJ400B partition, the above resulting status has to be one of
‘SCMP’ or ‘PCMP’.
6
If not, then a previous switch did not complete normally, and you have to
correct the status to ‘PCMP’.

FAILOVER-RETURN Unplanned Switch Tasks

These tasks are the actual switch from NSNJ400A to NSNJ400B:

Step Action Time

done
On the NSNJ400B, check if you still have MIMIX and ROLESWAP in the
7 library list.
8 On the NSNJ400B Disk perform the following command:
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 39 of 47
 MIMIX/RUNSWTFWK SWTFWK(SJDISKMFG) PRC(*BCKUP)
TYPE(*UNPLANNED)

This last command will check if the system is in the correct status. Next, it will confirm that
the user wishes to switch by issuing messages to QSYSOPR message queue.

On the NSNJ400B Disk answer the message in QSYSOPR message queue

9 to confirm the switchover. Use SysRq 6 to go to the QSYSOPR message
queue.
The RUNSWTFWK process will end the production infrastructure; end MIMIX replication
controlled, switch MIMIX data group direction, switch the network connections, and start
the production infrastructure on the new production system NSNJ400B.

This is done by internally calling the following programs: (This is shown here as documentation,
these are not steps you need to perform)

 SWTFWKCFM (running on Disk NSNJ400B)

This program will ask confirmation from the system operator, it also internally calls
programs DGSELECT to determine which data groups to switch. This program
performs no SJDISKMFG specific actions.

ENDBCKUP (running on Disk NSNJ400B)

This program ends the backup environment on NSNJ400B. This program performs no
SJDISKMFG specific actions
 Program ENDPROD is not called, as this is an unplanned switch, and the
production system is assumed crashed and unreachable.

 The replication to the NSNJ400B system is ended controlled and the data groups
are switched in their direction of replication. Reverse replication is not started yet.
 STRPROD (running on Disk NSNJ400A)
This program starts the production environment on NSNJ400A Disk. This program
performs the following SJDISKMFG specific actions:
 Sets the TCP/IP host name to SNJ400B
 Sets the local system role indicator data area MXSYSROLE to “SRC” (in
Production)
 Starts the TCP/IP interfaces 158.58.43.12
 Starts the required subsystem and controllers

Note: The source for the above programs is located in file QCLSRC in library ROLESWAP
on both systems.

FAILOVER-RETURN Post-Switch Tasks

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 40 of 47

These tasks should be performed immediately after to the switch:

On the NSNJ400A Disk, check that:

 The 158.58.43.12 TCP/IP interface is active
10 use command NETSTAT for this
 The TCP host name is set to SNJ400B
use command CFGTCP option 12 for this
 The NETSERVER job is started
Check for a job named QZLSSERVER
Record all current journal receiver names and first sequence
numbers of all journals on the new production Disk NSNJ400A, by
11 using the following command:
WRKDG OUTPUT(*OUTFILE)
OUTFILE(ROLESWAP/WRKDG)
OUTMBR(SWITCH)
Start the other subsystems on the new production Disk NSNJ400A
that you want to have active during this switch.
Have Applications Support checking and:
12
- starting job CAPTURE (once thee job is running, release the
MPLUS Capture monitor
- releasing MESA application usage by setting data area
MXUSROBJ/MXSYSSTS to ‘0’
Signal to the test users group they can start working. Ensure they
13 understand they should connect to the same production host. It
should automatically connect to new production Disk NSNJ400A

Step Action Time

done
Also notify the users that an unplanned switch has occurred and ask them
to check for the most recent transactions they entered just before the
14 system crashed. (Most likely they will have noticed the unplanned switch
already)
On the NSNJ400A Disk, physically disconnect the Ethernet cable, or at least
ensure when it starts, the 158.58.43.12 production IP addresses is not
15 started during IPL. Also ensure no DISK production environment or other
productive subsystem will not be started during IPL
The system is now switched and the production environment and host
SNJ400B is now active on the NSNJ400B partition.
16 However, there is no replication yet to the original production system, so
when appropriate, start reverse synchronization by following instruction 14
on the synchronize procedure on page 36

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 41 of 47

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 42 of 47
Appendix A: Runbook Hyperlinks
This appendix contains embedded links (Hyperlinks) that allow the reader to view related configuration
information contained in separate files. The information in the files referenced by the links must be retrieved
from the iSeries servers and stored on the workstation used to view this document.
To properly use the links, the files referenced by the links must be located precisely where the link specifies
and have the name identified by the link
The following table contains the links to be used to view the detailed information considered part of this
Runbook:
Link Detailed Information Shown
MIMIX HA Configuration Information
system_definitions.csv MIMIX System Definitions
transfer_definitions.csv MIMIX Transfer Definitions
journal_definitions.csv MIMIX Journal Definitions
remote_journal_links.csv MIMIX Remote Journal Link Information
data_group_definitions.csv MIMIX Data Groups
data_group_file_entries.csv MIMIX Data Group File Entries
data_group_object_entries.csv MIMIX Data Group Object Entries
data_group_object_tracking_entries.csv MIMIX Data Group Object Tracking Entries
data_group_ifs_entries.csv MIMIX Data Group IFS Entries
data_group_ifs_tracking_entries.csv MIMIX Data Group IFS Tracking Entries
data_group_dlo_entries.csv MIMIX Data Group DLO Entries
data_group_data_area_entries.csv MIMIX Data Group Data Area Entries
collision_resolution_class_definitions.csv MIMIX Collision Resolution Classes

Cluster_standard_application_groups.csv MIMIX Standard Application Groups

Cluster_parent_application_groups.csv MIMIX Parent Application Groups
Cluster_child_application_groups.csv MIMIX Child Application Groups

Mimix.policy.csv MIMIX Policies

mimix_rule.csv MIMIX Audit Rules
mimix_rulg.csv MIMIX Audit Rule Groups
mimix_grul.csv MIMIX Audit Rule Group Rules
S1111111 Server Information
S1111111_ASP_information Auxiliary Storage Pool (ASP) Information
S1111111_network_attributes.txt Network Attributes
S1111111_TCPIP_interface_table_entries.csv TCP/IP Interface Table Entries
S1111111_TCPIP_host_table_entries.csv TCP/IP Host Table Entries
S1111111_TCPIP_domain_information.csv TCP/IP Domain Information
S1111111_TCPIP_route_table_entries.csv TCP/IP Routing Table Entries
S1111111_TCPIP_service_table_entries.csv TCP/IP Service Table Entries
S2222222 Server Information
S2222222_ASP_information Auxiliary Storage Pool (ASP) Information
S2222222_network_attributes.txt Network Attributes
S2222222_TCPIP_interface_table_entries.csv TCP/IP Interface Table Entries
S2222222_TCPIP_host_table_entries.csv TCP/IP Host Table Entries
S2222222_TCPIP_domain_information.csv TCP/IP Domain Information
S2222222_TCPIP_route_table_entries.csv TCP/IP Routing Table Entries
S2222222_TCPIP_service_table_entries.csv TCP/IP Service Table Entries

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 43 of 47

Appendix B: The Runbook Data Capture Tool
Vision Solution provides a software tool to automatically create/refresh the files referenced by the
Hyperlinks in the Runbook. The software tool runs on a Windows workstation and uses FTP to interact with
the iSeries servers associated with the MIMIX solution. The tool will create/refresh the files and, if selected,
downloads the files from the iSeries server to a folder on the workstation.
The Runbook Data Capture tool includes functions that run on the user’s workstation and functions that run
on each of the iSeries servers from which data is to be retrieved. The intent is that the Runbook will contain
the following information retrieved from the iSeries servers:
Server Type Downloaded Files
Management System MIMIX configuration information, Policy, Rules, and Rule
Groups
Network attributes
TCP/IP configuration information
ASP information (for independent ASP environments)
Each Network System Network attributes
TCP/IP configuration information
ASP information (for independent ASP environments)

To use the Runbook Data Capture tool, do the following:

Note: These instructions apply to V5 MIMIX or later. If you are working with an earlier level of MIMIX,
contact your Vision Solution Manager for further information.
1. Sign on to the Support section of the www.mimix.com site and navigate to the following page
http://www.mimix.com/customs/downloads/mimixv4r4/scripts.asp. Click on the Read Me for the
Runbook Automated Collection Tool, and review the contents of the file. A copy of the file contents
appears below, but the version of the file on the web site may be more recent.
2. While signed on to the same section of the web site, click on the Runbook Automated Collection
Tool to download the runbooksetup.exe file. This file can be placed in any folder you choose on
your workstation.
3. Once the file is downloaded, run the runbooksetup.exe file. Doing this will create (if you accept the
defaults) the following folder structure on your workstation:
C:\Program Files\Vision Solutions\Runbook\runbookdata
C:\Program Files\Vision Solutions\Runbook\runbooktool
The runbookdata folder will be empty initially. It is the default location for data to be downloaded
later from the iSeries by the Runbook Data Capture tool.

The runbooktool folder will contain several objects, including the Read Me file and, most
importantly, the runRunbook batch file used to run the tool itself.
4. When you are ready to capture data for the Runbook, follow the instructions in the README.TXT
file.

Using and Updating the Links Used in this Document

When the configuration data files referenced by the links are downloaded from the iSeries by the Runbook
Data Capture tool, the user of the tool is able to specify the folder on the workstation where the files will be
placed.
 The default folder is:
C:\Program Files\Vision Solutions\Runbook\runbookdata
 You may specify any folder convenient to you
When the links in this document are used, they must refer to the folder where the downloaded files are
stored. By default, this document expects to find the files in the same folder where the Runbook document
itself is stored. For the links to work properly, one of the two following steps must be done:
 The files to be downloaded must either be downloaded into the folder where the Runbook Word
document resides, or moved from where they were downloaded to that folder.
 The links in the Runbook Word document must be modified to reference the location where the
downloaded files reside.
MIMIX HA DISK RunbookHGST Disk Availability Runbook page 44 of 47
Updating The Links
There are two methods to update the links
Method 1: Automatically
This method will alter all the links in the template to reference the folder in which the template is saved.
After ensuring that you have the .csv and .txt files generated by using the Runbook Data Capture Tool in
the same folder as the copy of this Runbook document into which you want those .csv and .txt files
embedded, perform the following steps:
 Move a copy of the Runbook document into the folder where you have (or will have) located
your .csv and .txt files
 Open the Runbook document
 Do a "CTRL-A". That is, press and hold the CONTROL key, then while you have the CONTROL
key depressed, press the A key also. Then release both keys. You should see that you have
selected the entire document.
 Press the F9 key. This should open up a window that looks something like the following:

 Select the UPDATE ENTIRE TABLE button, and press OK. This should update all links in your
entire document, as well as updating the table of contents. The links should now point to the
folder in which you have the Runbook document and .csv files located.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 45 of 47

Method 2: Manually
If it is preferred to manually modify the links, for example if for some reason you want to keep the Runbook
document and .csv and .txt files in different folders, there are two attributes of each link that typically need
to be modified:
1. The text displayed
This is a text string that appears in the document and is viewable when the document is printed.
While this text string is not used in accessing the file associated with the link, by convention, it often
consists of some of the characters in the file name. See the example below.
2. The file name
This is the full file name (including path information) of the file. It is used to access the file when the
link is selected. This file name does not appear when the document is printed. See the example
below.
To view these two link attributes, as well as to change them, do the following while editing the Runbook
Word document:
 Highlight the link (being careful not to click on it).
 Click on the Microsoft Word Insert pull-down.
 Click on the Hyperlink option. The following window will pop up [the data group definition link was
chosen for this example]:
 Click on the Browse for: File button and navigate to the location of the files
 Select the file
Text to
Display

File Name

Select
File

 Make the desired changes and press OK when completed.

Note: The Microsoft Word “Find/Replace” function may be used to globally change the hyperlink “text to
display” specifications in this template. To make that easier to do, the following character strings are used
only for hyperlinks in this entire template:
system_
transfer_
journal_
data_group_
collision_
S1111111_
S2222222_
The “Find/Replace” function has no effect on the “File Name” associated with a Hyperlink.

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 46 of 47

 Last page of the Hitachi GST DISK Availability Runbook document

MIMIX HA DISK RunbookHGST Disk Availability Runbook page 47 of 47