18CS822 Storage Area Networks Module-5 Local Replication
ACADEMIC YEAR (2021-2022)
EVEN SEMESTER
STORAGE AREA NETWORKS(18CS822)
Module 5
Module 5: Local Replica Technologies: Host-Based Local Replication, Storage Array- Based
Local Replication, Network Based Local Replication with Technologies, Local Replication Technologies,
Tracking Changes to Source and Replica, Restore and Restart Considerations Creating Multiple
Replicas.
Remote Replication: Modes of Remote Replication, Replication Technologies: Host-Based
Remote Replication ,Storage Array-Based Remote Replication, Network Based Remote Replication,
Securing the Storage Infrastructure: Information Security Framework, Risk Triad: Assets,
Treats, Vulnerability, Storage Security Domains: Securing the application Access, Domain,
Securing the management Access Domain ,Security Implementations in Storage Networking:
FCSAN, Security Implementations in Storage Networking: NAS,IP SAN.
Text Books:
1.EMC Education Services, “Information Storage and Management”, Wiley India
Publications, 2009. ISBN: 9781118094839.
Reference Books:
Paul Massiglia, Richard Barker, "Storage Area Network Essentials: A Complete Guide to
Understanding and Implementating SANs Paperback", 1st Edition, Wiley India
Publications, 2008.Text Book1:Ch 3.1 to 3.6, Ch4.1,4.3, Ch 5.1 to 5.3 .RBT L1,L2
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Chapter 11
Local Replication
Replication Terminology:
The common terms used to represent various entities and operations in a
replication environment are listed here:
■ Source: A host accessing the production data from one or more LUNs on
the storage array is called a production host, and these LUNs are known
as source LUNs (devices/volumes), production LUNs, or simply the
source.
■ Target: A LUN (or LUNs) on which the production data is replicated, is
called the target LUN or simply the target or replica.
■ Point-in-Time (PIT) and continuous replica: Replicas can be either
a PIT or a continuous copy. The PIT replica is an identical image of the
source at some specific timestamp.
■ For example, if a replica of a file system is created at 4:00 p.m. on
Monday, this replica is the Monday 4:00 p.m. PIT copy.
■ On the other hand, the continuous replica is in-sync with the pro-
duction data at all times.
■ Recoverability and restartability: Recoverability enables
restoration of data from the replicas to the source if data loss or
corruption occurs.
■ Restartability enables restarting business operations using the
replicas. The replica must be consistent with the source so that it is
usable for both recovery and restart operations.
■ Replica consistency is detailed in section “11.3 Replica Consistency
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Uses of Local Replicas
One or more local replicas of the source data may be created for
various purposes, including the following:
Alternative source for backup: Under normal backup operations,
data is read from the production volumes (LUNs) and written to the
backup device.
This places an additional burden on the production infrastructure
because production LUNs are simultaneously involved in
production operations and servicing data for backup operations.
The local replica contains an exact point-in-time (PIT) copy of the
source data, and there-fore can be used as a source to perform
backup operations.
This alleviates the backup I/O workload on the production volumes.
Another benefit of using local replicas for backup is that it reduces
the backup window to zero.
Fast recovery: If data loss or data corruption occurs on the source,
a local replica might be used to recover the lost or corrupted data.
If a complete failure of the source occurs, some replication solutions
enable a replica to be used to restore data onto a different set of
source devices, or production can be restarted on the replica.
In either case, this method provides faster recovery and minimal
RTO compared to traditional recovery from tape backups.
In many instances, business operations can be started using the
source device before the data is completely copied from the replica.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Decision-support activities, such as reporting or data
warehousing: Running the reports using the data on the replicas
greatly reduces the I/O burden placed on the production device.
Local replicas are also used for data-warehousing applications.
The data-warehouse application may be populated by the data on
the replica and thus avoid the impact on the production
environment.
Testing platform: Local replicas are also used for testing new
applications or upgrades.
o For example, an organization may use the replica to test
the production application upgrade;
o if the test is successful, the upgrade may be implemented
on the production environment.
Data migration: Another use for a local replica is data migration.
Data migrations are performed for various reasons, such as migrating
from a smaller capacity LUN to one of a larger capacity for newer
versions of the application.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Replica Consistency
Most file systems and databases buffer the data in the host before
writing it to the disk. A consistent replica ensures that the data
buffered in the host is captured on the disk when the replica is
created.
The data staged in the cache and not yet committed to the disk
should be flushed before taking the replica.
The storage array operating environment takes care of flushing
its cache before the replication operation is initiated.
Consistency ensures the usability of a replica and is a primary
requirement for all the replication technologies.
Consistency of a Replicated File System
File systems buffer the data in the host memory to improve the
application response time. The buffered data is periodically
written to the disk.
In UNIX operating systems, sync daemon is the process that flushes
the buffers to the disk at set intervals. In some cases, the replica
is created between the set intervals, which might result in the
creation of an inconsistent replica.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Therefore, host memory buffers must be flushed to ensure data
consistency on the replica, prior to its creation.
Figure 11-1 illustrates how the file system buffer is flushed to the
source device before replication.
If the host memory buffers are not flushed, the data on the replica
will not contain the information that was buffered in the host.
If the file system is unmounted before creating the replica, the
buffers will be automatically flushed and the data will be consistent
on the replica
If a mounted file system is replicated, some level of recovery, such as
fsck or log replay, is required on the replicated file system.
When the file system replica-tion and check process are completed,
the replica file system can be mounted for operational use.
Consistency of a Replicated Database
A database may be spread over numerous files, file systems, and
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
devices. All of these must be replicated consistently to ensure that the
replica is restorable and restart table.
Replication is performed with the database offline or online. If the
database is offline during the creation of the replica, it is not available
for I/O operations. Because no updates occur on the source, the
replica is consistent.
When a database is replicated while it is online, changes made to the
database at this time must be applied to the replica to make it
consistent.
A consistent replica of an online database is created by using the
dependent write I/O principle or by holding I/Os momentarily to the
source before creating the replica.
A dependent write I/O principle is inherent in many applications and
database management systems (DBMS) to ensure consistency.
According to this principle, a write I/O is not issued by an application
until a prior related write I/O has completed. For example, a data write
is dependent on the successful completion of the prior log write.
For a transaction to be deemed complete, databases require a series of
writes to have occurred in a particular order. These writes will be
recorded on the various devices or file systems.
Figure 11-2, illustrates the process of flushing the buffer from the host
to the source; I/Os 1 to 4 must complete for the transaction to be
considered complete. I/O 4 is dependent on I/O 3 and occurs only if
I/O 3 is complete. I/O 3 is dependent on I/O 2, which in turn depends
on I/O 1.
Each I/O completes only after completion of the previous I/O(s).
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Figure 11-2: Dependent write
consistency on sources
When the replica is created, all the writes to the source devices must
be captured on the replica devices to ensure data consistency.
Figure 11-3 illustrates the process of replication from the source to
the replica. I/O transactions 1 to 4 must be carried out for the data to be
consistent on the replica. It is possible that I/O transactions 3 and 4 were
copied to the replica devices, but I/O transactions 1 and 2 were not copied.
Figure 11-4 shows this situation.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
In this case, the data on the replica is inconsistent with the data on
the source. If a restart were to be performed on the replica devices, I/O
4, which is avail- able on the replica, might indicate that a particular
transaction is complete, but all the data associated with the transaction will
be unavailable on the replica, making the replica inconsistent.
Data Data
Log Log
Source Replica
Consistent
Figure 11-3: Dependent write consistency on replica
Da
ta Data
Log Log
Source Replica
Inconsistent
Figure 11-4: Inconsistent database replica
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Local Replication Technologies
Host-based, storage array-based, and network-based replications are
the major technologies used for local replication. File system
replication and LVM-based replication are examples of host-based
local replication.
Storage array-based replication can be implemented with distinct
solutions, namely, full-volume mirroring, pointer-based full-volume
replication, and pointer- based virtual replication. Continuous data
protection (CDP).
Host-Based Local Replication
LVM-based replication and file system (FS) snapshot are two common
methods of host- based local replication.
LVM-Based Replication
In LVM-based replication, the logical volume manager is responsible
for creating and controlling the host-level logical volumes.
An LVM has three components: physical volumes (physical disk), volume
groups, and logical volumes. A volume group is created by grouping
one or more physical volumes. Logical volumes are created within a
given volume group.
A volume group can have multiple logical volumes.
In LVM-based replication, each logical block in a logical volume is
mapped to two physical blocks on two different physical volumes, as
shown in Figure 11-5.
An application write to a logical volume is written to the two physical
volumes by the LVM device driver. This is also known as LVM
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
mirroring.
Mirrors can be split, and the data contained therein can be inde-
pendently accessed.
Advantages of LVM-Based Replication
The LVM-based replication technology is not dependent on a vendor-
specific storage system. Typically, LVM is part of the operating
system, and no additional license is required to deploy LVM
mirroring.
Physical Volume 1
Logical Volume
Host
Physical Volume 2
Figure 11-5: LVM-based mirroring
Limitations of LVM-Based Replication
Every write generated by an application translates into two writes on the
disk, and thus, an additional burden is placed on the host CPU. This can
degrade application performance. Presenting an LVM-based local replica to
another host is usually not possible because the replica will still be part of the
volume group, which is usually accessed by one host at any given time.
Tracking changes to the mirrors and performing incremental
resynchronization operations is also a challenge because all LVMs do not
support incremental resynchronization. If the devices are already
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
protected by some level of RAID on the array, then the additional
protection that the LVM mirroring provides is unnecessary. This solution
does not scale to provide replicas of federated data- bases and
applications. Both the replica and source are stored within the same
volume group. Therefore, the replica might become unavailable if there
is an error in the volume group. If the server fails, both the source and
replica are unavailable until the server is brought back online.
File System Snapshot
A file system (FS) snapshot is a pointer-based replica that requires a
fraction of the space used by the production FS.
This snapshot can be implemented by either FS or by LVM. It uses
the Copy on First Write (CoFW) principle to create snapshots.
When a snapshot is created, a bitmap and block map are created in
the metadata of the Snap FS. The bitmap is used to keep track
of blocks that are changed on the production FS after the snap
creation.
The block map is used to indicate the exact address from which the
data is to be read when the data is accessed from the Snap FS.
Immediately after the creation of the FS snapshot, all reads from the
snapshot are actually served by reading the production FS.
Figure 11-6 illustrates the write operations to the production file
system. For example, a write data “C” occurs on block 3 at the production
FS, which currently holds data “c”’ The snapshot application
holds the I/O to the production FS and first copies the old data.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
c” to an available data block on the Snap FS.
The bitmap and block map values for block 3 in the production
FS are changed in the snap metadata. The bitmap of block 3 is
changed to 1, indicating that this block has changed on the
production FS.
The block map of block 3 is changed and indicates the block
number where the data is written in Snap FS.
After this is done, the I/Os to the production FS are allowed to
complete. Any subsequent writes to block 3 on the production FS
occur as normal, and it does not initiate the CoFW operation.
Similarly, if an I/O is issued to block 4 on the production FS
to change the value of data “d” to “D,” the snapshot application
holds the I/Oto the production FS and copies the old data to an
available data block on the Snap FS. Then it changes the bitmap of
block 4 to 1, indicating that the data block has changed on the
production FS.
The block map for block 4 indicates the block number where
the data can be found on the Snap FS, in this case, data block 1 of
the Snap FS.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
After this is done, the I/O to the production FSis allowed to
Snap FS
Production FS Metadata
Metadata Blockmap Bitmap
1-0 1-0
1 Data a
2-0 2-0
2 Data b
3-2 3-1
3 Data C
STOP
New Writes
4-1 4-1
STOP 4 Data D
Data d
Data c
N Data N
No Data
No Data
complete.
Figure 11-6: Write to production FS
Storage Array-Based Local Replication
In storage array-based local replication, the array-operating
environment performs the local replication process.
The host resources, such as the CPU and memory, are not used
in the replication process.
Consequently, the host is not burdened by the replication
operations. The replica can be accessed by an alternative host for
other business operations.
In this replication, the required number of replica devices should be
selected on the same array and then data should be replicated between
the source-replica pairs.
Figure 11-7 shows a storage array-based local replication, where
the source.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
And target are in the same array and accessed by different hosts.
Read/Write Read/Write
Source Replica
Production Storage Array BC Host
Host
Figure 11-7: Storage array-based local replication
Storage array-based local replication is commonly
implemented in three ways: full- volume mirroring, pointer-
based full-volume replication, and pointer-based virtual
replication. Replica devices are also referred as target devices,
accessible by other hosts.
Full-Volume Mirroring
In full-volume mirroring, the target is attached to the source
and established as a mirror of the source (Figure 11-8 [a]).
The data on the source is copied to the target. New updates
to the source are also updated on the target. After all the data
is copied and both the source and the target contain identical
data, the target can be considered as a mirror of the source.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
While the target is attached to the source it remains unavailable to
any other host.
However, the production host continues to access the source.
After the synchronization is complete, the target can be
detached from the source and made available for other
business operations.
Figure 11-8 (b) shows full-volume mirroring when the target
is detached from the source.
Both the source and the target can be accessed for read and
write operations by the production and business continuity
hosts respectively.
After detaching from the source, the target becomes a point-
in-time (PIT) copy of the source. The PIT of a replica is
determined by the time when the target is detached from the
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
source.
For example, if the time of detachment is 4:00 p.m., the PIT for
the target is 4:00 p.m.
After detachment, changes made to both the source and replica
can be tracked at some predefined granularity. This enables
incremental resynchronization (source to target) or incremental
restore (target to source). The granularity of the data change can
range from 512 byte blocks to 64 KB blocks on higher.
Pointer-Based, Full-Volume Replication
Another method of array-based local replication is pointer-based
full-volume replication. Similar to full-volume mirroring, this
technology can provide full copies of the source data on the
targets.
Unlike full-volume mirroring, the target is immediately
accessible by the BC host after the replication session is
activate.
Therefore, data synchronization and detachment of the target
is not required to access it. Here, the time of replication session
activation defines the PIT copy of the source.
The data is then copied from the source to the target based on
the mode of activation.
In CoFA, after the replication session is initiated, the data is
copied from the source to the target only when the following
condition occurs:
■ A write I/O is issued to a specific address on the source for the
first time.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
■ A read or write I/O is issued to a specific address on
the target for the first time.
When a write is issued to the source for the first time after
replication session activation, the original data at that address
is copied to the target. After this operation, the new data is
updated on the source.
This ensures that the original data at the point-in-time of activation
is preserved on the target (see Figure 11-9).
When a read is issued to the target for the first time after
replication session activation, the original data is copied from
the source to the target and is made available to the BC host
(see Figure 11-10).
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
When a write is issued to the target for the first time after the replication
session activation, the original data is copied from the source to the target.
After this, the new data is updated on the target (see Figure 11-11).
The key difference between a pointer-based, Full Copy mode and
full-volume mirroring is that the target is immediately accessible
upon replication session activation in the Full Copy mode.
Both the full-volume mirroring and pointer- based full-volume
replication technologies require the target devices to be at least
as large as the source devices.
In addition, full-volume mirroring and pointer-based full-volume
replication in the Full Copy mode can provide incremental
resynchronization and restore capabilities
• Fig read from target device
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Pointer-Based Virtual Replication
In pointer-based virtual replication, at the time of the replication
session activation, the target contains pointers to the location of
the data on the source.
The target does not contain data at any time. Therefore, the target
is known as a virtual replica. Similar to pointer-based full-volume
replication, the target is immediately accessible after the
replication session activation.
A protection bitmap is created for all data blocks on the source
device. Granularity of data blocks can range from 512 byte blocks
to 64 KB blocks or greater.
Pointer-based virtual replication uses the CoFW technology. When
a write is issued to the source for the first time after the replication
session activation, the original data at that address is copied to a
predefined area in the array.
This area is generally known as the save location. The pointer in
the target is updated to point to this data in the save location.
After this, the new write is updated on the source. This process is
illustrated in Figure 11-12.
Subsequent writes to the same data block on the source or target
do not trigger a copy operation. This process is illustrated in
Figure 11-13.
When reads are issued to the target, unchanged data blocks since
the session activation are read from the source, whereas data
blocks that have changed are read from the save location.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Data on the target is a combined view of unchanged data on the
source and data on the save location. Unavailability of the source
device invalidates the data on the target.
The target contains only pointers to the data, and therefore, the
physical capacity required for the target is a fraction of the source
device. The capacity required for the save location depends on
the amount of the expected data change.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Network-Based Local Replication
In network-based replication, the replication occurs at the network
layer between the hosts and storage arrays. Network-based
replication combines the benefits of array-based and host-based
replications.
By offloading replication from servers and arrays, network-
based replication can work across a large number of server
platforms and storage arrays, making it ideal for highly
heterogeneous environments. Continuous data protection (CDP)
is a technology used for network-based local and remote
replications.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
18CS822 Storage Area Networks Module-5 Local Replication
Continuous Data Protection
In a data center environment, mission-critical applications often
require instant and unlimited data recovery points. Traditional
data protection technologies offer limited recovery points. If data
loss occurs, the system can be rolled back only to the last available
recovery point. Mirroring offers continuous replication; however,
if logical corruption occurs to the production data, the error might
propagate to the mirror, which makes the replica unusable.
In normal operation, CDP provides the ability to restore data to
any previous PIT. It enables this capability by tracking all the
changes to the production devices and maintaining consistent
point-in-time images.
In CDP, data changes are continuously captured and stored in a
separate location from the primary storage. Moreover, RPOs are
random and do not need to be defined in advance.
CDP appliance is an intelligent hardware platform that runs the CDP software
and manages local and remote data replications.
Write splitters intercept writes to the production volume from the
host and split each write into two copies. Write splitting can be performed
at the host, fabric, or storage array.
CDP Local Replication Operation
Figure 11-14 describes CDP local replication. In this method, before
the start of replication, the replica is synchronized with the source and
then the replication process starts. After the replication starts, all the
writes to the source are split into two copies. One of the copies is sent
to the CDP appliance and the other to the production volume. When
the CDP appliance receives a copy of a write, it is written to
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
the journal volume along with its timestamp. As a next step, data from
the journal volume is sent to the replica at predefined intervals.
Tracking Changes to Source and Replica
Updates can occur on the source device after the creation of PIT local
replicas. If the primary purpose of local replication is to have a viable.
PIT copy for data recovery or restore operations, then the replica
devices should not be modified.
Changes can occur on the replica device if it is used for other
business operations. To enable incremental resynchronization or restore
operations, changes to both the source and replica devices after the PIT
should be tracked. The bits in the source and target bitmaps are all set to
0 (zero) when the
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
replica is created. Any changes to the source or replica are then flagged
by set- ting the appropriate bits to 1 in the bitmap.
When resynchronization or restore is required, a logical OR operation
between the source bitmap and the target bitmap is performed. The
bitmap resulting from this operation references all blocks that have been
modified in either the source or replica (see Figure 11-15).
This enables an optimized resynchronization or a restore operation
because it eliminates the need to copy all the blocks between the source
and the replica. The direction of data movement depends on whether a
resynchronization or a restore operation is performed.
If resynchronization is required, changes to the replica are
overwritten with the corresponding blocks from the source. In this example,
that would be blocks labeled 2, 3, and 7 on the replica.
If a restore is required, changes to the source are overwritten with
the corresponding blocks from the replica. In this example, that would be
blocks labeled0, 3, and 5 on the source. In either case, changes to both the
source and the target cannot be simultaneously preserved.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Restore and Restart Considerations
Local replicas are used to restore data to production devices.
Alternatively, applications can be restarted using the consistent PIT
replicas.
Replicas are used to restore data to the production devices if logical
corruption of data on production devices occurs — that is, the devices are
available but the data on them is invalid. Examples of logical corruption
include accidental deletion of data (tables or entries in a database),
incorrect data entry, and incorrect data updates. Restore operations from
a replica are incremental and provide a small RTO. In some instances, the
applications can be resumed on the production devices prior to the completion
of the data copy. Prior to the restore operation, access to production and
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
replica devices should be stopped.
Full-volume replicas (both full-volume mirrors and pointer-based in
Full Copy mode) can be restored to the original source devices or to a new
set of source devices.
Restores to the original source devices can be incremental, but
restores to a new set of devices are full-volume copy operations.
pointer-based virtual and pointer-based full-volume replication in
CoFA mode, access to data on the replica is dependent on the health and
accessibility of the source volumes. If the source volume is inaccessible for
any reason, these replicas cannot be used for a restore or a restart
operation.
Table 11-1 presents a comparative analysis of the various storage
array-based replication technologies.
Creating Multiple Replicas
Most storage array-based replication technologies enable source
devices to maintain replication relationships with multiple targets.
Changes made to the source and each of the targets can be tracked.
This enables incremental resynchronization of the targets. Each PIT
copy can be used for different BC activities and as a restore points.
If the source is corrupted, the data can be restored from the latest
PIT copy. The maximum RPO in the example shown in Figure 11-
16 is 6 hours. More frequent replicas further reduce the RPO.
Array-based local replication technologies also enable the creation
of multiple concurrent PIT replicas. In this case, all replicas contain
identical data. One or more of the replicas can be
set aside for restore operations. Decision support activities can be
performed using the other replicas.
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Table 11-1: Comparison of Local Replication Technologies
POINTER- POINTER-
FULL- BASED, BASED
FACTOR VOLUME FULL- VIRTUAL
MIRROR VOLUME REPLICAT
ING REPLICAT ION
ION
Performan No impact CoFA mode — High impact
ce some
impact on impact
source
due to Full copy mode
replica — no
impact
Size of At least the At least the Small
target same as same as the fraction of
the source source the source
Availabil Not required CoFA mode — Required
ity of required
source
Full copy
for
mode — not
restorat
required
ion
Accessibili Only after Immediately Immediately
ty syn-
to target chronization accessible accessible
and
detachment
from
the source
CSE DEPT VEMANA IT
18CS822 Storage Area Networks Module-5 Local Replication
Figure 11-16 shows an example in which a copy is created every 6 hours
from the same source.
CSE DEPT VEMANA IT
CSE DEPT VEMANA IT