VDisk zero-detection - SVC_Support - itclin19 https://svcsupport.itc.ibm.com/wiki/index.php/VDisk_zero-detection#R...

VDisk zero-detection
From SVC_Support - itclin19
Jump to navigationJump to search

Thin provisioned / Space efficient VDisks have been with us since 430, where:-

1. Space is allocated when data is first written to a virtual LBA

2. Space is never de-allocated
If a host deletes a file the storage used by that file is not freed, hopefully the filesystem will re-use the storage
3. User can choose the granularity at which space is allocated (32, 64, 128 or 256KBytes)
When a grain is first allocated it is always formatted by writing 0x00’s
4. Space Efficient VDisks require more I/Os because of directory accesses
1. For truly random workloads a Space Efficient VDisk will require approximately 1 directory I/O for every user I/O so
performance will be 50% of a normal VDisk
2. The directory is 2-way write-back cached (just like the SVC fastwrite cache) so some applications will perform better
3. Space-Efficient VDisks require more CPU processing so the performance per I/O group will be lower

But as the product has evolved additional features have been introduced: ZERO DETECT ( zero-detection ) being introduced in SVC 510.

See DCR 93010 Zero-detection for conversion of fully-allocated vdisks, and for host writes

Contents
1 From DCR 93010
2 Background
3 What does this mean to us ?
4 zero-detect for host writes
5 510 DST Reclaiming unused disk space
6 Methods of reclaiming "stale" VMWare space
6.1 References
6.2 Redbook:- Thin provisioning and managing stale data
6.2.1 Microsoft
6.2.2 VMware
7 Andy Blog

From DCR 93010

Background
It is a property of space-efficient disks that storage of zero-valued data is optimized. In general, when an i/o request is made to write zero-
valued data to unallocated space on a space efficient disk, the software should optimize its processing such that a minimal (ideally zero)
amount of new data and metadata is written to the space efficient disk. When a subsequent read i/o request is made for the same region of the
disk, the software should return zero-valued data, ideally without requiring a read of the disk's real address space, thus obviating any
requirement for user write i/o's to explicitly write zero-valued data.

This DCR is to provide a mechanism to detect the presence of zero-valued data in write requests (under certain circumstances, see the
Requirements section) and optimize the subsequent processing so that no new metadata or user data is written to the SE disk when zero data is
present.

The concept is that writing zero-valued data to unallocated space essentially becomes a no-op.

This DCR builds upon some previous work in SVC SE disks, namely:-

a) reads from unallocated grains return zeros (part of the original SE design);

b) zero-page: a dedicated memory page pre-filled with zeros, can be used to express one or more pages of zeros occurring within an i/o buffer,

1 of 5 6/21/2022, 6:02 PM
VDisk zero-detection - SVC_Support - itclin19 https://svcsupport.itc.ibm.com/wiki/index.php/VDisk_zero-detection#R...

by including the page descriptor of the special page within the i/o buffer's page descriptor list;

c) 'formatting' (newly allocated data grains are block-filled with zeros); a write i/o may only write to some fraction of a newly allocated data
grain; subsequent reads to unwritten regions of the same grain are guaranteed to return zeroes, without requiring any additional write i/o's to
those same regions.

d) existing "write-zero" optimization: there is an existing optimization that can be applied to writes of zero data to SE disks; if the write is to
an unallocated grain, there will be no new allocation from the SE disk real address space resulting from the write, and no new metadata
created; however if the write is to an allocated grain, zero data is written to that grain; this DCR makes no change to the behavior provided by
this optimization, but seeks to exploit it under the circumstances described later.

This DCR has the aim of zero-optimizing one specific SVC use case, namely that of creating a SE copy from an existing non-SE vdisk.

Other potential use cases for the SE write-zero optimization may exist, but they will not be considered within the scope of this DCR.

The requirements for this DCR are:-

1. When a SE copy is made of a non-SE disk, zero data will be detected on the fly as it is submitted to the top of the SE layer within the i/o
stack (ie the algorithm must not require any separate pre-processing of the non-SE data).

2. The level of granularity used in the zero detection algorithm is the buffer size passed to the SE i/o submission interface.

3. The optimization will only be applied when a submitted i/o buffer is found to contain all zero data; otherwise a normal write of the i/o
buffer will occur.

4. The zero-detect algorithm must able to recognize the presence of both the special zero page and normal zero-valued data.

5. The processing involved in scanning input i/o buffers for zero data must have no significant effect on SVC i/o performance. Specifically, in
any one polling loop iteration, time spent in the zero scan algorithm must be < 1 millisecond.

6. No change to CLI, ie the optimization must be transparent to the user; no new error codes or messages will be introduced.

What does this mean to us ?

When converting ( using VDisk Mirroring ) from fully allocated ( Full Fat ) to Space Efficient ( Semi Skinned ) we can "dump" the zero's

This is achieved by the use of a zero-detection algorithm. ( and there's a level of H/W dependency to this feature ).
Notes
For all node types prior to CF8, this algorithm is used only for I/O generated by synchronization of Mirrored VDisks.
I/O from other components (e.g. FlashCopy) is written as normal.

But for ( and from ? ) CF8 nodes can perform zero-detect on host writes - see “zero-detect for host writes” below.

Requirements of the zero-detect function include:

• Granularity: zero-detection is per IO buffer submitted to the SE component. If the buffer consists entirely of 00’s, no space will be
allocated, otherwise, real space will be allocated for the whole IO buffer.
• Performance: zero-detection is intended to have no significant impact on SVC I/O performance. Specifically, in any one polling loop
iteration, time spent in the zero scan algorithm must be < 1 millisecond. This is achieved by implementing the comparison operation in
kernel space (PLMM driver) rather than user-space.
• Transparency: zero-detection is transparent to the user; no CLI changes or error codes are introduced.

zero-detect for host writes

(DCR 104292) (DCR 97241)

SVC node models from 2145-CF8 onwards and all enclosure-based system models use the same feature to detect host I/Os which contain full
grains containing only zeroes. No space is allocated on an SE vdisk for these grains. This feature is disabled on all node hardware earlier than
CF8 as the slower CPUs in those nodes causes a noticable drop in throughput for some I/O workloads

510 DST Reclaiming unused disk space

2 of 5 6/21/2022, 6:02 PM
VDisk zero-detection - SVC_Support - itclin19 https://svcsupport.itc.ibm.com/wiki/index.php/VDisk_zero-detection#R...

Enables customers to reclaim unused allocated disk space (zeros) when converting a fully allocated VDisk to a Space-Efficient Virtual
Disk (SEV) using VDisk Mirroring
Why it matters:-
Customers can more easily free up managed disk space�and make better use of their storage
Migration is accomplished without needing to purchase any additional function on SVC:
VDisk Mirroring and Space-Efficient VDisk functions are included in base virtualization license
Customers with thin-provisioned storage on an existing storage system can migrate their data into SVC management using SEV
without having to allocate additional storage space

Methods of reclaiming "stale" VMWare space

unmap scsi primitive

Discussing with Support Architect and trawling through the Redbooks, I

can confirm that one documented method od reclaiming stale space is the
same ( similar ) to the discussions on page 28 of the PMR i.e.
essentially fill the freed up space with zeros, delete at VMware level,
then and create a thin provisioned copy ( at Storwize ) if space allows.

1) Create a scraficial VMDK thick provisoned eager zeros

2) Delete the scrificial VMDK
i.e. deleted this sacrificial VM from the datastore, which results
free space again - but that's zeroed out.
3) Check how much used capacity you have using vCenter ( this wil
approximate to the size of mirrored thin provisioned volume you'll
need to create.
4) Next ( free space allowing ) create your mirrored thin provisoned
mirrored copy

Note: Volumes on Storwize can be seamlessly converted to thin, thick,

3 of 5 6/21/2022, 6:02 PM
VDisk zero-detection - SVC_Support - itclin19 https://svcsupport.itc.ibm.com/wiki/index.php/VDisk_zero-detection#R...

or compressed. The feature which allows this is called volume

mirroring. While synchronizing the copies, such as a source thin-volume
copy to a target thin-volume copy, the zero detect feature recognizes
zero-based data within the source copy and does not copy it the target.

5) You should now a volume copy that smaller than it's source

6) You should now be able to delete the larger source volume

7) Create a new volume from the now unused space

The IBM RedBook: IBM FlashSystem V9000 and VMware Best Practices Guide

Provides a more deatiled discussion / overview of this topic.

References

https://virtualstoragespeak.wordpress.com/2015/02/09/how-to-reclaim-thin-provisioned-space-on-ibm-flashsystem-v840/

Redbook:- Thin provisioning and managing stale data

Thin-provisioned volumes maximize storage use by only consuming physical storage space as it is needed. Over time and as data is generated,
thin-provisioned volumes grow and the amount of consumed capacity increases. If that data on the thin-provisioned volume is moved or
deleted (for example, if a VMware Storage vMotion operation or a virtual machine being deleted), the space on the thin-provisioned volume
remains fully used. The behavior can also occur as data within virtual machines VMDKs is created and then deleted.

With the current software, zeroing out areas of thin-provisioned volumes

will not free up space that’s already been allocated, and the SCSI

unmap command is not supported.

In “a future release”, the SCSI unmap command will be supported, and so hosts will be able to indicate regions of the vdisk that should be
discarded and that space freed back to the storage pool.

For *thin-provisioned* vdisks, there is no way to de-allocate storage in-place. If the volume only has a single copy, customer can (1)
overwrite unused data with zeroes and then (2) add a second vdisk copy which is thin-provisioned. Any zero data will not have space
allocated on the new

copy. When the sync has completed, the original copy can be deleted,

which will free up its allocated space in the storage pool.

When using thin-provisioned volumes it is important to understand that this behavior exists and for how to address it. FlashSystem V9000
includes a feature called zero detect, which provides clients with the ability to reclaim unused allocated disk space (zeros) by mirroring the
thin-provisioned volume to another volume copy. Follow this procedure to reclaim this capacity: 1. Capacity within the file systems must be
zeroed out so that zero detect can identify free space. In a VMware vSphere environment, there can be multiple file systems. For example, a
Microsoft Windows virtual machine might use the Microsoft New Technology file system (NTFS) within a VMDK, and that VMDK might be
on a VMware VMFS datastore. Either or both of these layers might contain stale data.

Microsoft

Microsoft has released a utility, at no change, that can be used to zero out the Microsoft Windows file system: https://technet.microsoft.com
/en-us/sysinternals/bb897443.aspx The utility is command-line based and can be executed within a Windows virtual machine to zero out any
unused capacity.

Versions of Windows starting with Windows Server 2008 R2 include an automatic reclamation feature that uses SCSI UNMAP. This feature is
not currently supported with FlashSystem V9000.

VMware

VMware vSphere has implemented reclaim functions by integrating with the SCSI-UNMAP

4 of 5 6/21/2022, 6:02 PM
VDisk zero-detection - SVC_Support - itclin19 https://svcsupport.itc.ibm.com/wiki/index.php/VDisk_zero-detection#R...

operation, which is not currently supported by FlashSystem V9000.

Without SCSI-UNMAP, the method for pruning VMFS of any stale data is to ensure that
any free capacity is zeroed out. This can be accomplished by creating a new virtual
machine with an eager zeroed thick VMDK that consumes about 90% of the free space on
the VMFS datastore. Zeros will be written for the entire capacity of the VMDK, which will
clear the VMware VMFS of stale pointers.
For example, a VMFS datastore is 2.00 TB, and it is thought to contain stale data that is
consuming capacity on a FlashSystem V9000 thin-provisioned volume:
– Total Capacity: 2.00 TB
– VMFS Consumed Capacity: 1.5 TB
– VMFS Free Capacity: 500 GB
To prune the stale data a VMDK should be created that consumes 450 GB (90% of the
available 500 GB). After the VMDK is created, the virtual machine can be immediately
deleted.
2. Space must be reclaimed on FlashSystem V9000 volumes.
Volumes on FlashSystem V9000 can be seamlessly converted to thin, thick, or
compressed. The feature which allows this is called volume mirroring. FlashSystem V9000
can maintain two copies of data for a single volume. While synchronizing the copies, such
as a source thin-volume copy to a target thin-volume copy, the zero detect feature
recognizes zero-based data within the source copy and does not copy it the target.

To create a thin-provisioned volume copy, complete the following steps:

a. Add the target thin-provisioned copy by right-clicking an existing volume in
FlashSystem V9000 GUI and selecting Volume Copy Actions → Add Mirrored Copy,
as shown in Figure 6-10.
Figure 6-10 Adding a mirrored copy
b. Select the destination Storage Pool for the new thin-provisioned copy, and click Add
Copy as shown in Figure 6-11.
Figure 6-11 Selecting storage pool for the mirrored copy
c. Wait for synchronization of the copy to complete.
d. Remove the original source volume copy.
The necessity and frequency for using these procedures varies, based on the frequency of
changes which occur in an environment.

Andy Blog
Note, Andrew Martin's blog <linked below> is not sufficient documentation for this.

https://www.ibm.com/developerworks/community/blogs/svcstorwize/entry
/thin_provisioning_what_should_you_think_about_from_an_operational_point_of_view?lang=en

above link no longer works but all developerworks blogs were moved to barrywhytestorage.blog and the three part A9000 tutorial is here:

https://barrywhytestorage.blog/?s=A9000

Retrieved from "https://svcsupport.itc.ibm.com/wiki/index.php?title=VDisk_zero-detection&oldid=92196"

This page was last edited on 18 June 2020, at 23:45.

5 of 5 6/21/2022, 6:02 PM