Space Management
Module 12 Data ONTAP 8.0 7-Mode Administration
Module Objectives
By the end of this module, you should be able to: List the factors that impact space consumption in Data ONTAP Describe how and when a volume consumes space from its containing aggregate Explain how to guarantee writes for a file Discuss how Data ONTAP can provide more space to a full volume State deduplication techniques available in Data ONTAP
2009 NetApp. All rights reserved.
Factors
2009 NetApp. All rights reserved.
Space Management Factors
Administrators have flexibility to manage their storage systems by allocating volumes as: Full provisioned volumes (Space guarantee)
Requires reserving space within the aggregate for the volume at volumes creation Default allocation Cannot over-commit an aggregate Simple storage management
Thin provisioned volumes (Non-space guarantee)
Does not require reserving space within the aggregate for the volume at the volumes creation Allows more aggressive allocation May over-commit an aggregate Complex storage management
2009 NetApp. All rights reserved.
Space Management Factors (Cont.)
Within a volume, whether full or thin provisioned, all files, whether a LUN or a NAS file, may:
Have space reservations (default for LUNs)
Requires reserving space within the volume so the entire file may be overwritten even if blocks are retained by a Snapshot copy Writes to the file will succeed Simpler management
Not have space reservations (default for NAS file)
Does not require reserving space within the volume Writes to the file might fail Active space monitoring required
See the SAN Administration on Data ONTAP course for more information on LUN reservations
2009 NetApp. All rights reserved.
Volume Space Guarantee
2009 NetApp. All rights reserved.
Space Guarantees
Space guarantee is an attribute of a volume that reserved or set aside out of the containing aggregate Space guarantee parameters: volume (default): Reserves the FlexVol total size within its containing aggregate and allows for space reserved or nonspace reserved files file: Allow the creation of a space guaranteed file total size within its containing FlexVol volume none: Does not reserves any space for the FlexVol volume within its containing aggregate and allows only for non-space guaranteed files Space guarantee may be set: At the volumes creation: vol create -s volume vol1 aggr1 5GB On existing volumes: vol options vol1 guarantee none
2009 NetApp. All rights reserved.
System Manager: Space Guarantees
To configure volumes
2009 NetApp. All rights reserved.
Space Guarantee Example
Create a five-disk aggregate:
aggr create aggr1 5 aggr1
df -Ag aggr1
Total AFS (95%) SSR (5%)
85 4
Used
0 0
Avail
85 4
Icon
Description
aggr1
72-GB disk
2009 NetApp. All rights reserved.
Space Guarantee Example (Cont.)
Create a 30 GB volume:
vol create vol1 aggr1 30g aggr1
df -Ag aggr1
Total AFS (95%) SSR (5%)
85 4
Used
30 0
Avail
54 4
df -g vol1
Total Used
0 0
Avail
24 6
AFS (80%)
SSR (20%) Icon
24 6
vol1
Description
aggr1
72-GB disk
Default is volume space guarantee
2009 NetApp. All rights reserved.
Space Guarantee Example (Cont.)
Create a 30 GB vol2:
vol create vol2 -s none aggr1 30g aggr1
df -Ag aggr1
Total AFS (95%) SSR (5%)
85 4
Used
30 0
Avail
54 4
vol2
df -g vol2
Total Used
0 0
Avail
24 6
AFS (80%)
SSR (20%) Icon
24 6
vol1
Description
aggr1
72-GB disk
With none space guarantee
2009 NetApp. All rights reserved.
Space Guarantee Example (Cont.)
Write 24 GB to vol2
Space taken from aggregate when written
aggr1
df -Ag aggr1
Total AFS (95%) SSR (5%)
85 4
Used
54 0
Avail
30 4
vol2
df -g vol2
Total Used
24 0
Avail
0 6
AFS (80%)
SSR (20%) Icon
24 6
vol1
Description
aggr1
72-GB disk
With none space guarantee
2009 NetApp. All rights reserved.
File Types
Files, within volumes, may be either:
Normal: The size of the file represents the amount of data within the file
foo Physical data
Sparse: The size of the file is greater than the amount of data within the file
foo2
2009 NetApp. All rights reserved.
File Type Example
Create a 30 GB vol3:
system> vol create vol3 -s file aggr1 30g
aggr1
df -Ag aggr1
Total AFS (95%) SSR (5%)
85 4
vol3
Used
54 0
Avail
30 4
vol2
df -g vol3
Total Used
0 0
Avail
24 6
AFS (80%)
SSR (20%) Icon
24 6
vol1
Description
aggr1
72-GB disk
With file space guarantee
2009 NetApp. All rights reserved.
File Type Example (Cont.)
Create a 10 GB file:
# dd if=/dev/zero of=foo bs=1000 count=10000000
aggr1
df -Ag aggr1
foo
vol3
Total AFS (95%) SSR (5%)
85 4
Used
64 0
Avail
20 4
vol2
df -g vol3
Total Used
10 0
Avail
14 6
AFS (80%)
SSR (20%) Icon
24 6
vol1
Description
aggr1
72-GB disk
With file space guarantee
2009 NetApp. All rights reserved.
File Type Example (Cont.)
Create a 10 GB sparse file:
# dd if=/dev/zero of=foo2 bs=1 count=0 seek=10000000
aggr1
df -Ag aggr1
foo
vol3
foo2
Total AFS (95%) SSR (5%)
85 4
Used
74 0
Avail
10 4
vol2
df -g vol3
Total Used
10 0
Avail
14 6
AFS (80%)
SSR (20%) Icon
24 6
vol1
Description
aggr1
72-GB disk
With file space guarantee
2009 NetApp. All rights reserved.
Solutions to Full Volumes
2009 NetApp. All rights reserved.
Solutions for Full Volumes
If a volume fills up, because of Snapshot copies, active file system data, or both Then administrators can:
Delete Snapshot copies manually or automatically Expand the volume Delete active file system data if the blocks are not part of a Snapshot copy
2009 NetApp. All rights reserved.
Snapshot Automatic Delete
Snapshot automatic delete determines when (if) Snapshot copies will be automatically deleted
Set at volume level snap autodelete vol[on|off|show|reset]
If autodelete is enabled, then options:
snap autodelete vol options option val Options commitment trigger target_free_space delete_order defer_delete prefix
2009 NetApp. All rights reserved.
Value try, disrupt volume, snap_reserve, space_reserve 1-100 oldest_first, newest_first scheduled, user_created, prefix, none <string>
Volume Autosize
Might want to grow the volume vol autosize determines if a volume should grow when nearly full
Set at volume level Possible values:
ON
Increment size (default 5% of original size) Maximum size (default 120% of original size)
OFF
vol autosize vol_name [-m size[k|m|g|t]] [-i size[k|m|g|t]] [on|off|reset]
2009 NetApp. All rights reserved.
Administrators Choice
Administrators may choose which procedure to employ first:
snapshot auto delete vol autosize Use the volume option:
try_first Possible values:
snap_delete volume_grow (default)
Example:
vol options vol_name try_first snap_delete
2009 NetApp. All rights reserved.
Deduplication
2009 NetApp. All rights reserved.
Deduplication
NetApp Deduplication
NetApp deduplication
20:1 or greater for backup
Integrated with Data ONTAP
General-purpose volume deduplication Identifies and removes redundant data blocks
Application agnostic
Before After Primary storage Backup data Archival data
Service
Runs as a background process and is transparent to any client
2009 NetApp. All rights reserved.
Deduplication in Action
[Link] = Identical blocks Original file 20 blocks [Link] With NetApp deduplication 30 total blocks
Identical file 20 blocks [Link]
Without NetApp deduplication 70 total blocks
Edited file 10 blocks added
2009 NetApp. All rights reserved.
NetApp Deduplication: Internals
Initialization (only necessary on pre-existing volume) Gather
Gatherer File
Gathering
qsort
qsort
...
qsort
Merge Sort Sorting
Fingerprint File
2009 NetApp. All rights reserved.
NetApp Deduplication: Internals (Cont.)
Byte-by-byte comparison Count file update Update inode
Sort by Inode
Update Inode
Duplicate Entry File qsort qsort ... qsort
Block Ref Count File
Merge Sort
Fingerprint File
2009 NetApp. All rights reserved.
NetApp Deduplication: Internals (Cont.)
Block Write
Log New FPs
Change Log File
Change Log File
Fingerprint File
2009 NetApp. All rights reserved.
NetApp Deduplication: Internals (Cont.)
Block Write
Log New FPs
Change Log File
Change Log File
qsort
qsort
...
qsort
Merge Sort
Fingerprint File
2009 NetApp. All rights reserved.
NetApp Deduplication: Internals (Cont.)
Block Write Byte-by-byte comparison Count file update Update inode
Log New FPs
Change Log File
Change Log File
Sort by Inode
Update Inode
Duplicate Entry File qsort qsort ... qsort
Block Ref Count File
Merge Sort
Fingerprint File
2009 NetApp. All rights reserved.
NetApp Deduplication: Internals (Cont.)
Block Write
Log New FPs
Change Log File
Change Log File
Fingerprint File
SIS Check
2009 NetApp. All rights reserved.
NetApp Deduplication: Stages
Initialization (only necessary on pre-existing volume) Gather Block Write Byte-by-byte comparison Count file update Update inode
Log New FPs
Gatherer File
Change Log File
Change Log File
Sort by Inode
Update Inode
Gathering
Duplicate Entry File qsort qsort ... qsort Deduplicating
Block Ref Count File
Merge Sort Sorting
Fingerprint File
SIS Check Checking
2009 NetApp. All rights reserved.
Configuration Overview
License it:
system> license add <license>
Turn it on:
system> sis on <vol>
Deduplicates existing data:
system> sis start -s <vol>
Schedule when to deduplicate or run manually:
system> sis config [-s schedule] <vol> system> sis start <vol>
For maintenance:
system> sis status [-l] <vol> system> sis check
View the space savings:
system> df s <vol>
2009 NetApp. All rights reserved.
Configuring Deduplication
system> sis on /vol/vol1
SIS for "/vol/vol1" is enabled. Already existing data could be processed by running "sis start -s /vol/vol1".
system> sis start -s /vol/vol1
The file system will be scanned to process existing data in /vol/vol1. This operation may initialize related existing metafiles. Are you sure you want to proceed with scan (y/n)? y Fri Nov 10 [Link] EST [[Link]:info]: Starting SIS volume scan on volume vol1. The SIS operation for "/vol/vol1" is started.
2009 NetApp. All rights reserved.
Configuring Deduplication (Cont.)
system> sis status /vol/vol1
Path /vol/vol1 ... Enabled State Active Status 12 GB Progress Scanned
system> sis status /vol/vol1
Path /vol/vol1 State Enabled Status Idle Progress Idle for [Link]
system> df -s /vol/vol1
Filesystem /vol/vol1 used 20568268 saved 3768732 %saved 15%
2009 NetApp. All rights reserved.
sis status Progress and Stages
system> sis status
Path State Enabled Status Progress Active 25 MB Scanned
Gathering
/vol/vol1
Sorting
Path /vol/vol1
State Enabled
Status Progress Active 25 MB Searched
Deduplicating
Path /vol/vol1
State Enabled
Status Progress Active 40MB (20%) done
Path
State Enabled
Status Progress Active 30MB Verified
Checking
/vol/vol1 OR /vol/vol1
Enabled
Active
10% Merged
2009 NetApp. All rights reserved.
Scheduling Deduplication
Default schedule:
system> sis on /vol/vol1 system> sis status /vol/vol1 sun-sat@0
To configure a schedule:
system> system> system> system> sis sis sis sis config config config config -s -s s s - /vol/vol1 23@sun-fri /vol/vol1 auto /vol/vol1 sat@6 /vol/vol1
2009 NetApp. All rights reserved.
Other Commands
vol status command
SIS keyword will be listed in the output for deduplication volumes system> vol status Volume State Vol0 online Vol1 online
Status raid_dp, flex raid_dp, flex
Options root sis
2009 NetApp. All rights reserved.
System Manager: Deduplication
Start the dedupe
Deduplication license was added
2009 NetApp. All rights reserved.
Module Summary
In this module, you should have learned to: List the factors that impact space consumption in Data ONTAP Describe how and when a volume consumes space from its containing aggregate Explain how to guarantee writes for a file Discuss how Data ONTAP can provide more space to a full volume State deduplication techniques available in Data ONTAP
2009 NetApp. All rights reserved.
Exercise
Module 12: Space Management Estimated Time: 45 minutes
Check Your Understanding
What is a fully provisioned volume?
A volume where space is taken from the containing aggregate at the moment of volume creation
What is a thin provisioned volume?
A volume where space is taken from the containing aggregate at the moment space is allocated to files
Why would you use thin provisioned volumes?
Allocating space to users that may use it is not efficient; use to over-allocate an aggregate to maximum space usage
2009 NetApp. All rights reserved.
