Tell us about your PDF experience.

Windows Server Storage documentation

Storage in Windows Server provides new and improved features for software-defined
datacenter (SDDC) customers focusing on virtualized workloads. Windows Server also
provides extensive support for enterprise customers using file servers with existing
workloads. Applies to: Windows Server 2019, Windows Server 2016.

About Windows Server Storage

ｈ WHAT'S NEW

What's new?

Software-defined storage for virtualized workloads

ｅ OVERVIEW

Storage Spaces Direct

Storage Replica

Storage Quality of Service (QoS)

Data Deduplication

General-purpose file servers

ｅ OVERVIEW

Storage Migration Service

Work Folders

Offline Files and Folder Redirection

Roaming User Profiles

DFS Namespaces

DFS Replication

File Server Resource Manager

iSCSI Target Server

iSCSI t tb t
iSCSI target boot

File systems, protocols, etc

ｅ OVERVIEW

SMB over QUIC

ReFS

Server Message Block (SMB) protocol

Storage-class memory

BitLocker Drive Encryption

NTFS

Network File System (NFS)

In Azure

ｅ OVERVIEW

Azure Storage

Azure StorSimple

Azure Shared Disk

Older versions of Windows Server

ｅ OVERVIEW

Windows previous versions documentation

Search for specific information

What's new in Storage in Windows
Server
Article • 03/29/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

This topic explains the new and changed functionality in storage in Windows Server
2019, Windows Server 2016, and Windows Server Semi-Annual Channel releases.

What's new in storage in Windows Server,

version 1903
This release of Windows Server adds the following changes and technologies.

Storage Migration Service now migrates local accounts,

clusters, and Linux servers
Storage Migration Service makes it easier to migrate servers to a newer version of
Windows Server. It provides a graphical tool that inventories data on servers and then
transfers the data and configuration to newer servers—all without apps or users having
to change anything.

When using this version of Windows Server to orchestrate migrations, we've added the
following abilities:

Migrate local users and groups to the new server

Migrate storage from failover clusters
Migrate storage from a Linux server that uses Samba
More easily sync migrated shares into Azure by using Azure File Sync
Migrate to new networks such as Azure

For more info about Storage Migration Service, see Storage Migration Service overview.

System Insights disk anomaly detection

System Insights is a predictive analytics feature that locally analyzes Windows Server
system data and provides insight into the functioning of the server. It comes with a
number of built-in capabilities, but we've added the ability to install additional
capabilities via Windows Admin Center, starting with disk anomaly detection.

Disk anomaly detection is a new capability that highlights when disks are behaving
differently than usual. While different isn't necessarily a bad thing, seeing these
anomalous moments can be helpful when troubleshooting issues on your systems.

This capability is also available for servers running Windows Server 2019.

Windows Admin Center enhancements

A new release of Windows Admin Center is out, adding new functionality to Windows
Server. For info on the latest features, see Windows Admin Center.

What's new in storage in Windows Server 2019

and Windows Server, version 1809
This release of Windows Server adds the following changes and technologies.

Manage storage with Windows Admin Center

Windows Admin Center is a new locally deployed, browser-based app for managing
servers, clusters, hyper-converged infrastructure with Storage Spaces Direct, and
Windows 10 PCs. It comes at no additional cost beyond Windows and is ready for
production use.

To be fair, Windows Admin Center is a separate download that runs on Windows Server
2019 and other versions of Windows, but it's new and we didn't want you to miss it...

Storage Migration Service

Storage Migration Service is a new technology that makes it easier to migrate servers to
a newer version of Windows Server. It provides a graphical tool that inventories data on
servers, transfers the data and configuration to newer servers, and then optionally
moves the identities of the old servers to the new servers so that apps and users don't
have to change anything. For more info, see Storage Migration Service.

Storage Spaces Direct (Windows Server 2019 only)

There are a number of improvements to Storage Spaces Direct in Windows Server 2019
(Storage Spaces Direct isn't included in Windows Server, Semi-Annual Channel):

Deduplication and compression for ReFS volumes

Store up to ten times more data on the same volume with deduplication and
compression for the ReFS filesystem. (It's just one click to turn on with Windows
Admin Center.) The variable-size chunk store with optional compression maximizes
savings rates, while the multi-threaded post-processing architecture keeps
performance impact minimal. Supports volumes up to 64 TB and will deduplicate
the first 4 TB of each file.

Native support for persistent memory

Unlock unprecedented performance with native Storage Spaces Direct support for
persistent memory modules, including Intel® Optane™ DC PM and NVDIMM-N.
Use persistent memory as cache to accelerate the active working set, or as capacity
to guarantee consistent low latency on the order of microseconds. Manage
persistent memory just as you would any other drive in PowerShell or Windows
Admin Center.

Nested resiliency for two-node hyper-converged infrastructure at the edge

Survive two hardware failures at once with an all-new software resiliency option
inspired by RAID 5+1. With nested resiliency, a two-node Storage Spaces Direct
cluster can provide continuously accessible storage for apps and virtual machines
even if one server node goes down and a drive fails in the other server node.

Two-server clusters using a USB flash drive as a witness

Use a low-cost USB flash drive plugged into your router to act as a witness in two-
server clusters. If a server goes down and then back up, the USB drive cluster
knows which server has the most up-to-date data. For more info, see the Storage
at Microsoft blog and documentation on how to deploy a file share witness.

Windows Admin Center

Manage and monitor Storage Spaces Direct with the new purpose-built Dashboard
and experience in Windows Admin Center. Create, open, expand, or delete
volumes with just a few clicks. Monitor performance like IOPS and IO latency from
the overall cluster down to the individual SSD or HDD. Available at no additional
cost for Windows Server 2016 and Windows Server 2019.

Performance history
 Get effortless visibility into resource utilization and performance with built-in
history. Over 50 essential counters spanning compute, memory, network, and
storage are automatically collected and stored on the cluster for up to one year.
Best of all, there's nothing to install, configure, or start – it just works. Visualize in
Windows Admin Center or query and process in PowerShell.

Scale up to 4 PB per cluster

Achieve multi-petabyte scale – great for media, backup, and archival use cases. In
Windows Server 2019, Storage Spaces Direct supports up to 4 petabytes (PB) =
4,000 terabytes of raw capacity per storage pool. Related capacity guidelines are
increased as well: for example, you can create twice as many volumes (64 instead
of 32), each twice as large as before (64 TB instead of 32 TB). Stitch multiple
clusters together into a cluster set for even greater scale within one storage
namespace. For more info, see the Storage at Microsoft blog .

Mirror-accelerated parity is 2X faster

With mirror-accelerated parity you can create Storage Spaces Direct volumes that
are part mirror and part parity, like mixing RAID-1 and RAID-5/6 to get the best of
both. (It's easier than you think in Windows Admin Center.) In Windows Server
2019, the performance of mirror-accelerated parity is more than doubled relative
to Windows Server 2016 thanks to optimizations.

Drive latency outlier detection

Easily identify drives with abnormal latency with proactive monitoring and built-in
outlier detection, inspired by Microsoft Azure's long-standing and successful
approach. Whether it's average latency or something more subtle like 99th
percentile latency that stands out, slow drives are automatically labeled in
PowerShell and Windows Admin Center with ‘Abnormal Latency' status.

Manually delimit the allocation of volumes to increase fault tolerance

This enables admins to manually delimit the allocation of volumes in Storage

Spaces Direct. Doing so can significantly increase fault tolerance under certain
conditions, but imposes some added management considerations and complexity.
For more info, see Delimit the allocation of volumes.

Storage Replica
There are a number of improvements to Storage Replica in this release:
Storage Replica in Windows Server, Standard Edition
You can now use Storage Replica with Windows Server, Standard Edition in addition to
Datacenter Edition. Storage Replica running on Windows Server, Standard Edition, has
the following limitations:

Storage Replica replicates a single volume instead of an unlimited number of

volumes.
Volumes can have a size of up to 2 TB instead of an unlimited size.

Storage Replica log performance improvements

We also made improvements to how the Storage Replica log tracks replication,
improving replication throughput and latency, especially on all-flash storage as well as
Storage Spaces Direct clusters that replicate between each other.

To gain the increased performance, all members of the replication group must run
Windows Server 2019.

Test failover

You can now temporarily mount a snapshot of the replicated storage on a destination
server for testing or backup purposes. For more information, see Frequently Asked
Questions about Storage Replica.

Windows Admin Center support

Support for graphical management of replication is now available in Windows Admin
Center via the Server Manager tool. This includes server-to-server replication, cluster-to-
cluster, as well as stretch cluster replication.

Miscellaneous improvements
Storage Replica also contains the following improvements:

Alters asynchronous stretch cluster behaviors so that automatic failovers now

occur
Multiple bug fixes

SMB
 SMB1 and guest authentication removal: Windows Server no longer installs the
SMB1 client and server by default. Additionally, the ability to authenticate as a
guest in SMB2 and later is off by default. For more information, review SMBv1 is
not installed by default in Windows 10, version 1709 and Windows Server, version
1709 .

SMB2/SMB3 security and compatibility: Additional options for security and

application compatibility were added, including the ability to disable oplocks in
SMB2+ for legacy applications, as well as require signing or encryption on per-
connection basis from a client. For more information, review the SMBShare
PowerShell module help.

Data Deduplication
Data Deduplication now supports ReFS: You no longer must choose between the
advantages of a modern file system with ReFS and the Data Deduplication: now,
you can enable Data Deduplication wherever you can enable ReFS. Increase
storage efficiency by upwards of 95% with ReFS.
DataPort API for optimized ingress/egress to deduplicated volumes: Developers
can now take advantage of the knowledge Data Deduplication has about how to
store data efficiently to move data between volumes, servers, and clusters
efficiently.

File Server Resource Manager

Windows Server 2019 includes the ability to prevent the File Server Resource Manager
service from creating a change journal (also known as a USN journal) on all volumes
when the service starts. This can conserve space on each volume, but will disable real-
time file classification. For more information, see File Server Resource Manager overview.

What's new in storage in Windows Server,

version 1803

File Server Resource Manager

Windows Server, version 1803 includes the ability to prevent the File Server Resource
Manager service from creating a change journal (also known as a USN journal) on all
volumes when the service starts. This can conserve space on each volume, but will
disable real-time file classification. For more information, see File Server Resource
Manager overview.

What's new in storage in Windows Server,

version 1709
Windows Server, version 1709 is the first Windows Server release in the Semi-Annual
Channel. The Semi-Annual Channel is a Software Assurance benefit and is fully
supported in production for 18 months, with a new version every six months.

For more information, see Windows Server Semi-annual Channel Overview.

Storage Replica
The disaster recovery protection added by Storage Replica is now expanded to include:

Test failover: the option to mount the destination storage is now possible through
the test failover feature. You can mount a snapshot of the replicated storage on
destination nodes temporarily for testing or backup purposes. For more
information, see Frequently Asked Questions about Storage Replica.
Windows Admin Center support: Support for graphical management of
replication is now available in Windows Admin Center via the Server Manager tool.
This includes server-to-server replication, cluster-to-cluster, as well as stretch
cluster replication.

Storage Replica also contains the following improvements:

Alters asynchronous stretch cluster behaviors so that automatic failovers now

occur
Multiple bug fixes

SMB
SMB1 and guest authentication removal: Windows Server, version 1709 no longer
installs the SMB1 client and server by default. Additionally, the ability to
authenticate as a guest in SMB2 and later is off by default. For more information,
review SMBv1 is not installed by default in Windows 10, version 1709 and Windows
Server, version 1709 .

SMB2/SMB3 security and compatibility: Additional options for security and

application compatibility were added, including the ability to disable oplocks in
 SMB2+ for legacy applications, as well as require signing or encryption on per-
connection basis from a client. For more information, review the SMBShare
PowerShell module help.

Data Deduplication
Data Deduplication now supports ReFS: You no longer must choose between the
advantages of a modern file system with ReFS and the Data Deduplication: now,
you can enable Data Deduplication wherever you can enable ReFS. Increase
storage efficiency by upwards of 95% with ReFS.
DataPort API for optimized ingress/egress to deduplicated volumes: Developers
can now take advantage of the knowledge Data Deduplication has about how to
store data efficiently to move data between volumes, servers, and clusters
efficiently.

What's new in storage in Windows Server 2016

Storage Spaces Direct

Storage Spaces Direct enables building highly available and scalable storage using
servers with local storage. It simplifies the deployment and management of software-
defined storage systems and unlocks use of new classes of disk devices, such as SATA
SSD and NVMe disk devices, that were previously not possible with clustered Storage
Spaces with shared disks.

What value does this change add? Storage Spaces Direct enables service providers and
enterprises to use industry standard servers with local storage to build highly available
and scalable software defined storage. Using servers with local storage decreases
complexity, increases scalability, and enables use of storage devices that were not
previously possible, such as SATA solid state disks to lower cost of flash storage, or
NVMe solid state disks for better performance.

Storage Spaces Direct removes the need for a shared SAS fabric, simplifying deployment
and configuration. Instead it uses the network as a storage fabric, leveraging SMB3 and
SMB Direct (RDMA) for high-speed, low-latency CPU efficient storage. To scale out,
simply add more servers to increase storage capacity and I/O performance For more
information, see the Storage Spaces Direct.

What works differently? This capability is new in Windows Server 2016.

Storage Replica
Storage Replica enables storage-agnostic, block-level, synchronous replication between
servers or clusters for disaster recovery, as well as stretching of a failover cluster
between sites. Synchronous replication enables mirroring of data in physical sites with
crash-consistent volumes to ensure zero data loss at the file-system level. Asynchronous
replication allows site extension beyond metropolitan ranges with the possibility of data
loss.

What value does this change add? Storage Replication enables you to do the following:

Provide a single vendor disaster recovery solution for planned and unplanned
outages of mission critical workloads.
Use SMB3 transport with proven reliability, scalability, and performance.
Stretch Windows failover clusters to metropolitan distances.
Use Microsoft software end to end for storage and clustering, such as Hyper-V,
Storage Replica, Storage Spaces, Cluster, Scale-Out File Server, SMB3,
Deduplication, and ReFS/NTFS.
Help reduce cost and complexity as follows:
Is hardware agnostic, with no requirement for a specific storage configuration
like DAS or SAN.
Allows commodity storage and networking technologies.
Features ease of graphical management for individual nodes and clusters
through Failover Cluster Manager.
Includes comprehensive, large-scale scripting options through Windows
PowerShell.
Help reduce downtime, and increase reliability and productivity intrinsic to
Windows.
Provide supportability, performance metrics, and diagnostic capabilities.

For more information, see the Storage Replica in Windows Server 2016.

What works differently? This capability is new in Windows Server 2016.

Storage Quality of Service

You can now use storage quality of service (QoS) to centrally monitor end-to-end
storage performance and create management policies using Hyper-V and CSV clusters
in Windows Server 2016.

What value does this change add? You can now create storage QoS policies on a CSV
cluster and assign them to one or more virtual disks on Hyper-V virtual machines.
Storage performance is automatically readjusted to meet policies as the workloads and
storage loads fluctuate.

Each policy can specify a reserve (minimum) and/or a limit (maximum) to be

applied to a collection of data flows, such as a virtual hard disk, a single virtual
machine or a group of virtual machines, a service, or a tenant.
Using Windows PowerShell or WMI, you can perform the following tasks:
Create policies on a CSV cluster.
Enumerate policies available on a CSV cluster.
Assign a policy to a virtual hard disk of a Hyper-V virtual machine.
Monitor the performance of each flow and status within the policy.
If multiple virtual hard disks share the same policy, performance is fairly distributed
to meet demand within the policy's minimum and maximum settings. Therefore, a
policy can be used to manage a virtual hard disk, a virtual machine, multiple virtual
machines comprising a service, or all virtual machines owned by a tenant.

What works differently? This capability is new in Windows Server 2016. Managing
minimum reserves, monitoring flows of all virtual disks across the cluster via a single
command, and centralized policy based management were not possible in previous
releases of Windows Server.

For more information, see Storage Quality of Service

Data Deduplication

Functionality New or Description

Updated

Support for Updated Prior to Windows Server 2016, volumes had to specifically sized for
Large the expected churn, with volume sizes above 10 TB not being good
Volumes candidates for deduplication. In Windows Server 2016, Data
Deduplication supports volume sizes up to 64 TB.

Support for Updated Prior to Windows Server 2016, files approaching 1 TB in size were not
Large Files good candidates for deduplication. In Windows Server 2016, files up
to 1 TB are fully supported.

Support for New Data Deduplication is available and fully supported in the new Nano
Nano Server Server deployment option for Windows Server 2016.
 Functionality New or Description
Updated

Simplified New In Windows Server 2012 R2, Virtualized Backup Applications, such as
Backup Microsoft's Data Protection Manager, were supported through a
Support series of manual configuration steps. In Windows Server 2016, a new
default Usage Type "Backup", has been added for seamless
deployment of Data Deduplication for Virtualized Backup
Applications.

Support for New Data Deduplication fully supports the new Cluster OS Rolling
Cluster OS Upgrade feature of Windows Server 2016.
Rolling
Upgrades

SMB hardening improvements for SYSVOL and

NETLOGON connections
In Windows 10 and Windows Server 2016 client connections to the Active Directory
Domain Services default SYSVOL and NETLOGON shares on domain controllers now
require SMB signing and mutual authentication (such as Kerberos).

What value does this change add? This change reduces the likelihood of man-in-the-
middle attacks.

What works differently? If SMB signing and mutual authentication are unavailable, a
Windows 10 or Windows Server 2016 computer won't process domain-based Group
Policy and scripts.

７ Note

The registry values for these settings aren't present by default, but the hardening
rules still apply until overridden by Group Policy or other registry values.

For more information on these security improvements - also referred to as UNC

hardening, see Microsoft Knowledge Base article 3000483 and MS15-011 & MS15-
014: Hardening Group Policy .

Work Folders
Improved change notification when the Work Folders server is running Windows Server
2016 and the Work Folders client is Windows 10.
What value does this change add?
For Windows Server 2012 R2, when file changes are synced to the Work Folders server,
clients are not notified of the change and wait up to 10 minutes to get the update.
When using Windows Server 2016, the Work Folders server immediately notifies
Windows 10 clients and the file changes are synced immediately.

What works differently?

This capability is new in Windows Server 2016. This requires a Windows Server 2016
Work Folders server and the client must be Windows 10.

If you're using an older client or the Work Folders server is Windows Server 2012 R2, the
client will continue to poll every 10 minutes for changes.

ReFS
The next iteration of ReFS provides support for large-scale storage deployments with
diverse workloads, delivering reliability, resiliency, and scalability for your data.

What value does this change add?

ReFS introduces the following improvements:

ReFS implements new storage tiers functionality, helping deliver faster

performance and increased storage capacity. This new functionality enables:
Multiple resiliency types on the same virtual disk (using mirroring in the
performance tier and parity in the capacity tier, for example).
Increased responsiveness to drifting working sets.
The introduction of block cloning substantially improves the performance of VM
operations, such as .vhdx checkpoint merge operations.
The new ReFS scan tool enables the recovery of leaked storage and helps salvage
data from critical corruptions.

What works differently?

These capabilities are new in Windows Server 2016.

Additional References
What's New in Windows Server 2016
Data Deduplication Overview
Article • 02/18/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Azure Stack HCI, versions 21H2 and 20H2

What is Data Deduplication?

Data Deduplication, often called Dedup for short, is a feature that can help reduce the
impact of redundant data on storage costs. When enabled, Data Deduplication
optimizes free space on a volume by examining the data on the volume by looking for
duplicated portions on the volume. Duplicated portions of the volume's dataset are
stored once and are (optionally) compressed for additional savings. Data Deduplication
optimizes redundancies without compromising data fidelity or integrity. More
information about how Data Deduplication works can be found in the 'How does Data
Deduplication work?' section of the Understanding Data Deduplication page.

） Important

KB4025334 contains a roll up of fixes for Data Deduplication, including

important reliability fixes, and we strongly recommend installing it when using Data
Deduplication with Windows Server 2016 and Windows Server 2019.

Why is Data Deduplication useful?

Data Deduplication helps storage administrators reduce costs that are associated with
duplicated data. Large datasets often have a lot of duplication, which increases the costs
of storing the data. For example:

User file shares may have many copies of the same or similar files.
Virtualization guests might be almost identical from VM-to-VM.
Backup snapshots might have minor differences from day to day.

The space savings that you can gain from Data Deduplication depend on the dataset or
workload on the volume. Datasets that have high duplication could see optimization
rates of up to 95%, or a 20x reduction in storage utilization. The following table
highlights typical deduplication savings for various content types:

Scenario Content Typical space savings

Scenario Content Typical space savings

User documents Office documents, photos, music, videos, etc. 30-50%

Deployment shares Software binaries, cab files, symbols, etc. 70-80%

Virtualization libraries ISOs, virtual hard disk files, etc. 80-95%

General file share All the above 50-60%

７ Note

If you're just looking to free up space on a volume, consider using Azure File Sync
with cloud tiering enabled. This allows you to cache your most frequently accessed
files locally and tier your least frequently accessed files to the cloud, saving local
storage space while maintaining performance. For details, see Planning for an
Azure File Sync deployment.

When can Data Deduplication be used?

Scenario Description
illustration

General purpose file servers: General purpose file servers are general use file
servers that might contain any of the following types of shares:

Team shares
User home folders
Work folders
Software development shares

General purpose file servers are a good candidate for Data Deduplication because
multiple users tend to have many copies or versions of the same file. Software
development shares benefit from Data Deduplication because many binaries remain
essentially unchanged from build to build.
Scenario Description
illustration

Virtual Desktop Infrastructure (VDI) deployments: VDI servers, such as Remote

Desktop Services, provide a lightweight option for organizations to provision
desktops to users. There are many reasons for an organization to rely on such
technology:

Application deployment: You can quickly deploy applications across your

enterprise. This is especially useful when you have applications that are
frequently updated, infrequently used, or difficult to manage.
Application consolidation: When you install and run applications from a set
of centrally managed virtual machines, you eliminate the need to update
applications on client computers. This option also reduces the amount of
network bandwidth that is required to access applications.
Remote Access: Users can access enterprise applications from devices such as
home computers, kiosks, low-powered hardware, and operating systems
other than Windows.
Branch office access: VDI deployments can provide better application
performance for branch office workers who need access to centralized data
stores. Data-intensive applications sometimes do not have client/server
protocols that are optimized for low-speed connections.

VDI deployments are great candidates for Data Deduplication because the virtual
hard disks that drive the remote desktops for users are essentially identical.
Additionally, Data Deduplication can help with the so-called VDI boot storm, which
is the drop in storage performance when many users simultaneously sign in to their
desktops to start the day.

Backup targets, such as virtualized backup applications: Backup applications, such

as Microsoft Data Protection Manager (DPM), are excellent candidates for Data
Deduplication because of the significant duplication between backup snapshots.

Other workloads: Other workloads may also be excellent candidates for Data
Deduplication.
What's New in Data Deduplication
Article • 02/18/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Azure Stack HCI, versions 21H2 and 20H2

Data Deduplication in Windows Server has been optimized to be highly performant,

flexible, and manageable at private cloud scale. For more information about the
software-defined storage stack in Windows Server, please see What's New in Storage in
Windows Server.

Windows Server 2022

Data Deduplication has no extra enhancements in Windows Server 2022.

Windows Server 2019

Data Deduplication has the following enhancements in Windows Server 2019:

Functionality New or Description

updated

ReFS support New Store up to 10X more data on the same volume with deduplication
and compression for the ReFS filesystem. (It's just one click to turn
on with Windows Admin Center.) The variable-size chunk store with
optional compression maximizes savings rates, while the multi-
threaded post-processing architecture keeps performance impact
minimal. Supports volumes up to 64 TB and will deduplicate the first 4
TB of each file.

Windows Server 2016

Data Deduplication has the following enhancements starting in Windows Server 2016:

Functionality New or Description

updated

Support for Updated Prior to Windows Server 2016, volumes had to be specifically sized for
large the expected churn, with volume sizes above 10 TB not being good
volumes candidates for deduplication. In Windows Server 2016, Data
Deduplication supports volume sizes up to 64 TB.
 Functionality New or Description
updated

Support for Updated Prior to Windows Server 2016, files approaching 1 TB in size were not
large files) good candidates for deduplication. In Windows Server 2016, files up
to 1 TB are fully supported.

Support for New Data Deduplication is available and fully supported in the new Nano
Nano Server Server deployment option for Windows Server 2016.

Simplified New Windows Server 2012 R2 supported Virtualized Backup Applications,

backup such as Microsoft's Data Protection Manager, through a series of
support manual configuration steps. Windows Server 2016 has added a new
default Usage Type (Backup) for seamless deployment of Data
Deduplication for Virtualized Backup Applications.

Support for New Data Deduplication fully supports the new Cluster OS Rolling Upgrade
Cluster OS feature of Windows Server 2016.
Rolling
Upgrade

Support for large volumes

What value does this change add?
To get the best performance out of Data Deduplication in Windows Server 2012 R2,
volumes must be sized properly to ensure that the Optimization job can keep up with
the rate of data changes, or churn. Typically, this means that Data Deduplication is only
performant on volumes of 10 TB or less, depending on the workload's write patterns.

Starting with Windows Server 2016, Data Deduplication is highly performant on volumes
up to 64 TB.

What works differently?

In Windows Server 2012 R2, the Data Deduplication Job Pipeline uses a single-thread
and I/O queue for each volume. To ensure that the Optimization jobs do not fall behind,
which would cause the overall savings rate for the volume to decrease, large datasets
must be broken up into smaller volumes. The appropriate volume size depends on the
expected churn for that volume. On average, the maximum is ~6-7 TB for high churn
volumes and ~9-10 TB for low churn volumes.

Starting with Windows Server 2016, the Data Deduplication Job pipeline has been
redesigned to run multiple threads in parallel using multiple I/O queues for each
volume. This results in performance that was previously only possible by dividing up
data into multiple smaller volumes. This change is represented in the following image:
These optimizations apply to all Data Deduplication Jobs, not just the Optimization Job.

Support for large files

What value does this change add?
In Windows Server 2012 R2, very large files are not good candidates for Data
Deduplication due to decreased performance of the Deduplication Processing Pipeline.
In Windows Server 2016, deduplication of files up to 1 TB is very performant, enabling
administrators to apply deduplication savings to a larger range of workloads. For
example, you can deduplicate very large files normally associated with backup
workloads.

What works differently?

Starting with Windows Server 2016, Data Deduplication makes use of new stream map
structures and other "under- the hood" improvements to increase optimization
throughput and access performance. Additionally, the Deduplication Processing Pipeline
can now resume optimization after a failover rather than restarting. These changes make
deduplication on files up to 1 TB highly performant.

Support for Nano Server

What value does this change add?
Nano Server is a new headless deployment option in Windows Server 2016 that requires
a far smaller system resource footprint, starts up significantly faster, and requires fewer
updates and restarts than the Windows Server Core deployment option. Data
Deduplication is fully supported on Nano Server. For more information about Nano
Server, see Getting Started with Nano Server.

Simplified configuration for Virtualized Backup

Applications
What value does this change add?
Data Deduplication for Virtualized Backup Applications is a supported scenario in
Windows Server 2012 R2, but it requires manually tuning of the deduplication settings.
Starting with Windows Server 2016, the configuration of Deduplication for Virtualized
Backup Applications is drastically simplified. It uses a predefined Usage Type option
when enabling Deduplication for a volume, just like our options for General Purpose File
Server and VDI.

Support for Cluster OS Rolling Upgrade

What value does this change add?
Windows Server Failover Clusters running Data Deduplication can have a mix of nodes
running Windows Server 2012 R2 versions of Data Deduplication alongside nodes
running Windows Server 2016 versions of Data Deduplication. This enhancement
provides full data access to all deduplicated volumes during a cluster rolling upgrade,
allowing for the gradual rollout of the new version of Data Deduplication on an existing
Windows Server 2012 R2 cluster without incurring downtime to upgrade all nodes at
once.

What works differently?

With previous versions of Windows Server, a Windows Server Failover Cluster required
all nodes in the cluster to have the same Windows Server version. Starting with the
Windows Server 2016, the cluster rolling upgrade functionality allows a cluster to run in
a mixed-mode. Data Deduplication supports this new mixed-mode cluster configuration
to enable full data access during a cluster rolling upgrade.
Understanding Data Deduplication
Article • 02/18/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Azure Stack HCI, versions 21H2 and 20H2

This document describes how Data Deduplication works.

How does Data Deduplication work?

Data Deduplication in Windows Server was created with the following two principles:

1. Optimization should not get in the way of writes to the disk Data Deduplication
optimizes data by using a post-processing model. All data is written unoptimized
to the disk and then optimized later by Data Deduplication.

2. Optimization should not change access semantics Users and applications that
access data on an optimized volume are completely unaware that the files they are
accessing have been deduplicated.

Once enabled for a volume, Data Deduplication runs in the background to:

Identify repeated patterns across files on that volume.

Seamlessly move those portions, or chunks, with special pointers called reparse
points that point to a unique copy of that chunk.

This occurs in the following four steps:

1. Scan the file system for files meeting the optimization policy.
2. Break files into variable-size chunks.

3. Identify unique chunks.

4. Place chunks in the chunk store and optionally compress.

5. Replace the original file stream of now optimized files with a reparse point to the
chunk store.
When optimized files are read, the file system sends the files with a reparse point to the
Data Deduplication file system filter (Dedup.sys). The filter redirects the read operation
to the appropriate chunks that constitute the stream for that file in the chunk store.
Modifications to ranges of a deduplicated files get written unoptimized to the disk and
are optimized by the Optimization job the next time it runs.

Usage Types
The following Usage Types provide reasonable Data Deduplication configuration for
common workloads:

Usage Ideal workloads What's different

Type

Default General purpose file server: Background optimization

Team shares Default optimization policy:
Work Folders Minimum file age = 3 days
Folder redirection Optimize in-use files = No
Software development shares Optimize partial files = No

Hyper- Virtualized Desktop Infrastructure (VDI) Background optimization

V servers Default optimization policy:
Minimum file age = 3 days
Optimize in-use files = Yes
Optimize partial files = Yes
"Under-the-hood" tweaks for
Hyper-V interop
 Usage Ideal workloads What's different
Type

Backup Virtualized backup applications, such as Priority optimization

Microsoft Data Protection Manager (DPM) Default optimization policy:
Minimum file age = 0 days
Optimize in-use files = Yes
Optimize partial files = No
"Under-the-hood" tweaks for
interop with DPM/DPM-like
solutions

Jobs
Data Deduplication uses a post-processing strategy to optimize and maintain a volume's
space efficiency.

Job name Job descriptions Default

schedule

Optimization The Optimization job deduplicates by chunking data on a volume Once

per the volume policy settings, (optionally) compressing those every
chunks, and storing chunks uniquely in the chunk store. The hour
optimization process that Data Deduplication uses is described in
detail in How does Data Deduplication work?.

Garbage The Garbage Collection job reclaims disk space by removing Every
Collection unnecessary chunks that are no longer being referenced by files Saturday
that have been recently modified or deleted. at 2:35
AM

Integrity The Integrity Scrubbing job identifies corruption in the chunk store Every
Scrubbing due to disk failures or bad sectors. When possible, Data Saturday
Deduplication can automatically use volume features (such as at 3:35
mirror or parity on a Storage Spaces volume) to reconstruct the AM
corrupted data. Additionally, Data Deduplication keeps backup
copies of popular chunks when they are referenced more than 100
times in an area called the hotspot.

Unoptimization The Unoptimization job, which is a special job that should only be On-
run manually, undoes the optimization done by deduplication and demand
disables Data Deduplication for that volume. only

Data Deduplication terminology

Term Definition

Chunk A chunk is a section of a file that has been selected by the Data Deduplication
chunking algorithm as likely to occur in other, similar files.

Chunk store The chunk store is an organized series of container files in the System Volume
Information folder that Data Deduplication uses to uniquely store chunks.

Dedup An abbreviation for Data Deduplication that's commonly used in PowerShell,

Windows Server APIs and components, and the Windows Server community.

File Every file contains metadata that describes interesting properties about the file
metadata that are not related to the main content of the file. For instance, Date Created, Last
Read Date, Author, etc.

File stream The file stream is the main content of the file. This is the part of the file that Data
Deduplication optimizes.

File system The file system is the software and on-disk data structure that the operating
system uses to store files on storage media. Data Deduplication is supported on
NTFS formatted volumes.

File system A file system filter is a plugin that modifies the default behavior of the file system.
filter To preserve access semantics, Data Deduplication uses a file system filter
(Dedup.sys) to redirect reads to optimized content completely transparently to the
user or application that makes the read request.

Optimization A file is considered optimized (or deduplicated) by Data Deduplication if it has

been chunked, and its unique chunks have been stored in the chunk store.

Optimization The optimization policy specifies the files that should be considered for Data
policy Deduplication. For example, files may be considered out-of-policy if they are
brand new, open, in a certain path on the volume, or a certain file type.

Reparse A reparse point is a special tag that notifies the file system to pass off I/O to a
point specified file system filter. When a file's file stream has been optimized, Data
Deduplication replaces the file stream with a reparse point, which enables Data
Deduplication to preserve the access semantics for that file.

Volume A volume is a Windows construct for a logical storage drive that may span
multiple physical storage devices across a one or more servers. Deduplication is
enabled on a volume-by-volume basis.

Workload A workload is an application that runs on Windows Server. Example workloads

include general purpose file server, Hyper-V, and SQL Server.

２ Warning

Unless instructed by authorized Microsoft Support Personnel, do not attempt to

manually modify the chunk store. Doing so may result in data corruption or loss.
Frequently asked questions
How does Data Deduplication differ from other optimization products? There are
several important differences between Data Deduplication and other common storage
optimization products:

How does Data Deduplication differ from Single Instance Store? Single Instance
Store, or SIS, is a technology that preceded Data Deduplication and was first
introduced in Windows Storage Server 2008 R2. To optimize a volume, Single
Instance Store identified files that were completely identical and replaced them
with logical links to a single copy of a file that's stored in the SIS common store.
Unlike Single Instance Store, Data Deduplication can get space savings from files
that are not identical but share many common patterns and from files that
themselves contain many repeated patterns. Single Instance Store was deprecated
in Windows Server 2012 R2 and removed in Windows Server 2016 in favor of Data
Deduplication.

How does Data Deduplication differ from NTFS compression? NTFS compression is a
feature of NTFS that you can optionally enable at the volume level. With NTFS
compression, each file is optimized individually via compression at write-time.
Unlike NTFS compression, Data Deduplication can get spacing savings across all
the files on a volume. This is better than NTFS compression because files may have
both internal duplication (which is addressed by NTFS compression) and have
similarities with other files on the volume (which is not addressed by NTFS
compression). Additionally, Data Deduplication has a post-processing model,
which means that new or modified files will be written to disk unoptimized and will
be optimized later by Data Deduplication.

How does Data Deduplication differ from archive file formats like zip, rar, 7z, cab,
etc.? Archive file formats, like zip, rar, 7z, cab, etc., perform compression over a
specified set of files. Like Data Deduplication, duplicated patterns within files and
duplicated patterns across files are optimized. However, you have to choose the
files that you want to include in the archive. Access semantics are different, too. To
access a specific file within the archive, you have to open the archive, select a
specific file, and decompress that file for use. Data Deduplication operates
transparently to users and administrators and requires no manual kick-off.
Additionally, Data Deduplication preserves access semantics: optimized files
appear unchanged after optimization.
Can I change the Data Deduplication settings for my selected Usage Type? Yes.
Although Data Deduplication provides reasonable defaults for Recommended
workloads, you might still want to tweak Data Deduplication settings to get the most
out of your storage. Additionally, other workloads will require some tweaking to ensure
that Data Deduplication does not interfere with the workload.

Can I manually run a Data Deduplication job? Yes, all Data Deduplication jobs may be
run manually. This may be desirable if scheduled jobs did not run due to insufficient
system resources or because of an error. Additionally, the Unoptimization job can only
be run manually.

Can I monitor the historical outcomes of Data Deduplication jobs? Yes, all Data
Deduplication jobs make entries in the Windows Event Log.

Can I change the default schedules for the Data Deduplication jobs on my system?
Yes, all schedules are configurable. Modifying the default Data Deduplication schedules
is particularly desirable to ensure that the Data Deduplication jobs have time to finish
and do not compete for resources with the workload.
Install and enable Data Deduplication
Article • 07/05/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Azure Stack HCI, versions 21H2 and 20H2

This topic explains how to install Data Deduplication, evaluate workloads for
deduplication, and enable Data Deduplication on specific volumes.

７ Note

If you're planning to run Data Deduplication in a Failover Cluster, every node in the
cluster must have the Data Deduplication server role installed.

Install Data Deduplication

） Important

KB4025334 contains a roll up of fixes for Data Deduplication, including

important reliability fixes, and we strongly recommend installing it when using Data
Deduplication with Windows Server 2016.

Install Data Deduplication by using Server Manager

1. In the Add Roles and Feature wizard, select Server Roles, and then select Data
Deduplication.
 2. Click Next until the Install button is active, and then click Install.

Install Data Deduplication by using PowerShell

To install Data Deduplication, run the following PowerShell command as an
administrator: Install-WindowsFeature -Name FS-Data-Deduplication

To install Data Deduplication:

 From a server running Windows Server 2016 or later, or from a Windows PC with
the Remote Server Administration Tools (RSAT) installed, install Data
Deduplication with an explicit reference to the server name (replace 'MyServer'
with the real name of the server instance):

PowerShell

Install-WindowsFeature -ComputerName <MyServer> -Name FS-Data-

Deduplication

Or

Connect remotely to the server instance with PowerShell remoting and install Data
Deduplication by using DISM:

PowerShell

Enter-PSSession -ComputerName MyServer

dism /online /enable-feature /featurename:dedup-core /all

Enable Data Deduplication

Determine which workloads are candidates for Data

Deduplication
Data Deduplication can effectively minimize the costs of a server application's data
consumption by reducing the amount of disk space consumed by redundant data.
Before enabling deduplication, it is important that you understand the characteristics of
your workload to ensure that you get the maximum performance out of your storage.
There are two classes of workloads to consider:

Recommended workloads that have been proven to have both datasets that benefit
highly from deduplication and have resource consumption patterns that are
compatible with Data Deduplication's post-processing model. We recommend that
you always enable Data Deduplication on these workloads:
General purpose file servers (GPFS) serving shares such as team shares, user
home folders, work folders, and software development shares.
Virtualized desktop infrastructure (VDI) servers.
Virtualized backup applications, such as Microsoft Data Protection Manager
(DPM).
 Workloads that might benefit from deduplication, but aren't always good
candidates for deduplication. For example, the following workloads could work
well with deduplication, but you should evaluate the benefits of deduplication first:
General purpose Hyper-V hosts
SQL servers
Line-of-business (LOB) servers

Evaluate workloads for Data Deduplication

） Important

If you are running a recommended workload, you can skip this section and go to
Enable Data Deduplication for your workload.

To determine whether a workload works well with deduplication, answer the following
questions. If you're unsure about a workload, consider doing a pilot deployment of Data
Deduplication on a test dataset for your workload to see how it performs.

1. Does my workload's dataset have enough duplication to benefit from enabling

deduplication? Before enabling Data Deduplication for a workload, investigate
how much duplication your workload's dataset has by using the Data
Deduplication Savings Evaluation tool, or DDPEval. After installing Data
Deduplication, you can find this tool at C:\Windows\System32\DDPEval.exe . DDPEval
can evaluate the potential for optimization against directly connected volumes
(including local drives or Cluster Shared Volumes) and mapped or unmapped
network shares.

Running DDPEval.exe will return an output similar to the following:

Data Deduplication Savings Evaluation Tool

Copyright 2011-2012 Microsoft Corporation. All Rights Reserved.

Evaluated folder: E:\Test

Processed files: 34
Processed files size: 12.03MB
Optimized files size: 4.02MB
Space savings: 8.01MB
Space savings percent: 66
Optimized files size (no compression): 11.47MB
Space savings (no compression): 571.53KB
Space savings percent (no compression): 4
Files with duplication: 2
 Files excluded by policy: 20
Files excluded by error: 0

2. What do my workload's I/O patterns to its dataset look like? What performance
do I have for my workload? Data Deduplication optimizes files as a periodic job,
rather than when the file is written to disk. As a result, it is important to examine is
a workload's expected read patterns to the deduplicated volume. Because Data
Deduplication moves file content into the Chunk Store and attempts to organize
the Chunk Store by file as much as possible, read operations perform best when
they are applied to sequential ranges of a file.

Database-like workloads typically have more random read patterns than sequential
read patterns because databases do not typically guarantee that the database
layout will be optimal for all possible queries that may be run. Because the sections
of the Chunk Store may exist all over the volume, accessing data ranges in the
Chunk Store for database queries may introduce additional latency. High
performance workloads are particularly sensitive to this extra latency, but other
database-like workloads might not be.

７ Note

These concerns primarily apply to storage workloads on volumes made up of

traditional rotational storage media (also known as Hard Disk drives, or
HDDs). All-flash storage infrastructure (also known as Solid State Disk drives,
or SSDs), is less affected by random I/O patterns because one of the
properties of flash media is equal access time to all locations on the media.
Therefore, deduplication will not introduce the same amount of latency for
reads to a workload's datasets stored on all-flash media as it would on
traditional rotational storage media.

3. What are the resource requirements of my workload on the server? Because Data
Deduplication uses a post-processing model, Data Deduplication periodically
needs to have sufficient system resources to complete its optimization and other
jobs. This means that workloads that have idle time, such as in the evening or on
weekends, are excellent candidates for deduplication, and workloads that run all
day, every day may not be. Workloads that have no idle time may still be good
candidates for deduplication if the workload does not have high resource
requirements on the server.

Enable Data Deduplication

Before enabling Data Deduplication, you must choose the Usage Type that most closely
resembles your workload. There are three Usage Types included with Data
Deduplication.

Default - tuned specifically for general purpose file servers

Hyper-V - tuned specifically for VDI servers
Backup - tuned specifically for virtualized backup applications, such as Microsoft
DPM

Enable Data Deduplication by using Server Manager

1. Select File and Storage Services in Server Manager.

2. Select Volumes from File and Storage Services.

3. Right-click the desired volume and select Configure Data Deduplication.

 4. Select the desired Usage Type from the drop-down box and select OK.

5. If you are running a recommended workload, you're done. For other workloads,
see Other considerations.

７ Note

You can find more information on excluding file extensions or folders and selecting
the deduplication schedule, including why you would want to do this, in
Configuring Data Deduplication.

Enable Data Deduplication by using PowerShell

1. With an administrator context, run the following PowerShell command:

PowerShell

Enable-DedupVolume -Volume <Volume-Path> -UsageType <Selected-Usage-

Type>

2. If you are running a recommended workload, you're done. For other workloads,
see Other considerations.
 ７ Note

The Data Deduplication PowerShell cmdlets, including Enable-DedupVolume, can

be run remotely by appending the -CimSession parameter with a CIM Session. This
is particularly useful for running the Data Deduplication PowerShell cmdlets
remotely against a server instance. To create a new CIM Session run New-
CimSession.

Other considerations

） Important

If you are running a recommended workload, you can skip this section.

Data Deduplication's Usage Types give sensible defaults for recommended

workloads, but they also provide a good starting point for all workloads. For
workloads other than the recommended workloads, it is possible to modify Data
Deduplication's advanced settings to improve deduplication performance.
If your workload has high resource requirements on your server, the Data
Deduplication jobs should be scheduled to run during the expected idle times for
that workload. This is particularly important when running deduplication on a
hyper-converged host, because running Data Deduplication during expected
working hours can starve VMs.
If your workload does not have high resource requirements, or if it is more
important that optimization jobs complete than workload requests be served, the
memory, CPU, and priority of the Data Deduplication jobs can be adjusted.

Frequently asked questions (FAQ)

I want to run Data Deduplication on the dataset for X workload. Is this supported?
Aside from workloads that are known not to interoperate with Data Deduplication, we
fully support the data integrity of Data Deduplication with any workload. Recommended
workloads are supported by Microsoft for performance as well. The performance of
other workloads depends greatly on what they are doing on your server. You must
determine what performance impacts Data Deduplication has on your workload, and if
this is acceptable for this workload.

What are the volume sizing requirements for deduplicated volumes? In Windows
Server 2012 and Windows Server 2012 R2, volumes had to be carefully sized to ensure
that Data Deduplication could keep up with the churn on the volume. This typically
meant that the average maximum size of a deduplicated volume for a high-churn
workload was 1-2 TB, and the absolute maximum recommended size was 10 TB. In
Windows Server 2016, these limitations were removed. For more information, see What's
new in Data Deduplication.

Do I need to modify the schedule or other Data Deduplication settings for

recommended workloads? No, the provided Usage Types were created to provide
reasonable defaults for recommended workloads.

What are the memory requirements for Data Deduplication? At a minimum, Data
Deduplication should have 300 MB + 50 MB for each TB of logical data. For instance, if
you are optimizing a 10 TB volume, you would need a minimum of 800 MB of memory
allocated for deduplication ( 300 MB + 50 MB * 10 = 300 MB + 500 MB = 800 MB ). While
Data Deduplication can optimize a volume with this low amount of memory, having
such constrained resources will slow down Data Deduplication's jobs.

Optimally, Data Deduplication should have 1 GB of memory for every 1 TB of logical

data. For instance, if you are optimizing a 10 TB volume, you would optimally need 10
GB of memory allocated for Data Deduplication ( 1 GB * 10 ). This ratio will ensure the
maximum performance for Data Deduplication jobs.

What are the storage requirements for Data Deduplication? In Windows Server 2016,
Data Deduplication can support volume sizes up to 64 TB. For more information, view
What's new in Data Deduplication.
Running Data Deduplication
Article • 02/18/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Azure Stack HCI, versions 21H2 and 20H2

Running Data Deduplication jobs manually

You can run every scheduled Data Deduplication job manually by using the following
PowerShell cmdlets:

Start-DedupJob: Starts a new Data Deduplication job

Stop-DedupJob: Stops a Data Deduplication job already in progress (or removes it
from the queue)
Get-DedupJob: Shows all the active and queued Data Deduplication jobs

All settings that are available when you schedule a Data Deduplication job are also
available when you start a job manually except for the scheduling-specific settings. For
example, to start an Optimization job manually with high priority, maximum CPU usage,
and maximum memory usage, execute the following PowerShell command with
administrator privilege:

PowerShell

Start-DedupJob -Type Optimization -Volume <Your-Volume-Here> -Memory 100 -

Cores 100 -Priority High

Monitoring Data Deduplication

Job successes
Because Data Deduplication uses a post-processing model, it is important that Data
Deduplication jobs succeed. An easy way to check the status of the most recent job is to
use the Get-DedupStatus PowerShell cmdlet. Periodically check the following fields:

For the Optimization job, look at LastOptimizationResult (0 = Success),

LastOptimizationResultMessage , and LastOptimizationTime (should be recent).
For the Garbage Collection job, look at LastGarbageCollectionResult (0 = Success),
LastGarbageCollectionResultMessage , and LastGarbageCollectionTime (should be
 recent).
For the Integrity Scrubbing job, look at LastScrubbingResult (0 = Success),
LastScrubbingResultMessage , and LastScrubbingTime (should be recent).

７ Note

More detail on job successes and failures can be found in the Windows Event
Viewer under \Applications and Services
Logs\Windows\Deduplication\Operational .

Optimization rates
One indicator of Optimization job failure is a downward-trending optimization rate
which might indicate that the Optimization jobs are not keeping up with the rate of
changes, or churn. You can check the optimization rate by using the Get-DedupStatus
PowerShell cmdlet.

） Important

Get-DedupStatus has two fields that are relevant to the optimization rate:

OptimizedFilesSavingsRate and SavingsRate . These are both important values to

track, but each has a unique meaning.

OptimizedFilesSavingsRate applies only to the files that are 'in-policy' for

optimization ( space used by optimized files after optimization / logical

size of optimized files ).

SavingsRate applies to the entire volume ( space used by optimized files

after optimization / total logical size of the optimization ).

Disabling Data Deduplication

To turn off Data Deduplication, run the Unoptimization job. To undo volume
optimization, run the following command:

PowerShell

Start-DedupJob -Type Unoptimization -Volume <Desired-Volume>

 ） Important

The Unoptimization job will fail if the volume does not have sufficient space to hold
the unoptimized data.

Frequently Asked Questions

Is there a System Center Operations Manager Management Pack available to monitor
Data Deduplication? Yes. Data Deduplication can be monitored through the System
Center Management Pack for File Server. For more information, see the Guide for
System Center Management Pack for File Server 2012 R2 document.
Advanced Data Deduplication settings
Article • 02/18/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Azure Stack HCI, versions 21H2 and 20H2

This document describes how to modify advanced Data Deduplication settings. For
recommended workloads, the default settings should be sufficient. The main reason to
modify these settings is to improve Data Deduplication's performance with other kinds
of workloads.

Modifying Data Deduplication job schedules

The default Data Deduplication job schedules are designed to work well for
recommended workloads and be as non-intrusive as possible (excluding the Priority
Optimization job that is enabled for the Backup usage type). When workloads have
large resource requirements, it is possible to ensure that jobs run only during idle hours,
or to reduce or increase the amount of system resources that a Data Deduplication job
is allowed to consume.

Changing a Data Deduplication schedule

Data Deduplication jobs are scheduled via Windows Task Scheduler and can be viewed
and edited there under the path Microsoft\Windows\Deduplication. Data Deduplication
includes several cmdlets that make scheduling easy.

Get-DedupSchedule shows the current scheduled jobs.

New-DedupSchedule creates a new scheduled job.
Set-DedupSchedule modifies an existing scheduled job.
Remove-DedupSchedule removes a scheduled job.

The most common reason for changing when Data Deduplication jobs run is to ensure
that jobs run during off hours. The following step-by-step example shows how to
modify the Data Deduplication schedule for a sunny day scenario: a hyper-converged
Hyper-V host that is idle on weekends and after 7:00 PM on week nights. To change the
schedule, run the following PowerShell cmdlets in an Administrator context.

1. Disable the scheduled hourly Optimization jobs.

PowerShell
 Set-DedupSchedule -Name BackgroundOptimization -Enabled $false
Set-DedupSchedule -Name PriorityOptimization -Enabled $false

2. Remove the currently scheduled Garbage Collection and Integrity Scrubbing jobs.

PowerShell

Get-DedupSchedule -Type GarbageCollection | ForEach-Object { Remove-

DedupSchedule -InputObject $_ }
Get-DedupSchedule -Type Scrubbing | ForEach-Object { Remove-
DedupSchedule -InputObject $_ }

3. Create a nightly Optimization job that runs at 7:00 PM with high priority and all the
CPUs and memory available on the system.

PowerShell

New-DedupSchedule -Name "NightlyOptimization" -Type Optimization -

DurationHours 11 -Memory 100 -Cores 100 -Priority High -Days
@(1,2,3,4,5) -Start (Get-Date "2016-08-08 19:00:00")

７ Note

The date part of the System.Datetime provided to -Start is irrelevant (as long
as it's in the past), but the time part specifies when the job should start.

4. Create a weekly Garbage Collection job that runs on Saturday starting at 7:00 AM
with high priority and all the CPUs and memory available on the system.

PowerShell

New-DedupSchedule -Name "WeeklyGarbageCollection" -Type

GarbageCollection -DurationHours 23 -Memory 100 -Cores 100 -Priority
High -Days @(6) -Start (Get-Date "2016-08-13 07:00:00")

5. Create a weekly Integrity Scrubbing job that runs on Sunday starting at 7 AM with
high priority and all the CPUs and memory available on the system.

PowerShell

New-DedupSchedule -Name "WeeklyIntegrityScrubbing" -Type Scrubbing -

DurationHours 23 -Memory 100 -Cores 100 -Priority High -Days @(0) -
Start (Get-Date "2016-08-14 07:00:00")
Available job-wide settings
You can toggle the following settings for new or scheduled Data Deduplication jobs:

Parameter name Definition Accepted values Why would you

want to set this
value?

Type The type of the job Optimization This value is

that should be GarbageCollection required because it
scheduled Scrubbing is the type of job
that you want to
schedule. This value
cannot be changed
after the task has
been scheduled.

Priority The system priority of High This value helps the

the scheduled job Medium system determine
Low how to allocate CPU
time. High will use
more CPU time, low
will use less.

Days The days that the job An array of integers 0-6 Scheduled tasks
is scheduled representing the days of have to run on at
the week: least one day.
0 = Sunday
1 = Monday
2 = Tuesday
3 = Wednesday
4 = Thursday
5 = Friday
6 = Saturday

Cores The percentage of Integers 0-100 (indicates To control what level

cores on the system a percentage) of impact a job will
that a job should use have on the
compute resources
on the system

DurationHours The maximum number Positive integers To prevent a job for

of hours a job should running into a
be allowed to run workload's non-idle
hours

Enabled Whether the job will True/false To disable a job

run without removing it
Parameter name Definition Accepted values Why would you
want to set this
value?

Full For scheduling a full Switch (true/false) By default, every

Garbage Collection fourth job is a full
job Garbage Collection
job. With this switch,
you can schedule
full Garbage
Collection to run
more frequently.

InputOutputThrottle Specifies the amount Integers 0-100 (indicates Throttling ensures

of input/output a percentage) that jobs don't
throttling applied to interfere with other
the job I/O-intensive
processes.

Memory The percentage of Integers 0-100 (indicates To control what level

memory on the a percentage) of impact the job
system that a job will have on the
should use memory resources
of the system

Name The name of the String A job must have a

scheduled job uniquely identifiable
name.

ReadOnly Indicates that the Switch (true/false) You want to

scrubbing job manually restore
processes and reports files that sit on bad
on corruptions that it sections of the disk.
finds, but does not
run any repair actions

Start Specifies the time a System.DateTime The date part of the

job should start System.Datetime
provided to Start is
irrelevant (as long as
it's in the past), but
the time part
specifies when the
job should start.
 Parameter name Definition Accepted values Why would you
want to set this
value?

StopWhenSystemBusy Specifies whether Switch (True/False) This switch gives

Data Deduplication you the ability to
should stop if the control the behavior
system is busy of Data
Deduplication--this
is especially
important if you
want to run Data
Deduplication while
your workload is not
idle.

Modifying Data Deduplication volume-wide

settings

Toggling volume settings

You can set the volume-wide default settings for Data Deduplication via the usage type
that you select when you enable a deduplication for a volume. Data Deduplication
includes cmdlets that make editing volume-wide settings easy:

Get-DedupVolume
Set-DedupVolume

The main reasons to modify the volume settings from the selected usage type are to
improve read performance for specific files (such as multimedia or other file types that
are already compressed) or to fine-tune Data Deduplication for better optimization for
your specific workload. The following example shows how to modify the Data
Deduplication volume settings for a workload that most closely resembles a general
purpose file server workload, but uses large files that change frequently.

1. See the current volume settings for Cluster Shared Volume 1.

PowerShell

Get-DedupVolume -Volume C:\ClusterStorage\Volume1 | Select *

2. Enable OptimizePartialFiles on Cluster Shared Volume 1 so that the

MinimumFileAge policy applies to sections of the file rather than the whole file.
 This ensures that the majority of the file gets optimized even though sections of
the file change regularly.

PowerShell

Set-DedupVolume -Volume C:\ClusterStorage\Volume1 -

OptimizePartialFiles

Available volume-wide settings

Setting name Definition Accepted Why would you want to

values modify this value?

ChunkRedundancyThreshold The number of times Positive The main reason to

that a chunk is integers modify this number is to
referenced before a increase the savings rate
chunk is duplicated into for volumes with high
the hotspot section of duplication. In general,
the Chunk Store. The the default value (100) is
value of the hotspot the recommended
section is that so-called setting, and you
"hot" chunks that are shouldn't need to modify
referenced frequently this.
have multiple access
paths to improve access
time.

ExcludeFileType File types that are Array of Some file types,

excluded from file particularly multimedia or
optimization extensions files that are already
compressed, do not
benefit very much from
being optimized. This
setting allows you to
configure which types are
excluded.

ExcludeFolder Specifies folder paths Array of If you want to improve

that should not be folder performance or keep
considered for paths content in particular
optimization paths from being
optimized, you can
exclude certain paths on
the volume from
consideration for
optimization.
Setting name Definition Accepted Why would you want to
values modify this value?

InputOutputScale Specifies the level of IO Positive The main reason to

parallelization (IO integers modify this value is to
queues) for Data ranging 1- decrease the impact on
Deduplication to use on 36 the performance of a
a volume during a post- high IO workload by
processing job restricting the number of
IO queues that Data
Deduplication is allowed
to use on a volume. Note
that modifying this
setting from the default
may cause Data
Deduplication's post-
processing jobs to run
slowly.

MinimumFileAgeDays Number of days after the Positive The Default and Hyper-V
file is created before the integers usage types set this value
file is considered to be (inclusive to 3 to maximize
in-policy for of zero) performance on hot or
optimization. recently created files. You
may want to modify this
if you want Data
Deduplication to be more
aggressive or if you do
not care about the extra
latency associated with
deduplication.

MinimumFileSize Minimum file size that a Positive The main reason to

file must have to be integers change this value is to
considered in-policy for (bytes) exclude small files that
optimization greater may have limited
than 32 optimization value to
KB conserve compute time.
Setting name Definition Accepted Why would you want to
values modify this value?

NoCompress Whether the chunks True/False Some types of files,

should be compressed particularly multimedia
before being put into the files and already
Chunk Store compressed file types,
may not compress well.
This setting allows you to
turn off compression for
all files on the volume.
This would be ideal if you
are optimizing a dataset
that has a lot of files that
are already compressed.

NoCompressionFileType File types whose chunks Array of Some types of files,

should not be file particularly multimedia
compressed before extensions files and already
going into the Chunk compressed file types,
Store may not compress well.
This setting allows
compression to be
turned off for those files,
saving CPU resources.

OptimizeInUseFiles When enabled, files that True/false Enable this setting if your
have active handles workload keeps files
against them will be open for extended
considered as in-policy periods of time. If this
for optimization. setting is not enabled, a
file would never get
optimized if the workload
has an open handle to it,
even if it's only
occasionally appending
data at the end.
 Setting name Definition Accepted Why would you want to
values modify this value?

OptimizePartialFiles When enabled, the True/false Enable this setting if your

MinimumFileAge value workload works with
applies to segments of a large, often edited files
file rather than to the where most of the file
whole file. content is untouched. If
this setting is not
enabled, these files
would never get
optimized because they
keep getting changed,
even though most of the
file content is ready to be
optimized.

Verify When enabled, if the True/false This is an integrity

hash of a chunk matches feature that ensures that
a chunk we already have the hashing algorithm
in our Chunk Store, the that compares chunks
chunks are compared does not make a mistake
byte-by-byte to ensure by comparing two
they are identical. chunks of data that are
actually different but
have the same hash. In
practice, it is extremely
improbable that this
would ever happen.
Enabling the verification
feature adds significant
overhead to the
optimization job.

Modifying Data Deduplication system-wide

settings
Data Deduplication has additional system-wide settings that can be configured via the
registry. These settings apply to all of the jobs and volumes that run on the system. Extra
care must be given whenever editing the registry.

For example, you may want to disable full Garbage Collection. More information about
why this may be useful for your scenario can be found in Frequently asked questions. To
edit the registry with PowerShell:

If Data Deduplication is running in a cluster:

 PowerShell

Set-ItemProperty -Path
HKLM:\System\CurrentControlSet\Services\ddpsvc\Settings -Name
DeepGCInterval -Type DWord -Value 0xFFFFFFFF
Set-ItemProperty -Path HKLM:\CLUSTER\Dedup -Name DeepGCInterval -Type
DWord -Value 0xFFFFFFFF

If Data Deduplication is not running in a cluster:

PowerShell

Set-ItemProperty -Path
HKLM:\System\CurrentControlSet\Services\ddpsvc\Settings -Name
DeepGCInterval -Type DWord -Value 0xFFFFFFFF

Available system-wide settings

Setting name Definition Accepted Why would

values you want to
change this?

WlmMemoryOverPercentThreshold This setting allows jobs to use Positive If you have

more memory than Data integers another task
Deduplication judges to (a value that will stop
actually be available. For of 300 if Data
example, a setting of 300 means Deduplication
would mean that the job 300% or takes more
would have to use three times 3 times) memory
the assigned memory to get
canceled.
 Setting name Definition Accepted Why would
values you want to
change this?

DeepGCInterval This setting configures the Integers See this

interval at which regular (-1 frequently
Garbage Collection jobs indicates asked
become full Garbage disabled) question
Collection jobs. A setting of n
would mean that every nth job
was a full Garbage Collection
job. Note that full Garbage
Collection is always disabled
(regardless of the registry
value) for volumes with the
Backup Usage Type. Start-
DedupJob -Type
GarbageCollection -Full may
be used if full Garbage
Collection is desired on a
Backup volume.

Frequently asked questions

I changed a Data Deduplication setting, and now jobs are slow or don't finish, or my
workload performance has decreased. Why? These settings give you a lot of power to
control how Data Deduplication runs. Use them responsibly, and monitor performance.

I want to run a Data Deduplication job right now, but I don't want to create a new
schedule--can I do this? Yes, all jobs can be run manually.

What is the difference between full and regular Garbage Collection? There are two
types of Garbage Collection:

Regular Garbage Collection uses a statistical algorithm to find large unreferenced

chunks that meet a certain criteria (low in memory and IOPs). Regular Garbage
Collection compacts a chunk store container only if a minimum percentage of the
chunks is unreferenced. This type of Garbage Collection runs much faster and uses
fewer resources than full Garbage Collection. The default schedule of the regular
Garbage Collection job is to run once a week.
Full Garbage Collection does a much more thorough job of finding unreferenced
chunks and freeing more disk space. Full Garbage Collection compacts every
container even if just a single chunk in the container is unreferenced. Full Garbage
Collection will also free space that may have been in use if there was a crash or
 power failure during an Optimization job. Full Garbage Collection jobs will recover
100 percent of the available space that can be recovered on a deduplicated
volume at the cost of requiring more time and system resources compared to a
regular Garbage Collection job. The full Garbage Collection job will typically find
and release up to 5 percent more of the unreferenced data than a regular Garbage
Collection job. The default schedule of the full Garbage Collection job is to run
every fourth time Garbage Collection is scheduled.

Why would I want to disable full Garbage Collection?

Garbage Collection could adversely affect the volume's lifetime shadow copies and
the size of incremental backup. High churn or I/O-intensive workloads may see a
degradation in performance by full Garbage Collection jobs.
You can manually run a full Garbage Collection job from PowerShell to clean up
leaks if you know your system crashed.
Data Deduplication interoperability
Article • 03/29/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Azure Stack HCI, versions 21H2 and 20H2

Supported

ReFS
Data Deduplication is supported starting with Windows Server 2019.

Failover Clustering
Failover Clustering is fully supported, if every node in the cluster has the Data
Deduplication feature installed. Other important notes:

Manually started Data Deduplication jobs must be run on the Owner node for the
Cluster Shared Volume.
Scheduled Data Deduplication jobs are stored in the cluster task scheduled so that
if a deduplicated volume is taken over by another node, the scheduled job will be
applied on the next scheduled interval.
Data Deduplication fully interoperates with the Cluster OS Rolling Upgrade feature.
Data Deduplication is fully supported on Storage Spaces Direct with ReFS or NTFS-
formatted volumes (mirror or parity). ReFS-formatted volumes are supported
starting with Windows Server 2019. Deduplication is not supported on volumes
with multiple tiers.

Storage Replica
Storage Replica is fully supported. Data Deduplication should be configured to not run
on the secondary copy.

BranchCache
You can optimize data access over the network by enabling BranchCache on servers and
clients. When a BranchCache-enabled system communicates over a WAN with a remote
file server that is running data deduplication, all of the deduplicated files are already
indexed and hashed. Therefore, requests for data from a branch office are quickly
computed. This is similar to preindexing or prehashing a BranchCache-enabled server.

DFS Replication
Data Deduplication works with Distributed File System (DFS) Replication. Optimizing or
unoptimizing a file will not trigger a replication because the file does not change. DFS
Replication uses Remote Differential Compression (RDC), not the chunks in the chunk
store, for over-the-wire savings. The files on the replica can also be optimized by using
deduplication if the replica is using Data Deduplication.

Quotas
Data Deduplication does not support creating a hard quota on a volume root folder that
also has deduplication enabled. When a hard quota is present on a volume root, the
actual free space on the volume and the quota-restricted space on the volume are not
the same. This may cause deduplication optimization jobs to fail. It is possible however
to creating a soft quota on a volume root that has deduplication enabled.

When quota is enabled on a deduplicated volume, quota uses the logical size of the file
rather than the physical size of the file. Quota usage (including any quota thresholds)
does not change when a file is processed by deduplication. All other quota functionality,
including volume-root soft quotas and quotas on subfolders, works normally when
using deduplication.

Windows Server Backup

Windows Server Backup can back up an optimized volume as-is (that is, without
removing deduplicated data). The following steps show how to back up a volume and
how to restore a volume or selected files from a volume.

1. Install Windows Server Backup.

PowerShell

Install-WindowsFeature -Name Windows-Server-Backup

2. Back up the E: volume to another volume by running the following command,

substituting the correct volume names for your situation.

PowerShell
 wbadmin start backup –include:E: -backuptarget:F: -quiet

3. Get the version ID of the backup you just created.

PowerShell

wbadmin get versions

This output version ID will be a date and time string, for example: 08/18/2016-
06:22.

4. Restore the entire volume.

PowerShell

wbadmin start recovery –version:02/16/2012-06:22 -itemtype:Volume -

items:E: -recoveryTarget:E:

--OR--

Restore a particular folder (in this case, the E:\Docs folder):

PowerShell

wbadmin start recovery –version:02/16/2012-06:22 -itemtype:File -

items:E:\Docs -recursive

Unsupported

Windows 10 (client OS)

Data Deduplication is not supported on Windows 10. There are several popular blog
posts in the Windows community describing how to remove the binaries from Windows
Server 2016 and install on Windows 10, but this scenario has not been validated as part
of the development of Data Deduplication.

Windows Search
Windows Search doesn't support Data Deduplication. Data Deduplication uses reparse
points, which Windows Search can't index, so Windows Search skips all deduplicated
files, excluding them from the index. As a result, search results might be incomplete for
deduplicated volumes.

Robocopy
Running Robocopy with Data Deduplication is not recommended because certain
Robocopy commands can corrupt the Chunk Store. The Chunk Store is stored in the
System Volume Information folder for a volume. If the folder is deleted, the optimized
files (reparse points) that are copied from the source volume become corrupted because
the data chunks are not copied to the destination volume.
DFS Namespaces overview
Article • 01/05/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

DFS (Distributed File System) Namespaces is a role service in Windows Server that
enables you to group shared folders located on different servers into one or more
logically structured namespaces. This makes it possible to give users a virtual view of
shared folders, where a single path leads to files located on multiple servers, as shown in
the following figure:

Here's a description of the elements that make up a DFS namespace:

Namespace server - A namespace server hosts a namespace. The namespace

server can be a member server or a domain controller.
Namespace root - The namespace root is the starting point of the namespace. In
the previous figure, the name of the root is Public, and the namespace path is
\\Contoso\Public. This type of namespace is a domain-based namespace because
it begins with a domain name (for example, Contoso) and its metadata is stored in
Active Directory Domain Services (AD DS). Although a single namespace server is
shown in the previous figure, a domain-based namespace can be hosted on
multiple namespace servers to increase the availability of the namespace.
Folder - Folders without folder targets add structure and hierarchy to the
namespace, and folders with folder targets provide users with actual content.
When users browse a folder that has folder targets in the namespace, the client
computer receives a referral that transparently redirects the client computer to one
of the folder targets.
 Folder targets - A folder target is the UNC path of a shared folder or another
namespace that is associated with a folder in a namespace. The folder target is
where data and content is stored. In the previous figure, the folder named Tools
has two folder targets, one in London and one in New York, and the folder named
Training Guides has a single folder target in New York. A user who browses to
\\Contoso\Public\Software\Tools is transparently redirected to the shared folder
\\LDN-SVR-01\Tools or \\NYC-SVR-01\Tools, depending on which site the user is
currently located in.

This article discusses how to install DFS, what's new, and where to find evaluation and
deployment information.

You can administer namespaces by using DFS Management, the DFS Namespace (DFSN)
Cmdlets in Windows PowerShell, the DfsUtil command, or scripts that call WMI.

Server requirements and limits

There are no additional hardware or software requirements for running DFS
Management or using DFS Namespaces.

A namespace server is a domain controller or member server that hosts a namespace.

The number of namespaces you can host on a server is determined by the operating
system running on the namespace server.

Servers that are running the following operating systems can host multiple domain-
based namespaces in addition to a single stand-alone namespace.

Windows Server 2022

Windows Server 2019
Windows Server 2016
Windows Server 2012 R2
Windows Server 2012
Windows Server 2008 R2 Datacenter and Enterprise Editions
Windows Server (Semi-Annual Channel)

Servers that are running the following operating systems can host a single stand-alone
namespace:

Windows Server 2008 R2 Standard

The following table describes additional factors to consider when choosing servers to
host a namespace.
 Server Hosting Server Hosting Domain-Based Namespaces
Stand-Alone
Namespaces

Must contain an NTFS Must contain an NTFS volume to host the namespace.
volume to host the
namespace.

Can be a member Must be a member server or domain controller in the domain in which
server or domain the namespace is configured. (This requirement applies to every
controller. namespace server that hosts a given domain-based namespace.)

Can be hosted by a The namespace cannot be a clustered resource in a failover cluster.

failover cluster to However, you can locate the namespace on a server that also functions
increase the as a node in a failover cluster if you configure the namespace to use only
availability of the local resources on that server.
namespace.

Installing DFS Namespaces

DFS Namespaces and DFS Replication are a part of the File and Storage Services role.
The management tools for DFS (DFS Management, the DFS Namespaces module for
Windows PowerShell, and command-line tools) are installed separately as part of the
Remote Server Administration Tools.

Install DFS Namespaces by using Windows Admin Center, Server Manager, or

PowerShell, as described in the next sections.

To install DFS by using Server Manager

1. Open Server Manager, click Manage, and then click Add Roles and Features. The
Add Roles and Features Wizard appears.

2. On the Server Selection page, select the server or virtual hard disk (VHD) of an
offline virtual machine on which you want to install DFS.

3. Select the role services and features that you want to install.

To install the DFS Namespaces service, on the Server Roles page, select DFS
Namespaces.

To install only the DFS Management Tools, on the Features page, expand
Remote Server Administration Tools, Role Administration Tools, expand File
Services Tools, and then select DFS Management Tools.
 DFS Management Tools installs the DFS Management snap-in, the DFS
Namespaces module for Windows PowerShell, and command-line tools, but
it does not install any DFS services on the server.

To install DFS by using Windows PowerShell

Open a Windows PowerShell session with elevated user rights, and then type the
following command, where <name> is the role service or feature that you want to install
(see the following table for a list of relevant role service or feature names):

PowerShell

Install-WindowsFeature <name>

Role service or feature Name

DFS Namespaces FS-DFS-Namespace

DFS Management Tools RSAT-DFS-Mgmt-Con

For example, to install the Distributed File System Tools portion of the Remote Server
Administration Tools feature, type:

PowerShell

Install-WindowsFeature "RSAT-DFS-Mgmt-Con"

To install the Distributed File System Tools portion for a client device, type:

PowerShell

Add-WindowsCapability -Name Rsat.FileServices.Tools~~~~0.0.1.0 -Online

To install the DFS Namespaces, and the Distributed File System Tools portions of the
Remote Server Administration Tools feature, type:

PowerShell

Install-WindowsFeature "FS-DFS-Namespace", "RSAT-DFS-Mgmt-Con"

Interoperability with Azure virtual machines

Using DFS Namespaces on a virtual machine in Microsoft Azure has been tested.

You can host domain-based namespaces in Azure virtual machines, including

environments with Azure Active Directory.
You can cluster stand-alone namespaces in Azure virtual machines using failover
clusters that use Shared Disk or Ultra Disks.

To learn about how to get started with Azure virtual machines, see Azure virtual
machines documentation.

Additional References
For additional related information, see the following resources.

Content type References

Product evaluation What's New in DFS Namespaces and DFS Replication in Windows Server

Deployment DFS Namespace Scalability Considerations

Operations DFS Namespaces: Frequently Asked Questions

Community resources The File Services and Storage TechNet Forum

Protocols File Services Protocols in Windows Server (Deprecated)

Related technologies Failover Clustering

Support Windows IT Pro Support

Checklist: Deploy DFS Namespaces
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

Distributed File System (DFS) Namespaces and DFS Replication can be used to publish
documents, software, and line-of-business data to users throughout an organization.
Although DFS Replication alone is sufficient to distribute data, you can use
DFS Namespaces to configure the namespace so that a folder is hosted by multiple
servers, each of which holds an updated copy of the folder. This increases data
availability and distributes the client load across servers.

When browsing a folder in the namespace, users are not aware that the folder is hosted
by multiple servers. When a user opens the folder, the client computer is automatically
referred to a server on its site. If no same-site servers are available, you can configure
the namespace to refer the client to a server that has the lowest connection cost as
defined in Active Directory Directory Services (AD DS).

To deploy DFS Namespaces, perform the following tasks:

Review the concepts, and requirements of DFS Namespaces. Overview of DFS

Namespaces
Choose a namespace type
Create a DFS namespace
Migrate existing domain-based namespaces to Windows Server 2008 mode
domain-based namespaces. Migrate a Domain-based Namespace to Windows
Server 2008 mode
Increase availability by adding namespace servers to a domain-based namespace.
Add Namespace Servers to a Domain-based DFS Namespace
Add folders to a namespace. Create a Folder in a DFS Namespace
Add folder targets to folders in a namespace. Add Folder Targets
Replicate content between folder targets using DFS Replication (optional).
Replicate Folder Targets Using DFS Replication

Additional References
Namespaces
Checklist: Tune a DFS Namespace
Replication
Checklist: Tune a DFS namespace
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

After creating a namespace and adding folders and targets, use the following checklist
to tune or optimize the way the DFS namespace handles referrals and polls Active
Directory Domain Services (AD DS) for updated namespace data.

Prevent users from seeing folders in a namespace that they do not have
permissions to access. Enable Access-Based Enumeration on a Namespace
Enable or prevent users from being referred to a namespace or folder target when
they access a folder in the namespace. Enable or Disable Referrals and Client
Failback
Adjust how long clients cache a referral before requesting a new one. Change the
Amount of Time That Clients Cache Referrals
Optimize how namespace servers poll AD DS to obtain the most current
namespace data. Optimize Namespace Polling
Use inherited permissions to control which users can view folders in a namespace
for which access-based enumeration is enabled. Using Inherited Permissions with
Access-Based Enumeration

In addition, by using a DFS Namespaces enhancement known as target priority, you can
specify the priority of servers so that a specific server is always placed first or last in the
list of servers (known as a referral) that the client receives when it accesses a folder with
targets in the namespace.

Specify in what order users should be referred to folder targets. Set the Ordering
Method for Targets in Referrals
Override referral ordering for a specific namespace server or folder target. Set
Target Priority to Override Referral Ordering

Additional References
Namespaces
Checklist: Deploy DFS Namespaces
Tuning DFS Namespaces
Deploying DFS Namespaces
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

To deploy DFS Namespaces, refer to the following topics:

Choose a Namespaces Type

Create a DFS Namespace
Migrate a Domain-based Namespace to Windows Server 2008 Mode
Add Namespace Servers to a Domain-based DFS Namespace
Create a Folder in a DFS Namespace
Add Folder Targets
Replicate Folder Targets Using DFS Replication
Delegate Management Permissions for DFS Namespaces
Choose a namespace type
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016, Windows
Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows Server 2008

When creating a namespace, you must choose one of two namespace types: a stand-alone
namespace or a domain-based namespace. In addition, if you choose a domain-based
namespace, you must choose a namespace mode: Windows 2000 Server mode or Windows
Server 2008 mode.

Choosing a namespace type

Choose a stand-alone namespace if any of the following conditions apply to your environment:

Your organization does not use Active Directory Domain Services (AD DS).
You want to increase the availability of the namespace by using a failover cluster.
You need to create a single namespace with more than 5,000 DFS folders in a domain that
does not meet the requirements for a domain-based namespace (Windows Server 2008
mode) as described later in this topic.

７ Note

To check the size of a namespace, right-click the namespace in the DFS Management
console tree, click Properties, and then view the namespace size in the Namespace
Properties dialog box. For more information about DFS Namespace scalability, see the
Microsoft website File Services.

Choose a domain-based namespace if any of the following conditions apply to your environment:

You want to ensure the availability of the namespace by using multiple namespace servers.
You want to hide the name of the namespace server from users. This makes it easier to
replace the namespace server or migrate the namespace to another server.

Choosing a domain-based namespace mode

If you choose a domain-based namespace, you must choose whether to use the
Windows 2000 Server mode or the Windows Server 2008 mode. The Windows Server 2008 mode
includes support for access-based enumeration and increased scalability. The domain-based
namespace introduced in Windows 2000 Server is now referred to as "domain-based namespace
(Windows 2000 Server mode)."
To use the Windows Server 2008 mode, the domain and namespace must meet the following
minimum requirements:

The forest uses the Windows Server 2003 or higher forest functional level.
The domain uses the Windows Server 2008 or higher domain functional level.
All namespace servers are running Windows Server 2012 R2, Windows Server 2012,
Windows Server 2008 R2, or Windows Server 2008.

If your environment supports it, choose the Windows Server 2008 mode when you create new
domain-based namespaces. This mode provides additional features and scalability, and also
eliminates the possible need to migrate a namespace from the Windows 2000 Server mode.

For information about migrating a namespace to Windows Server 2008 mode, see Migrate a
Domain-based Namespace to Windows Server 2008 Mode.

If your environment does not support domain-based namespaces in Windows Server 2008 mode,
use the existing Windows 2000 Server mode for the namespace.

Comparing namespace types and modes

The characteristics of each namespace type and mode are described in the following table.

Characteristic Stand-Alone Domain-based Namespace Domain-based Namespace

Namespace (Windows 2000 Server Mode) (Windows Server 2008 Mode)

Path to \\ \\ \\
namespace ServerName\RootName NetBIOSDomainName\RootName NetBIOSDomainName\RootName
\\ DNSDomainName\RootName \\ DNSDomainName\RootName

Namespace In the registry and in a In AD DS and in a memory cache In AD DS and in a memory cache
information memory cache on the on each namespace server on each namespace server
storage location namespace server

Namespace size The namespace can The size of the namespace object The namespace can contain
recommendations contain more than in AD DS should be less than 5 more than 5,000 folders with
5,000 folders with megabytes (MB) to maintain targets; the recommended limit
targets; the compatibility with domain is 50,000 folders with targets
recommended limit is controllers that are not running
50,000 folders with Windows Server 2008. This
targets means no more than
approximately 5,000 folders with
targets.

Minimum AD DS AD DS is not required Windows 2000 Windows Server 2003

forest functional
level

Minimum AD DS AD DS is not required Windows 2000 mixed Windows Server 2008

domain
functional level
Characteristic Stand-Alone Domain-based Namespace Domain-based Namespace
Namespace (Windows 2000 Server Mode) (Windows Server 2008 Mode)

Minimum Windows 2000 Server Windows 2000 Server Windows Server 2008
supported
namespace
servers

Support for Yes, requires Windows No Yes

access-based Server 2008
enumeration (if namespace server
enabled)

Supported Create a stand-alone Use multiple namespace servers Use multiple namespace servers
methods to namespace on a to host the namespace. (The to host the namespace. (The
ensure failover cluster. namespace servers must be in namespace servers must be in
namespace the same domain.) the same domain.)
availability

Support for using Supported when Supported Supported

DFS Replication joined to an AD DS
to replicate folder domain
targets

Additional References
Deploying DFS Namespaces
Migrate a Domain-based Namespace to Windows Server 2008 Mode
Create a DFS namespace
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

To create a new namespace, you can use Server Manager to create the namespace when
you install the DFS Namespaces role service. You can also use the New-DfsnRoot cmdlet
from a Windows PowerShell session.

The DFSN Windows PowerShell module was introduced in Windows Server 2012.

Alernatively, you can use the following procedure to create a namespace after installing
the role service.

To create a namespace
1. Click Start, point to Administrative Tools, and then click DFS Management.

2. In the console tree, right-click the Namespaces node, and then click New
Namespace.

3. Follow the instructions in the New Namespace Wizard.

To create a stand-alone namespace on a failover cluster, specify the name of a

clustered file server instance on the Namespace Server page of the New
Namespace Wizard .

） Important

Do not attempt to create a domain-based namespace using the Windows

Server 2008 mode unless the forest functional level is Windows Server 2003 or
higher. Doing so can result in a namespace for which you cannot delete DFS
folders, yielding the following error message: "The folder cannot be deleted.
Cannot complete this function".

Additional References
Deploying DFS Namespaces
Choose a Namespace Type
Add Namespace Servers to a Domain-based DFS Namespace
Delegate Management Permissions for DFS Namespaces.
Migrate a domain-based namespace to
Windows Server 2008 Mode
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

The Windows Server 2008 mode for domain-based namespaces includes support for
access-based enumeration and increased scalability.

To migrate a domain-based namespace to

Windows Server 2008 mode
To migrate a domain-based namespace from Windows 2000 Server mode to Windows
Server 2008 mode, you must export the namespace to a file, delete the namespace, re-
create it in Windows Server 2008 mode, and then import the namespace settings. To do
so, use the following procedure:

1. Open a Command Prompt window and type the following command to export the
namespace to a file, where \\domain\namespace is the name of the appropriate
domain, and namespace and path\filename is the path and file name of the file for
export:

Dfsutil root export \\domain\namespace path\filename.xml

2. Write down the path ( \\server \share ) for each namespace server. You must
manually add namespace servers to the re-created namespace because Dfsutil
cannot import namespace servers.

3. In DFS Management, right-click the namespace and then click Delete, or type the
following command at a command prompt,
where \\domain\namespace is the name of the appropriate domain and
namespace:
 Dfsutil root remove \\domain\namespace

4. In DFS Management, re-create the namespace with the same name, but use the
Windows Server 2008 mode, or type the following command at a command
prompt, where
\\server\namespace is the name of the appropriate server and share for the
namespace root:

Dfsutil root adddom \\server\namespace v2

5. To import the namespace from the export file, type the following command at a
command prompt, where
\\domain\namespace is the name of the appropriate domain and namespace and
path\filename is the path and file name of the file to import:

Dfsutil root import merge path\filename.xml \\domain\namespace

７ Note

To minimize the time required to import a large namespace, run the Dfsutil
root import command locally on a namespace server.

6. Add any remaining namespace servers to the re-created namespace by right-

clicking the namespace in DFS Management and then clicking Add Namespace
Server, or by typing the following command at a command prompt, where
\\server\share is the name of the appropriate server and share for the namespace
root:

Dfsutil target add \\server\share

７ Note

You can add namespace servers before importing the namespace, but doing
so causes the namespace servers to incrementally download the metadata for
 the namespace instead of immediately downloading the entire namespace
after being added as a namespace server.

Additional References
Deploying DFS Namespaces
Choose a Namespace Type
Add namespace servers to a domain-
based DFS namespace
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

You can increase the availability of a domain-based namespace by specifying additional

namespace servers to host the namespace.

To add a namespace server to a domain-based

namespace
To add a namespace server to a domain-based namespace using DFS Management, use
the following procedure:

1. Click Start, point to Administrative Tools, and then click DFS Management.

2. In the console tree, under the Namespaces node, right-click a domain-based

namespace, and then click Add Namespace Server.

3. Enter the path to another server, or click Browse to locate a server.

７ Note

This procedure is not applicable for stand-alone namespaces because they support
only a single namespace server. To increase the availability of a stand-alone
namespace, specify a failover cluster as the namespace server in the New
Namespace Wizard.

 Tip

To add a namespace server by using Windows PowerShell, use the New-

DfsnRootTarget cmdlet. The DFSN Windows PowerShell module was introduced in
Windows Server 2012.
Additional References
Deploying DFS Namespaces
Review DFS Namespaces Server Requirements
Create a DFS Namespace
Delegate Management Permissions for DFS Namespaces
Create a folder in a DFS namespace
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

You can use folders to create additional levels of hierarchy in a namespace. You can also
create folders with folder targets to add shared folders to the namespace. DFS folders
with folder targets cannot contain other DFS folders, so if you want to add a level of
hierarchy to the namespace, do not add folder targets to the folder.

Use the following procedure to create a folder in a namespace using DFS Management:

To create a folder in a DFS namespace

1. Click Start, point to Administrative Tools, and then click DFS Management.

2. In the console tree, under the Namespaces node, right-click a namespace or a

folder within a namespace, and then click New Folder.

3. In the Name text box, type the name of the new folder.

4. To add one or more folder targets to the folder, click Add and specify the Universal
Naming Convention (UNC) path of the folder target, and then click OK .

 Tip

To create a folder in a namespace by using Windows PowerShell, use the New-

DfsnFolder cmdlet. The DFSN Windows PowerShell module was introduced in
Windows Server 2012.

Additional References
Deploying DFS Namespaces
Delegate Management Permissions for DFS Namespaces
Add folder targets
Article • 05/11/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

A folder target is the Universal Naming Convention (UNC) path of a shared folder or
another namespace that is associated with a folder in a namespace. Adding multiple
folder targets increases the availability of the folder in the namespace.

Prerequisites
The following must be installed to use this feature:

A Windows Server with the DFS Namespaces role service installed as part of the
File and Storage Services server role. To learn more, see Install or Uninstall Roles,
Role Services, or Features.
An account with Administrative privileges.
A server to host the DFS folder target.

Add a folder target

To add a folder target by using DFS Management, perform the following:

1. Click Start > Windows Administrative Tools > select DFS Management.
Alternatively, click Start > type dfsmgmt.msc > hit Enter .
2. In the console tree, under the Namespaces node, right-click on the namespace
where you want to add the folder, then select New Folder.
3. In the popup box, provide a name for this folder in the Name field, then click Add.
4. Type the path to the folder target, or click Browse to locate the folder target, click
OK, then click OK again.

The following animation demonstrates the steps to add a folder target by using DFS
Management.
If the folder is replicated using DFS Replication, you can specify whether to add the new
folder target to the replication group.

 Tip

To add a folder target by using Windows PowerShell, use the New-

DfsnFolderTarget cmdlet. The DFSN Windows PowerShell module was introduced
in Windows Server 2012.

７ Note

Folders can contain folder targets or other DFS folders, but not both, at the same
level in the folder hierarchy.

Additional references
Deploying DFS Namespaces
Delegate Management Permissions for DFS Namespaces
Replicate Folder Targets Using DFS Replication
Replicate folder targets using DFS
Replication
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and
Windows Server 2008

You can use DFS Replication to keep the contents of folder targets in sync so that users
see the same files regardless of which folder target the client computer is referred to.

To replicate folder targets using DFS

Replication
1. Click Start, point to Administrative Tools, and then click DFS Management.

2. In the console tree, under the Namespaces node, right-click a folder that has two
or more folder targets, and then click Replicate Folder.

3. Follow the instructions in the Replicate Folder Wizard.

７ Note

Configuration changes are not applied immediately to all members except when
using the Suspend-DfsReplicationGroup and Sync-DfsReplicationGroup cmdlets.
The new configuration must be replicated to all domain controllers, and each
member in the replication group must poll its closest domain controller to obtain
the changes. The amount of time this takes depends on the Active Directory
Directory Services (AD DS) replication latency and the long polling interval (60
minutes) on each member. To poll immediately for configuration changes, open a
Command Prompt window and then type the following command once for each
member of the replication group:
dfsrdiag.exe PollAD /Member:DOMAIN\Server1
To do so from a Windows PowerShell session, use the Update-
DfsrConfigurationFromAD cmdlet, which was introduced in Windows
Server 2012 R2.
Additional References
Deploying DFS Namespaces
Delegate Management Permissions for DFS Namespaces
DFS Replication
Delegate management permissions for
DFS Namespaces
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

The following table describes the groups that can perform basic namespace tasks by
default, and the method for delegating the ability to perform these tasks:

Task Groups that Delegation Method

Can Perform
this Task by
Default

Create a Domain Right-click the Namespaces node in the console tree, and then
domain- Admins group click Delegate Management Permissions. Or use the Set-
based in the domain DfsnRoot GrantAdminAccounts and Set-DfsnRoot
namespace where the RevokeAdminAccounts. Windows PowerShell cmdlets (introduced
namespace is in Windows Server 2012). You must also add the user to the local
configured Administrators group on the namespace server.

Add a Domain Right-click the domain-based namespace in the console tree, and
namespace Admins group then click Delegate Management Permissions. Or use the Set-
server to a in the domain DfsnRoot GrantAdminAccounts and Set-DfsnRoot
domain- where the RevokeAdminAccounts. Windows PowerShell cmdlets (introduced
based namespace is in Windows Server 2012). You must also add the user to the local
namespace configured Administrators group on the namespace server to be added.

Manage a Local Right-click the domain-based namespace in the console tree, and
domain- Administrators then click Delegate Management Permissions.
based group on each
namespace namespace
server

Create a Local Add the user to the local Administrators group on the namespace
stand-alone Administrators server.
namespace group on the
namespace
server
 Task Groups that Delegation Method
Can Perform
this Task by
Default

Manage a Local Right-click the stand-alone namespace in the console tree, and
stand-alone Administrators then click Delegate Management Permissions. Or use the Set-
namespace* group on the DfsnRoot GrantAdminAccounts and Set-DfsnRoot
namespace RevokeAdminAccounts. Windows PowerShell cmdlets (introduced
server in Windows Server 2012).

Create a Domain Right-click the Replication node in the console tree, and then
replication Admins group click Delegate Management Permissions.
group or in the domain
enable DFS where the
Replication namespace is
on a folder configured

*Delegating management permissions to manage a stand-alone namespace does not

grant the user the ability to view and manage security by using the Delegation tab
unless the user is a member of the local Administrators group on the namespace server.
This issue occurs because the DFS Management snap-in cannot retrieve the
discretionary access control lists (DACLs) for the stand-alone namespace from the
registry. To enable the snap-in to display delegation information, you must follow the
steps in the Microsoft® Knowledge Base article: KB314837: How to Manage Remote
Access to the Registry
Tuning DFS Namespaces
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

After creating a namespace and adding folders and targets, refer to the following
sections to tune or optimize the way DFS Namespace handles referrals and polls Active
Directory Domain Services (AD DS) for updated namespace data:

Enable Access-Based Enumeration on a Namespace

Enable or Disable Referrals and Client Failback
Change the Amount of Time That Clients Cache Referrals
Set the Ordering Method for Targets in Referrals
Set Target Priority to Override Referral Ordering
Optimize Namespace Polling
Using Inherited Permissions with Access-Based Enumeration

７ Note

To search for folders or folder targets, select a namespace, click the Search tab,
type your search string in the text box, and then click Search.
Enable or Disable Referrals and Client
Failback
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

A referral is an ordered list of servers that a client computer receives from a domain
controller or namespace server when the user accesses a namespace root or DFS folder
with targets. After the computer receives the referral, the computer attempts to access
the first server in the list. If the server is not available, the client computer attempts to
access the next server. If a server becomes unavailable, you can configure clients to fail
back to the preferred server after it becomes available.

The following sections provide information about how to enable or disable referrals or
enable client failback:

Enable or disable referrals

By disabling a namespace server's or folder target's referral, you can prevent users from
being directed to that namespace server or folder target. This is useful if you need to
temporarily take a server offline for maintenance.

To enable or disable referrals to a folder target, use the following steps:

1. In the DFS Management console tree, under the Namespaces node, click a
folder containing targets, and then click the Folder Targets tab in the Details
pane.
2. Right-click the folder target, and then click either Disable Folder Target or
Enable Folder Target.

To enable or disable referrals to a namespace server, use the following steps:

1. In the DFS Management console tree, select the appropriate namespace and
then click the Namespace Servers tab.
2. Right-click the appropriate namespace server and then click either Disable
Namespace Server or Enable Namespace Server.

 Tip
 To enable or disable referrals by using Windows PowerShell, use the Set-
DfsnRootTarget –State or Set-DfsnServerConfiguration cmdlets, which were
introduced in Windows Server 2012.

Enable client failback

If a target becomes unavailable, you can configure clients to fail back to the target after
it is restored. For failback to work, client computers must meet the requirements listed
in the following topic: Review DFS Namespaces Client Requirements.

７ Note

To enable client failback on a namespace root by using Windows PowerShell, use

the Set-DfsnRoot cmdlet. To enable client failback on a DFS folder, use the Set-
DfsnFolder cmdlet.

To enable client failback for a namespace root

1. Click Start, point to Administrative Tools, and then click DFS Management.

2. In the console tree, under the Namespaces node, right-click a namespace, and
then click Properties.

3. On the Referrals tab, select the Clients fail back to preferred targets check box.

Folders with targets inherit client failback settings from the namespace root. If client
failback is disabled on the namespace root, you can use the following procedure to
enable the client to fail back on a folder with targets.

To enable client failback for a folder with

targets
1. Click Start, point to Administrative Tools, and then click DFS Management.

2. In the console tree, under the Namespaces node, right-click a folder with targets,
and then click Properties.

3. On the Referrals tab, click the Clients fail back to preferred targets check box.
Additional References
Tuning DFS Namespaces
Review DFS Namespaces Client Requirements
Delegate Management Permissions for DFS Namespaces
Change the amount of time that clients
cache referrals
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

A referral is an ordered list of targets that a client computer receives from a domain
controller or namespace server when the user accesses a namespace root or folder with
targets in the namespace. You can adjust how long clients cache a referral before
requesting a new one.

To change the amount of time that clients

cache namespace root referrals
1. Click Start, point to Administrative Tools, and then click DFS Management.

2. In the console tree, under the Namespaces node, right-click a namespace, and
then click Properties.

3. On the Referrals tab, in the Cache duration (in seconds) text box, type the amount
of time (in seconds) that clients cache namespace root referrals. The default setting
is 300 seconds (five minutes).

 Tip

To change the amount of time that clients cache namespace root referrals by using
Windows PowerShell, use the Set-DfsnRoot TimeToLiveSec cmdlet. These cmdlets
were introduced in Windows Server 2012.

To change the amount of time that clients

cache folder referrals
1. Click Start , point to Administrative Tools, and then click DFS Management.
 2. In the console tree, under the Namespaces node, right-click a folder that has
targets, and then click Properties.

3. On the Referrals tab, in the Cache duration (in seconds) text box, type the amount
of time (in seconds) that clients cache folder referrals. The default setting is 1800
seconds (30 minutes).

Additional References
Tuning DFS Namespaces
Delegate Management Permissions for DFS Namespaces
Set the Ordering Method for Targets in
Referrals
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

A referral is an ordered list of targets that a client computer receives from a domain
controller or namespace server when the user accesses a namespace root or folder with
targets. After the client receives the referral, the client attempts to access the first target
in the list. If the target is not available, the client attempts to access the next target.
Targets on the client's site are always listed first in a referral. Targets outside of the
client's site are listed according to the ordering method.

Use the following sections to specify in what order targets should be referred to clients
and to understand the different methods of ordering target referrals:

To set the ordering method for targets in

namespace root referrals
Use the following procedure to set the ordering method on the namespace root:

1. Click Start, point to Administrative Tools, and then click DFS Management.

2. In the console tree, under the Namespaces node, right-click a namespace, and
then click Properties.

3. On the Referrals tab, select an ordering method.

７ Note

To use Windows PowerShell to set the ordering method for targets in namespace
root referrals, use the Set-DfsnRoot cmdlet with one of the following parameters:

EnableSiteCosting specifies the Lowest cost ordering method

EnableInsiteReferrals specifies the Exclude targets outside of the client's site
ordering method
 Omitting either parameter specifies the Random order referral ordering
method.

The DFSN Windows PowerShell module was introduced in Windows Server 2012.

To set the ordering method for targets in folder

referrals
Folders with targets inherit the ordering method from the namespace root. You can
override the ordering method by using the following procedure:

1. Click Start, point to Administrative Tools, and then click DFS Management.

2. In the console tree, under the Namespaces node, right-click a folder with targets,
and then click Properties.

3. On the Referrals tab, select the Exclude targets outside of the client's site check
box.

７ Note

To use Windows PowerShell to exclude folder targets outside of the client's site, use
the Set-DfsnFolder –EnableInsiteReferrals cmdlet.

Target referral ordering methods

The three ordering methods are:

Random order
Lowest cost
Exclude targets outside of the client's site

Random order
In this method, targets are ordered as follows:

1. Targets in the same Active Directory Directory Services (AD DS) site as the client
are listed in random order at the top of the referral.
2. Targets outside of the client's site are listed in random order.
If no same-site target servers are available, the client computer is referred to a random
target server regardless of how expensive the connection is or how distant the target.

Lowest cost
In this method, targets are ordered as follows:

1. Targets in the same site as the client are listed in random order at the top of the
referral.
2. Targets outside of the client's site are listed in order of lowest cost to highest cost.
Referrals with the same cost are grouped together, and the targets are listed in
random order within each group.

７ Note

Site link costs are not shown in the DFS Management snap-in. To view site link
costs, use the Active Directory Sites and Services snap-in.

Exclude targets outside of the client's site

In this method, the referral contains only the targets that are in the same site as the
client. These same-site targets are listed in random order. If no same-site targets exist,
the client does not receive a referral and cannot access that portion of the namespace.

７ Note

Targets that have target priority set to "First among all targets" or "Last among all
targets" are still listed in the referral, even if the ordering method is set to Exclude
targets outside of the client's site.

Additional References
Tuning DFS Namespaces
Delegate Management Permissions for DFS Namespaces
Set target priority to override referral
ordering
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

A referral is an ordered list of targets that a client computer receives from a domain
controller or namespace server when the user accesses a namespace root or folder with
targets in the namespace. Each target in a referral is ordered according to the ordering
method for the namespace root or folder.

To refine how targets are ordered, you can set priority on individual targets. For
example, you can specify that the target is first among all targets, last among all targets,
or first or last among all targets of equal cost.

To set target priority on a root target for a

domain-based namespace
To set target priority on a root target for a domain-based namespace, use the following
procedure:

1. Click Start, point to Administrative Tools, and then click DFS Management.

2. In the console tree, under the Namespaces node, click the domain-based
namespace for the root targets for which you want to set priority.

3. In the Details pane, on the Namespace Servers tab, right-click the root target with
the priority that you want to change, and then click Properties.

4. On the Advanced tab, click Override referral ordering, and then click the priority
you want.

First among all targets Specifies that users should always be referred to this
target if the target is available.
Last among all targets Specifies that users should never be referred to this
target unless all other targets are unavailable.
First among targets of equal cost Specifies that users should be referred to
this target before other targets of equal cost (which usually means other
 targets in the same site).
Last among targets of equal cost Specifies that users should never be
referred to this target if there are other targets of equal cost available (which
usually means other targets in the same site).

To set target priority on a folder target

To set target priority on a folder target, use the following procedure:

1. Click Start, point to Administrative Tools, and then click DFS Management.

2. In the console tree, under the Namespaces node, click the folder of the targets for
which you want to set priority.

3. In the Details pane, on the Folder Targets tab, right-click the folder target with the
priority that you want to change, and then click Properties .

4. On the Advanced tab, click Override referral ordering and then click the priority
that you want.

７ Note

To set target priorities by using Windows PowerShell, use the Set-DfsnRootTarget

and Set-DfsnFolderTarget cmdlets with the ReferralPriorityClass and
ReferralPriorityRank parameters. These cmdlets were introduced in Windows
Server 2012.

Additional References
Tuning DFS Namespaces
Delegate Management Permissions for DFS Namespaces
Optimize Namespace Polling
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

To maintain a consistent domain-based namespace across namespace servers, it is

necessary for namespace servers to periodically poll Active Directory Domain Services
(AD DS) to obtain the most current namespace data.

To optimize namespace polling

Use the following procedure to optimize how namespace polling occurs:

1. Click Start, point to Administrative Tools, and then click DFS Management.

2. In the console tree, under the Namespaces node, right-click a domain-based

namespace, and then click Properties .

3. On the Advanced tab, select whether you want the namespace optimized for
consistency or scalability.

Choose Optimize for consistency if there are 16 or fewer namespace servers

hosting the namespace.
Choose Optimize for scalability if there are more than 16 namespace servers.
This reduces the load on the Primary Domain Controller (PDC) Emulator, but
increases the time required for changes to the namespace to replicate to all
namespace servers. Until changes replicate to all servers, users might have an
inconsistent view of the namespace.

７ Note

To set the namespace polling mode by using Windows PowerShell, use the Set-
DfsnRoot EnableRootScalability cmdlet, which was introduced in Windows
Server 2012.

Additional References
Tuning DFS Namespaces
Delegate Management Permissions for DFS Namespaces
Enable access-based enumeration on a
namespace
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

Access-based enumeration hides files and folders that users do not have permissions to
access. By default, this feature is not enabled for DFS namespaces. You can enable
access-based enumeration of DFS folders by using DFS Management. To control access-
based enumeration of files and folders in folder targets, you must enable access-based
enumeration on each shared folder by using Share and Storage Management.

To enable access-based enumeration on a namespace, all namespace servers must be

running Windows Server 2008 or newer. Additionally, domain-based namespaces must
use the Windows Server 2008 mode. For information about the requirements of the
Windows Server 2008 mode, see Choose a Namespace Type.

In some environments, enabling access-based enumeration can cause high CPU

utilization on the server and slow response times for users.

７ Note

If you upgrade the domain functional level to Windows Server 2008 while there are
existing domain-based namespaces, DFS Management will allow you to enable
access-based enumeration on these namespaces. However, you will not be able to
edit permissions to hide folders from any groups or users unless you migrate the
namespaces to the Windows Server 2008 mode. For more information, see Migrate
a Domain-based Namespace to Windows Server 2008 Mode.

To use access-based enumeration with DFS Namespaces, you must follow these steps:

Enable access-based enumeration on a namespace

Control which users and groups can view individual DFS folders

２ Warning
 Access-based enumeration does not prevent users from getting a referral to a
folder target if they already know the DFS path. Only the share permissions or the
NTFS file system permissions of the folder target (shared folder) itself can prevent
users from accessing a folder target. DFS folder permissions are used only for
displaying or hiding DFS folders, not for controlling access, making Read access the
only relevant permission at the DFS folder level. For more information, see Using
Inherited Permissions with Access-Based Enumeration

You can enable access-based enumeration on a namespace either by using the Windows
interface or by using a command line.

To enable access-based enumeration by using

the Windows interface
1. In the console tree, under the Namespaces node, right-click the appropriate
namespace and then click Properties .

2. Click the Advanced tab and then select the Enable access-based enumeration for
this namespace check box.

To enable access-based enumeration by using a

command line
1. Open a command prompt window on a server that has the Distributed File System
role service or Distributed File System Tools feature installed.

2. Type the following command, where <namespace_root> is the root of the

namespace:

dfsutil property abe enable \\ <namespace_root>

 Tip

To manage access-based enumeration on a namespace by using Windows

PowerShell, use the Set-DfsnRoot, Grant-DfsnAccess, and Revoke-DfsnAccess
cmdlets. The DFSN Windows PowerShell module was introduced in Windows Server
2012.
You can control which users and groups can view individual DFS folders either by using
the Windows interface or by using a command line.

To control folder visibility by using the

Windows interface
1. In the console tree, under the Namespaces node, locate the folder with targets for
which you want to control visibility, right-click it and then click Properties.

2. Click the Advanced tab.

3. Click Set explicit view permissions on the DFS folder and then Configure view
permissions.

4. Add or remove groups or users by clicking Add or Remove.

5. To allow users to see the DFS folder, select the group or user, and then select the
Allow check box.

To hide the folder from a group or user, select the group or user, and then select
the Deny check box.

To control folder visibility by using a command

line
1. Open a Command Prompt window on a server that has the Distributed File
System role service or Distributed File System Tools feature installed.

2. Type the following command, where <DFSPath> is the path of the DFS folder
(link), <DOMAIN\Account> is the name of the group or user account, and (...) is
replaced with additional Access Control Entries (ACEs):

dfsutil property sd grant <DFSPath> DOMAIN\Account:R (...) Protect

Replace

For example, to replace existing permissions with permissions that allows the
Domain Admins and CONTOSO\Trainers groups Read (R) access to the
\contoso.office\public\training folder, type the following command:
 dfsutil property sd grant \\contoso.office\public\training
"CONTOSO\Domain Admins":R CONTOSO\Trainers:R Protect Replace

3. To perform additional tasks from the command prompt, use the following
commands:

Command Description

Dfsutil property sd deny Denies a group or user the ability to view the folder.

Dfsutil property sd reset Removes all permissions from the folder.

Dfsutil property sd revoke Removes a group or user ACE from the folder.

Additional References
Create a DFS Namespace
Delegate Management Permissions for DFS Namespaces
Installing DFS
Using Inherited Permissions with Access-Based Enumeration
Using inherited permissions with
Access-based Enumeration
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

By default, the permissions used for a DFS folder are inherited from the local file system
of the namespace server. The permissions are inherited from the root directory of the
system drive and grant the DOMAIN\Users group Read permissions. As a result, even
after enabling access-based enumeration, all folders in the namespace remain visible to
all domain users.

Advantages and limitations of inherited

permissions
There are two primary benefits to using inherited permissions to control which users can
view folders in a DFS namespace:

You can quickly apply inherited permissions to many folders without having to use
scripts.
You can apply inherited permissions to namespace roots and folders without
targets.

Despite the benefits, inherited permissions in DFS Namespaces have many limitations
that make them inappropriate for most environments:

Modifications to inherited permissions are not replicated to other namespace

servers. Therefore, use inherited permissions only on stand-alone namespaces or in
environments where you can implement a third-party replication system to keep
the Access Control Lists (ACLs) on all namespace servers synchronized.
DFS Management and Dfsutil cannot view or modify inherited permissions.
Therefore, you must use Windows Explorer or the Icacls command in addition to
DFS Management or Dfsutil to manage the namespace.
When using inherited permissions, you cannot modify the permissions of a folder
with targets except by using the Dfsutil command. DFS Namespaces automatically
removes permissions from folders with targets set using other tools or methods.
 If you set permissions on a folder with targets while you are using inherited
permissions, the ACL that you set on the folder with targets combines with
inherited permissions from the folder's parent in the file system. You must examine
both sets of permissions to determine what the net permissions are.

７ Note

When using inherited permissions, it is simplest to set permissions on namespace

roots and folders without targets. Then use inherited permissions on folders with
targets so that they inherit all permissions from their parents.

Using inherited permissions

To limit which users can view a DFS folder, you must perform one of the following tasks:

Set explicit permissions for the folder, disabling inheritance. To set explicit
permissions on a folder with targets (a link) using DFS Management or the Dfsutil
command, see Enable Access-Based Enumeration on a Namespace.
Modify inherited permissions on the parent in the local file system. To modify the
permissions inherited by a folder with targets, if you have already set explicit
permissions on the folder, switch to inherited permissions from explicit
permissions, as discussed in the following procedure. Then use Windows Explorer
or the Icacls command to modify the permissions of the folder from which the
folder with targets inherits its permissions.

７ Note

Access-based enumeration does not prevent users from obtaining a referral to a

folder target if they already know the DFS path of the folder with targets.
Permissions set using Windows Explorer or the Icacls command on namespace
roots or folders without targets control whether users can access the DFS folder or
namespace root. However, they do not prevent users from directly accessing a
folder with targets. Only the share permissions or the NTFS file system permissions
of the shared folder itself can prevent users from accessing folder targets.

To switch from explicit permissions to inherited

permissions
 1. In the console tree, under the Namespaces node, locate the folder with targets
whose visibility you want to control, right-click the folder and then click Properties.

2. Click the Advanced tab.

3. Click Use inherited permissions from the local file system and then click OK in the
Confirm Use of Inherited Permissions dialog box. Doing this removes all explicitly
set permissions on this folder, restoring inherited NTFS permissions from the local
file system of the namespace server.

4. To change the inherited permissions for folders or namespace roots in a DFS

namespace, use Windows Explorer or the ICacls command.

Additional References
Create a DFS Namespace
DFS Replication overview
Article • 03/28/2023

Applies To: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

Distributed File System Replication, or DFS Replication, is a role service in Windows

Server that enables you to efficiently replicate folders across multiple servers and sites.
You can replicate all types of folders, including folders referred to by a DFS namespace
path.

DFS Replication is an efficient, multiple-master replication engine that you can use to
keep folders synchronized between servers across limited bandwidth network
connections. The service replaces the File Replication Service (FRS) as the replication
engine for DFS namespaces.

 Tip

Consider using Azure File Sync to reduce your on-premises storage footprint.
Azure File Sync can keep multiple Windows file servers in sync. Each server needs to
only keep a cache on-premises while the full copy of the data is in the cloud. Azure
File Sync also provides the benefit of cloud backups with integrated snapshots. For
more information, see Planning for an Azure File Sync deployment.

DFS Replication uses a compression algorithm known as remote differential compression,

or RDC. RDC detects changes to the data in a file and enables DFS Replication to
replicate only the changed file blocks instead of the entire file.

Active Directory Domain Services (AD DS) uses DFS Replication to replicate the sysvol
folder in domains that use the Windows Server 2008 or later domain functional level. For
more information about replicating the sysvol folder by using DFS Replication, see
Migrate the sysvol folder replication to DFS Replication.

Understand replication groups

To use DFS Replication, you create replication groups and add replicated folders to the
groups. Replicated folders are stored on servers in the group, which are referred to as
members. DFS Replication establishes connections between the members of a group.
The following figure illustrates the relationship between a replication group, the
members in the group, and the replicated folders.

A replicated folder stays synchronized on each member in a group. In the figure, there
are two replicated folders: Projects and Proposals. As the data changes in each
replicated folder, the changes are replicated across connections between the members
of the replication group. The connections between all members form the replication
topology.

Creating multiple replicated folders in a single replication group simplifies the process
of deploying replicated folders. The topology, schedule, and bandwidth throttling for
the replication group are applied to each replicated folder. To deploy more replicated
folders, you can run the Dfsradmin.exe tool or use a wizard to define the local path and
permissions for the new replicated folder.

Each replicated folder has unique settings, such as file and subfolder filters. The settings
let you filter out different files and subfolders for each replicated folder.

The replicated folders stored on each member can be located on different volumes in
the member, and the replicated folders don't need to be shared folders or part of a
namespace. However, the DFS Management snap-in simplifies sharing replicated folders
and optionally publishing them in an existing namespace.

Deploy and manage DFS Replication

DFS Replication is a part of the File and Storage Services role for Windows Server. The
management tools for DFS (DFS Management, the DFS Replication module for Windows
PowerShell, and command-line tools) are installed separately as part of the Remote
Server Administration Tools (RSAT).

You can install DFS Replication by using Server Manager, Windows PowerShell, or
Windows Admin Center.
You can administer DFS Replication by using DFS Management, the dfsradmin and
dfsrdiag commands, or scripts that call WMI.

Deployment requirements
Before you can deploy DFS Replication, you must configure your servers as follows:

Confirm file system and volume format. Determine the folders that you want to
replicate, and identify any folders located on volumes that are formatted with the
NTFS file system. DFS Replication doesn't support the Resilient File System (ReFS)
or the FAT file system. DFS Replication also doesn't support replicating content
stored on Cluster Shared Volumes.

Verify antivirus compatibility. Contact your antivirus software vendor to confirm

your antivirus software is compatible with DFS Replication.

Update AD DS schema. Update the AD DS schema to include Windows Server

2003 R2 or later schema additions. You can't use read-only replicated folders with
the Windows Server 2003 R2 or later schema additions.

Prepare replication group servers. Install DFS Replication on all servers that you
plan to use as members of a replication group.

Check forest location. Ensure all servers in a replication group are located in the
same forest. You can't enable replication across servers in different forests.

Interoperability with Azure virtual machines

DFS Replication on an Azure virtual machine is a verified scenario for Windows Server.
However, there are some limitations and requirements for this implementation.

Snapshots and saved states. To restore a server that's running DFS Replication,
don't use snapshots or saved states to replicate anything other than the sysvol
folder. If you attempt this type of restore, DFS Replication fails. This restoration
requires special database recovery steps. Also, don't export, clone, or copy the
virtual machines. For more information, see DFSR no longer replicates files after
restoring a virtualized server's snapshot and Safely virtualizing DFSR .

DFS Replication backups. To back up data in a replicated folder that's stored in a

virtual machine, use backup software that's located on the guest virtual machine.
Don't back up or restore a virtualized DFS Replication server from the host virtual
machine.
 Domain controller access. DFS Replication requires access to physical or
virtualized domain controllers. The DFS Replication service can't communicate
directly with Azure Active Directory.

VPN connection. DFS Replication requires a VPN connection between your on-
premises replication group members and any members hosted in Azure virtual
machines. You also need to configure the on-premises router (such as Forefront
Threat Management Gateway) to allow the RPC Endpoint Mapper (port 135) and a
randomly assigned port between 49152 and 65535 to pass over the VPN
connection. You can use the Set-DfsrMachineConfiguration cmdlet or the dfsrdiag
command-line tool to specify a static port instead of the random port. For more
information about how to specify a static port for DFS Replication, see Set-
DfsrServiceConfiguration. For information about related ports to open for
managing Windows Server, see Service overview and network port requirements
for Windows.

To learn how to get started with Azure virtual machines, visit the Microsoft Azure
website.

Install DFS Replication from Server Manager

To install DFS Replication by using Server Manager, follow these steps:

1. Open Server Manager.

2. Select Manage, and then select Add Roles and Features. The Add Roles and
Features wizard opens.

3. Under Server Selection, select the server or virtual hard disk (VHD) where you want
to install DFS Replication. The server or VHD should be an offline virtual machine.

4. To install the DFS Replication service, go to Server Roles.

5. Expand File and Storage Services > File and iSCSI Services, and then select DFS
Replication.

6. To install the DFS Management Tools, go to Features.

a. Expand Remote Server Administration Tools, Role Administration Tools, and

then expand File Services Tools.

b. Select DFS Management Tools.

 The DFS Management Tools option installs the DFS Management snap-in, the DFS
Replication and DFS Namespaces modules for Windows PowerShell, and
command-line tools. The option doesn't install any DFS services on the server.

Install DFS Replication from PowerShell

To install the DFS Replication by using Windows PowerShell, follow these steps:

1. Open a Windows PowerShell session with elevated user rights.

2. Enter the following command to install the desired RSAT role services or features
to support DFS replication.

For the <name\> parameter, enter of the names of the RSAT role services or
features that you want to install. You can install one or multiple services and
features in a single command. The table lists the names of the relevant RSAT role
services and features.

PowerShell

Install-WindowsFeature <name>

RSAT role service/feature Value for <name> parameter

DFS Replication FS-DFS-Replication

DFS Management Tools RSAT-DFS-Mgmt-Con

To install the DFS Replication service only, enter the following command:

PowerShell

Install-WindowsFeature "RSAT-DFS-Mgmt-Con"

To install both the DFS Replication service and the DFS Management Tools,
enter the following command:

PowerShell

Install-WindowsFeature "FS-DFS-Replication", "RSAT-DFS-Mgmt-Con"

Related links
DFS Namespaces and DFS Replication overview
Checklist: Deploy DFS Replication
Checklist: Manage DFS Replication
Deploying DFS Replication
Troubleshooting DFS Replication
Migrate SYSVOL replication to DFS
Replication
Article • 04/25/2023

Applies To: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and
Windows Server 2008

Domain controllers use a special shared folder named SYSVOL to replicate sign-in
scripts and Group Policy object files to other domain controllers. Windows 2000 Server
and Windows Server 2003 use the File Replication Service (FRS) to replicate SYSVOL.
Windows Server 2008 uses the newer Distributed File System Replication (DFS
Replication) service for domains that use the Windows Server 2008 domain functional
level. Windows Server 2008 uses FRS for domains that run older domain functional
levels.

To use DFS Replication to replicate the SYSVOL folder, you can create a new domain that
uses the Windows Server 2008 domain functional level. You can also use the procedure
described in this article to upgrade an existing domain and migrate replication to DFS
Replication.

Prerequisites
This article assumes you have a basic knowledge of Active Directory Domain Services
(AD DS), FRS, and Distributed File System Replication (DFS Replication). For more
information, see:

Active Directory Domain Services overview

FRS overview
Overview of DFS Replication

Printable download
To download a printable version of this guide, go to SYSVOL Replication Migration
Guide: FRS to DFS Replication .

Migration topics
The SYSVOL migration guide provides topics that describe a range of concepts and
procedures from the use of FRS to the use DFS. Use the following list to access articles
about migrating the SYSVOL folder to use DFS Replication.

Concepts
Review these concepts about SYSVOL migration states for a basic understanding of the
migration tasks.

SYSVOL migration conceptual information

SYSVOL migration states
Overview of the SYSVOL migration procedure

Procedures
Follow these SYSVOL migration procedures for a basic understanding of the migration
states.

SYSVOL migration procedure

Migrating to the Prepared state
Migrating to the Redirected state
Migrating to the Eliminated state

Troubleshooting
Access these articles that describe known issues and provide troubleshooting guidance.

Troubleshooting SYSVOL migration

Troubleshooting SYSVOL migration issues
Rolling back SYSVOL migration to a previous stable state

References
The following resources offer supplemental reference information:

SYSVOL migration reference information

Supported SYSVOL migration scenarios
Verifying the state of SYSVOL migration
Dfsrmig
SYSVOL migration tool actions
Related links
SYSVOL Migration Series: Part 1 – Introduction to the SYSVOL migration process
SYSVOL Migration Series: Part 2 – Dfsrmig.exe: The SYSVOL migration tool
SYSVOL Migration Series: Part 3 - Migrating to the 'PREPARED' state
SYSVOL Migration Series: Part 4 – Migrating to the 'REDIRECTED' state
SYSVOL Migration Series: Part 5 – Migrating to the 'ELIMINATED' state
Distributed File System step-by-step guide for Windows Server 2008
FRS technical reference
Use Robocopy to pre-seed files for DFS
Replication
Article • 12/23/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

This topic explains how to use the command-line tool, Robocopy.exe, to pre-seed files
when setting up replication for Distributed File System (DFS) Replication (also known as
DFSR or DFS-R) in Windows Server. By pre-seeding files before you set up DFS
Replication, add a new replication partner, or replace a server, you can speed up initial
synchronization and enable cloning of the DFS Replication database in Windows Server
2012 R2. The Robocopy method is one of several pre-seeding methods; for an overview,
see Step 1: pre-seed files for DFS Replication.

The Robocopy (Robust File Copy) command-line utility is included with Windows Server.
The utility provides extensive options that include copying security, backup API support,
retry capabilities, and logging. Later versions include multi-threading and un-buffered
I/O support.

） Important

Robocopy does not copy exclusively locked files. If users tend to lock many files for
long periods on your file servers, consider using a different pre-seeding method.
pre-seeding does not require a perfect match between file lists on the source and
destination servers, but the more files that do not exist when initial synchronization
is performed for DFS Replication, the less effective pre-seeding is. To minimize lock
conflicts, use Robocopy during non-peak hours for your organization. Always
examine the Robocopy logs after pre-seeding to ensure that you understand which
files were skipped because of exclusive locks.

To use Robocopy to pre-seed files for DFS Replication, follow these steps:

1. Download and install the latest version of Robocopy.

2. Stabilize files that will be replicated.
3. Copy the replicated files to the destination server.
Prerequisites
Because pre-seeding does not directly involve DFS Replication, you only need to meet
the requirements for performing a file copy with Robocopy.

You need an account that's a member of the local Administrators group on both
the source and destination servers.

Install the most recent version of Robocopy on the server that you will use to copy
the files—either the source server or the destination server; you will need to install
the most recent version for the operating system version. For instructions, see Step
2: Stabilize files that will be replicated. Unless you are pre-seeding files from a
server running Windows Server 2003 R2, you can run Robocopy on either the
source or destination server. The destination server, which typically has the more
recent operating system version, gives you access to the most recent version of
Robocopy.

Ensure that sufficient storage space is available on the destination drive. Do not
create a folder on the path that you plan to copy to: Robocopy must create the
root folder.

７ Note

When you decide how much space to allocate for the pre-seeded files,
consider expected data growth over time and storage requirements for DFS
Replication. For planning help, see Edit the Quota Size of the Staging Folder
and Conflict and Deleted Folder in Managing DFS Replication.

On the source server, optionally install Process Monitor or Process Explorer, which
you can use to check for applications that are locking files. For download
information, see Process Monitor and Process Explorer.

Step 1: Download and install the latest version

of Robocopy
Before you use Robocopy to pre-seed files, you should download and install the latest
version of Robocopy.exe. This ensures that DFS Replication doesn't skip files because of
issues within Robocopy's shipping versions.

The source for the latest compatible Robocopy version depends on the version of
Windows Server that is running on the server. For information about downloading the
hotfix with the most recent version of Robocopy for Windows Server 2008 R2 or
Windows Server 2008, see List of currently available hotfixes for Distributed File System
(DFS) technologies in Windows Server 2008 and in Windows Server 2008 R2 .

Alternatively, you can locate and install the latest hotfix for an operating system by
taking the following steps.

Locate and install the latest version of Robocopy for a

specific version of Windows Server
1. In a web browser, open https://support.microsoft.com .

2. In Search Support, enter the following string, replacing <operating system

version> with the appropriate operating system, then press the Enter key:

robocopy.exe kbqfe "<operating system version>"

For example, enter robocopy.exe kbqfe "Windows Server 2008 R2".

3. Locate and download the hotfix with the highest ID number (that is, the latest
version).

4. Install the hotfix on the server.

Step 2: Stabilize files that will be replicated

After you install the latest version of Robocopy on the server, you should prevent locked
files from blocking copying by using the methods described in the following table. Most
applications do not exclusively lock files. However, during normal operations, a small
percentage of files might be locked on file servers.

Source of Explanation Mitigation

the lock
 Source of Explanation Mitigation
the lock

Users Employees connect to Only perform Robocopy operations during off-peak,

remotely a standard file server non-business hours. This minimizes the number of files
open files and edit documents, that Robocopy must skip during pre-seeding.
on shares. multimedia content, or
other files. Sometimes Consider temporarily setting Read-only access on the file
referred to as the shares that will be replicated by using the Windows
traditional home folder PowerShell Grant-SmbShareAccess and Close-
or shared data SmbSession cmdlets. If you set permissions for a
workloads. common group such as Everyone or Authenticated Users
to READ, standard users might be less likely to open files
with exclusive locks (if their applications detect the Read-
only access when files are opened).

You might also consider setting a temporary firewall rule

for SMB port 445 inbound to that server to block access
to files or use the Block-SmbShareAccess cmdlet.
However, both of these methods are very disruptive to
user operations.

Applications Application workloads Temporarily disable or uninstall the applications that are
open files running on a file server locking files. You can use Process Monitor or Process
local. sometimes lock files. Explorer to determine which applications are locking
files. To download Process Monitor or Process Explorer,
visit the Process Monitor and Process Explorer pages.

Step 3: Copy the replicated files to the

destination server
After you minimize locks on the files that will be replicated, you can pre-seed the files
from the source server to the destination server.

７ Note

You can run Robocopy on either the source computer or the destination computer.
The following procedure describes running Robocopy on the destination server,
which typically is running a more recent operating system, to take advantage of any
additional Robocopy capabilities that the more recent operating system might
provide.
pre-seed the replicated files onto the destination server
with Robocopy
1. Sign in to the destination server with an account that's a member of the local
Administrators group on both the source and destination servers.

2. Open an elevated command prompt.

3. To pre-seed the files from the source to destination server, run the following
command, substituting your own source, destination, and log file paths for the
bracketed values:

PowerShell

robocopy "<source replicated folder path>" "<destination replicated

folder path>" /e /b /copyall /r:6 /w:5 /MT:64 /xd DfsrPrivate /tee
/log:<log file path> /v

This command copies all contents of the source folder to the destination folder,
with the following parameters:

Parameter Description

"<source Specifies the source folder to pre-seed on the destination server.

replicated
folder path>"

"<destination Specifies the path to the folder that will store the pre-seeded files.
replicated
folder path>" The destination folder must not already exist on the destination server. To
get matching file hashes, Robocopy must create the root folder when it
pre-seeds the files.

/e Copies subdirectories and their files, as well as empty subdirectories.

/b Copies files in Backup mode.

/copyall Copies all file information, including data, attributes, time stamps, the
NTFS access control list (ACL), owner information, and auditing
information.

/r:6 Retries the operation six times when an error occurs.

/w:5 Waits 5 seconds between retries.

MT:64 Copies 64 files simultaneously.

/xd DfsrPrivate Excludes the DfsrPrivate folder.

 Parameter Description

/tee Writes status output to the console window, as well as to the log file.

/log <log file Specifies the log file to write. Overwrites the file's existing contents. (To
path> append the entries to the existing log file, use /log+ <log file path> .)

/v Produces verbose output that includes skipped files.

For example, the following command replicates files from the source replicated
folder, E:\RF01, to data drive D on the destination server:

PowerShell

robocopy.exe "\\srv01\e$\rf01" "d:\rf01" /e /b /copyall /r:6 /w:5

/MT:64 /xd DfsrPrivate /tee /log:c:\temp\pre-seedsrv02.log

７ Note

We recommend that you use the parameters described above when you use
Robocopy to pre-seed files for DFS Replication. However, you can change
some of their values or add additional parameters. For example, you might
find out through testing that you have the capacity to set a higher value
(thread count) for the /MT parameter. Also, if you'll primarily replicate larger
files, you might be able to increase copy performance by adding the /j option
for unbuffered I/O. For more information about Robocopy parameters, see
the Robocopy command-line reference.

２ Warning

To avoid potential data loss when you use Robocopy to pre-seed files for DFS
Replication, do not make the following changes to the recommended
parameters:

Do not use the /mir parameter (that mirrors a directory tree) or the /mov
parameter (that moves the files, then deletes them from the source).
Do not remove the /e, /b, and /copyall options.

4. After copying completes, examine the log for any errors or skipped files. Use
Robocopy to copy any skipped files individually instead of recopying the entire set
of files. If files were skipped because of exclusive locks, either try copying
 individual files with Robocopy later, or accept that those files will require over-the-
wire replication by DFS Replication during initial synchronization.

Next step
After you complete the initial copy, and use Robocopy to resolve issues with as many
skipped files as possible, you will use the Get-DfsrFileHash cmdlet in Windows
PowerShell or the Dfsrdiag command to validate the pre-seeded files by comparing file
hashes on the source and destination servers. For detailed instructions, see Step 2:
Validate pre-seeded Files for DFS Replication.
DFS Replication FAQ
FAQ

Updated: January 30, 2019

Applies To: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
Server 2008

Try our Virtual Agent - It can help you quickly identify and fix common File

replication issues.

This FAQ answers questions about Distributed File System (DFS) Replication (also known
as DFS-R or DFSR) for Windows Server.

For information about DFS Namespaces, see DFS Namespaces: Frequently Asked
Questions.

For information about what's new in DFS Replication, see the following topics:

DFS Namespaces and DFS Replication Overview (in Windows Server 2012)

What's New in Distributed File System topic in Changes in Functionality from

Windows Server 2008 to Windows Server 2008 R2

Distributed File System topic in Changes in Functionality from Windows Server

2003 with SP1 to Windows Server 2008

For a list of recent changes to this topic, see the Change history section of this topic.

Interoperability
Can DFS Replication communicate with FRS?
No. DFS Replication does not communicate with File Replication Service (FRS). DFS
Replication and FRS can run on the same server at the same time, but they must never
be configured to replicate the same folders or subfolders because doing so can cause
data loss.

Can DFS Replication replace FRS for SYSVOL

replication
Yes, DFS Replication can replace FRS for SYSVOL replication on servers running Windows
Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, or Windows Server
2008. Servers running Windows Server 2003 R2 don't support using DFS Replication to
replicate the SYSVOL folder.

For more information about replicating SYSVOL by using DFS Replication, see the
Migrate SYSVOL replication to DFS Replication.

Can I upgrade from FRS to DFS Replication

without losing configuration settings?
Yes. To migrate replication from FRS to DFS Replication, see the following documents:

To migrate replication of folders other than the SYSVOL folder, see DFS Operations
Guide: Migrating from FRS to DFS Replication and FRS2DFSR – An FRS to DFSR
Migration Utility (https://go.microsoft.com/fwlink/?LinkID=195437 ).

To migrate replication of the SYSVOL folder to DFS Replication, see Migrate

SYSVOL replication to DFS Replication.

Can I use DFS Replication in a mixed

Windows/UNIX environment?
Yes. Although DFS Replication only supports replicating content between servers
running Windows Server, UNIX clients can access file shares on the Windows servers. To
do so, install Services for Network File Systems (NFS) on the DFS Replication server.

You can also use the SMB/CIFS client functionality included in many UNIX clients to
directly access the Windows file shares, although this functionality is often limited or
requires modifications to the Windows environment (such as disabling SMB Signing by
using Group Policy).

DFS Replication interoperates with NFS on a server running a Windows Server operating
system, but you can't replicate an NFS mount point.

Can I use the Volume Shadow Copy Service with

DFS Replication?
Yes. DFS Replication is supported on Volume Shadow Copy Service (VSS) volumes and
previous snapshots can be restored successfully with the Previous Versions Client.
Can I use Windows Backup (Ntbackup.exe) to
remotely back up a replicated folder?
No, using Windows Backup (Ntbackup.exe) on a computer running Windows
Server 2003 or earlier to back up the contents of a replicated folder on a computer
running Windows Server 2012, Windows Server 2008 R2, or Windows Server 2008 isn't
supported.

To back up files that are stored in a replicated folder, use Windows Server Backup or
Microsoft® System Center Data Protection Manager. For information about Backup and
Recovery functionality in Windows Server 2008 R2 and Windows Server 2008, see
Backup and Recovery. For more information, see System Center Data Protection
Manager (https://go.microsoft.com/fwlink/?LinkId=182261 ).

Do file system policies impact DFS Replication?

Yes. Don't configure file system policies on replicated folders. The file system policy
reapplies NTFS permissions at every Group Policy refresh interval. This can result in
sharing violations because an open file isn't replicated until the file is closed.

Does DFS Replication replicate mailboxes hosted

on Microsoft Exchange Server?
No. DFS Replication can't be used to replicate mailboxes hosted on Microsoft Exchange
Server.

Does DFS Replication support file screens

created by File Server Resource Manager?
Yes. However, the File Server Resource Manager (FSRM) file screening settings must
match on both ends of the replication. In addition, DFS Replication has its own filter
mechanism for files and folders that you can use to exclude certain files and file types
from replication.

The following are best practices for implementing file screens or quotas:

The hidden DfsrPrivate folder must not be subject to quotas or file screens.

Screened files must not exist in any replicated folder before screening is enabled.

No folders may exceed the quota before the quota is enabled.

 You must use hard quotas with caution. It's possible for individual members of a
replication group to stay within a quota before replication, but exceed it when files
are replicated. For example, if a user copies a 10 megabyte (MB) file onto server A
(which is then at the hard limit) and another user copies a 5 MB file onto server B,
when the next replication occurs, both servers will exceed the quota by 5
megabytes. This can cause DFS Replication to continually retry replicating the files,
causing holes in the version vector and possible performance problems.

Is DFS Replication cluster aware?

Yes, DFS Replication in Windows Server 2012 R2, Windows Server 2012 and Windows
Server 2008 R2 includes the ability to add a failover cluster as a member of a replication
group. For more information, see Add a Failover Cluster to a Replication Group
(https://go.microsoft.com/fwlink/?LinkId=155085 ). The DFS Replication service on
versions of Windows prior to Windows Server 2008 R2 isn't designed to coordinate with
a failover cluster, and the service won't fail over to another node.

７ Note

DFS Replication doesn't support replicating files on Cluster Shared Volumes.

Is DFS Replication compatible with Data

Deduplication?
Yes, DFS Replication can replicate folders on volumes that use Data Deduplication in
Windows Server.

Is DFS Replication compatible with RIS and

WDS?
Yes. DFS Replication replicates volumes on which Single Instance Storage (SIS) is
enabled. SIS is used by Remote Installation Services (RIS), Windows Deployment Services
(WDS), and Windows Storage Server.

Is it possible to use DFS Replication with Offline

Files?
You can safely use DFS Replication and Offline Files together in scenarios when there's
only one user at a time who writes to the files. This is useful for users who travel
between two branch offices and want to be able to access their files at either branch or
while offline. Offline Files caches the files locally for offline use and DFS Replication
replicates the data between each branch office.

Don't use DFS Replication with Offline Files in a multi-user environment because DFS
Replication doesn't provide any distributed locking mechanism or file checkout
capability. If two users modify the same file at the same time on different servers, DFS
Replication moves the older file to the DfsrPrivate\ConflictandDeleted folder (located
under the local path of the replicated folder) during the next replication.

What antivirus applications are compatible with

DFS Replication?
Antivirus applications can cause excessive replication if their scanning activities alter the
files in a replicated folder. For more information, Testing Antivirus Application
Interoperability with DFS Replication (https://go.microsoft.com/fwlink/?
LinkId=73990 ).

What are the benefits of using DFS Replication

instead of Windows SharePoint Services?
Windows® SharePoint® Services provides tight coherency in the form of file check-out
functionality that DFS Replication doesn't. If you're concerned about multiple people
editing the same file, we recommend using Windows SharePoint Services. Windows
SharePoint Services 2.0 with Service Pack 2 is available as part of Windows
Server 2003 R2. Windows SharePoint Services can be downloaded from the Microsoft
Web site; it isn't included in newer versions of Windows Server. However, if you're
replicating data across multiple sites and users won't edit the same files at the same
time, DFS Replication provides greater bandwidth and simpler management.

Limitations and requirements

Can DFS Replication replicate between branch
offices without a VPN connection?
Yes—assuming that there's a private Wide Area Network (WAN) link (not the Internet)
connecting the branch offices. However, you must open the proper ports in external
firewalls. DFS Replication uses the RPC Endpoint Mapper (port 135) and a randomly
assigned ephemeral port above 1024. You can use the Dfsrdiag command line tool to
specify a static port instead of the ephemeral port. For more information about how to
specify the RPC Endpoint Mapper, see article 154596 in the Microsoft Knowledge Base
(https://go.microsoft.com/fwlink/?LinkId=73991 ).

Can DFS Replication replicate files encrypted

with the Encrypting File System?
No. DFS Replication won't replicate files or folders that are encrypted using the
Encrypting File System (EFS). If a user encrypts a file that was previously replicated, DFS
Replication deletes the file from all other members of the replication group. This ensures
that the only available copy of the file is the encrypted version on the server.

Can DFS Replication replicate Outlook .pst or

Microsoft Office Access database files?
DFS Replication can safely replicate Microsoft Outlook personal folder files (.pst) and
Microsoft Access files only if they are stored for archival purposes and are not accessed
across the network by using a client such as Outlook or Access (to open .pst or Access
files, first copy the files to a local storage device). The reasons for this are as follows:

Opening .pst files over network connections could lead to data corruption in the
.pst files. For more information about why .pst files cannot be safely accessed from
across a network, see article 297019 in the Microsoft Knowledge Base
(https://go.microsoft.com/fwlink/?LinkId=125363 ).

.pst and Access files tend to stay open for long periods of time while being
accessed by a client such as Outlook or Office Access. This prevents DFS
Replication from replicating these files until they are closed.

Can I use DFS Replication in a workgroup?

No. DFS Replication relies on Active Directory® Domain Services for configuration. It will
only work in a domain.

Can more than one folder be replicated on a

single server?
Yes. DFS Replication can replicate numerous folders between servers. Ensure that each of
the replicated folders has a unique root path and that they do not overlap. For example,
D:\Sales and D:\Accounting can be the root paths for two replicated folders, but D:\Sales
and D:\Sales\Reports cannot be the root paths for two replicated folders.

Does DFS Replication require DFS Namespaces?

No. DFS Replication and DFS Namespaces can be used separately or together. In
addition, DFS Replication can be used to replicate standalone DFS namespaces, which
was not possible with FRS.

Does DFS Replication require time

synchronization between servers?
No. DFS Replication does not explicitly require time synchronization between servers.
However, DFS Replication does require that the server clocks match closely. The server
clocks must be set within five minutes of each other (by default) for Kerberos
authentication to function properly. For example, DFS Replication uses time stamps to
determine which file takes precedence in the event of a conflict. Accurate times are also
important for garbage collection, schedules, and other features.

Does DFS Replication support replicating an

entire volume?
Yes. However, replicating an entire volume can cause the following problems:

If the volume contains a Windows paging file, replication fails and logs DFSR event
4312 in the system event log.

DFS Replication sets the System and Hidden attributes on the replicated folder on
the destination server(s). This occurs because Windows applies the System and
Hidden attributes to the volume root folder by default. If the local path of the
replicated folder on the destination server(s) is also a volume root, no further
changes are made to the folder attributes.

When replicating a volume that contains the Windows system folder, DFS
Replication recognizes the %WINDIR% folder and does not replicate it. However,
DFS Replication does replicate folders used by non-Microsoft applications, which
might cause the applications to fail on the destination server(s) if the applications
have interoperability issues with DFS Replication.

Does DFS Replication support RPC over HTTP?

No.

Does DFS Replication work across wireless

networks?
Yes. DFS Replication is independent of the connection type.

Does DFS Replication work on ReFS or FAT

volumes?
No. DFS Replication supports volumes formatted with the NTFS file system only; the
Resilient File System (ReFS) and the FAT file system are not supported. DFS Replication
requires NTFS because it uses the NTFS change journal and other features of the NTFS
file system.

Does DFS Replication work with sparse files?

Yes. You can replicate sparse files. The Sparse attribute is preserved on the receiving
member.

Do I need to log in as administrator to replicate

files?
No. DFS Replication is a service that runs under the local system account, so you do not
need to log in as administrator to replicate. However, you must be a domain
administrator or local administrator of the affected file servers to make changes to the
DFS Replication configuration.

For more information, see "DFS Replication security requirements and delegation" in the
Delegate the Ability to Manage DFS Replication (https://go.microsoft.com/fwlink/?
LinkId=182294 ).

How can I upgrade or replace a DFS Replication

member?
To upgrade or replace a DFS Replication member, see this blog post on the Ask the
Directory Services Team blog: Replacing DFSR Member Hardware or OS.
Is DFS Replication suitable for replicating
roaming profiles?
Yes. Certain scenarios are supported when replicating roaming user profiles. For
information about the supported scenarios, see Microsoft's Support Statement Around
Replicated User Profile Data (https://go.microsoft.com/fwlink/?LinkId=201282 ).

Is there a file character limit or limit to the folder

depth?
Windows and DFS Replication support folder paths with up to 32 thousand characters.
DFS Replication is not limited to folder paths of 260 characters.

Must members of a replication group reside in

the same domain?
No. Replication groups can span across domains within a single forest but not across
different forests.

What are the supported limits of DFS

Replication?
The following list provides a set of scalability guidelines that have been tested by
Microsoft and apply to Windows Server 2012 R2, Windows Server 2016, and Windows
Server 2019

Size of all replicated files on a server: 100 terabytes.

Number of replicated files on a volume: 70 million.

Maximum file size: 250 gigabytes.

） Important

When creating replication groups with a large number or size of files we

recommend exporting a database clone and using pre-seeding techniques to
minimize the duration of initial replication. For more information, see DFS
Replication Initial Sync in Windows Server 2012 R2: Attack of the Clones .
The following list provides a set of scalability guidelines that have been tested by
Microsoft on Windows Server 2012, Windows Server 2008 R2, and Windows
Server 2008:

Size of all replicated files on a server: 10 terabytes.

Number of replicated files on a volume: 11 million.

Maximum file size: 64 gigabytes.

７ Note

There is no longer a limit to the number of replication groups, replicated folders,

connections, or replication group members.

For a list of scalability guidelines that have been tested by Microsoft for Windows
Server 2003 R2, see DFS Replication scalability guidelines
(https://go.microsoft.com/fwlink/?LinkId=75043 ).

When should I not use DFS Replication?

Do not use DFS Replication in an environment where multiple users update or modify
the same files simultaneously on different servers. Doing so can cause DFS Replication
to move conflicting copies of the files to the hidden DfsrPrivate\ConflictandDeleted
folder.

When multiple users need to modify the same files at the same time on different
servers, use the file check-out feature of Windows SharePoint Services to ensure that
only one user is working on a file. Windows SharePoint Services 2.0 with Service Pack 2
is available as part of Windows Server 2003 R2. Windows SharePoint Services can be
downloaded from the Microsoft Web site; it is not included in newer versions of
Windows Server.

Why is a schema update required for DFS

Replication?
DFS Replication uses new objects in the domain-naming context of Active Directory
Domain Services to store configuration information. These objects are created when you
update the Active Directory Domain Services schema. For more information, see Review
Requirements for DFS Replication (https://go.microsoft.com/fwlink/?LinkId=182264 ).
Monitoring and management tools
Can I automate the health report to receive
warnings?
Yes. There are three ways to automate health reports:

Use the DFSR Windows PowerShell module included in Windows Server 2012 R2 or
DfsrAdmin.exe in conjunction with Scheduled Tasks to regularly generate health
reports. For more information, see Automating DFS Replication Health Reports
(https://go.microsoft.com/fwlink/?LinkId=74010 ).

Use the DFS Replication Management Pack for System Center Operations Manager
to create alerts that are based on specified conditions.

Use the DFS Replication WMI provider to script alerts.

Can I use Microsoft System Center Operations

Manager to monitor DFS Replication?
Yes. For more information, see the DFS Replication Management Pack for System Center
Operations Manager 2007 in the Microsoft Download Center
(https://go.microsoft.com/fwlink/?LinkId=182265 ).

Does DFS Replication support remote

management?
Yes. DFS Replication supports remote management using the DFS Management console
and the Add Replication Group command. For example, on server A, you can connect to
a replication group defined in the forest with servers A and B as members.

DFS Management is included with Windows Server 2012 R2, Windows Server 2012,
Windows Server 2008 R2, Windows Server 2008, and Windows Server 2003 R2. To
manage DFS Replication from other versions of Windows, use Remote Desktop or the
Remote Server Administration Tools for Windows 7.

） Important

To view or manage replication groups that contain read-only replicated folders or

members that are failover clusters, you must use the version of DFS Management
 that is included with Windows Server 2012 R2, Windows Server 2012, Windows
Server 2008 R2, the Remote Server Administration Tools for Windows 8 , or the
Remote Server Administration Tools for Windows 7.

Do Ultrasound and Sonar work with DFS

Replication?
No. DFS Replication has its own set of monitoring and diagnostics tools. Ultrasound and
Sonar are only capable of monitoring FRS.

How can files be recovered from the

ConflictAndDeleted or PreExisting folders?
To recover lost files, restore the files from the file system folder or shared folder using
File History, the Restore previous versions command in File Explorer, or by restoring the
files from backup. To recover files directly from the ConflictAndDeleted or PreExisting
folder, use the Get-DfsrPreservedFiles and Restore-DfsrPreservedFiles Windows
PowerShell cmdlets (included with the DFSR module in Windows Server 2012 R2), or the
RestoreDFSR sample script from the MSDN Code Gallery. This script is intended only
for disaster recovery and is provided AS-IS, without warranty.

Is there a way to know the state of replication?

Yes. There are a number of ways to monitor replication:

DFS Replication has a management pack for System Center Operations Manager
that provides proactive monitoring.

DFS Management has an in-box diagnostic report for the replication backlog,
replication efficiency, and the number of files and folders in a given replication
group.

The DFSR Windows PowerShell module in Windows Server 2012 R2 contains

cmdlets for starting propagation tests and writing propagation and health reports.
For more information, see Distributed File System Replication Cmdlets in Windows
PowerShell.

Dfsrdiag.exe is a command-line tool that can generate a backlog count or trigger a

propagation test. Both show the state of replication. Propagation shows you if files
are being replicated to all nodes. Backlog shows you how many files still need to
 replicate before two computers are in sync. The backlog count is the number of
updates that a replication group member has not processed. On computers
running Windows Server 2012 R2, Windows Server 2012 or Windows
Server 2008 R2, Dfsrdiag.exe can also display the updates that DFS Replication is
currently replicating.

Scripts can use WMI to collect backlog information—manually or through MOM.

Performance
Does DFS Replication support dial-up
connections?
Although DFS Replication will work at dial-up speeds, it can get backlogged if there are
large numbers of changes to replicate. If small changes are made to existing files, DFS
Replication with Remote Differential Compression (RDC) will provide a much higher
performance than copying the file directly.

Does DFS Replication perform bandwidth

sensing?
No. DFS Replication does not perform bandwidth sensing. You can configure DFS
Replication to use a limited amount of bandwidth on a per-connection basis (bandwidth
throttling). However, DFS Replication does not further reduce bandwidth utilization if
the network interface becomes saturated, and DFS Replication can saturate the link for
short periods. Bandwidth throttling with DFS Replication is not completely accurate
because DFS Replication throttles bandwidth by throttling RPC calls. As a result, various
buffers in lower levels of the network stack (including RPC) may interfere, causing bursts
of network traffic.

Does DFS Replication throttle bandwidth per

schedule, per server, or per connection?
If you configure bandwidth throttling when specifying the schedule, all connections for
that replication group will use that setting for bandwidth throttling. Bandwidth
throttling can be also set as a connection-level setting using DFS Management.
Does DFS Replication use Active Directory
Domain Services to calculate site links and
connection costs?
No. DFS Replication uses the topology defined by the administrator, which is
independent of Active Directory Domain Services site costing.

How can I improve replication performance?

To learn about different methods of tuning replication performance, see Tuning
Replication Performance in DFSR on the Ask the Directory Services Team blog.

How does DFS Replication avoid saturating a

connection?
In DFS Replication you set the maximum bandwidth you want to use on a connection,
and the service maintains that level of network usage. This is different from the
Background Intelligent Transfer Service (BITS), and DFS Replication does not saturate the
connection if you set it appropriately.

Nonetheless, the bandwidth throttling is not 100% accurate and DFS Replication can
saturate the link for short periods of time. This is because DFS Replication throttles
bandwidth by throttling RPC calls. Because this process relies on various buffers in lower
levels of the network stack, including RPC, the replication traffic tends to travel in bursts
which may at times saturate the network links.

DFS Replication in Windows Server 2008 includes several performance enhancements, as

discussed in Distributed File System, a topic in Changes in Functionality from Windows
Server 2003 with SP1 to Windows Server 2008.

How does DFS Replication performance compare

with FRS?
DFS Replication is much faster than FRS, particularly when small changes are made to
large files and RDC is enabled. For example, with RDC, a small change to a 2 MB
PowerPoint® presentation can result in only 60 kilobytes (KB) being sent across the
network—a 97 percent savings in bytes transferred.

RDC is not used on files smaller than 64 KB and might not be beneficial on high-speed
LANs where network bandwidth is not contended. RDC can be disabled on a per-
connection basis using DFS Management.

How frequently does DFS Replication replicate

data?
Data replicates according to the schedule you set. For example, you can set the schedule
to 15-minute intervals, seven days a week. During these intervals, replication is enabled.
Replication starts soon after a file change is detected (generally within seconds).

The replication group schedule may be set to Universal Time Coordinate (UTC) while the
connection schedule is set to the local time of the receiving member. Take this into
account when the replication group spans multiple time zones. Local time means the
time of the member hosting the inbound connection. The displayed schedule of the
inbound connection and the corresponding outbound connection reflect time zone
differences when the schedule is set to local time.

How much of my server's system resources will

DFS Replication consume?
The disk, memory, and CPU resources used by DFS Replication depend on a number of
factors, including the number and size of the files, rate of change, number of replication
group members, and number of replicated folders. In addition, some resources are
harder to estimate. For example, the Extensible Storage Engine (ESE) technology used
for the DFS Replication database can consume a large percentage of available memory,
which it releases on demand. Applications other than DFS Replication can be hosted on
the same server depending on the server configuration. However, when hosting multiple
applications or server roles on a single server, it is important that you test this
configuration before implementing it in a production environment.

What happens if a WAN link fails during

replication?
If the connection goes down, DFS Replication will keep trying to replicate while the
schedule is open. There will also be connectivity errors noted in the DFS Replication
event log that can be harvested using MOM (proactively through alerts) and the DFS
Replication Health Report (reactively, such as when an administrator runs it).

Remote Differential Compression details

What is RDC?
Remote differential compression (RDC) is a client-server protocol that can be used to
efficiently update files over a limited-bandwidth network. RDC detects insertions,
removals, and rearrangements of data in files, enabling DFS Replication to replicate only
the changes when files are updated. RDC is used only for files that are 64 KB or larger by
default. RDC can use an older version of a file with the same name in the replicated
folder or in the DfsrPrivate\ConflictandDeleted folder (located under the local path of
the replicated folder).

When is RDC used for replication?

RDC is used when the file exceeds a minimum size threshold. This size threshold is 64 KB
by default. After a file exceeding that threshold has been replicated, updated versions of
the file always use RDC, unless a large portion of the file is changed or RDC is disabled.

Which editions of the Windows operating

system support cross-file RDC?
To use cross-file RDC, one member of the replication connection must be running an
edition of the Windows operating system that supports cross-file RDC. The following
table shows which editions of the Windows operating system support cross-file RDC.

Cross-file RDC availability in editions of the

Windows operating system

Operating System Standard Enterprise Datacenter

Version Edition Edition Edition

Windows Server 2012 Yes Not available Yes

R2

Windows Server 2012 Yes Not available Yes

Windows Server 2008 No Yes Yes

R2

Windows Server 2008 No Yes No

 Operating System Standard Enterprise Datacenter
Version Edition Edition Edition

Windows Server 2003 No Yes No

R2

* You can optionally disable cross-file RDC on Windows Server 2012 R2.

Are changes compressed before being

replicated?
Yes. Changed portions of files are compressed before being sent for all file types except
the following (which are already compressed): .wma, .wmv, .zip, .jpg, .mpg, .mpeg, .m1v,
.mp2, .mp3, .mpa, .cab, .wav, .snd, .au, .asf, .wm, .avi, .z, .gz, .tgz, and .frx. Compression
settings for these file types are not configurable in Windows Server 2003 R2.

Can an administrator turn off RDC or change the

threshold?
Yes. You can turn off RDC through the property page of a given connection. Disabling
RDC can reduce CPU utilization and replication latency on fast local area network (LAN)
links that have no bandwidth constraints or for replication groups that consist primarily
of files smaller than 64 KB. If you choose to disable RDC on a connection, test the
replication efficiency before and after the change to verify that you have improved
replication performance.

You can change the RDC size threshold by using the Dfsradmin Connection Set
command, the DFS Replication WMI Provider, or by manually editing the configuration
XML file.

Does RDC work on all file types?

Yes. RDC computes differences at the block level irrespective of file data type. However,
RDC works more efficiently on certain file types such as Word docs, PST files, and VHD
images.

How does RDC work on a compressed file?

DFS Replication uses RDC, which computes the blocks in the file that have changed and
sends only those blocks over the network. DFS Replication does not need to know
anything about the contents of the file—only which blocks have changed.

Is cross-file RDC enabled when upgrading to

Windows Server Enterprise Edition or Datacenter
Edition?
The Standard Editions of Windows Server do not support cross-file RDC. However, it is
automatically enabled when you upgrade to an edition that supports cross-file RDC, or
if a member of the replication connection is running a supported edition. For a list of
editions that support cross-file RDC, see Which editions of the Windows operating
system support cross-file RDC?

Is RDC true block-level replication?

No. RDC is a general purpose protocol for compressing file transfer. DFS Replication
uses RDC on blocks at the file level, not at the disk block level. RDC divides a file into
blocks. For each block in a file, it calculates a signature, which is a small number of bytes
that can represent the larger block. The set of signatures is transferred from server to
client. The client compares the server signatures to its own. The client then requests the
server send only the data for signatures that are not already on the client.

What happens if I rename a file?

DFS Replication renames the file on all other members of the replication group during
the next replication. Files are tracked using a unique ID, so renaming a file and moving
the file within the replica has no effect on the ability of DFS Replication to replicate a
file.

What is cross-file RDC?

Cross-file RDC allows DFS Replication to use RDC even when a file with the same name
does not exist at the client end. Cross-file RDC uses a heuristic to determine files that
are similar to the file that needs to be replicated, and uses blocks of the similar files that
are identical to the replicating file to minimize the amount of data transferred over the
WAN. Cross-file RDC can use blocks of up to five similar files in this process.

To use cross-file RDC, one member of the replication connection must be running an
edition of Windows that supports cross-file RDC. For a list of editions that support
cross-file RDC, see Which editions of the Windows operating system support cross-file
RDC?
Replication details
Can I change the path for a replicated folder
after it is created?
No. If you need to change the path of a replicated folder, you must delete it in DFS
Management and add it back as a new replicated folder. DFS Replication then uses
Remote Differential Compression (RDC) to perform a synchronization that determines
whether the data is the same on the sending and receiving members. It does not
replicate all the data in the folder again.

Can I configure which file attributes are

replicated?
No, you cannot configure which file attributes that DFS Replication replicates.

For a list of attribute values and their descriptions, see File Attributes on MSDN
(https://go.microsoft.com/fwlink/?LinkId=182268 ).

The following attribute values are set by using the SetFileAttributes dwFileAttributes
function, and they are replicated by DFS Replication. Changes to these attribute values
trigger replication of the attributes. The contents of the file are not replicated unless the
contents change as well. For more information, see SetFileAttributes Function in the
MSDN library (https://go.microsoft.com/fwlink/?LinkId=182269 ).

FILE_ATTRIBUTE_HIDDEN

FILE_ATTRIBUTE_READONLY

FILE_ATTRIBUTE_SYSTEM

FILE_ATTRIBUTE_NOT_CONTENT_INDEXED

FILE_ATTRIBUTE_OFFLINE

The following attribute values are replicated by DFS Replication, but they do not trigger
replication.

FILE_ATTRIBUTE_ARCHIVE

FILE_ATTRIBUTE_NORMAL
The following file attribute values also trigger replication, although they cannot be set
by using the SetFileAttributes function (use the GetFileAttributes function to view
the attribute values).

FILE_ATTRIBUTE_REPARSE_POINT

７ Note

DFS Replication does not replicate reparse point attribute values unless the reparse
tag is IO_REPARSE_TAG_SYMLINK. Files with the IO_REPARSE_TAG_DEDUP,
IO_REPARSE_TAG_SIS or IO_REPARSE_TAG_HSM reparse tags are replicated as
normal files. However, the reparse tag and reparse data buffers are not replicated
to other servers because the reparse point only works on the local system.

FILE_ATTRIBUTE_COMPRESSED

FILE_ATTRIBUTE_ENCRYPTED

７ Note

DFS Replication does not replicate files that are encrypted by using the Encrypting
File System (EFS). DFS Replication does replicate files that are encrypted by using
non-Microsoft software, but only if it does not set the FILE_ATTRIBUTE_ENCRYPTED
attribute value on the file.

FILE_ATTRIBUTE_SPARSE_FILE

FILE_ATTRIBUTE_DIRECTORY

DFS Replication does not replicate the FILE_ATTRIBUTE_TEMPORARY value.

Can I control which member is replicated?

Yes. You can choose a topology when you create a replication group. Or you can select
No topology and manually configure connections after the replication group has been
created.

Can I seed a replication group member with data

prior to the initial replication?
Yes. DFS Replication supports copying files to a replication group member before the
initial replication. This "prestaging" can dramatically reduce the amount of data
replicated during the initial replication.

The initial replication does not need to replicate contents when files differ only by real
attributes or time stamps. A real attribute is an attribute that can be set by the Win32
function SetFileAttributes . For more information, see SetFileAttributes Function in the
MSDN library (https://go.microsoft.com/fwlink/?LinkId=182269 ). If two files differ by
other attributes, such as compression, then the contents of the file are replicated.

To prestage a replication group member, copy the files to the appropriate folder on the
destination server(s), create the replication group, and then choose a primary member.
Choose the member that has the most up-to-date files that you want to replicate
because the primary member's content is considered "authoritative." This means that
during initial replication, the primary member's files will always overwrite other versions
of the files on other members of the replication group.

For information about pre-seeding and cloning the DFSR database, see DFS Replication
Initial Sync in Windows Server 2012 R2: Attack of the Clones .

For more information about the initial replication, see Create a Replication Group.

Does DFS Replication overcome common File

Replication Service issues?
Yes. DFS Replication overcomes three common FRS issues:

Journal wraps: DFS Replication recovers from journal wraps on the fly. Each existing
file or folder will be marked as journalWrap and verified against the file system
before replication is enabled again. During the recovery, this volume is not
available for replication in either direction.

Excessive replication: To prevent excessive replication, DFS Replication uses a

system of credits.

Morphed folders: To prevent morphed folder names, DFS Replication stores

conflicting data in a hidden DfsrPrivate\ConflictandDeleted folder (located under
the local path of the replicated folder). For example, creating multiple folders
simultaneously with identical names on different servers replicated using FRS
causes FRS to rename the older folder(s). DFS Replication instead moves the older
folder(s) to the local Conflict and Deleted folder.
Does DFS Replication replicate files in
chronological order?
No. Files may be replicated out of order.

Does DFS Replication replicate files that are

being used by another application?
If an application opens a file and creates a file lock on it (preventing it from being used
by other applications while it is open), DFS Replication will not replicate the file until it is
closed. If the application opens the file with read-share access, the file can still be
replicated.

Does DFS Replication replicate NTFS file

permissions, alternate data streams, hard links,
and reparse points?
DFS Replication replicates NTFS file permissions and alternate data streams.

Microsoft does not support creating NTFS hard links to or from files in a replicated
folder – doing so can cause replication issues with the affected files. Hard link files
are ignored by DFS Replication and are not replicated. Junction points also are not
replicated, and DFS Replication logs event 4406 for each junction point it
encounters.

The only reparse points replicated by DFS Replication are those that use the
IO_REPARSE_TAG_SYMLINK tag; however, DFS Replication does not guarantee that
the target of a symlink is also replicated. For more information, see the Ask the
Directory Services Team blog.

Files with the IO_REPARSE_TAG_DEDUP, IO_REPARSE_TAG_SIS, or

IO_REPARSE_TAG_HSM reparse tags are replicated as normal files. The reparse tag
and reparse data buffers are not replicated to other servers because the reparse
point only works on the local system. As such, DFS Replication can replicate folders
on volumes that use Data Deduplication in Windows Server 2012, or Single
Instance Storage (SIS), however, data deduplication information is maintained
separately by each server on which the role service is enabled.
Does DFS Replication replicate timestamp
changes if no other changes are made to the
file?
No, DFS Replication does not replicate files for which the only change is a change to the
timestamp. Additionally, the changed timestamp is not replicated to other members of
the replication group unless other changes are made to the file.

Does DFS Replication replicate updated

permissions on a file or folder?
Yes. DFS Replication replicates permission changes for files and folders. Only the part of
the file associated with the Access Control List (ACL) is replicated, although DFS
Replication must still read the entire file into the staging area.

７ Note

Changing ACLs on a large number of files can have an impact on replication

performance. However, when using RDC, the amount of data transferred is
proportionate to the size of the ACLs, not the size of the entire file. The amount of
disk traffic is still proportional to the size of the files because the files must be read
to and from the staging folder.

Does DFS Replication support merging text files

in the event of a conflict?
DFS Replication does not merge files when there is a conflict. However, it does attempt
to preserve the older version of the file in the hidden DfsrPrivate\ConflictandDeleted
folder on the computer where the conflict was detected.

Does DFS Replication use encryption when

transmitting data?
Yes. DFS Replication uses Remote Procedure Call (RPC) connections with encryption.

Is it possible to disable the use of encrypted

RPC?
No. The DFS Replication service uses remote procedure calls (RPC) over TCP to replicate
data. To secure data transfers across the Internet, the DFS Replication service is designed
to always use the authentication-level constant, RPC_C_AUTHN_LEVEL_PKT_PRIVACY . This
ensures that the RPC communication across the Internet is always encrypted. Therefore,
it is not possible to disable the use of encrypted RPC by the DFS Replication service.

For more information, see the following Microsoft Web sites:

RPC Technical Reference

About Remote Differential Compression

Authentication-Level Constants

How are simultaneous replications handled?

There is one update manager per replicated folder. Update managers work
independently of one another.

By default, a maximum of 16 (four in Windows Server 2003 R2) concurrent downloads

are shared among all connections and replication groups. Because connections and
replication group updates are not serialized, there is no specific order in which updates
are received. If two schedules are opened, updates are generally received and installed
from both connections at the same time.

How do I force replication or polling?

You can force replication immediately by using DFS Management, as described in Edit
Replication Schedules. You can also force replication by using the Sync-
DfsReplicationGroup cmdlet, included in the DFSR PowerShell module introduced with

Windows Server 2012 R2, or the Dfsrdiag SyncNow command. You can force polling by
using the Update-DfsrConfigurationFromAD cmdlet, or the Dfsrdiag PollAD command.

Is it possible to configure a quiet time between

replications for files that change frequently?
No. If the schedule is open, DFS Replication will replicate changes as it notices them.
There is no way to configure a quiet time for files.

Is it possible to configure one-way replication

with DFS Replication?
Yes. If you are using Windows Server 2012 or Windows Server 2008 R2, you can create a
read-only replicated folder that replicates content through a one-way connection. For
more information, see Make a Replicated Folder Read-Only on a Particular Member
(https://go.microsoft.com/fwlink/?LinkId=156740 ).

We do not support creating a one-way replication connection with DFS Replication in

Windows Server 2008 or Windows Server 2003 R2. Doing so can cause numerous
problems including health-check topology errors, staging issues, and problems with the
DFS Replication database.

If you are using Windows Server 2008 or Windows Server 2003 R2, you can simulate a
one-way connection by performing the following actions:

Train administrators to make changes only on the server(s) that you want to
designate as primary servers. Then let the changes replicate to the destination
servers.

Configure the share permissions on the destination servers so that end users do
not have Write permissions. If no changes are allowed on the branch servers, then
there is nothing to replicate back, simulating a one-way connection and keeping
WAN utilization low.

Is there a way to force a complete replication of

all files including unchanged files?
No. If DFS Replication considers the files identical, it will not replicate them. If changed
files have not been replicated, DFS Replication will automatically replicate them when
configured to do so. To overwrite the configured schedule, use the WMI method
ForceReplicate(). However, this is only a schedule override, and it does not force
replication of unchanged or identical files.

What happens if the primary member suffers a

database loss during initial replication?
During initial replication, the primary member's files will always take precedence in the
conflict resolution that occurs if the receiving members have different versions of files
on the primary member. The primary member designation is stored in Active Directory
Domain Services, and the designation is cleared after the primary member is ready to
replicate, but before all members of the replication group replicate.

If the initial replication fails or the DFS Replication service restarts during the replication,
the primary member sees the primary member designation in the local DFS Replication
database and retries the initial replication. If the primary member's DFS Replication
database is lost after clearing the primary designation in Active Directory Domain
Services, but before all members of the replication group complete the initial replication,
all members of the replication group fail to replicate the folder because no server is
designated as the primary member. If this happens, use the Dfsradmin membership
/set /isprimary:true command on the primary member server to restore the primary
member designation manually.

For more information about initial replication, see Create a Replication Group.

２ Warning

The primary member designation is used only during the initial replication process.
If you use the Dfsradmin command to specify a primary member for a replicated
folder after replication is complete, DFS Replication does not designate the server
as a primary member in Active Directory Domain Services. However, if the DFS
Replication database on the server subsequently suffers irreversible corruption or
data loss, the server attempts to perform an initial replication as the primary
member instead of recovering its data from another member of the replication
group. Essentially, the server becomes a rogue primary server, which can cause
conflicts. For this reason, specify the primary member manually only if you are
certain that the initial replication has irretrievably failed.

What happens if the replication schedule closes

while a file is being replicated?
If remote differential compression (RDC) is enabled on the connection, inbound
replication of a file larger than 64 KB that began replicating immediately prior to the
schedule closing (or changing to No bandwidth) continues when the schedule opens (or
changes to something other than No bandwidth). The replication continues from the
state it was in when replication stopped.

If RDC is turned off, DFS Replication completely restarts the file transfer. This can delay
when the file is available on the receiving member.

What happens when two users simultaneously

update the same file on different servers?
When DFS Replication detects a conflict, it uses the version of the file that was saved
last. It moves the other file into the DfsrPrivate\ConflictandDeleted folder (under the
local path of the replicated folder on the computer that resolved the conflict). It remains
there until Conflict and Deleted folder cleanup, which occurs when the Conflict and
Deleted folder exceeds the configured size or DFS Replication encounters an Out of disk
space error. The Conflict and Deleted folder is not replicated, and this method of conflict
resolution avoids the problem of morphed directories that was possible in FRS.

When a conflict occurs, DFS Replication logs an informational event to the DFS
Replication event log. This event does not require user action for the following reasons:

It is not visible to users (it is visible only to server administrators).

DFS Replication treats the Conflict and Deleted folder as a cache. When a quota
threshold is reached, it cleans out some of those files. There is no guarantee that
conflicting files will be saved.

The conflict could reside on a server different from the origin of the conflict.

Staging
Does DFS Replication continue staging files
when replication is disabled by a schedule or
bandwidth throttling quota, or when a
connection is manually disabled?
No. DFS Replication does not continue to stage files outside of scheduled replication
times, if the bandwidth throttling quota has been exceeded, or when connections are
disabled.

Does DFS Replication prevent other applications

from accessing a file during staging?
No. DFS Replication opens files in a way that does not block users or applications from
opening files in the replication folder. This method is known as "opportunistic locking."

Is it possible to change the location of the

staging folder with the DFS Management Tool?
Yes. The staging folder location is configured on the Advanced tab of the Properties
dialog box for each member of a replication group.
When are files staged?
Files are staged on the sending member when the receiving member requests the file
(unless the file is 64 KB or smaller) as shown in the following table. If Remote Differential
Compression (RDC) is disabled on the connection, the file is staged unless it is 256 KB or
smaller. Files are also staged on the receiving member as they are transferred if they are
less than 64 KB in size, although you can configure this setting between 16 KB and 1 MB.
If the schedule is closed, files are not staged.

The minimum file sizes for staging files

RDC enabled RDC disabled

Sending member 64 KB 256 KB

Receiving member 64 KB by default 64 KB by default

What happens if a file is changed after it is

staged but before it is completely transmitted to
the remote site?
If any part of the file is already being transmitted, DFS Replication continues the
transmission. If the file is changed before DFS Replication begins transmitting the file,
then the newer version of the file is sent.

Change history
Date Description Reason

November Updated for Windows Server 2019. New operating system.

15, 2018

October Updated the What are the supported limits Updates for the latest
9th, 2013 of DFS Replication? section with results version of Windows Server
from tests on Windows Server 2012 R2.
Date Description Reason

January Added the Does DFS Replication continue Customer questions

30th, 2013 staging files when replication is disabled by
a schedule or bandwidth throttling quota,
or when a connection is manually disabled?
entry.

October Edited the What are the supported limits of Customer feedback
31st, 2012 DFS Replication? entry to increase the
tested number of replicated files on a
volume.

August 15, Edited the Does DFS Replication replicate Feedback from Customer
2012 NTFS file permissions, alternate data Support Services
streams, hard links, and reparse points?
entry to further clarify how DFS Replication
handles hard links and reparse points.

June 13, Edited the Does DFS Replication work on Customer feedback
2012 ReFS or FAT volumes? entry to add
discussion of ReFS.

April 25, Edited the Does DFS Replication replicate Reduce potential
2012 NTFS file permissions, alternate data confusion
streams, hard links, and reparse points?
entry to clarify how DFS Replication handles
hard links.

March 30, Edited the Can DFS Replication replicate Customer questions about
2011 Outlook .pst or Microsoft Office Access the previous entry, which
database files? entry to correct the incorrectly indicated that
potential impact of using DFS Replication replicating .pst or Access
with .pst and Access files. Added How can I files could corrupt the DFS
improve replication performance? Replication database.

January Added How can files be recovered from the Customer feedback
26, 2011 ConflictAndDeleted or PreExisting folders?

October Added How can I upgrade or replace a DFS Customer feedback

20, 2010 Replication member?
How to determine the minimum staging
area DFSR needs for a replicated folder
Article • 04/28/2023

This article is a quick reference guide on how to calculate the minimum staging area
needed for DFSR to function properly. Values lower than these may cause replication to
go slowly or stop altogether.

Keep in mind these are minimums only. When considering staging area size, the bigger
the staging area the better, up to the size of the Replicated Folder. See the section "How
to determine if you have a staging area problem" and the blog posts linked at the end
of this article for more details on why it is important to have a properly sized staging
area.

General guidance
The staging area quota must be as large as the 32 largest files in the Replicated Folder.

Initial Replication will make much more use of the staging area than day-to-day
replication. Setting the staging area higher than the minimum during initial replication is
strongly encouraged if you have the drive space available.

How do you find these X largest files?

Use a PowerShell script to find the 32 or 9 largest files and determine how many
gigabytes they add up to. Before beginning, enable maximum path length support, first
added in Windows Server 2016 using Maximum Path Length Limitation

1. Run the following command:

Powershell

Get-ChildItem c:\\temp -recurse | Sort-Object length -descending |

select-object -first 32 | ft name,length -wrap –auto

This command will return the file names and the size of the files in bytes. Useful if
you want to know what 32 files are the largest in the Replicated Folder so you can
“visit” their owners.

2. Run the following command:

 Powershell

Get-ChildItem c:\\temp -recurse | Sort-Object length -descending |

select-object -first 32 | measure-object -property length –sum

This command will return the total number of bytes of the 32 largest files in the
folder without listing the file names.

3. Run the following command:

Powershell

$big32 = Get-ChildItem c:\\temp -recurse | Sort-Object length -

descending | select-object -first 32 | measure-object -property length
–sum

$big32.sum /1gb

This command will get the total number of bytes of 32 largest files in the folder
and do the math to convert bytes to gigabytes for you. This command is two
separate lines. You can paste both them into the PowerShell command shell at
once or run them back to back.

Manual Walkthrough
Running command 1 will return results similar to the output below. This example only
uses 16 files for brevity. Always use 32 for Windows 2008 and later operating systems.

Example Data returned by PowerShell

Name Length

File5.zip 10286089216

archive.zip 6029853696

BACKUP.zip 5751522304

file9.zip 5472683008

MENTOS.zip 5241586688

File7.zip 4321264640

file2.zip 4176765952
 frd2.zip 4176765952

BACKUP.zip 4078994432

File44.zip 4058424320

file11.zip 3858056192

Backup2.zip 3815138304

BACKUP3.zip 3815138304

Current.zip 3576931328

Backup8.zip 3307488256

File999.zip 3274982400

How to use this data to determine the minimum staging

area size:
Name = Name of the file.
Length = bytes
One Gigabyte = 1073741824 Bytes

First, sum the total number of bytes. Next divide the total by 1073741824. Microsoft
Excel is an easy way to do this.

Example
From the example above the total number of bytes = 75241684992. To get the
minimum staging area quota needed you need to divide 75241684992 by 1073741824.

75241684992 / 1073741824 = 70.07 GB

Based on this data you would set my staging area to 71 GB if you round up to the
nearest whole number.

Real World Scenario:

While a manual walkthrough is interesting it is likely not the best use of your time to do
the math yourself. To automate the process, use command 3 from the examples above.
The results will look like this
Using the example command 3 without any extra effort except for rounding to the
nearest whole number, you can determine that you need a 6 GB staging area quota for
d:\docs.

Do you need to reboot or restart the DFSR

service for the changes to be picked Up?
Changes to the staging area quota do not require a reboot or restart of the service to
take effect. You will need to wait on AD replication and DFSR’s AD polling cycle for the
changes to be applied.

How to determine if you have a staging area

problem
You detect staging area problems by monitoring for specific events IDs on your DFSR
servers. The list of events is 4202, 4204, 4206, 4208 and 4212. The texts of these events
are listed below. It is important to distinguish between 4202 and 4204 and the other
events. It is possible to log a high number of 4202 and 4204 events under normal
operating conditions.

Staging Area Events

Event ID: 4202 Severity: Warning

The DFS Replication service has detected that the staging space in use for the
replicated folder at local path (path) is above the high watermark. The service will
attempt to delete the oldest staging files. Performance may be affected.

Event ID: 4204 Severity: Informational

The DFS Replication service has successfully deleted old staging files for the
replicated folder at local path (path). The staging space is now below the high
watermark.

Event ID: 4206 Severity: Warning

The DFS Replication service failed to clean up old staging files for the replicated
folder at local path (path). The service might fail to replicate some large files and the
replicated folder might get out of sync. The service will automatically retry staging
space cleanup in (x) minutes. The service may start cleanup earlier if it detects some
staging files have been unlocked.
 Event ID: 4208 Severity: Warning

The DFS Replication service detected that the staging space usage is above the
staging quota for the replicated folder at local path (path). The service might fail to
replicate some large files and the replicated folder might get out of sync. The
service will attempt to clean up staging space automatically.

Event ID: 4212 Severity: Error

The DFS Replication service could not replicate the replicated folder at local path
(path) because the staging path is invalid or inaccessible.

What is the difference between 4202 and 4208?

Events 4202 and 4208 have similar text; i.e. DFSR detected the staging area usage
exceeds the high watermark. The difference is that 4208 is logged after staging area
cleanup has run and the staging quota is still exceeded. 4202 is a normal and expected
event whereas 4208 is abnormal and requires intervention.

How many 4202, 4204 events are too many?

There is no single answer to this question. Unlike 4206, 4208 or 4212 events, which are
always bad and indicate action is needed, 4202 and 4204 events occur under normal
operating conditions. Seeing many 4202 and 4204 events may indicate a problem.
Things to consider:

1. Is the Replicated Folder (RF) logging 4202 performing initial replication? If so, it is
normal to log 4202 and 4204 events. You will want to keep these to down to as few
as possible during Initial Replication by providing as much staging area as possible
2. Simply checking the total number of 4202 events is not sufficient. You have to
know how many were logged per RF. If you log twenty 4202 events for one RF in a
24 hour period that is high. However if you have 20 Replicated Folders and there is
one event per folder, you are doing well.
3. You should examine several days of data to establish trends.

We usually counsel customers to allow no more than one 4202 event per Replicated
Folder per day under normal operating conditions. “Normal” meaning no Initial
Replication is occurring. We base this on the reasoning that:

1. Time spent cleaning up the staging area is time spent not replicating files.
Replication is paused while the staging area is cleared.
 2. DFSR benefits from a full staging area using it for RDC and cross-file RDC or
replicating the same files to other members
3. The more 4202 and 4204 events you log the greater the odds you will run into the
condition where DFSR cannot clean up the staging area or will have to prematurely
purge files from the staging area.
4. 4206, 4208 and 4212 events are, in my experience, always preceded and followed
by a high number of 4202 and 4204 events.

While allowing for only one 4202 event per RF per day is conservative, it greatly
decreases your odds of running into staging area problems and better utilizes your
DFSR server’s resources for the intended purpose of replicating files.
Understanding (the Lack of) Distributed
File Locking in DFSR
Article • 06/21/2022

This article discusses the absence of a multi-host distributed file locking mechanism
within Windows, and specifically within folders replicated by DFSR.

Some Background
Distributed File Locking – this refers to the concept of having multiple copies of a
file on several computers and when one file is opened for writing, all other copies
are locked. This prevents a file from being modified on multiple servers at the
same time by several users.
Distributed File System Replication – DFSR operates in a multi-master, state-based
design. In state-based replication, each server in the multi-master system applies
updates to its replica as they arrive, without exchanging log files (it instead uses
version vectors to maintain “up-to-dateness” information). No one server is ever
arbitrarily authoritative after initial sync, so it is highly available and very flexible on
various network topologies.
Server Message Block - SMB is the common protocol used in Windows for
accessing files over the network. In simplified terms, it's a client-server protocol
that makes use of a redirector to have remote file systems appear to be local file
systems. It is not specific to Windows and is quite common – a well known non-
Microsoft example is Samba, which allows Linux, Mac, and other operating systems
to act as SMB clients/servers and participate in Windows networks.

It's important to make a clear delineation of where DFSR and SMB live in your replicated
data environment. SMB allows users to access their files, and it has no awareness of
DFSR. Likewise, DFSR (using the RPC protocol) keeps files in sync between servers and
has no awareness of SMB. Don't confuse distributed locking as defined in this post and
Opportunistic Locking.

So here's where things can go pear-shaped, as the Brits say.

Since users can modify data on multiple servers, and since each Windows server only
knows about a file lock on itself, and since DFSR doesn't know anything about those
locks on other servers, it becomes possible for users to overwrite each other's changes.
DFSR uses a “last writer wins” conflict algorithm, so someone has to lose and the person
to save last gets to keep their changes. The losing file copy is chucked into the
ConflictAndDeleted folder.
Now, this is far less common than people like to believe. Typically, true shared files are
modified in a local environment; in the branch office or in the same row of cubicles.
They are usually worked on by people on the same team, so people are generally aware
of colleagues modifying data. And since they are usually in the same site, the odds are
much higher that all the users working on a shared doc will be using the same server.
Windows SMB handles the situation here. When a user has a file locked for modification
and his coworker tries to edit it, the other user will get an error like:

And if the application opening the file is really clever, like Word 2007, it might give you:

DFSR does have a mechanism for locked files, but it is only within the server's own
context. DFSR will not replicate a file in or out if its local copy has an exclusive lock. But
this doesn't prevent anyone on another server from modifying the file.

Back on topic, the issue of shared data being modified geographically does exist, and for
some folks it's pretty gnarly. We're occasionally asked why DFSR doesn't handle this
locking and take of everything with a wave of the magic wand. It turns out this is an
interesting and difficult scenario to solve for a multi-master replication system. Let's
explore.

Third-Party Solutions
There are some vendor solutions that take on this problem, which they typically tackle
through one or more of the following methods*:
 Use of a broker mechanism

Having a central ‘traffic cop' allows one server to be aware of all the other servers
and which files they have locked by users. Unfortunately this also means that there
is often a single point of failure in the distributed locking system.

Requirement for a fully routed network

Since a central broker must be able to talk to all servers participating in file
replication, this removes the ability to handle complex network topologies. Ring
topologies and multi hub-and-spoke topologies are not usually possible. In a non-
fully routed network, some servers may not be able to directly contact each other or
a broker, and can only talk to a partner who himself can talk to another server – and
so on. This is fine in a multi-master environment, but not with a brokering
mechanism.
 Are limited to a pair of servers

Some solutions limit the topology to a pair of servers in order to simplify their
distributed locking mechanism. For larger environments this is may not be feasible.

Make use of agents on clients and servers

Do not use multi-master replication
Do not make use of MS clustering
Make use of specialty appliances

* Note that I say typically! Please do not post death threats because you have a solution
that does/does not implement one or more of those methods!

Deeper Thoughts
As you think further about this issue, some fundamental issues start to crop up. For
example, if we have four servers with data that can be modified by users in four sites,
and the WAN connection to one of them goes offline, what do we do? The users can still
access their individual servers – but should we let them? We don't want them to make
changes that conflict, but we definitely want them to keep working and making our
company money. If we arbitrarily block changes at that point, no users can work even
though there may not actually be any conflicts happening! There's no way to tell the
other servers that the file is in use and you're back at square one.

Then there's SMB itself and the error handling of reporting locks. We can't really change
how SMB reports sharing violations as we'd break a ton of applications and clients
wouldn't understand new extended error messages anyways. Applications like Word
2007 do some undercover trickery to figure out who is locking files, but the vast
majority of applications don't know who has a file in use (or even that SMB exists.
Really.). So when a user gets the message ‘This file is in use' it's not particularly
actionable – should they all call the help desk? Does the help desk have access to all the
file servers to see which users are accessing files? Messy.

Since we want multi-master for high availability, a broker system is less desirable; we
might need to have something running on all servers that allows them all to
communicate even through non-fully routed networks. This will require very complex
synchronization techniques. It will add some overhead on the network (although
probably not much) and it will need to be lightning fast to make sure that we are not
holding up the user in their work; it needs to outrun file replication itself - in fact, it
might need to actually be tied to replication somehow. It will also have to account for
server outages that are network related and not server crashes, somehow.
And then we're back to special client software for this scenario that better understands
the locks and can give the user some useful info (“Go call Susie in accounting and tell
her to release that doc”, “Sorry, the file locking topology is broken and your
administrator is preventing you from opening this file until it's fixed”, etc). Getting this to
play nicely with the millions of applications running in Windows will definitely be
interesting. There are plenty of OS's that would not be supported or get the software –
Windows 2000 is out of mainstream support and XP soon will be. Linux and Mac clients
wouldn't have this software until they felt it was important, so the customer would have
to hope their vendors made something analogous.

More information
Right now the easiest way to control this situation in DFSR is to use DFS Namespaces to
guide users to predictable locations, with a consistent namespace. By correctly
configuring your DFSN site topology and server links, you force users to all share the
same local server and only allow them to access remote computers when their ‘main'
server is down. For most environments, this works quite well. Alternative to DFSR,
SharePoint is an option because of its check-out/check-in system. BranchCache (coming
in Windows Server 2008 R2 and Windows 7) may be an option for you as it is designed
for easing the reading of files in a branch scenario, but in the end the authoritative data
will still live on one server only – more on this here. And again, those vendors have their
solutions.
Overview of Disk Management
Article • 03/22/2023

Applies To: Windows 11, Windows 10, Windows Server 2022, Windows Server 2019,
Windows Server 2016

Disk Management is a system utility in Windows for advanced storage operations. Here
are some tasks you can complete with Disk Management:

Set up a new drive. For more information, see Initialize new disks.

Extend a volume into space that's not already part of a volume on the same drive.
For more information, see Extend a basic volume.

Shrink a partition, such as to enable extending into a neighboring partition. For

more information, see Shrink a basic volume.

Change a drive letter or assign a new drive letter. For more information, see
Change a drive letter.

Review drives and partitions

Disk Management shows the details for each drive on your PC and all partitions for each
drive. The details include statistics about the partitions, including the amount of space
allocated or used.

The following image shows the Disk Management overview for several drives. Disk 0 has
three partitions, and Disk 1 has two partitions. On Disk 0, the C: drive for Windows uses
the most disk space. Two other partitions for system operations and recovery use a
smaller amount of disk space.
Windows typically includes three partitions on your main drive (usually the C:\ drive).
These partitions include the EFI System Partition, the Local Disk (C:) Partition, and a
Recovery Partition.

The Windows operating system is installed on the Local Disk (C:) Partition. This
partition is the common storage location for your other apps and files.

Modern PCs use the EFI System Partition to start (boot) your PC and your
operating system.

The Recovery Partition stores special tools to help you recover Windows, in case
there's a problem starting the PC or other serious issues.

） Important

Disk Management might show the EFI System Partition and Recovery Partition as
100 percent free. However, these partitions store critical files that your PC needs to
operate properly, and the partitions are generally nearly full. It's recommended to
not modify these partitions in any way.

Troubleshoot issues
Sometimes a Disk Management task reports an error, or a procedure doesn't work as
expected. There are several options available to help you resolve the issue.

Review suggestions in the Troubleshooting Disk Management article.

 Search the Microsoft Community website for posts about files, folders, and
storage.

If you don't find an answer on the site, you can post a question for input from
Microsoft or other members of the community. You can also Contact Microsoft
Support .

Complete related tasks

Disk Management supports a wide range of drive tasks, but some tasks need to be
completed by using a different tool. Here are some common disk management tasks to
complete with other tools in Windows:

Free up disk space. For more information, see Free up drive space in Windows .

Defragment or optimize your drives. For more information, see Ways to improve
your computer's performance .

Pool multiple hard drives together, similar to a RAID (redundant array of

independent disks). For more information, see Storage Spaces in Windows .

Related links
Manage disks
Manage basic volumes
Troubleshooting Disk Management
Recovery options in Windows
Find lost files after the upgrade to Windows
Backup and Restore in Windows
Create a recovery drive
Create a system restore point
Where to look for your BitLocker recovery key
Overview of file sharing using the SMB
3 protocol in Windows Server
Article • 01/26/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012

This topic describes the SMB 3 feature in Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, and Windows Server 2012—practical uses for the feature, the
most significant new or updated functionality in this version compared to previous
versions, and the hardware requirements. SMB is also a fabric protocol used by
software-defined data center (SDDC) solutions such as Storage Spaces Direct, Storage
Replica, and others. SMB version 3.0 was introduced with Windows Server 2012 and has
been incrementally improved in subsequent releases.

Feature description
The Server Message Block (SMB) protocol is a network file sharing protocol that allows
applications on a computer to read and write to files and to request services from server
programs in a computer network. The SMB protocol can be used on top of its TCP/IP
protocol or other network protocols. Using the SMB protocol, an application (or the user
of an application) can access files or other resources at a remote server. This allows
applications to read, create, and update files on the remote server. SMB can also
communicate with any server program that is set up to receive an SMB client request.
SMB is a fabric protocol that is used by Software-defined Data Center (SDDC)
computing technologies, such as Storage Spaces Direct, Storage Replica. For more
information, see Windows Server software-defined datacenter.

Practical applications
This section discusses some new practical ways to use the new SMB 3.0 protocol.

File storage for virtualization (Hyper-V™ over SMB). Hyper-V can store virtual
machine files, such as configuration, Virtual hard disk (VHD) files, and snapshots, in
file shares over the SMB 3.0 protocol. This can be used for both stand-alone file
servers and clustered file servers that use Hyper-V together with shared file
storage for the cluster.
 Microsoft SQL Server over SMB. SQL Server can store user database files on SMB
file shares. Currently, this is supported with SQL Server 2008 R2 for stand-alone
SQL servers. Upcoming versions of SQL Server will add support for clustered SQL
servers and system databases.
Traditional storage for end-user data. The SMB 3.0 protocol provides
enhancements to the Information Worker (or client) workloads. These
enhancements include reducing the application latencies experienced by branch
office users when accessing data over wide area networks (WAN) and protecting
data from eavesdropping attacks.

７ Note

If you need to conserve storage space on an SMB file share, consider using Azure
File Sync with cloud tiering enabled. This allows you to cache your most frequently
accessed files locally and tier your least frequently accessed files to the cloud,
saving local storage space while maintaining performance. For details, see Planning
for an Azure File Sync deployment.

New and changed functionality

The following sections describe functionality that was added in SMB 3 and subsequent
updates.

Features added in Windows Server 2019 and

Windows 10, version 1809
Feature/functionality New or Summary
updated

Ability to require New To provide some added assurance that writes to a file share
write-through to disk make it all the way through the software and hardware stack
on file shares that to the physical disk prior to the write operation returning as
aren't continuously completed, you can enable write-through on the file share
available using either the NET USE /WRITETHROUGH command or the
New-SMBMapping -UseWriteThrough PowerShell cmdlet. There's
some amount of performance hit to using write-through; see
the blog post Controlling write-through behaviors in SMB
for further discussion.
Features added in Windows Server, version
1709, and Windows 10, version 1709
Feature/functionality New or Summary
updated

Guest access to file New The SMB client no longer allows the following actions: Guest
shares is disabled account access to a remote server; Fallback to the Guest
account after invalid credentials are provided. For details, see
Guest access in SMB2 disabled by default in Windows .

SMB global mapping New Maps a remote SMB share to a drive letter that is accessible
to all users on the local host, including containers. This is
required to enable container I/O on the data volume to
traverse the remote mount point. Be aware that when using
SMB global mapping for containers, all users on the
container host can access the remote share. Any application
running on the container host also have access to the
mapped remote share. For details, see Container Storage
Support with Cluster Shared Volumes (CSV), Storage Spaces
Direct, SMB Global Mapping .

SMB dialect control New You can now set registry values to control the minimum SMB
version (dialect) and maximum SMB version used. For details,
see Controlling SMB Dialects .

Features added in SMB 3.1.1 with Windows

Server 2016 and Windows 10, version 1607
Feature/functionality New or Summary
updated

SMB Encryption Updated SMB 3.1.1 encryption with Advanced Encryption

Standard-Galois/Counter Mode (AES-GCM) is
faster than SMB Signing or previous SMB
encryption using AES-CCM.

Directory Caching New SMB 3.1.1 includes enhancements to directory

caching. Windows clients can now cache much
larger directories, approximately 500K entries.
Windows clients will attempt directory queries with
1 MB buffers to reduce round trips and improve
performance.
 Feature/functionality New or Summary
updated

Pre-Authentication Integrity New In SMB 3.1.1, pre-authentication integrity provides

improved protection from a man-in-the-middle
attacker tampering with SMB’s connection
establishment and authentication messages. For
details, see SMB 3.1.1 Pre-authentication integrity
in Windows 10.

SMB Encryption Improvements New SMB 3.1.1 offers a mechanism to negotiate the
crypto algorithm per connection, with options for
AES-128-CCM and AES-128-GCM. AES-128-GCM is
the default for new Windows versions, while older
versions will continue to use AES-128-CCM.

Rolling cluster upgrade support New Enables rolling cluster upgrades by letting SMB
appear to support different max versions of SMB
for clusters in the process of being upgraded. For
more details on letting SMB communicate using
different versions (dialects) of the protocol, see the
blog post Controlling SMB Dialects .

SMB Direct client support in New Windows 10 Enterprise, Windows 10 Education,

Windows 10 and Windows 10 Pro for Workstations now include
SMB Direct client support.

Native support for New Adds native support for querying the normalized
FileNormalizedNameInformation name of a file. For details, see
API calls FileNormalizedNameInformation.

For additional details, see the blog post What’s new in SMB 3.1.1 in the Windows Server
2016 Technical Preview 2.

Features added in SMB 3.02 with Windows

Server 2012 R2 and Windows 8.1
Feature/functionality New or Summary
updated
 Feature/functionality New or Summary
updated

Automatic rebalancing New Improves scalability and manageability for Scale-Out File
of Scale-Out File Servers. SMB client connections are tracked per file share
Server clients (instead of per server), and clients are then redirected to the
cluster node with the best access to the volume used by the
file share. This improves efficiency by reducing redirection
traffic between file server nodes. Clients are redirected
following an initial connection and when cluster storage is
reconfigured.

Performance over Updated Windows 8.1 and Windows 10 provide improved CopyFile
WAN SRV_COPYCHUNK over SMB support when you use File
Explorer for remote copies from one location on a remote
machine to another copy on the same server. You will copy
only a small amount of metadata over the network (1/2KiB
per 16MiB of file data is transmitted). This results in a
significant performance improvement. This is an OS-level and
File Explorer-level distinction for SMB.

SMB Direct Updated Improves performance for small I/O workloads by increasing
efficiency when hosting workloads with small I/Os (such as
an online transaction processing (OLTP) database in a virtual
machine). These improvements are evident when using
higher speed network interfaces, such as 40 Gbps Ethernet
and 56 Gbps InfiniBand.

SMB bandwidth limits New You can now use Set-SmbBandwidthLimit to set bandwidth
limits in three categories: VirtualMachine (Hyper-V over SMB
traffic), LiveMigration (Hyper-V Live Migration traffic over
SMB), or Default (all other types of SMB traffic).

For more information on new and changed SMB functionality in Windows Server 2012
R2, see What's New in SMB in Windows Server.

Features added in SMB 3.0 with Windows

Server 2012 and Windows 8
Feature/functionality New or Summary
updated
Feature/functionality New or Summary
updated

SMB Transparent New Enables administrators to perform hardware or software

Failover maintenance of nodes in a clustered file server without
interrupting server applications storing data on these file
shares. Also, if a hardware or software failure occurs on a
cluster node, SMB clients transparently reconnect to another
cluster node without interrupting server applications that are
storing data on these file shares.

SMB Scale Out New Support for multiple SMB instances on a Scale-Out File
Server. Using Cluster Shared Volumes (CSV) version 2,
administrators can create file shares that provide
simultaneous access to data files, with direct I/O, through all
nodes in a file server cluster. This provides better utilization
of network bandwidth and load balancing of the file server
clients, and optimizes performance for server applications.

SMB Multichannel New Enables aggregation of network bandwidth and network fault
tolerance if multiple paths are available between the SMB
client and server. This enables server applications to take full
advantage of all available network bandwidth and be resilient
to a network failure.

SMB Multichannel in SMB 3 contributes to a substantial

increase in performance compared to previous versions of
SMB.

SMB Direct New Supports the use of network adapters that have RDMA
capability and can function at full speed with very low
latency, while using very little CPU. For workloads such as
Hyper-V or Microsoft SQL Server, this enables a remote file
server to resemble local storage.

SMB Direct in SMB 3 contributes to a substantial increase in

performance compared to previous versions of SMB.

Performance Counters New The new SMB performance counters provide detailed, per-
for server applications share information about throughput, latency, and I/O per
second (IOPS), allowing administrators to analyze the
performance of SMB file shares where their data is stored.
These counters are specifically designed for server
applications, such as Hyper-V and SQL Server, which store
files on remote file shares.
Feature/functionality New or Summary
updated

Performance Updated Both the SMB client and server have been optimized for
optimizations small random read/write I/O, which is common in server
applications such as SQL Server OLTP. In addition, large
Maximum Transmission Unit (MTU) is turned on by default,
which significantly enhances performance in large sequential
transfers, such as SQL Server data warehouse, database
backup or restore, deploying or copying virtual hard disks.

SMB-specific Windows New With Windows PowerShell cmdlets for SMB, an administrator
PowerShell cmdlets can manage file shares on the file server, end to end, from
the command line.

SMB Encryption New Provides end-to-end encryption of SMB data and protects
data from eavesdropping occurrences on untrusted
networks. Requires no new deployment costs, and no need
for Internet Protocol security (IPsec), specialized hardware, or
WAN accelerators. It may be configured on a per share basis,
or for the entire file server, and may be enabled for a variety
of scenarios where data traverses untrusted networks.

SMB Directory Leasing New Improves application response times in branch offices. With
the use of directory leases, roundtrips from client to server
are reduced since metadata is retrieved from a longer living
directory cache. Cache coherency is maintained because
clients are notified when directory information on the server
changes. Directory leases work with scenarios for
HomeFolder (read/write with no sharing) and Publication
(read-only with sharing).

Performance over New Directory opportunistic locks (oplocks) and oplock leases
WAN were introduced in SMB 3.0. For typical office/client
workloads, oplocks/leases are shown to reduce network
round trips by approximately 15%.

In SMB 3, the Windows implementation of SMB has been

refined to improve the caching behavior on the client as well
as the ability to push higher throughputs.

SMB 3 features improvements to the CopyFile() API, as well

as to associated tools such as Robocopy, to push significantly
more data over the network.
 Feature/functionality New or Summary
updated

Secure dialect New Helps protect against man-in-the-middle attempt to

negotiation downgrade dialect negotiation. The idea is to prevent an
eavesdropper from downgrading the initially negotiated
dialect and capabilities between the client and the server. For
details, see SMB3 Secure Dialect Negotiation. Note that this
has been superceded by the SMB 3.1.1 Pre-authentication
integrity in Windows 10 feature in SMB 3.1.1.

Hardware requirements
SMB Transparent Failover has the following requirements:

A failover cluster running Windows Server 2012 or Windows Server 2016 with at
least two nodes configured. The cluster must pass the cluster validation tests
included in the validation wizard.
File shares must be created with the Continuous Availability (CA) property, which is
the default.
File shares must be created on CSV volume paths to attain SMB Scale-Out.
Client computers must be running Windows® 8 or Windows Server 2012, both of
which include the updated SMB client that supports continuous availability.

７ Note

Down-level clients can connect to file shares that have the CA property, but
transparent failover will not be supported for these clients.

SMB Multichannel has the following requirements:

At least two computers running Windows Server 2012 are required. No extra
features need to be installed—the technology is on by default.
For information on recommended network configurations, see the See Also section
at the end of this overview topic.

SMB Direct has the following requirements:

At least two computers running Windows Server 2012 are required. No extra
features need to be installed—the technology is on by default.
Network adapters with RDMA capability are required. Currently, these adapters are
available in three different types: iWARP, Infiniband, or RoCE (RDMA over
Converged Ethernet).
More information
The following list provides additional resources on the web about SMB and related
technologies in Windows Server 2012 R2, Windows Server 2012, and Windows Server
2016.

Storage in Windows Server

Scale-Out File Server for Application Data
Improve Performance of a File Server with SMB Direct
Deploy Hyper-V over SMB
Deploy SMB Multichannel
Deploying Fast and Efficient File Servers for Server Applications
SMB: Troubleshooting Guide
SMB Direct
Article • 01/25/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Azure Stack HCI, version 21H2

Windows Server includes a feature called SMB Direct, which supports the use of network
adapters that have Remote Direct Memory Access (RDMA) capability. Network adapters
that have RDMA can function at full speed with very low latency, while using very little
CPU. For workloads such as Hyper-V or Microsoft SQL Server, this enables a remote file
server to resemble local storage. SMB Direct includes:

Increased throughput: Leverages the full throughput of high speed networks

where the network adapters coordinate the transfer of large amounts of data at
line speed.
Low latency: Provides extremely fast responses to network requests, and, as a
result, makes remote file storage feel as if it is directly attached block storage.
Low CPU utilization: Uses fewer CPU cycles when transferring data over the
network, which leaves more power available to server applications.

SMB Direct is automatically configured by Windows Server.

SMB Multichannel and SMB Direct

SMB Multichannel is the feature responsible for detecting the RDMA capabilities of
network adapters to enable SMB Direct. Without SMB Multichannel, SMB uses regular
TCP/IP with the RDMA-capable network adapters (all network adapters provide a TCP/IP
stack along with the new RDMA stack).

With SMB Multichannel, SMB detects whether a network adapter has the RDMA
capability, and then creates multiple RDMA connections for that single session (two per
interface). This allows SMB to use the high throughput, low latency, and low CPU
utilization offered by RDMA-capable network adapters. It also offers fault tolerance if
you are using multiple RDMA interfaces.

You can team RDMA-capable network adapters using Switch Embedded Teaming (SET)
starting with Windows Server 2016. After at least one RDMA network connection is
created, the TCP/IP connection used for the original protocol negotiation is no longer
used. However, the TCP/IP connection is retained in case the RDMA network
connections fail.
SMB Encryption with SMB Direct
Beginning in Windows Server 2022 and Windows 11, SMB Direct now supports
encryption. Previously, enabling SMB encryption disabled direct data placement, making
RDMA performance as slow as TCP. Now data is encrypted before placement, leading to
relatively minor performance degradation while adding AES-128 and AES-256 protected
packet privacy. For more information on configuring SMB encryption, review SMB
security enhancements.

Furthermore, Windows Server failover clusters now support granular control of

encrypting intra-node storage communications for Cluster Shared Volumes (CSV) and
the storage bus layer (SBL). This means that when using Storage Spaces Direct and SMB
Direct, you can decide to encrypt east-west communications within the cluster itself for
higher security.

Requirements
SMB Direct requires the following:

At least two computers running Windows Server 2012 or newer.

One or more network adapters with RDMA capability.

Considerations when using SMB Direct

You can use SMB Direct in a failover cluster; however, you need to make sure that
the cluster networks used for client access are adequate for SMB Direct. Failover
clustering supports using multiple networks for client access, along with network
adapters that are RSS (Receive Side Scaling)-capable and RDMA-capable.
You can use SMB Direct on the Hyper-V management operating system to support
using Hyper-V over SMB, and to provide storage to a virtual machine that uses the
Hyper-V storage stack. However, RDMA-capable network adapters are not directly
exposed to a Hyper-V client. If you connect an RDMA-capable network adapter to
a virtual switch, the virtual network adapters from the switch will not be RDMA-
capable.
If you disable SMB Multichannel, SMB Direct is also disabled. Since SMB
Multichannel detects network adapter capabilities and determines whether a
network adapter is RDMA-capable, SMB Direct cannot be used by the client if SMB
Multichannel is disabled.
SMB Direct is not supported on Windows RT. SMB Direct requires support for
RDMA-capable network adapters, which is available starting with Windows Server
2012.
 SMB Direct is not supported on down-level versions of Windows Server. It is
supported starting with Windows Server 2012.

Enabling and disabling SMB Direct

SMB Direct is enabled by default starting with Windows Server 2012. The SMB client
automatically detects and uses multiple network connections if an appropriate
configuration is identified.

Disable SMB Direct

Typically, you will not need to disable SMB Direct, however, you can disable it by
running one of the following Windows PowerShell scripts.

To disable RDMA for a specific interface, type:

PowerShell

Disable-NetAdapterRdma <name>

To disable RDMA for all interfaces, type:

PowerShell

Set-NetOffloadGlobalSetting -NetworkDirect Disabled

When you disable RDMA on either the client or the server, the systems cannot use it.
Network Direct is the internal name for Windows Server basic networking support for
RDMA interfaces.

Re-enable SMB Direct

After disabling RDMA, you can re-enable it by running one of the following Windows
PowerShell scripts.

To re-enable RDMA for a specific interface, type:

PowerShell

Enable-NetAdapterRDMA <name>

To re-enable RDMA for all interfaces, type:

 PowerShell

Set-NetOffloadGlobalSetting -NetworkDirect Enabled

You need to enable RDMA on both the client and the server to start using it again.

Test performance of SMB Direct

You can test how the performance is working by using one of the following procedures.

Compare a file copy with and without using SMB Direct

Here's how to measure the increased throughput of SMB Direct:

1. Configure SMB Direct

2. Measure the amount of time to run a large file copy using SMB Direct.
3. Disable RDMA on the network adapter, see Enabling and disabling SMB Direct.
4. Measure the amount of time to run a large file copy without using SMB Direct.
5. Re-enable RDMA on the network adapter, and then compare the two results.
6. To avoid the impact of caching, you should do the following:
a. Copy a large amount of data (more data than memory is capable of handling).
b. Copy the data twice, with the first copy as practice and then timing the second
copy.
c. Restart both the server and the client before each test to make sure they
operate under similar conditions.

Fail one of multiple network adapters during a file copy

with SMB Direct
Here's how to confirm the failover capability of SMB Direct:

1. Ensure that SMB Direct is functioning in a multiple network adapter configuration.

2. Run a large file copy. While the copying is run, simulate a failure of one of the
network paths by disconnecting one of the cables (or by disabling one of the
network adapters).
3. Confirm that the file copying continues using one of the remaining network
adapters, and that there are no file copy errors.

７ Note
 To avoid failures of a workload that does not use SMB Direct, make sure there are
no other workloads using the disconnected network path.

More information
Server Message Block overview
Increasing Server, Storage, and Network Availability: Scenario Overview
Deploy Hyper-V over SMB
SMB over QUIC
Article • 05/18/2023

Applies to: Windows Server 2022 Datacenter: Azure Edition, Windows 11

SMB over QUIC introduces an alternative to the TCP network transport, providing
secure, reliable connectivity to edge file servers over untrusted networks like the
Internet. QUIC is an IETF-standardized protocol with many benefits when compared with
TCP:

All packets are always encrypted and handshake is authenticated with TLS 1.3
Parallel streams of reliable and unreliable application data
Exchanges application data in the first round trip (0-RTT)
Improved congestion control and loss recovery
Survives a change in the clients IP address or port

SMB over QUIC offers an "SMB VPN" for telecommuters, mobile device users, and high
security organizations. The server certificate creates a TLS 1.3-encrypted tunnel over the
internet-friendly UDP port 443 instead of the legacy TCP port 445. All SMB traffic,
including authentication and authorization within the tunnel is never exposed to the
underlying network. SMB behaves normally within the QUIC tunnel, meaning the user
experience doesn't change. SMB features like multichannel, signing, compression,
continuous availability, directory leasing, and so on, work normally.

A file server administrator must opt in to enabling SMB over QUIC. It isn't on by default
and a client can't force a file server to enable SMB over QUIC. Windows SMB clients still
use TCP by default and will only attempt SMB over QUIC if the TCP attempt first fails or
if intentionally requiring QUIC using NET USE /TRANSPORT:QUIC or New-SmbMapping -
TransportType QUIC .

Prerequisites
To use SMB over QUIC, you need the following things:

A file server running Windows Server 2022 Datacenter: Azure Edition (Microsoft
Server Operating Systems )
A Windows 11 computer (Windows for business )
Windows Admin Center (Homepage )
A Public Key Infrastructure to issue certificates like Active Directory Certificate
Server or access to a trusted third party certificate issuer like Verisign, Digicert,
 Let's Encrypt, and so on.

Deploy SMB over QUIC

Step 1: Install a server certificate

1. Create a Certificate Authority-issued certificate with the following properties:

Key usage: digital signature

Purpose: Server Authentication (EKU 1.3.6.1.5.5.7.3.1)
Signature algorithm: SHA256RSA (or greater)
Signature hash: SHA256 (or greater)
Public key algorithm: ECDSA_P256 (or greater. Can also use RSA with at least
2048 length)
Subject Alternative Name (SAN): (A DNS name entry for each fully qualified
DNS name used to reach the SMB server)
Subject: (CN= anything, but must exist)
Private key included: yes
 If using a Microsoft Enterprise Certificate Authority, you can create a certificate
template and allow the file server administrator to supply the DNS names when
requesting it. For more information on creating a certificate template, review
Designing and Implementing a PKI: Part III Certificate Templates . For a
demonstration of creating a certificate for SMB over QUIC using a Microsoft
Enterprise Certificate Authority, watch this video:
https://www.youtube-nocookie.com/embed/L0yl5Z5wToA

For requesting a third-party certificate, consult your vendor documentation.

2. If using a Microsoft Enterprise Certificate Authority:

a. Start MMC.EXE on the file server.
b. Add the Certificates snap-in, and select the Computer account.
c. Expand Certificates (Local Computer), Personal, then right-click Certificates
and click Request New Certificate.
d. Click Next
e. Select Active Directory Enrollment Policy
f. Click Next
g. Select the certificate template for SMB over QUIC that was published in Active
Directory.
h. Click More information is required to enroll for this certificate. Click here to
configure settings.
i. So users can use to locate the file server, fill in the value Subject with a common
name and Subject Alternative Name with one or more DNS names.
j. Click Ok and click Enroll.
 ７ Note

If you're using a certificate file issued by a third party certificate authority, you can
use the Certificates snap-in or Windows Admin Center to import it.

Step 2: Configure SMB over QUIC

1. Deploy a Windows Server 2022 Datacenter: Azure Edition server.

2. Install the latest version of Windows Admin Center on a management PC or the file
server. You need the latest version of the Files & File Sharing extension. It's
installed automatically by Windows Admin Center if Automatically update
extensions is enabled in Settings > Extensions.

3. Join your Windows Server 2022 Datacenter: Azure Edition file server to your Active
Directory domain and make it accessible to Windows Insider clients on the Azure
public interface by adding a firewall allow rule for UDP/443 inbound. Do not allow
TCP/445 inbound to the file server. The file server must have access to at least one
domain controller for authentication, but no domain controller requires any
internet access.

4. Connect to the server with Windows Admin Center and click the Settings icon in
the lower left. In the File shares (SMB server) section, under File sharing across
the internet with SMB over QUIC, click Configure.

5. Click a certificate under Select a computer certificate for this file server, click the
server addresses clients can connect to or click Select all, and click Enable.
 6. Ensure that the certificate and SMB over QUIC report are healthy.

7. Click on the Files and File Sharing menu option. Note your existing SMB shares or
create a new one.

For a demonstration of configuring and using SMB over QUIC, watch this video:
https://www.youtube-nocookie.com/embed/OslBSB8IkUw

Step 3: Connect to SMB shares

1. Join your Windows 11 device to your domain. Be certain the names of the SMB
over QUIC file server's certificate subject alternative names are published to DNS
and are fully qualified OR added to the HOST files for your Windows 11. Ensure
that the server's certificate subject alternative names are published to DNS OR
added to the HOSTS files for your Windows 11 .

2. Move your Windows 11 device to an external network where it no longer has any
network access to domain controllers or the file server's internal IP addresses.

3. In Windows File Explorer, in the Address Bar, type the UNC path to a share on the
file server and confirm you can access data in the share. Alternatively, you can use
NET USE /TRANSPORT:QUIC or New-SmbMapping -TransportType QUIC with a UNC
path. Examples:

NET USE * \\fsedge1.contoso.com\sales (automatically tries TCP then QUIC)

NET USE * \\fsedge1.contoso.com\sales /TRANSPORT:QUIC (tries only QUIC)

New-SmbMapping -LocalPath 'Z:' -RemotePath '\\fsedge1.contoso.com\sales' -

TransportType QUIC (tries only QUIC)

Configure the KDC Proxy (Optional, but recommended)

By default, a Windows 11 device won't have access to an Active Directory domain
controller when connecting to an SMB over QUIC file server. This means authentication
uses NTLMv2, where the file server authenticates on behalf of the client. No NTLMv2
authentication or authorization occurs outside the TLS 1.3-encrypted QUIC tunnel.
However, we still recommend using Kerberos as a general security best practice and
don't recommend creating new NTLMv2 dependencies in deployments. To allow this,
you can configure the KDC proxy to forward ticket requests on the user's behalf, all
while using an internet-friendly HTTPS encrypted communication channel. The KDC
Proxy is fully supported by SMB over QUIC and highly recommended.

７ Note

You cannot configure the Windows Admin Center (WAC) in gateway mode using
TCP port 443 on a file server where you are configuring KDC Proxy. When
configuring WAC on the file server, change the port to one that is not in use and is
 not 443. If you have already configured WAC on port 443, re-run the WAC setup
MSI and choose a different port when prompted.

Windows Admin Center method

1. Ensure you're using at least Windows Admin Center version 2110.

2. Configure SMB over QUIC normally. Starting in Windows Admin Center 2110, the
option to configure KDC proxy in SMB over QUIC is automatically enabled and you
don't need to perform extra steps on the file servers. The default KDC proxy port is
443 and assigned automatically by Windows Admin Center.

７ Note

You cannot configure an SMB over QUIC server joined to a Workgroup using
Windows Admin Center. You must join the server to an Active Directory
domain or use the step in Manual Method section.

3. Configure the following group policy setting to apply to the Windows 11 device:

Computers > Administrative templates > System > Kerberos > Specify KDC
proxy servers for Kerberos clients

The format of this group policy setting is a value name of your fully qualified
Active Directory domain name and the value will be the external name you
specified for the QUIC server. For example, where the Active Directory domain is
named corp.contoso.com and the external DNS domain is named contoso.com:

value name: corp.contoso.com

value: <https fsedge1.contoso.com:443:kdcproxy />

This Kerberos realm mapping means that if user [email protected] tried to

connect to a file server name fs1edge.contoso.com, the KDC proxy will know to
forward the kerberos tickets to a domain controller in the internal corp.contoso.com
domain. The communication with the client will be over HTTPS on port 443 and
user credentials aren't directly exposed on the client-file server network.

4. Ensure that edge firewalls allow HTTPS on port 443 inbound to the file server.

5. Apply the group policy and restart the Windows 11 device.

Manual Method
1. On the file server, in an elevated PowerShell prompt, run:

NETSH http add urlacl url=https://+:443/KdcProxy user="NT authority\Network

Service"

REG ADD "HKLM\SYSTEM\CurrentControlSet\Services\KPSSVC\Settings" /v

HttpsClientAuth /t REG_DWORD /d 0x0 /f

REG ADD "HKLM\SYSTEM\CurrentControlSet\Services\KPSSVC\Settings" /v

DisallowUnprotectedPasswordAuth /t REG_DWORD /d 0x0 /f

Get-SmbServerCertificateMapping

2. Copy the thumbprint value from the certificate associated with SMB over QUIC
certificate (there may be multiple lines but they will all have the same thumbprint)
and paste it as the Certhash value for the following command:

$guid = [Guid]::NewGuid()

Add-NetIPHttpsCertBinding -ipport 0.0.0.0:443 -CertificateHash <thumbprint> -

CertificateStoreName "my" -ApplicationId "{$guid}" -NullEncryption $false

3. Add the file server's SMB over QUIC names as SPNs in Active Directory for
Kerberos. For example:

NETDOM computername ws2022-quic.corp.contoso.com /add fsedge1.contoso.com

4. Set the KDC Proxy service to automatic and start it:

Set-Service -Name kpssvc -StartupType Automatic

Start-Service -Name kpssvc

5. Configure the following group policy to apply to the Windows 11 device:

Computers > Administrative templates > System > Kerberos > Specify KDC
proxy servers for Kerberos clients

The format of this group policy setting is a value name of your fully qualified
Active Directory domain name and the value will be the external name you
specified for the QUIC server. For example, where the Active Directory domain is
named "corp.contoso.com" and the external DNS domain is named "contoso.com":

value name: corp.contoso.com

 value: <https fsedge1.contoso.com:443:kdcproxy />

This Kerberos realm mapping means that if user [email protected] tried to

connect to a file server name fs1edge.contoso.com" , the KDC proxy will know to
forward the kerberos tickets to a domain controller in the internal
corp.contoso.com domain. The communication with the client will be over HTTPS

on port 443 and user credentials aren't directly exposed on the client-file server
network.

6. Create a Windows Defender Firewall rule that inbound-enables TCP port 443 for
the KDC Proxy service to receive authentication requests.

7. Ensure that edge firewalls allow HTTPS on port 443 inbound to the file server.

8. Apply the group policy and restart the Windows 11 device.

７ Note

Automatic configuration of the KDC Proxy will come later in the SMB over QUIC
and these server steps will not be necessary.

Certificate expiration and renewal

An expired SMB over QUIC certificate that you replace with a new certificate from the
issuer will contain a new thumbprint. While you can automatically renew SMB over QUIC
certificates when they expire using Active Directory Certificate Services, a renewed
certificate gets a new thumbprint as well. This effectively means that SMB over QUIC
must be reconfigured when the certificate expires, as a new thumbprint must be
mapped. You can simply select your new certificate in Windows Admin Center for the
existing SMB over QUIC configuration or use the Set-SMBServerCertificateMapping
PowerShell command to update the mapping for the new certificate. You can use Azure
Automanage for Windows Server to detect impending certificate expiration and prevent
an outage. For more information, review Azure Automanage for Windows Server.

Notes
Windows Server 2022 Datacenter: Azure Edition will also eventually be available on
Azure Stack HCI 21H2, for customers not using Azure public cloud.
We recommend read-only domain controllers configured only with passwords of
mobile users be made available to the file server.
 Users should have strong passwords or, ideally, be configured using a
passwordless strategy with Windows Hello for Business MFA or smart cards.
Configure an account lockout policy for mobile users through fine-grained
password policy and you should deploy intrusion protection software to detect
brute force or password spray attacks.
You can't configure SMB over QUIC using WAC when the SMB server is in a
workgroup (that is, not AD domain joined). In that scenario you must use the New-
SMBServerCertificateMapping cmdlet and the Manual Method steps for KDC proxy
configuration.

More references
Storage at Microsoft blog

QUIC Working Group homepage

Microsoft MsQuic GitHub homepage

QUIC Wikipedia

TLS 1.3 Working Group homepage

Taking Transport Layer Security (TLS) to the next level with TLS 1.3
SMB compression
Article • 05/18/2023

Applies to: Windows Server 2022, Windows 11

SMB compression allows an administrator, user or application to request compression of

files as they transfer over the network. This removes the need to first manually deflate a
file with an application, copy it, then inflate on the destination computer. Compressed
files will consume less network bandwidth and take less time to transfer, at the cost of
slightly increased CPU usage during transfers. SMB compression is most effective on
networks with less bandwidth, such as a client's 1 Gbps ethernet or Wi-Fi network; a file
transfer over an uncongested 100 Gbps ethernet network between two servers with
flash storage may be as fast without SMB compression in practice, but will still create
less congestion for other applications.

SMB compression in Windows has the following characteristics:

Supports compression algorithms XPRESS (LZ77), XPRESS Huffman

(LZ77+Huffman), LZNT1, or PATTERN_V1*. XPRESS is used automatically
Supports SMB signing and SMB encryption
Supports SMB over QUIC
Supports SMB Multichannel
Doesn't support SMB Direct over RDMA

For a demonstration of SMB compression, watch this video:

https://www.youtube-nocookie.com/embed/zpMS6w33H7U

Requirements
To use SMB compression in a traditional client-file server workload, you need the
following:

A file server running Windows Server 2022 on-premises or in Azure

A Windows 11 (Windows for business ) computer
Windows Admin Center - (Homepage )

Configuring SMB compression

You can configure SMB compression from both a client and server perspective. Client
and server don't refer to a particular edition like Windows Server 2022 or Windows 11
Insider Preview but instead to the architecture of a file transfer between two computers.
Both Windows Server 2022 and Windows 11 support being a client or server of SMB
compression.

Requesting SMB compression on file shares

You can configure shares to always request compression when connected to by clients.
You can use Windows Admin Center or PowerShell.

Using Windows Admin Center

1. Install Windows Admin Center and connect to a Windows Server 2022 file server.
2. Click on the Files and file sharing menu item.
3. Click on File shares.
4. Edit an existing share or create a new share.
5. Select Compress data and then click Add or Edit.

Using PowerShell
1. Open an elevated PowerShell command prompt as an administrator.

2. Create a new share with compression using New-SMBShare with the -CompressData
$true parameter and argument. For example:

PowerShell

New-SmbShare -Name "Sales" -Path "C:\sales" -CompressData $true

 3. Edit an existing share with compression using Set-SMBShare with the -CompressData
$true parameter and argument. For example:

PowerShell

Set-SmbShare -Name "Sales" -CompressData $true

Requesting SMB compression on mapped drives

You can request that all data copied over a mapped drive to be compressed. This can be
done as part of a logon script or when run manually.

PowerShell

1. Open a PowerShell command prompt.

2. Map a drive using New-SMBMapping with the -CompressNetworkTraffic $true

parameter and argument. For example:

PowerShell

New-SmbMapping -LocalPath "Z:" -RemotePath

"\\fs1.corp.contoso.com\sales" -CompressNetworkTraffic $true

Requesting SMB compression with copy tools

You can request that SMB compression is attempted for particular files using robocopy
or xcopy.

７ Note

If you want File Explorer, third party copy tools, or applications to use compression,
map drives with compression, enable compression on shares, or set SMB clients to
always compress.

Robocopy
 1. Open a CMD prompt or PowerShell command prompt.

2. Copy with the /COMPRESS flag. For example:

ROBOCOPY c:\hypervdisks \\hypervcluster21.corp.contoso.com\disks$

*.vhdx /COMPRESS

Always require or always reject compression requests

Starting in Windows Server 2022 with update KB5016693 (OS Build 20348.946) and
Windows 11 with update KB5016691 (OS Build 22000.918) you can configure an SMB
client or SMB server to always request compression and to always reject requests for
compression. You can now use Group Policy or PowerShell; in the initial release of
Windows 11 and Windows Server 2022, you could only use registry settings to control
most of these behaviors and you could not configure an SMB server to always request
compression despite its share settings. An SMB client and SMB server refers to the SMB
services, not to a Windows edition or SKU. All of these SMB changes take effect
immediately without a reboot.

Always attempt compression (SMB client)

Group Policy

1. Run the Group Policy Management Console for your Active Directory domain
and create or navigate to a group policy.
2. Expand policy Computer Configuration\Policies\Administrative
Templates\Network\Lanman Workstation.
3. Enable policy Use SMB Compression by Default.
4. Close the policy editor.

Never compress (SMB client)

Group Policy

1. Run the Group Policy Management Console for your Active Directory domain
and create or navigate to a group policy.
 2. Expand policy Computer Configuration\Policies\Administrative
Templates\Network\Lanman Workstation.
3. Enable policy Disable SMB Compression.
4. Close the policy editor.

Always attempt compression (SMB server)

Group Policy

1. Run the Group Policy Management Console for your Active Directory domain
and create or navigate to a group policy.
2. Expand policy Computer Configuration\Policies\Administrative
Templates\Network\Lanman Server.
3. Enable policy Request traffic compression for all shares.
4. Close the policy editor.

Never compress (SMB server)

Group Policy

1. Run the Group Policy Management Console for your Active Directory domain
and create or navigate to a group policy.
2. Expand policy Computer Configuration\Policies\Administrative
Templates\Network\Lanman Server.
3. Enable policy Disable SMB Compression.
4. Close the policy editor.

Understanding and controlling compression

behaviors
Starting in Windows Server 2022 with update KB5016693 (OS Build 20348.946) and
Windows 11 with update KB5016691 (OS Build 22000.918), SMB by default always
attempts to compress a file when a client or server requests it, without using
compression sampling.
 ７ Note

In the original release of Windows Server 2022 and Windows 11, SMB compression
defaulted to use of an algorithm where it attempted to compress the first
524,288,000 bytes (500 MiB) of a file during transfer and track that at least
104,857,600 bytes (100 MiB) compressed within that 500 MiB range. If fewer than
100 MiB was compressible, SMB compression stopped trying to compress the rest
of the file. If at least 100 MiB compressed, SMB compression attempted to
compress the rest of the file. With this behavior change, sampling is now disabled
by default and SMB always attempts to compress the entire file when a client or
server requests it.

Testing SMB compression

A simple way to test your compression configuration is using VHDX files. You can create
and mount a VHDX, add some files to it, then dismount the VHDX and copy it as a file.
Alternatively, you can just copy an existing dismounted virtual machine VHDX file, as
much of its file contents will compress. For an example of creating a VHDX test file:

1. Start Diskmgmt.msc.

2. Select Local Disk (C:) by clicking on it.

3. Click Action then Create VHD.

4. In Diskmgmt, right-click your VHDX now shown as "Not initialized" and click
Initialize disk and click OK. Right-click on the disks Unallocated section and click
New Simple Volume, then Next for all menu prompts, then click Finish.

5. Specify a file path, set the size to "25 GB", select VHDX and Fixed size, then click
OK.
 6. Right-click on the disk and click Detach VHD, then click OK.

7. In File Explorer, double-click that VHDX file to mount it. Copy a few MB of
uncompressible files, such as JPG format, then right-click the mounted disk and
click Eject.

You now have a large test file with compressed contents.

Testing SMB compression between a pair of VMs running on the same Hyper-V host
may not show time savings because the virtual switch is 10 Gbps and has no congestion,
plus modern hypervisors often use flash storage. Test your compression over the real
networks you plan to use. You can also reduce the network bandwidth on Hyper-V VMs
for testing purposes using Set-VMNetworkAdapter with -MaximumBandwidth set to 1Gb ,
for example.

To see how well compression is working, you can robocopy the same file to a server
twice, once with the /compress flag and again without compression, deleting the server
file between each test. If the file is compressing, you should see less network utilization
in Task Manager and a lower copy time. You can also observe the SMB server's
Performance Monitor object "SMB Server Shares" for its "Compressed Requests/sec" and
"Compressed Responses/sec" counters.
RDMA and SMB Direct
SMB compression doesn't support SMB Direct and RDMA. This means that even if the
client requests compression and the server supports it, compression will not be
attempted with SMB Direct and RDMA. Support for SMB compression with SMB Direct
and RDMA will come after the Windows Server 2022 and Windows 11 public previews.
SMB security enhancements
Article • 05/18/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Azure Stack HCI version 21H2,
Windows 11, Windows 10

This article explains the SMB security enhancements in Windows Server and Windows.

SMB Encryption
SMB Encryption provides SMB data end-to-end encryption and protects data from
eavesdropping occurrences on untrusted networks. You can deploy SMB Encryption with
minimal effort, but it might require other costs for specialized hardware or software. It
has no requirements for Internet Protocol security (IPsec) or WAN accelerators. SMB
Encryption can be configured on a per share basis, for the entire file server, or when
mapping drives.

７ Note

SMB Encryption does not cover security at rest, which is typically handled by
BitLocker Drive Encryption.

You can consider SMB Encryption for any scenario in which sensitive data needs to be
protected from interception attacks. Possible scenarios include:

You move an information worker’s sensitive data by using the SMB protocol. SMB
Encryption offers an end-to-end privacy and integrity assurance between the file
server and the client. It provides this security regardless of the networks traversed,
such as wide area network (WAN) connections maintained by non-Microsoft
providers.
SMB 3.0 enables file servers to provide continuously available storage for server
applications, such as SQL Server or Hyper-V. Enabling SMB Encryption provides an
opportunity to protect that information from snooping attacks. SMB Encryption is
simpler to use than the dedicated hardware solutions that are required for most
storage area networks (SANs).

Windows Server 2022 and Windows 11 introduce AES-256-GCM and AES-256-CCM

cryptographic suites for SMB 3.1.1 encryption. Windows automatically negotiates this
more advanced cipher method when connecting to another computer that supports it.
You can also mandate this method through Group Policy. Windows still supports AES-
128-GCM and AES-128-CCM. By default, AES-128-GCM is negotiated with SMB 3.1.1,
bringing the best balance of security and performance.

Windows Server 2022 and Windows 11 SMB Direct now support encryption. Previously,
enabling SMB encryption disabled direct data placement, making RDMA performance as
slow as TCP. Now data is encrypted before placement, leading to relatively minor
performance degradation while adding AES-128 and AES-256 protected packet privacy.
You can enable encryption using Windows Admin Center, Set-SmbServerConfiguration,
or UNC Hardening group policy .

Furthermore, Windows Server failover clusters now support granular control of

encrypting intra-node storage communications for Cluster Shared Volumes (CSV) and
the storage bus layer (SBL). This support means that when using Storage Spaces Direct
and SMB Direct, you can encrypt east-west communications within the cluster itself for
higher security.

） Important

There is a notable performance operating cost with any end-to-end encryption

protection when compared to non-encrypted.

Enable SMB Encryption

You can enable SMB Encryption for the entire file server or only for specific file shares.
Use one of the following procedures to enable SMB Encryption.

Enable SMB Encryption with Windows Admin Center

1. Download and install Windows Admin Center.
2. Connect to the file server.
3. Select Files & file sharing.
4. Select the File shares tab.
5. To require encryption on a share, select the share name and choose Enable SMB
encryption.
6. To require encryption on the server, select File server settings.
7. Under SMB 3 encryption, select Required from all clients (others are rejected),
and then choose Save.
Enable SMB Encryption with UNC Hardening
UNC Hardening lets you configure SMB clients to require encryption regardless of server
encryption settings. This feature helps prevent interception attacks. To configure UNC
Hardening, see MS15-011: Vulnerability in Group Policy could allow remote code
execution . For more information on interception attack defenses, see How to Defend
Users from Interception Attacks via SMB Client Defense .

Enable SMB Encryption with Windows PowerShell

1. Sign into your server and run PowerShell on your computer in an elevated session.

2. To enable SMB Encryption for an individual file share, run the following command.

PowerShell

Set-SmbShare –Name <sharename> -EncryptData $true

3. To enable SMB Encryption for the entire file server, run the following command.

PowerShell

Set-SmbServerConfiguration –EncryptData $true

4. To create a new SMB file share with SMB Encryption enabled, run the following
command.

PowerShell

New-SmbShare –Name <sharename> -Path <pathname> –EncryptData $true

Map drives with encryption

1. To enable SMB Encryption when mapping a drive using PowerShell, run the
following command.

PowerShell

New-SMBMapping -LocalPath <drive letter> -RemotePath <UNC path> -

RequirePrivacy $TRUE
 2. To enable SMB Encryption when mapping a drive using CMD, run the following
command.

Windows Command Prompt

NET USE <drive letter> <UNC path> /REQUIREPRIVACY

Considerations for deploying SMB Encryption

By default, when SMB Encryption is enabled for a file share or server, only SMB 3.0, 3.02,
and 3.1.1 clients are allowed to access the specified file shares. This limit enforces the
administrator's intent of safeguarding the data for all clients that access the shares.

However, in some circumstances, an administrator might want to allow unencrypted

access for clients that don't support SMB 3.x. This situation could occur during a
transition period when mixed client operating system versions are being used. To allow
unencrypted access for clients that don't support SMB 3.x, enter the following script in
Windows PowerShell:

PowerShell

Set-SmbServerConfiguration –RejectUnencryptedAccess $false

７ Note

We do not recommend allowing unencrypted access when you have deployed

encryption. Update the clients to support encryption instead.

The preauthentication integrity capability described in the next section prevents an

interception attack from downgrading a connection from SMB 3.1.1 to SMB 2.x (which
would use unencrypted access). However, it doesn't prevent a downgrade to SMB 1.0,
which would also result in unencrypted access.

To guarantee that SMB 3.1.1 clients always use SMB Encryption to access encrypted
shares, you must disable the SMB 1.0 server. For instructions, connect to the server with
Windows Admin Center and open the Files & File Sharing extension, and then select the
File shares tab to be prompted to uninstall. For more information, see How to detect,
enable and disable SMBv1, SMBv2, and SMBv3 in Windows.

If the –RejectUnencryptedAccess setting is left at its default setting of $true, only

encryption-capable SMB 3.x clients are allowed to access the file shares (SMB 1.0 clients
are also rejected).

Consider the following issues as you deploy SMB Encryption:

SMB Encryption uses the Advanced Encryption Standard (AES)-GCM and CCM
algorithm to encrypt and decrypt the data. AES-CMAC and AES-GMAC also
provide data integrity validation (signing) for encrypted file shares, regardless of
the SMB signing settings. If you want to enable SMB signing without encryption,
you can continue to do so. For more information, see Configure SMB Signing with
Confidence .
You might encounter issues when you attempt to access the file share or server if
your organization uses wide area network (WAN) acceleration appliances.
With a default configuration (where there's no unencrypted access allowed to
encrypted file shares), if clients that don't support SMB 3.x attempt to access an
encrypted file share, Event ID 1003 is logged to the Microsoft-Windows-
SmbServer/Operational event log, and the client receives an Access denied error
message.
SMB Encryption and the Encrypting File System (EFS) in the NTFS file system are
unrelated, and SMB Encryption doesn't require or depend on using EFS.
SMB Encryption and the BitLocker Drive Encryption are unrelated, and SMB
Encryption doesn't require or depend on using BitLocker Drive Encryption.

Preauthentication integrity
SMB 3.1.1 is capable of detecting interception attacks that attempt to downgrade the
protocol or the capabilities that the client and server negotiate by use of
preauthentication integrity. Preauthentication integrity is a mandatory feature in SMB
3.1.1. It protects against any tampering with Negotiate and Session Setup messages by
using cryptographic hashing. The resulting hash is used as input to derive the session’s
cryptographic keys, including its signing key. This process enables the client and server
to mutually trust the connection and session properties. When the client or the server
detects such an attack, the connection is disconnected, and event ID 1005 is logged in
the Microsoft-Windows-SmbServer/Operational event log.

Because of this protection, and to take advantage of the full capabilities of SMB
Encryption, we strongly recommend that you disable the SMB 1.0 server. For
instructions, connect to the server with Windows Admin Center and open the Files &
File Sharing extension, and then select the File shares tab to be prompted to uninstall.
For more information, see How to detect, enable and disable SMBv1, SMBv2, and SMBv3
in Windows.
New signing algorithm
SMB 3.0 and 3.02 use a more recent encryption algorithm for signing: Advanced
Encryption Standard (AES)-cipher-based message authentication code (CMAC). SMB 2.0
used the older HMAC-SHA256 encryption algorithm. AES-CMAC and AES-CCM can
significantly accelerate data encryption on most modern CPUs that have AES instruction
support.

Windows Server 2022 and Windows 11 introduce AES-128-GMAC for SMB 3.1.1 signing.
Windows automatically negotiates this better-performing cipher method when
connecting to another computer that supports it. Windows still supports AES-128-
CMAC. For more information, see Configure SMB Signing with Confidence .

Disabling SMB 1.0

SMB 1.0 isn't installed by default starting in Windows Server version 1709 and Windows
10 version 1709. For instructions on removing SMB1, connect to the server with
Windows Admin Center, open the Files & File Sharing extension, and then select the File
shares tab to be prompted to uninstall. For more information, see How to detect, enable
and disable SMBv1, SMBv2, and SMBv3 in Windows.

If it's still installed, you should disable SMB1 immediately. For more information on
detecting and disabling SMB 1.0 usage, see Stop using SMB1 . For a clearinghouse of
software that previously or currently requires SMB 1.0, see SMB1 Product
Clearinghouse .

Related links
Overview of file sharing using the SMB 3 protocol in Windows Server
Windows Server Storage documentation
Scale-Out File Server for application data overview
SMB: File and printer sharing ports
should be open
Article • 03/22/2023

Applies To: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

When a Best Practices Analyzer scan for Server Message Block (SMB)-based network
services identifies that firewall ports for file and printer sharing aren't open, follow the
steps in this article to resolve the issue.

Operating system Product/Feature Severity Category

Windows Server File Services Error Configuration

７ Note

This article addresses a specific issue identified by a Best Practices Analyzer scan.
Apply the information in this article only to computers that have a File Services Best
Practices Analyzer scan that reports the specific port issue. For more information
about best practices and scans, see Best Practices Analyzer.

Identify the issue

A File Services Best Practices Analyzer scan reports that firewall ports necessary for file
and printer sharing aren't open (ports 445 and 139).

The issue prevents computer access to shared folders and other SMB-based network
services on the server.

Resolve the issue

To resolve the issue, enable file and printer sharing to communicate through the
computer's firewall. To complete the procedure, you must be a member of the
Administrators group (or equivalent), at a minimum.

To open the firewall ports and enable file and printer sharing, complete the following
steps:
 1. Open Control Panel, select System and Security, and then select Windows
Defender Firewall.

2. On the left, select Advanced settings. The Windows Defender Firewall console
opens and shows the advanced settings.

3. In the Windows Defender Firewall console on the left, select Inbound Rules.

4. Under Inbound Rules, locate the following two rules:

File and Printer Sharing (NB-Session-In)

File and Printer Sharing (SMB-In)

5. For each rule, select and hold (or right-click) the rule, and then select Enable Rule.

Related links
Understanding shared folders and the Windows Firewall
Secure SMB Traffic in Windows Server
Article • 04/01/2022

As a defense in depth measure, you can use segmentation and isolation techniques to
secure SMB traffic and reduce threats between devices on your network.

SMB is used for file sharing, printing, and inter-process communication such as named
pipes and RPC. It's also used as a network data fabric for technologies such as Storage
Spaces Direct, Storage Replica, Hyper-V Live Migration, and Cluster Shared Volumes. Use
the following sections to configure SMB traffic segmentation and endpoint isolation to
help prevent outbound and lateral network communications.

Block inbound SMB access

Block TCP port 445 inbound from the internet at your corporate hardware firewalls.
Blocking inbound SMB traffic protects devices inside your network by preventing access
from the internet.

If you want users to access their files inbound at the edge of your network, you can use
SMB over QUIC. This uses UDP port 443 by default and provides a TLS 1.3-encrypted
security tunnel like a VPN for SMB traffic. The solution requires Windows 11 and
Windows Server 2022 Datacenter: Azure Edition file servers running on Azure Stack HCI.
For more information, see SMB over QUIC .

Block outbound SMB access

Block TCP port 445 outbound to the internet at your corporate firewall. Blocking
outbound SMB traffic prevents devices inside your network from sending data using
SMB to the internet.

It is unlikely you need to allow any outbound SMB using TCP port 445 to the internet
unless you require it as part of a public cloud offering. The primary scenarios include
Azure Files and Office 365.

If you are using Azure Files SMB, use a VPN for outbound VPN traffic. By using a VPN,
you restrict the outbound traffic to the required service IP ranges. For more information
about Azure Cloud and Office 365 IP address ranges, see:

Azure IP ranges and service tags: public cloud ,US government cloud , Germany
cloud , or China cloud . The JSON files are updated weekly and include
versioning both for the full file and each individual service tag. The AzureCloud tag
 provides the IP ranges for the cloud (Public, US government, Germany, or China)
and is grouped by region within that cloud. Service tags in the file will increase as
Azure services are added.
Office 365 URLs and IP address ranges.

With Windows 11 and Windows Server 2022 Datacenter: Azure Edition, you can use SMB
over QUIC to connect to file servers in Azure. This uses UDP port 443 by default and
provides a TLS 1.3-encrypted security tunnel like a VPN for the SMB traffic. For more
information, see SMB over QUIC .

Inventory SMB usage and shares

By inventorying your network's SMB traffic, you get an understanding of traffic that is
occurring and can determine if it's necessary. Use the following checklist of questions to
help identify unnecessary SMB traffic.

For server endpoints:

1. Which server endpoints require inbound SMB access to do their role? Do they
need inbound access from all clients, certain networks, or certain nodes?
2. Of the remaining server endpoints, is inbound SMB access necessary?

For client endpoints:

1. Which client endpoints (for example, Windows 10) require inbound SMB access?
Do they need inbound access from all clients, certain networks, or certain nodes?
2. Of the remaining client endpoints, is inbound SMB access necessary?
3. Of the remaining client endpoints, do they need to run the SMB server service?

For all endpoints, determine if you allow outbound SMB in the safest and most minimal
fashion.

Review server built-in roles and features that require SMB inbound. For example, file
servers and domain controllers require SMB inbound to do their role. For more
information on built-in roles and feature network port requirements, see Service
overview and network port requirements for Windows.

Review servers that need to be accessed from inside the network. For example, domain
controllers and file servers likely need to be accessed anywhere in the network.
However, application server access may be limited to a set of other application servers
on the same subnet. You can use the following tools and features to help you inventory
SMB access:

Use Get-FileShares script to examine shares on servers and clients.

 Enable an audit trail of SMB inbound access using the registry key Security
Settings\Advanced Audit Policy Configuration\Audit Policies\Object Access\File
Share . Since the number of events may be large, consider enabling for a specified

amount of time or use Azure Monitor .

Examining SMB logs lets you know which nodes are communicating with endpoints over
SMB. You can decide if an endpoint's shares are in use and understand which to exist.

Configure Windows Defender Firewall

Use firewall rules to add extra connection security. Configure rules to block both
inbound and outbound communications that include exceptions. An outbound firewall
policy that prevents use of SMB connections both outside and inside your managed
network while allowing access to the minimum set of servers and no other devices is a
lateral defense-in-depth measure.

For information on the SMB firewall rules you need to set for inbound and outbound
connections, see the support article Preventing SMB traffic from lateral connections and
entering or leaving the network .

The support article includes templates for:

Inbound rules that are based on any kind of network profile.

Outbound rules for private/domain (trusted) networks.
Outbound rules for guest/public (untrusted) networks. This template is important
to enforce on mobile devices and home-based telecommuters that are not behind
your firewall that is blocking outbound traffic. Enforcing these rules on laptops
reduces the odds of phishing attacks that send users to malicious servers to
harvest credentials or run attack code.
Outbound rules that contain an override allowlist for domain controllers and file
servers called Allow the connection if secure.

To use the null encapsulation IPSEC authentication, you must create a Security
Connection rule on all computers in your network that are participating in the rules.
Otherwise, the firewall exceptions won't work and you'll only be arbitrarily blocking.

Ｕ Caution

You should test the Security Connection rule before broad deployment. An
incorrect rule could prevent users from accessing their data.
To create a Connection Security rule, use Windows Defender Firewall with Advanced
Security control panel or snap-in:

1. In Windows Defender Firewall, select Connection Security Rules and choose a New
rule.
2. In Rule Type, select Isolation then select Next.
3. In Requirements, select Request authentication for inbound and outbound
connections then select Next.
4. In Authentication Method, select Computer and User (Kerberos V5) then select
Next.
5. In Profile, check all profiles (Domain, Private, Public) then select Next.
6. Enter a name your rule then select Finish.

Remember, the Connection Security rule must be created on all clients and servers
participating in your inbound and outbound rules or they will be blocked from
connecting SMB outbound. These rules may already be in place from other security
efforts in your environment and like the firewall inbound/outbound rules, can be
deployed via group policy.

When configuring rules based on the templates in the Preventing SMB traffic from
lateral connections and entering or leaving the network support article, set the
following to customize the Allow the connection if secure action:

1. In the Action step, select Allow the connection if it is secure then select
Customize.
2. In Customize Allow if Secure Settings, select Allow the connection to use null
encapsulation.

The Allow the connection if it is secure option allows override of a global block rule. You
can use the easy but least secure Allow the connection to use null encapsulation with
*override block rules, which relies on Kerberos and domain membership for
authentication. Windows Defender Firewall allows for more secure options like IPSEC.

For more information about configuring the firewall, see Windows Defender Firewall
with Advanced Security deployment overview.

Disable SMB Server if unused

Windows clients and some of your Windows Servers on your network may not require
the SMB Server service to be running. If the SMB Server service isn't required, you can
disable the service. Before disabling SMB Server service, be sure no applications and
processes on the computer require the service.
You can use Group Policy Preferences to disable the service on a large number of
machines when you are ready to implement. For more information about configuring
Group Policy Preferences, see Configure a Service Item.

Test and deploy using policy

Begin by testing using small-scale, hand-made deployments on select servers and
clients. Use phased group policy rollouts to make these changes. For example, start with
the heaviest user of SMB such as your own IT team. If your team's laptops and apps and
file share access work well after deploying your inbound and outbound firewall rules,
create test group policy within your broad test and QA environments. Based on results,
start sampling some departmental machines, then expand out.

Next steps
Watch Jessica Payne's Ignite conference session Demystifying the Windows Firewall
Protect SMB traffic from interception
Article • 12/13/2022

In this article, you'll learn about some of the ways an attacker might use interception
techniques against the SMB protocol and how you might mitigate an attack. The
concepts will support you with developing your own defense-in-depth strategy for the
SMB protocol.

What is an interception attack?

An adversary-in-the-middle (AITM) attack intends to modify the network
communication between a client and server, allowing a threat actor to intercept traffic.
After interception, a malicious actor may have the ability to spoof, tamper, disclose, or
deny access to your organizations data or account credentials.

Many organizations rely on SMB to share files between users and to support other
applications or technologies like Active Directory Domain Services. With such broad
adoption, SMB is both a popular target for attackers and has the potential for business-
wide impact.

For example, an AITM attack might be used for industrial or state-level espionage,
extortion, or finding sensitive security data stored in files. It could also be used as part of
a wider attack to enable the attacker to move laterally within your network or to target
multiple endpoints.

Attacks are constantly evolving, with attackers often using a combination of established
and new techniques. When protecting your system against SMB interception, there are
two main goals:

Reduce the number of attack methods available.

Secure the pathways you present to your users.

Due to the diversity of technology and clients within many organizations, a well-
rounded defense will combine multiple methods and will follow the Zero Trust
principles. Learn more about Zero Trust in the What is Zero Trust? article.

Now you'll learn about some of the typical good practice configurations to reduce the
risk of SMB interception.

Reducing the attack methods available

To protect your system against SMB interception attacks, your first step should be to
reduce the attack surface. Attack surfaces are places where your system is vulnerable to
cyberthreats and compromise.

In the following sections, we'll discuss some of the basic steps you should take to reduce
the attack surface.

Install updates
Regularly install all available security updates on both your Windows Server and client
systems as close to their release as your organization allows. Installing the latest security
updates is the quickest and easiest way to protect your systems from the current known
security vulnerabilities affecting not just SMB, but all Microsoft products and services.

You can install security updates using a few different methods depending on your
organizations requirements. Typical methods are:

Azure Update Management

Windows Update
Windows Server Update Services (WSUS)
Software updates in Endpoint Configuration Manager

Consider subscribing to notifications in the Microsoft Security Response Center (MSRC)

Security Update Guide . The Security Update Guide Notification System will let you
know when software updates are published to address new and existing Common
Vulnerabilities and Exposures (CVEs).

Remove SMB 1.0

You should remove or disable the SMB 1.0 feature from all Windows Servers and clients
that don't require it. For systems that do require SMB 1.0, you should move to SMB 2.0
or higher as soon as possible. Starting in the Windows 10 Fall Creators Update and
Windows Server 2019, SMB 1.0 is no longer installed by default.

 Tip

Windows 10 Home and Windows 10 Pro still contain the SMB 1.0 client by default
after a clean installation or in-place upgrade. This behavior is changing in Windows
11, you can read more in the article SMB1 now disabled by default for Windows 11
Home Insiders builds .
Removing SMB 1.0 protects your systems by eliminating several well known security
vulnerabilities. SMB 1.0 lacks the security features of SMB 2.0 and later that help protect
against interception. For example, to prevent a compromised connection SMB 3.0 or
later uses pre-authentication integrity, encryption, and signing. Learn more in the SMB
security enhancements article.

Before removing the SMB 1.0 feature, be sure no applications and processes on the
computer require it. For more information on how to detect and disable SMB 1.0, see
the article How to detect, enable and disable SMBv1, SMBv2, and SMBv3 in Windows.

You can also use the Windows Admin Center Files and file sharing tool to both quickly
enable the auditing of SMB1 client connections and to uninstall SMB 1.

Disable guest authentication and fallback

In SMB 1.0 when a user's credentials fail, the SMB client will try guest access. Starting in
Windows 10, version 1709, and Windows Server 2019, SMB2 and SMB3 clients no longer
allow guest account access or fallback to the guest account by default. You should use
SMB 2.0 or higher and disable the use of SMB guest access on any systems where guest
access isn't disabled by default.

 Tip

Windows 11 Home and Pro editions are unchanged from their previous default
behavior; they allow guest authentication by default.

When guest access is disabled, it prevents a malicious actor from creating a server and
tricking users into accessing it using guest access. For example, when a user accesses
the spoofed share, their credentials would fail and SMB 1.0 would fall back to using
guest access. Disabling guest access stops the SMB session from connecting, preventing
the user from accessing the share and any malicious files.

To prevent the use of guest fallback on Windows SMB clients where guest access isn't
disabled by default (including Windows Server):

Group Policy

1. Open the Group Policy Management Console.

2. In the console tree, select Computer Configuration > Administrative
Templates > Network > Lanman Workstation.
3. For the setting, right-click Enable insecure guest logons and select Edit.
 4. Select Enabled and select OK.

To learn more about guest access default behavior, read the article Guest access in
SMB2 and SMB3 disabled by default in Windows.

Disable the WebDAV protocol

Windows clients may not require the WebClient service to be running. The service
provides the Web Distributed Authoring and Versioning (WebDAV) protocol. If your
clients aren't accessing SMB shares over HTTP or HTTPS using WebDAV, you can disable
the service.

When your users are accessing files using WebDAV, there's no method to force a TLS
based connection over HTTPS. For example, your server may be configured to require
SMB signing or encryption, however the Webclient could connect to HTTP/80 if
WebDAV has been enabled. Any resulting connection would be unencrypted, regardless
of your SMB configuration.

You can use Group Policy Preferences to disable the service on a large number of
machines when you're ready to implement. For more information about configuring
Group Policy Preferences, see Configure a Service Item.

Restrict outbound SMB destinations

Block outbound SMB traffic to devices outside your network as a minimum. Blocking
outbound SMB prevents data being sent to external endpoints. Malicious actors often
try spoofing, tampering or phishing attacks that attempt to send users to malicious
endpoints disguised as friendly links or shortcuts within emails or other files. To learn
more about blocking outbound SMB access, read the Secure SMB Traffic in Windows
Server article.

Take this principle further by introducing micro-perimeters and micro-segmentation into

your architecture. Blocking outbound SMB traffic to external networks helps to prevent
the direct exfiltration of data to internet, however modern attacks use advanced
techniques to indirectly gain access by attacking other systems, then moving laterally
within your network. Micro-perimeters and micro-segmentation aim to reduce the
number of systems and users being able to directly connect to your SMB share unless
that explicitly need to. Learn more about Network segmentation as part of our Zero
Trust Guidance.
Secure the protocol
Your second goal is to secure the pathways between your users and their data, known as
data-in-transit protection. Data-in-transit protection typically involves the use of
encryption, interface hardening, and removing insecure protocols to improve your
resistance to attack.

In the following sections, we'll discuss some of the basic steps you should take to secure
the SMB protocol.

Use SMB 3.1.1

Windows always negotiates to the highest available protocol, ensure your devices and
machines support SMB 3.1.1.

SMB 3.1.1 is available beginning with Windows 10 and Windows Server 2016. SMB 3.1.1
includes a new mandatory security feature called pre-authentication integrity. Pre-
authentication integrity signs or encrypts the early phases of SMB connections to
prevent the tampering of Negotiate and Session Setup messages by using
cryptographic hashing.

Cryptographic hashing means the client and server can mutually trust the connection
and session properties. Pre-authentication integrity supersedes secure dialect
negotiation introduced in SMB 3.0. You can’t turn off pre-authentication integrity, but if
a client uses an older dialect, it won’t be used.

You can enhance your security posture further by forcing the use of SMB 3.1.1 as a
minimum. To set the minimum SMB dialect to 3.1.1, from an elevated PowerShell
prompt, run the following commands:

PowerShell

Set-ItemProperty -Path
"HKLM:\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters" -Name
"MinSMB2Dialect" -Value 0x000000311
Set-ItemProperty -Path
"HKLM:\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters" -Name
"MaxSMB2Dialect" -Value 0x000000311

To learn more about how to set the minimum SMB dialect used, see Controlling SMB
dialects .
Use UNC hardening to require signing, encryption, and
mutual authentication
Enable UNC hardening for all SMB shares by requiring at least mutual authentication
(Kerberos) and integrity (SMB signing). You should also consider evaluating privacy (SMB
encryption) instead of SMB signing. There's no need to configure both SMB signing and
encryption because encryption implicitly includes the signatures used by signing.

Ｕ Caution

SMB encryption was introduced with SMB 3 in Windows 8 and Windows Server
2012. You shouldn't require encryption unless all your machines support SMB 3.0 or
later, or are third parties with SMB 3 and encryption support. If you configure SMB
encryption on clients or UNC paths hosted by servers that do not support SMB
Encryption, the SMB client will be unable to access the specified path. Also, if you
configure your server for SMB encryption and it is accessed by clients that don't
support it, those clients will again be unable to access the path.

UNC Hardening gives the ability to check UNC paths for mandated security settings and
will refuse to connect if a server couldn't meet them. Beginning with Windows Server
2016 and Windows 10, UNC Hardening is enabled by default for SYSVOL and
NETLOGON shares on domain controllers. It's a highly effective tool against spoofing
and tampering because the client can authenticate the identity of the server and
validate the integrity of the SMB payloads.

When configuring UNC hardening, you can specify various UNC path patterns. For
example:

\\<Server>\<Share> - The configuration entry applies to the share that has the
specified name on the specified server.
\\*\<Share> - The configuration entry applies to the share that has the specified
name on any server.
\\<Server>\* - The configuration entry applies to any share on the specified

server.

You can use Group Policy to apply the UNC hardening feature to a large number of
machines when you're ready to implement it. For more information about configuring
UNC hardening through Group Policy, see the security bulletin MS15-011 .
Map drives on demand with mandated signing or
encryption
In addition to UNC hardening, you can use signing or encryption when mapping
network drives. Beginning in Windows version 1709 and later, you can create encrypted
or signed mapped drives on demand using Windows PowerShell or Command Prompt.
You can use the NET USE command or the PowerShell New-SmbMapping command to map
drives by specifying RequireIntegrity (signing) or RequirePrivacy (encryption)
parameters.

The commands can be used by administrators or included in scripts to automate the

mapping of drives that require encryption or integrity checks.

The parameters don't change how signing or encryption work, or the dialect
requirements. If you try to map a drive and the server refuses to honor your requirement
for signing or encryption, the drive mapping will fail rather than connect unsafely.

Learn about the syntax and parameters for the New-SmbMapping command in New-
SmbMapping reference article.

Beyond SMB
Stop using NTLM and increase your Kerberos security. You can start by enabling
auditing for NTLM usage, then reviewing the logs to find where NTLM is used.

Removing NTLM helps to protect you against common attacks like pass-the-hash,
brute-force or rainbow hash tables due to its use of older MD4/MD5 cryptography hash
function. NTLM also isn't able to verify the server identity, unlike more recent protocols
like Kerberos, making it vulnerable to NTLM relay attacks as well. Many of these
common attacks are easily mitigated with Kerberos.

To learn how to audit NTLM as part of your effort to begin the transition to Kerberos,
see the Assessing NTLM usage article. You can also read learn about detecting insecure
protocols using Azure Sentinel in the Azure Sentinel Insecure Protocols Workbook
Implementation Guide blog article.

In parallel to removing NTLM, you should consider adding more layers of protection for
offline and ticket passing attacks. Use the following items as a guide when enhancing
Kerberos security.

1. Deploy Windows Hello for Business or smart cards - Two-factor authentication

with Windows Hello for Business adds an entire new layer of protection. Learn
about Windows Hello for Business.
 2. Enforce long passwords and phrases - We encourage using longer password
lengths such as 15 characters or more to reduce your resistance to brute force
attacks. You should also avoid common words or phrases to make your password
even stronger.
3. Deploy Azure AD Password Protection for Active Directory Domain Services -
Use Azure AD Password Protect to block known weak passwords and terms that
are specific to your organization. To learn more, review Enforce on-premises Azure
AD Password Protection for Active Directory Domain Services.
4. Use group Managed Service Accounts (gMSA) - gMSA enabled services with their
127 random character construction, makes brute force and dictionary attacks to
crack passwords incredibly time consuming. Read about what gMSAs are in the
article Group Managed Service Accounts Overview.
5. Kerberos Armoring, known as Flexible Authentication Secure Tunneling (FAST) -
FAST prevents Kerberoasting because the user’s pre-authentication data is
protected and no longer subject to offline brute force or dictionary attacks. It also
prevents downgrade attacks from spoofed KDCs, to learn more review Kerberos
Armoring.
6. Use Windows Defender Credential Guard - Credential Guard makes the local
compromise of tickets harder by preventing ticket-granting and cached service
tickets from being stolen. Learn more in the How Windows Defender Credential
Guard works article.
7. Consider requiring SCRIL: Smart Card Required for Interactive Logon - When
deploying SCRIL, AD changes the user's password to a random 128-bit set which
users can no longer use to sign in interactively. SCRIL is typically only suitable for
environments with specific security requirements. To learn more about
passwordless strategies, review Configuring user accounts to disallow password
authentication.

Next steps
Now you've learned about some of the security controls and mitigations to prevent SMB
interception, you'll understand there’s no single step to prevent all interception attacks.
The goal is to create a thoughtful, holistic, and prioritized combination of risk
mitigations spanning multiple technologies through layered defenses.

You can continue to learn more about these concepts in the articles below.

Secure SMB traffic in Windows Server.

SMB security enhancements
SMB 3.1.1 Pre-authentication integrity in Windows 10
SMB 2 and SMB 3 security in Windows 10: the anatomy of signing and
cryptographic keys
Zero Trust Guidance Center
Network File System overview
Article • 12/07/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012

This article describes the Network File System role service and features included with the
File and Storage Services server role in Windows Server. Network File System (NFS)
provides a file sharing solution for enterprises that have heterogeneous environments
that include both Windows and non-Windows computers.

Feature description
Using the NFS protocol, you can transfer files between computers running Windows and
other non-Windows operating systems, such as Linux or UNIX.

NFS in Windows Server includes Server for NFS and Client for NFS. A computer running
Windows Server can use Server for NFS to act as a NFS file server for other non-
Windows client computers. Client for NFS allows a Windows-based computer running
Windows Server to access files stored on a non-Windows NFS server.

Windows and Windows Server versions

Windows supports multiple versions of the NFS client and server, depending on
operating system version and family.

Operating Systems NFS Server NFS Client

Versions Versions

Windows 7, Windows 8.1, Windows 10, Windows 11 N/A NFSv2,

NFSv3

Windows Server 2008, Windows Server 2008 R2 NFSv2, NFSv2,

NFSv3 NFSv3

Windows Server 2012, Windows Server 2012 R2, Windows Server NFSv2, NFSv2,
2016, Windows Server 2019, Windows Server 2022 NFSv3, NFSv3
NFSv4.1

Practical applications
Here are some ways you can use NFS:

Use a Windows NFS file server to provide multi-protocol access to the same file
share over both SMB and NFS protocols from multi-platform clients.
Deploy a Windows NFS file server in a predominantly non-Windows operating
system environment to provide non-Windows client computers access to NFS file
shares.
Migrate applications from one operating system to another by storing the data on
file shares accessible through both SMB and NFS protocols.

New and changed functionality

New and changed functionality in Network File System includes support for the NFS
version 4.1 and improved deployment and manageability. For information about
functionality that is new or changed in Windows Server 2012, review the following table:

Feature/functionality New or Description

updated

NFS version 4.1 New Increased security, performance, and interoperability

compared to NFS version 3.

NFS infrastructure Updated Improves deployment and manageability, and increases

security.

NFS version 3 Updated Improves continuous availability on NFS version 3 clients.

continuous availability

Deployment and Updated Enables you to easily deploy and manage NFS with new
manageability Windows PowerShell cmdlets and a new WMI provider.
improvements

NFS version 4.1

NFS version 4.1 implements all of the required aspects, in addition to some of the
optional aspects, of RFC 5661 :

Pseudo file system, a file system that separates physical and logical namespace
and is compatible with NFS version 3 and NFS version 2. An alias is provided for
the exported file system, which is part of the pseudo file system.
Compound RPCs combine relevant operations and reduce chattiness.
Sessions and session trunking enables just one semantic and allows continuous
availability and better performance while utilizing multiple networks between NFS
4.1 clients and the NFS Server.
NFS infrastructure
Improvements to the overall NFS infrastructure in Windows Server 2012 are detailed
below:

The Remote Procedure Call (RPC)/External Data Representation (XDR) transport

infrastructure, powered by the WinSock network protocol, is available for both
Server for NFS and Client for NFS. This replaces Transport Device Interface (TDI),
offers better support, and provides better scalability and Receive Side Scaling
(RSS).
The RPC port multiplexer feature is firewall-friendly (less ports to manage) and
simplifies deployment of NFS.
Auto-tuned caches and thread pools are resource management capabilities of the
new RPC/XDR infrastructure that are dynamic, automatically tuning caches and
thread pools based on workload. This completely removes the guesswork involved
when tuning parameters, providing optimal performance as soon as NFS is
deployed.
New Kerberos privacy implementation and authentication options with the
addition of Kerberos privacy (Krb5p) support along with the existing krb5 and krb5i
authentication options.
Identity Mapping Windows PowerShell module cmdlets make it easier to manage
identity mapping, configure Active Directory Lightweight Directory Services (AD
LDS), and set up UNIX and Linux passwd and flat files.
Volume mount point lets you access volumes mounted under an NFS share with
NFS version 4.1.
The Port Multiplexing feature supports the RPC port multiplexer (port 2049),
which is firewall-friendly and simplifies NFS deployment.

NFS version 3 continuous availability

NFS version 3 clients can have fast and transparent planned failovers with more
availability and reduced downtime. The failover process is faster for NFS version 3 clients
because:

The clustering infrastructure now allows one resource per network name instead of
one resource per share, which significantly improves resources' failover time.
Failover paths within an NFS server are tuned for better performance.
Wildcard registration in an NFS server is no longer required, and the failovers are
more fine-tuned.
Network Status Monitor (NSM) notifications are sent out after a failover process,
and clients no longer need to wait for TCP timeouts to reconnect to the failed over
 server.

Note that Server for NFS supports transparent failover only when manually initiated,
typically during planned maintenance. If an unplanned failover occurs, NFS clients lose
their connections. Server for NFS also doesn't have any integration with the Resume Key
filter. This means that if a local app or SMB session attempts to access the same file that
an NFS client is accessing immediately after a planned failover, the NFS client might lose
its connections (transparent failover wouldn't succeed).

Deployment and manageability improvements

Deploying and managing NFS has improved in the following ways:

Over forty new Windows PowerShell cmdlets make it easier to configure and
manage NFS file shares. For more information, see NFS Cmdlets in Windows
PowerShell.
Identity mapping is improved with a local flat file mapping store and new Windows
PowerShell cmdlets for configuring identity mapping.
The Server Manager graphical user interface is easier to use.
The new WMI version 2 provider is available for easier management.
The RPC port multiplexer (port 2049) is firewall-friendly and simplifies deployment
of NFS.

Server Manager information

In Server Manager - or the newer Windows Admin Center - use the Add Roles and
Features Wizard to add the Server for NFS role service (under the File and iSCSI Services
role). For general information about installing features, see Install or Uninstall Roles, Role
Services, or Features. Server for NFS tools includes the Services for Network File System
MMC snap-in to manage the Server for NFS and Client for NFS components. Using the
snap-in, you can manage the Server for NFS components installed on the computer.
Server for NFS also contains several Windows command-line administration tools:

Mount mounts a remote NFS share (also known as an export) locally and maps it
to a local drive letter on the Windows client computer.
Nfsadmin manages configuration settings of the Server for NFS and Client for NFS
components.
Nfsshare configures NFS share settings for folders that are shared using Server for
NFS.
Nfsstat displays or resets statistics of calls received by Server for NFS.
Showmount displays mounted file systems exported by Server for NFS.
 Umount removes NFS-mounted drives.

NFS in Windows Server 2012 introduces the NFS module for Windows PowerShell with
several new cmdlets specifically for NFS. These cmdlets provide an easy way to
automate NFS management tasks. For more information, see NFS cmdlets in Windows
PowerShell.

Additional information
The following table provides additional resources for evaluating NFS.

Content type References

Deployment Deploy Network File System

Operations NFS cmdlets in Windows PowerShell

Related technologies Storage in Windows Server

Deploy Network File System
Article • 03/29/2023

Applies to: Windows Server 2022, Windows Server 2019, and Windows Server 2016.

Network File System (NFS) provides a file sharing solution that lets you transfer files
between computers running Windows Server and UNIX operating systems by using the
NFS protocol. This article describes the steps you should follow to deploy NFS.

What's new in Network File System

Here's what's changed for NFS in Windows Server:

Support for NFS version 4.1: This protocol version includes the following
enhancements.
Makes navigating firewalls easier, which improves accessibility.
Supports the RPCSEC_GSS protocol, providing stronger security and allowing
clients and servers to negotiate security.
Supports UNIX and Windows file semantics.
Takes advantage of clustered file server deployments.
Supports WAN-friendly compound procedures.

NFS module for Windows PowerShell: The availability of built-in NFS cmdlets
makes it easier to automate various operations. The cmdlet names are consistent
with other Windows PowerShell cmdlets (with verbs such as "Get" and "Set"),
making it easier for users familiar with Windows PowerShell to learn to use new
cmdlets.

NFS management improvements: A new centralized UI-based management

console simplifies configuration and management of SMB and NFS shares, quotas,
file screens, and classification, in addition to managing clustered file servers.

Identity Mapping improvements: This improvement includes new UI support and

task-based Windows PowerShell cmdlets for configuring identity mapping.
Administrators can quickly configure an identity mapping source, and then create
individual mapped identities for users. Improvements make it easy for
administrators to set up a share for multi-protocol access over both NFS and SMB.

Cluster resource model restructure: This improvement brings consistency between

the cluster resource model for the Windows NFS and SMB protocol servers and
simplifies administration. For NFS servers that have many shares, the resource
 network and the number of WMI calls required fail over a volume containing a
large number of NFS shares are reduced.

Integration with Resume Key Manager: The Resume Key Manager tracks file
server and file system state. The tool enables the Windows SMB and NFS protocol
servers to fail over without disrupting clients or server applications that store their
data on the file server. This improvement is a key component of the continuous
availability capability of the file server running Windows Server.

Scenarios for using Network File System

NFS supports a mixed environment of Windows-based and UNIX-based operating
systems. The following deployment scenarios are examples of how you can deploy a
continuously available Windows Server file server by using NFS.

Provision file shares in heterogeneous environments

This scenario applies to organizations with heterogeneous environments that consist of
both Windows and other operating systems, such as UNIX or Linux-based client
computers. With this scenario, you can provide multi-protocol access to the same file
share over both the SMB and NFS protocols. Typically, when you deploy a Windows file
server in this scenario, you want to facilitate collaboration between users on Windows
and UNIX-based computers. When a file share is configured, it's shared with both the
SMB and NFS protocols. Windows users access their files over the SMB protocol, and
users on UNIX-based computers typically access their files over the NFS protocol.

For this scenario, you must have a valid identity mapping source configuration. Windows
Server supports the following identity mapping stores:

Mapping File
Active Directory Domain Services (AD DS)
RFC 2307-compliant LDAP stores such as Active Directory Lightweight Directory
Services (AD LDS)
User Name Mapping (UNM) server

Provision file shares in UNIX-based environments

In this scenario, Windows file servers are deployed in a predominantly UNIX-based
environment to provide access to NFS file shares for UNIX-based client computers. An
Unmapped UNIX User Access (UUUA) option was initially implemented for NFS shares in
Windows Server 2008 R2. This option enables Windows servers to store NFS data
without creating UNIX-to-Windows account mapping. UUUA allows administrators to
quickly provision and deploy NFS without having to configure account mapping. When
it is enabled for NFS, UUUA creates custom security identifiers (SIDs) to represent
unmapped users. Mapped user accounts use standard Windows SIDs, and unmapped
user accounts use custom NFS SIDs.

System requirements
Server for NFS can be installed on any version of Windows Server. You can use NFS with
UNIX-based computers that are running an NFS server or NFS client, if these NFS server
and client implementations comply with one of the following protocol specifications:

NFS Version 4.1 Protocol Specification (as defined in RFC 5661 )

NFS Version 3 Protocol Specification (as defined in RFC 1813 )
NFS Version 2 Protocol Specification (as defined in RFC 1094 )

Deploy NFS infrastructure

You need to deploy the following computers and connect them on a local area network
(LAN):

One or more computers running Windows Server on which you'll install the two
main Services for NFS components: Server for NFS and Client for NFS. You can
install these components on the same computer or on different computers.
One or more UNIX-based computers that are running NFS server and NFS client
software. The UNIX-based computer that is running NFS server hosts an NFS file
share or export, which is accessed by a computer that is running Windows Server
as a client by using Client for NFS. You can install NFS server and client software
either in the same UNIX-based computer or on different UNIX-based computers,
as desired.
A domain controller running at the Windows Server 2008 R2 functional level. The
domain controller provides user authentication information and mapping for the
Windows environment.
When a domain controller isn't deployed, you can use a Network Information
Service (NIS) server to provide user authentication information for the UNIX
environment. Or, if you prefer, you can use password and group files that are
stored on the computer that's running the User Name Mapping service.

Install Network File System on the server with Server

Manager
 1. From the Add Roles and Features Wizard, under Server Roles, expand File and
Storage Services > expand File and iSCSI Services.

2. Select File Server and Server for NFS, select Next.

3. A dialog box lets you know what other tools are required for the selected feature.

Check the box for the required features, select Add Features.

4. Select Next, and then choose any other preferred features. When you're ready,
select Next.

5. To install the NFS components on the server, select Install.

Install Network File System on the server with Windows

PowerShell
1. Start Windows PowerShell. Select and hold (or right-click) the PowerShell icon on
the taskbar, and select Run as Administrator.

2. Run the following Windows PowerShell commands:

PowerShell

Import-Module ServerManager
Add-WindowsFeature FS-NFS-Service
Import-Module NFS

Configure NFS authentication

For the NFS version 4.1 and NFS version 3.0 protocols, we recommend that you use
Kerberos (RPCSEC_GSS). There are three options with increasing levels of security
protection:

Krb5: Uses the Kerberos version 5 protocol to authenticate users before granting
them access to the file share.

Krb5i: Uses the Kerberos version 5 protocol to authenticate with integrity checking
(checksums), which verifies that the data hasn't been altered.

Krb5p: Uses the Kerberos version 5 protocol, which authenticates NFS traffic with
encryption for privacy. This option is the most secure Kerberos option.
 ７ Note

You can also choose not to use the preceding Kerberos authentication
methods and instead enable unmapped user access through AUTH_SYS. We
strongly discourage using this option as it removes all authentication
protections and allows any user with access to the NFS server to access data.
When you use unmapped user access, you can specify to allow unmapped
user access by UID or GID, which is the default. You can also allow anonymous
access.

Instructions for configuring NFS authentication are discussed in the following section.

Create an NFS file share

You can create an NFS file share by using either Server Manager or Windows PowerShell
NFS cmdlets.

Create an NFS file share with Server Manager

1. Sign in to the server as a member of the local Administrators group.

2. Server Manager starts automatically.

If the tool doesn't automatically start, select Start. Enter servermanager.exe,

and then select Server Manager.

3. On the left, select File and Storage Services, then select Shares.

4. Under the Shares column, select To create a file share, start the New Share
Wizard.

5. On the Select Profile page, select either NFS Share - Quick or NFS Share -
Advanced, then select Next.

6. On the Share Location page, select a server and a volume, then select Next.

7. On the Share Name page, enter a name for the new share, then select Next.

8. On the Authentication page, specify the authentication method you want to use,
then select Next.

9. On the Share Permissions page, select Add. The Add Permissions dialog opens.
 a. Choose the level of user permissions to grant: Host, Netgroup, Client group, or
All Machines.

b. For the selected user level, enter the name for the user(s) to grant permission to
the share.

c. Use the drop-down menu to select the preferred Language encoding.

d. Use the drop-down menu to select the preferred Share permissions.

e. (Optional) Select the Allow root access checkbox. This option isn't
recommended.

f. Select Add. The dialog closes. Select Next.

10. On the Permissions page, configure access control for your selected users. When
you're ready, select Next.

11. On the Confirmation page, review your configuration, and select Create to create
the NFS file share.

Windows PowerShell equivalent commands

The following Windows PowerShell cmdlet can also create an NFS file share (where nfs1
is the name of the share and C:\\shares\\nfsfolder is the file path):

PowerShell

New-NfsShare -Name nfs1 -Path C:\shares\nfsfolder

Known issue
NFS version 4.1 allows the file names to be created or copied with illegal characters. If
you attempt to open the files with vi editor, it shows the files as being corrupt. You can't
save the file from vi, rename, move it, or change permissions. So avoid using illegal
characters.
NTFS overview
Article • 03/24/2023

Applies to: Windows Server 2022, Windows 10, Windows Server 2019, Windows
Server 2016, Windows Server 2012 R2, Windows Server 2012, Windows Server 2008
R2, Windows Server 2008

NTFS, the primary file system for recent versions of Windows and Windows Server,
provides a full set of features including security descriptors, encryption, disk quotas, and
rich metadata. It can be used with Cluster Shared Volumes (CSV) to provide continuously
available volumes that can be accessed simultaneously from multiple nodes of a failover
cluster.

For more feature information, see the Related links section of this article. To learn about
the newer Resilient File System (ReFS), see Resilient File System (ReFS) overview.

Increased reliability
NTFS uses its log file and checkpoint information to restore the consistency of the file
system when the computer is restarted after a system failure. After a bad-sector error,
NTFS dynamically remaps the cluster that contains the bad sector, and allocates a new
cluster for the data. It also marks the original cluster as bad, and no longer uses the old
cluster. For example, after a server crash, NTFS can recover data by replaying its log files.

NTFS continuously monitors and corrects transient corruption issues in the background
without taking the volume offline. This feature is known as self-healing NTFS, which was
introduced in Windows Server 2008.

For larger corruption issues, the Chkdsk utility, in Windows Server 2012 and later, scans
and analyzes the drive while the volume is online, limiting time offline to the time
required to restore data consistency on the volume. When you use NTFS with Cluster
Shared Volumes, no downtime is required. For more information, see NTFS Health and
Chkdsk.

Increased security
Access Control List (ACL)-based security for files and folders: NTFS lets you set
permissions on a file or folder, specify the groups and users whose access you
want to restrict or allow, and select access type.
 Support for BitLocker Drive Encryption: BitLocker Drive Encryption provides more
security for critical system information and other data stored on NTFS volumes.
Beginning in Windows Server 2012 R2 and Windows 8.1, BitLocker provides
support for device encryption on x86 and x64-based computers with a Trusted
Platform Module (TPM) that supports connected stand-by (previously available
only on Windows RT devices). Device encryption helps protect data on Windows-
based computers, and it helps block malicious users from accessing the system
files they rely on to discover the user's password. It also prevents malicious users
from accessing a drive by physically removing it from the PC and installing it on a
different one. For more information, see What's new in BitLocker.

Support for large volumes

NTFS can support volumes as large as 8 petabytes on Windows Server 2019 and newer
and Windows 10, version 1709 and newer (older versions support up to 256 TB).
Supported volume sizes are affected by the cluster size and the number of clusters. With
(232 – 1) clusters (the maximum number of clusters that NTFS supports), the following
volume and file sizes are supported.

Cluster size Largest volume and file

4 KB (default size) 16 TB

8 KB 32 TB

16 KB 64 TB

32 KB 128 TB

64 KB (earlier max) 256 TB

128 KB 512 TB

256 KB 1 PB

512 KB 2 PB

1024 KB 4 PB

2048 KB (max size) 8 PB

If you try to mount a volume with a cluster size larger than the supported maximum of
the Windows version you're using, you get the error STATUS_UNRECOGNIZED_VOLUME.

） Important
 Services and apps might impose additional limits on file and volume sizes. For
example, the volume size limit is 64 TB if you're using the Previous Versions feature
or a backup app that makes use of Volume Shadow Copy Service (VSS) snapshots
(and you're not using a SAN or RAID enclosure). However, you might need to use
smaller volume sizes depending on your workload and the performance of your
storage.

Formatting requirements for large files

To allow proper extension of large VDHX files, there are new recommendations for
formatting volumes. When formatting volumes that you use with Data Deduplication or
that host very large files, such as VDHX files larger than 1 TB, use the Format-Volume
cmdlet in Windows PowerShell with the following parameters.

Parameter Description

- Sets a 64-KB NTFS allocation unit size.

AllocationUnitSize
64KB

-UseLargeFRS Enables support for large file record segments (FRS). Using this parameter
increases the number of extents allowed per file on the volume. For large FRS
records, the limit increases from about 1.5 million extents to about 6 million
extents.

For example, the following cmdlet formats drive D as an NTFS volume, with FRS enabled
and an allocation unit size of 64 KB.

PowerShell

Format-Volume -DriveLetter D -FileSystem NTFS -AllocationUnitSize 64KB -

UseLargeFRS

You also can use the format command. At a system command prompt, enter the
following command, where /L formats a large FRS volume and /A:64k sets a 64-KB
allocation unit size:

PowerShell

format /L /A:64k
Maximum file name and path
NTFS supports long file names and extended-length paths, with the following maximum
values:

Support for long file names, with backward compatibility: NTFS supports long file
names, storing an 8.3 alias on disk (in Unicode) to provide compatibility with file
systems that impose an 8.3 limit on file names and extensions. If needed (for
performance reasons), you can selectively disable 8.3 aliasing on individual NTFS
volumes in Windows Server 2008 R2, Windows 8, and more recent versions of the
Windows operating system. In Windows Server 2008 R2 and later systems, short
names are disabled by default when a volume is formatted using the operating
system. For application compatibility, short names still are enabled on the system
volume.

Support for extended-length paths: Many Windows API functions have Unicode
versions that allow an extended-length path of approximately 32,767 characters.
That total is well beyond the 260-character path limit defined by the MAX_PATH
setting. For detailed file name and path format requirements, and guidance for
implementing extended-length paths, see Naming files, paths, and namespaces.

Clustered storage: When used in failover clusters, NTFS supports continuously

available volumes that can be accessed by multiple cluster nodes simultaneously
when used with the Cluster Shared Volumes (CSV) file system. For more
information, see Use Cluster Shared Volumes in a failover cluster.

Flexible allocation of capacity

If the space on a volume is limited, NTFS provides the following ways to work with the
storage capacity of a server:

Use disk quotas to track and control disk space usage on NTFS volumes for
individual users.
Use file system compression to maximize the amount of data that can be stored.
Increase the size of an NTFS volume by adding unallocated space from the same
disk or from a different disk.
Mount a volume at any empty folder on a local NTFS volume if you run out of
drive letters or need to create extra space that is accessible from an existing folder.

Related links
Cluster size recommendations for ReFS and NTFS
Resilient File System (ReFS) overview
What's New in NTFS for Windows Server (Windows Server 2012 R2)
What's New in NTFS (Windows Server 2008 R2, Windows 7)
NTFS Health and Chkdsk
Self-Healing NTFS (introduced in Windows Server 2008)
Transactional NTFS (introduced in Windows Server 2008)
Windows Server Storage documentation
Volume Shadow Copy Service
Article • 12/07/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, and Windows Server 2008 R2,
Windows Server 2008, Windows 10, Windows 8.1, Windows 8, Windows 7

Backing up and restoring critical business data can be very complex due to the following
issues:

The data usually needs to be backed up while the applications that produce the
data are still running. This means that some of the data files might be open or they
might be in an inconsistent state.

If the data set is large, it can be difficult to back up all of it at one time.

Correctly performing backup and restore operations requires close coordination

between the backup applications, the line-of-business applications that are being
backed up, and the storage management hardware and software. The Volume Shadow
Copy Service (VSS), which was introduced in Windows Server® 2003, facilitates the
conversation between these components to allow them to work better together. When
all the components support VSS, you can use them to back up your application data
without taking the applications offline.

VSS coordinates the actions that are required to create a consistent shadow copy (also
known as a snapshot or a point-in-time copy) of the data that is to be backed up. The
shadow copy can be used as-is, or it can be used in scenarios such as the following:

You want to back up application data and system state information, including
archiving data to another hard disk drive, to tape, or to other removable media.

You are data mining.

You are performing disk-to-disk backups.

You need a fast recovery from data loss by restoring data to the original Logical
Unit Number (LUN) or to an entirely new LUN that replaces an original LUN that
failed.

Windows features and applications that use VSS include the following:

Windows Server Backup (https://go.microsoft.com/fwlink/?LinkId=180891 )

 Shadow Copies of Shared Folders (https://go.microsoft.com/fwlink/?
LinkId=142874 )

System Center Data Protection Manager (https://go.microsoft.com/fwlink/?

LinkId=180892 )

System Restore (https://support.microsoft.com/windows/use-system-restore-

a5ae3ed9-07c4-fd56-45ee-096777ecd14e )

How Volume Shadow Copy Service Works

A complete VSS solution requires all of the following basic parts:

VSS service Part of the Windows operating system that ensures the other components
can communicate with each other properly and work together.

VSS requester The software that requests the actual creation of shadow copies (or
other high-level operations like importing or deleting them). Typically, this is the backup
application. The Windows Server Backup utility and the System Center Data Protection
Manager application are VSS requesters. Non-Microsoft® VSS requesters include nearly
all backup software that runs on Windows.

VSS writer The component that guarantees we have a consistent data set to back up.
This is typically provided as part of a line-of-business application, such as SQL Server®
or Exchange Server. VSS writers for various Windows components, such as the registry,
are included with the Windows operating system. Non-Microsoft VSS writers are
included with many applications for Windows that need to guarantee data consistency
during back up.

VSS provider The component that creates and maintains the shadow copies. This can
occur in the software or in the hardware. The Windows operating system includes a VSS
provider that uses copy-on-write. If you use a storage area network (SAN), it is
important that you install the VSS hardware provider for the SAN, if one is provided. A
hardware provider offloads the task of creating and maintaining a shadow copy from
the host operating system.

The following diagram illustrates how the VSS service coordinates with requesters,
writers, and providers to create a shadow copy of a volume.
Figure 1 Architectural diagram of Volume Shadow Copy Service

How a Shadow Copy Is Created

This section puts the various roles of the requester, writer, and provider into context by
listing the steps that need to be taken to create a shadow copy. The following diagram
shows how the Volume Shadow Copy Service controls the overall coordination of the
requester, writer, and provider.

Figure 2 Shadow copy creation process

To create a shadow copy, the requester, writer, and provider perform the following
actions:
1. The requester asks the Volume Shadow Copy Service to enumerate the writers,
gather the writer metadata, and prepare for shadow copy creation.

2. Each writer creates an XML description of the components and data stores that
need to be backed up and provides it to the Volume Shadow Copy Service. The
writer also defines a restore method, which is used for all components. The Volume
Shadow Copy Service provides the writer's description to the requester, which
selects the components that will be backed up.

3. The Volume Shadow Copy Service notifies all the writers to prepare their data for
making a shadow copy.

4. Each writer prepares the data as appropriate, such as completing all open
transactions, rolling transaction logs, and flushing caches. When the data is ready
to be shadow-copied, the writer notifies the Volume Shadow Copy Service.

5. The Volume Shadow Copy Service tells the writers to temporarily freeze application
write I/O requests (read I/O requests are still possible) for the few seconds that are
required to create the shadow copy of the volume or volumes. The application
freeze is not allowed to take longer than 60 seconds. The Volume Shadow Copy
Service flushes the file system buffers and then freezes the file system, which
ensures that the file system metadata is recorded correctly and the data to be
shadow-copied is written in a consistent order.

6. The Volume Shadow Copy Service tells the provider to create the shadow copy.
The shadow copy creation period lasts no more than 10 seconds, during which all
write I/O requests to the file system remain frozen.

7. The Volume Shadow Copy Service releases file system write I/O requests.

8. VSS tells the writers to thaw application write I/O requests. At this point
applications are free to resume writing data to the disk that is being shadow-
copied.

７ Note

The shadow copy creation can be aborted if the writers are kept in the freeze state
for longer than 60 seconds or if the providers take longer than 10 seconds to
commit the shadow copy.

9. The requester can retry the process (go back to step 1) or notify the administrator
to retry at a later time.
 10. If the shadow copy is successfully created, the Volume Shadow Copy Service
returns the location information for the shadow copy to the requester. In some
cases, the shadow copy can be temporarily made available as a read-write volume
so that VSS and one or more applications can alter the contents of the shadow
copy before the shadow copy is finished. After VSS and the applications make their
alterations, the shadow copy is made read-only. This phase is called Auto-recovery,
and it is used to undo any file-system or application transactions on the shadow
copy volume that were not completed before the shadow copy was created.

How the Provider Creates a Shadow Copy

A hardware or software shadow copy provider uses one of the following methods for
creating a shadow copy:

Complete copy This method makes a complete copy (called a "full copy" or "clone") of
the original volume at a given point in time. This copy is read-only.

Copy-on-write This method does not copy the original volume. Instead, it makes a
differential copy by copying all changes (completed write I/O requests) that are made to
the volume after a given point in time.

Redirect-on-write This method does not copy the original volume, and it does not
make any changes to the original volume after a given point in time. Instead, it makes a
differential copy by redirecting all changes to a different volume.

Complete copy
A complete copy is usually created by making a "split mirror" as follows:

1. The original volume and the shadow copy volume are a mirrored volume set.

2. The shadow copy volume is separated from the original volume. This breaks the
mirror connection.

After the mirror connection is broken, the original volume and the shadow copy volume
are independent. The original volume continues to accept all changes (write I/O
requests), while the shadow copy volume remains an exact read-only copy of the
original data at the time of the break.

Copy-on-write method
In the copy-on-write method, when a change to the original volume occurs (but before
the write I/O request is completed), each block to be modified is read and then written
to the volume's shadow copy storage area (also called its "diff area"). The shadow copy
storage area can be on the same volume or a different volume. This preserves a copy of
the data block on the original volume before the change overwrites it.

Time Source data (status and data) Shadow copy (status and data)

T0 Original data: 1 2 3 4 5 No copy: —

T1 Data changed in cache: 3 to 3' Shadow copy created (differences only): 3

T2 Original data overwritten: 1 2 3' 4 5 Differences and index stored on shadow copy: 3

Table 1 The copy-on-write method of creating shadow copies

The copy-on-write method is a quick method for creating a shadow copy, because it
copies only data that is changed. The copied blocks in the diff area can be combined
with the changed data on the original volume to restore the volume to its state before
any of the changes were made. If there are many changes, the copy-on-write method
can become expensive.

Redirect-on-write method
In the redirect-on-write method, whenever the original volume receives a change (write
I/O request), the change is not applied to the original volume. Instead, the change is
written to another volume's shadow copy storage area.

Time Source data (status and data) Shadow copy (status and data)

T0 Original data: 1 2 3 4 5 No copy: —

T1 Data changed in cache: 3 to 3' Shadow copy created (differences only): 3'

T2 Original data unchanged: 1 2 3 4 5 Differences and index stored on shadow copy: 3'

Table 2 The redirect-on-write method of creating shadow copies

Like the copy-on-write method, the redirect-on-write method is a quick method for
creating a shadow copy, because it copies only changes to the data. The copied blocks
in the diff area can be combined with the unchanged data on the original volume to
create a complete, up-to-date copy of the data. If there are many read I/O requests, the
redirect-on-write method can become expensive.
Shadow Copy Providers
There are two types of shadow copy providers: hardware-based providers and software-
based providers. There is also a system provider, which is a software provider that is
built in to the Windows operating system.

Hardware-based providers
Hardware-based shadow copy providers act as an interface between the Volume
Shadow Copy Service and the hardware level by working in conjunction with a hardware
storage adapter or controller. The work of creating and maintaining the shadow copy is
performed by the storage array.

Hardware providers always take the shadow copy of an entire LUN, but the Volume
Shadow Copy Service only exposes the shadow copy of the volume or volumes that
were requested.

A hardware-based shadow copy provider makes use of the Volume Shadow Copy
Service functionality that defines the point in time, allows data synchronization,
manages the shadow copy, and provides a common interface with backup applications.
However, the Volume Shadow Copy Service does not specify the underlying mechanism
by which the hardware-based provider produces and maintains shadow copies.

Software-based providers
Software-based shadow copy providers typically intercept and process read and write
I/O requests in a software layer between the file system and the volume manager
software.

These providers are implemented as a user-mode DLL component and at least one
kernel-mode device driver, typically a storage filter driver. Unlike hardware-based
providers, software-based providers create shadow copies at the software level, not the
hardware level.

A software-based shadow copy provider must maintain a "point-in-time" view of a

volume by having access to a data set that can be used to re-create volume status
before the shadow copy creation time. An example is the copy-on-write technique of
the system provider. However, the Volume Shadow Copy Service places no restrictions
on what technique the software-based providers use to create and maintain shadow
copies.
A software provider is applicable to a wider range of storage platforms than a hardware-
based provider, and it should work with basic disks or logical volumes equally well. (A
logical volume is a volume that is created by combining free space from two or more
disks.) In contrast to hardware shadow copies, software providers consume operating
system resources to maintain the shadow copy.

For more information about basic disks, see What Are Basic Disks and Volumes?.

System provider
One shadow copy provider, the system provider, is supplied in the Windows operating
system. Although a default provider is supplied in Windows, other vendors are free to
supply implementations that are optimized for their storage hardware and software
applications.

To maintain the "point-in-time" view of a volume that is contained in a shadow copy, the
system provider uses a copy-on-write technique. Copies of the blocks on volume that
have been modified since the beginning of the shadow copy creation are stored in a
shadow copy storage area.

The system provider can expose the production volume, which can be written to and
read from normally. When the shadow copy is needed, it logically applies the differences
to data on the production volume to expose the complete shadow copy.

For the system provider, the shadow copy storage area must be on an NTFS volume. The
volume to be shadow copied does not need to be an NTFS volume, but at least one
volume mounted on the system must be an NTFS volume.

The component files that make up the system provider are swprv.dll and volsnap.sys.

In-Box VSS Writers

The Windows operating system includes a set of VSS writers that are responsible for
enumerating the data that is required by various Windows features.

For more information about these writers, see In-Box VSS Writers.

How Shadow Copies Are Used

In addition to backing up application data and system state information, shadow copies
can be used for a number of purposes, including the following:

Restoring LUNs (LUN resynchronization and LUN swapping)

 Restoring individual files (Shadow Copies for Shared Folders)

Data mining by using transportable shadow copies

Restoring LUNs (LUN resynchronization and LUN

swapping)
In Windows Server 2008 R2 and Windows 7, VSS requesters can use a hardware shadow
copy provider feature called LUN resynchronization (or "LUN resync"). This is a fast-
recovery scheme that allows an application administrator to restore data from a shadow
copy to the original LUN or to a new LUN.

The shadow copy can be a full clone or a differential shadow copy. In either case, at the
end of the resync operation, the destination LUN will have the same contents as the
shadow copy LUN. During the resync operation, the array performs a block-level copy
from the shadow copy to the destination LUN.

７ Note

The shadow copy must be a transportable hardware shadow copy.

Most arrays allow production I/O operations to resume shortly after the resync
operation begins. While the resync operation is in progress, read requests are redirected
to the shadow copy LUN, and write requests to the destination LUN. This allows arrays
to recover very large data sets and resume normal operations in several seconds.

LUN resynchronization is different from LUN swapping. A LUN swap is a fast recovery
scenario that VSS has supported since Windows Server 2003 SP1. In a LUN swap, the
shadow copy is imported and then converted into a read-write volume. The conversion
is an irreversible operation, and the volume and underlying LUN cannot be controlled
with the VSS APIs after that. The following list describes how LUN resynchronization
compares with LUN swapping:

In LUN resynchronization, the shadow copy is not altered, so it can be used several
times. In LUN swapping, the shadow copy can be used only once for a recovery.
For the most safety-conscious administrators, this is important. When LUN
resynchronization is used, the requester can retry the entire restore operation if
something goes wrong the first time.

At the end of a LUN swap, the shadow copy LUN is used for production I/O
requests. For this reason, the shadow copy LUN must use the same quality of
storage as the original production LUN to ensure that performance is not impacted
 after the recovery operation. If LUN resynchronization is used instead, the
hardware provider can maintain the shadow copy on storage that is less expensive
than production-quality storage.

If the destination LUN is unusable and needs to be recreated, LUN swapping may
be more economical because it doesn't require a destination LUN.

２ Warning

All of the operations listed are LUN-level operations. If you attempt to recover a
specific volume by using LUN resynchronization, you are unwittingly going to revert
all the other volumes that are sharing the LUN.

Restoring individual files (Shadow Copies for Shared

Folders)
Shadow Copies for Shared Folders uses the Volume Shadow Copy Service to provide
point-in-time copies of files that are located on a shared network resource, such as a file
server. With Shadow Copies for Shared Folders, users can quickly recover deleted or
changed files that are stored on the network. Because they can do so without
administrator assistance, Shadow Copies for Shared Folders can increase productivity
and reduce administrative costs.

For more information about Shadow Copies for Shared Folders, see Shadow Copies for
Shared Folders (https://go.microsoft.com/fwlink/?LinkId=180898 ) on TechNet.

Data mining by using transportable shadow copies

With a hardware provider that is designed for use with the Volume Shadow Copy
Service, you can create transportable shadow copies that can be imported onto servers
within the same subsystem (for example, a SAN). These shadow copies can be used to
seed a production or test installation with read-only data for data mining.

With the Volume Shadow Copy Service and a storage array with a hardware provider
that is designed for use with the Volume Shadow Copy Service, it is possible to create a
shadow copy of the source data volume on one server, and then import the shadow
copy onto another server (or back to the same server). This process is accomplished in a
few minutes, regardless of the size of the data. The transport process is accomplished
through a series of steps that use a shadow copy requester (a storage-management
application) that supports transportable shadow copies.
To transport a shadow copy
1. Create a transportable shadow copy of the source data on a server.

2. Import the shadow copy to a server that is connected to the SAN (you can import
to a different server or the same server).

3. The data is now ready to be used.

Figure 3 Shadow copy creation and transport between two servers

７ Note
 A transportable shadow copy that is created on Windows Server 2003 cannot be
imported onto a server that is running Windows Server 2008 or Windows Server
2008 R2. A transportable shadow copy that was created on Windows Server 2008
or Windows Server 2008 R2 cannot be imported onto a server that is running
Windows Server 2003. However, a shadow copy that is created on Windows Server
2008 can be imported onto a server that is running Windows Server 2008 R2 and
vice versa.

Shadow copies are read-only. If you want to convert a shadow copy to a read/write LUN,
you can use a Virtual Disk Service-based storage-management application (including
some requesters) in addition to the Volume Shadow Copy Service. By using this
application, you can remove the shadow copy from Volume Shadow Copy Service
management and convert it to a read/write LUN.

Volume Shadow Copy Service transport is an advanced solution on computers running

Windows Server 2003 Enterprise Edition, Windows Server 2003 Datacenter Edition,
Windows Server 2008, or Windows Server 2008 R2. It works only if there is a hardware
provider on the storage array. Shadow copy transport can be used for a number of
purposes, including tape backups, data mining, and testing.

Frequently Asked Questions

This FAQ answers questions about Volume Shadow Copy Service (VSS) for system
administrators. For information about VSS application programming interfaces, see
Volume Shadow Copy Service (https://go.microsoft.com/fwlink/?LinkId=180899 ) in the
Windows Developer Center Library.

When was Volume Shadow Copy Service introduced? On

which Windows operating system versions is it available?
VSS was introduced in Windows XP. It is available on Windows XP, Windows
Server 2003, Windows Vista®, Windows Server 2008, Windows 7, and Windows
Server 2008 R2.

What is the difference between a shadow copy and a

backup?
In the case of a hard disk drive backup, the shadow copy created is also the backup.
Data can be copied off the shadow copy for a restore or the shadow copy can be used
for a fast recovery scenario—for example, LUN resynchronization or LUN swapping.
When data is copied from the shadow copy to tape or other removable media, the
content that is stored on the media constitutes the backup. The shadow copy itself can
be deleted after the data is copied from it.

What is the largest size volume that Volume Shadow

Copy Service supports?
Volume Shadow Copy Service supports a volume size of up to 64 TB.

I made a backup on Windows Server 2008. Can I restore it

on Windows Server 2008 R2?
It depends on the backup software that you used. Most backup programs support this
scenario for data but not for system state backups.

Shadow copies that are created on either of these versions of Windows can be used on
the other.

I made a backup on Windows Server 2003. Can I restore it

on Windows Server 2008?
It depends on the backup software you used. If you create a shadow copy on Windows
Server 2003, you cannot use it on Windows Server 2008. Also, if you create a shadow
copy on Windows Server 2008, you cannot restore it on Windows Server 2003.

How can I disable VSS?

It is possible to disable the Volume Shadow Copy Service by using the Microsoft
Management Console. However, you should not do this. Disabling VSS adversely affects
any software you use that depends on it, such as System Restore and Windows Server
Backup.

For more information, see the following Microsoft TechNet Web sites:

System Restore (https://go.microsoft.com/fwlink/?LinkID=157113 )

Windows Server Backup (https://go.microsoft.com/fwlink/?LinkID=180891 )

Can I exclude files from a shadow copy to save space?

VSS is designed to create shadow copies of entire volumes. Temporary files, such as
paging files, are automatically omitted from shadow copies to save space.

To exclude specific files from shadow copies, use the following registry key:
FilesNotToSnapshot.

７ Note

The FilesNotToSnapshot registry key is intended to be used only by applications.

Users who attempt to use it will encounter limitations such as the following:

It cannot delete files from a shadow copy that was created on a Windows
Server by using the Previous Versions feature.

It cannot delete files from shadow copies for shared folders.

It can delete files from a shadow copy that was created by using the
Diskshadow utility, but it cannot delete files from a shadow copy that was
created by using the Vssadmin utility.

Files are deleted from a shadow copy on a best-effort basis. This means that
they are not guaranteed to be deleted.

For more information, see Excluding Files from Shadow Copies

(https://go.microsoft.com/fwlink/?LinkId=180904 ) on MSDN.

My non-Microsoft backup program failed with a VSS

error. What can I do?
Check the product support section of the Web site of the company that created the
backup program. There may be a product update that you can download and install to
fix the problem. If not, contact the company's product support department.

System administrators can use the VSS troubleshooting information on the following
Microsoft TechNet Library Web site to gather diagnostic information about VSS-related
issues.

For more information, see Volume Shadow Copy Service

(https://go.microsoft.com/fwlink/?LinkId=180905 ) on TechNet.
What is the "diff area"?
The shadow copy storage area (or "diff area") is the location where the data for the
shadow copy that is created by the system software provider is stored.

Where is the diff area located?

The diff area can be located on any local volume. However, it must be located on an
NTFS volume that has enough space to store it.

How is the diff area location determined?

The following criteria are evaluated, in this order, to determine the diff area location:

If a volume already has an existing shadow copy, that location is used.

If there is a preconfigured manual association between the original volume and the
shadow copy volume location, then that location is used.

If the previous two criteria do not provide a location, the shadow copy service
chooses a location based on available free space. If more than one volume is being
shadow copied, the shadow copy service creates a list of possible snapshot
locations based on the size of free space, in descending order. The number of
locations provided is equal to the number of volumes being shadow copied.

If the volume being shadow copied is one of the possible locations, then a local
association is created. Otherwise an association with the volume with the most
available space is created.

Can VSS create shadow copies of non-NTFS volumes?

Yes. However, persistent shadow copies can be made only for NTFS volumes. In addition,
at least one volume mounted on the system must be an NTFS volume.

What's the maximum number of shadow copies I can

create at one time?
The maximum number of shadow copied volumes in a single shadow copy set is 64.
Note that this is not the same as the number of shadow copies.
What's the maximum number of software shadow copies
created by the system provider that I can maintain for a
volume?
The max number is of software shadow copies for each volume is 512. However, by
default you can only maintain 64 shadow copies that are used by the Shadow Copies of
Shared Folders feature. To change the limit for the Shadow Copies of Shared Folders
feature, use the following registry key: MaxShadowCopies.

How can I control the space that is used for shadow copy
storage space?
Type the vssadmin resize shadowstorage command.

For more information, see Vssadmin resize shadowstorage

(https://go.microsoft.com/fwlink/?LinkId=180906 ) on TechNet.

What happens when I run out of space?

Shadow copies for the volume are deleted, beginning with the oldest shadow copy.

Volume Shadow Copy Service Tools

The Windows operating system provides the following tools for working with VSS:

DiskShadow (https://go.microsoft.com/fwlink/?LinkId=180907 )

VssAdmin (https://go.microsoft.com/fwlink/?LinkId=84008 )

DiskShadow
DiskShadow is a VSS requester that you can use to manage all the hardware and
software snapshots that you can have on a system. DiskShadow includes commands
such as the following:

list: Lists VSS writers, VSS providers, and shadow copies

create: Creates a new shadow copy

import: Imports a transportable shadow copy

expose: Exposes a persistent shadow copy (as a drive letter, for example)
 revert: Reverts a volume back to a specified shadow copy

This tool is intended for use by IT professionals, but developers might also find it useful
when testing a VSS writer or VSS provider.

DiskShadow is available only on Windows Server operating systems. It is not available

on Windows client operating systems.

VssAdmin
VssAdmin is used to create, delete, and list information about shadow copies. It can also
be used to resize the shadow copy storage area ("diff area").

VssAdmin includes commands such as the following:

create shadow: Creates a new shadow copy

delete shadows: Deletes shadow copies

list providers: Lists all registered VSS providers

list writers: Lists all subscribed VSS writers

resize shadowstorage: Changes the maximum size of the shadow copy storage
area

VssAdmin can only be used to administer shadow copies that are created by the system
software provider.

VssAdmin is available on Windows client and Windows Server operating system

versions.

Volume Shadow Copy Service Registry Keys

The following registry keys are available for use with VSS:

VssAccessControl

MaxShadowCopies

MinDiffAreaFileSize

VssAccessControl
This key is used to specify which users have access to shadow copies.
For more information, see the following entries on the MSDN Web site:

Security Considerations for Writers (https://go.microsoft.com/fwlink/?

LinkId=157739 )

Security Considerations for Requesters (https://go.microsoft.com/fwlink/?

LinkId=180908 )

MaxShadowCopies
This key specifies the maximum number of client-accessible shadow copies that can be
stored on each volume of the computer. Client-accessible shadow copies are used by
Shadow Copies for Shared Folders.

For more information, see the following entry on the MSDN Web site:

MaxShadowCopies under Registry Keys for Backup and Restore

(https://go.microsoft.com/fwlink/?LinkId=180909 )

MinDiffAreaFileSize
This key specifies the minimum initial size, in MB, of the shadow copy storage area.

For more information, see the following entry on the MSDN Web site:

MinDiffAreaFileSize under Registry Keys for Backup and Restore

(https://go.microsoft.com/fwlink/?LinkId=180910 )

Supported Operating System Versions

The following table lists the minimum supported operating system versions for VSS
features.

VSS feature Minimum Minimum

supported supported
client server

LUN resynchronization None Windows

supported Server 2008 R2

FilesNotToSnapshot registry key Windows Windows

Vista Server 2008
 VSS feature Minimum Minimum
supported supported
client server

Transportable shadow copies None Windows

supported Server 2003
with SP1

Hardware shadow copies None Windows

supported Server 2003

Previous versions of Windows Server Windows Windows

Vista Server 2003

Fast recovery using LUN swap None Windows

supported Server 2003
with SP1

Multiple imports of hardware shadow copies None Windows

supported Server 2008

Note

This is the ability to import a shadow copy more than

once. Only one import operation can be performed at a
time.

Shadow Copies for Shared Folders None Windows

supported Server 2003

Transportable auto-recovered shadow copies None Windows

supported Server 2008

Concurrent backup sessions (up to 64) Windows XP Windows

Server 2003

Single restore session concurrent with backups Windows Windows

Vista Server 2003
with SP2

Up to 8 restore sessions concurrent with backups Windows 7 Windows

Server 2003 R2

Additional References
Volume Shadow Copy Service in Windows Developer Center
Use Disk Cleanup on Windows Server
Article • 03/13/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016, Windows
Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

The Disk Cleanup tool clears unnecessary files in a Windows Server environment. This tool is
available by default on Windows Server 2019 and Windows Server 2016, but you might have to
take a few manual steps to enable it on earlier versions of Windows Server.

To start the Disk Cleanup tool, either run the Cleanmgr.exe file, or select Start > Windows
Administrative Tools > Disk Cleanup.

You can also run Disk Cleanup by using the cleanmgr Windows command, and use command-
line options to direct Disk Cleanup to clean certain files.

７ Note

If you're just looking to free up disk space, consider using Azure File Sync with cloud tiering
enabled. This method lets you cache your most frequently accessed files locally and tier
your least frequently accessed files to the cloud, saving local storage space while
maintaining performance. For more information, see Planning for an Azure File Sync
deployment.

Enable Disk Cleanup on an earlier version of

Windows Server
Follow these steps to use the Add Roles and Features Wizard to install the Desktop Experience
on a server running Windows Server 2012 R2 or earlier. This process also installs the Disk
Cleanup tool.

1. If Server Manager is already open, go to the next step. If Server Manager isn't open yet,
launch it by doing one of the following options.

On the Windows desktop, select Server Manager in the Windows taskbar.

On the Windows Start menu, select the Server Manager tile.

2. On the Manage menu, select Add Roles and Features.

3. On the Before you begin page, verify that your destination server and network
environment are prepared for the feature that you want to install. Select Next.
 4. On the Select installation type page, select Role-based or feature-based installation to
install all parts features on a single server. Select Next.

5. On the Select destination server page, select a server from the server pool, or select an
offline VHD. Select Next.

6. On the Select server roles page, select Next.

7. On the Select features page, select User Interface and Infrastructure, and then select
Desktop Experience.

8. In Add features that are required for Desktop Experience?, select Add Features.

9. Finish the installation, and then reboot the system.

10. Verify that the Disk Cleanup button appears in the Properties dialog box.

Manually add Disk Cleanup to an earlier version of

Windows Server
The Disk Cleanup tool (cleanmgr.exe) isn't present on Windows Server 2012 R2 or earlier unless
you have the Desktop Experience feature installed.

To use cleanmgr.exe, install the Desktop Experience as described earlier, or copy two files that
are already present on the server, cleanmgr.exe and cleanmgr.exe.mui. Use the following table
to locate the files for your operating system.

Operating Architecture File Location

System

Windows 64-bit C:\Windows\winsxs\amd64_microsoft-windows-

Server cleanmgr_31bf3856ad364e35_6.1.7600.16385_none_c9392808773cd7da\cleanmgr.exe
2008 R2

Windows 64-bit C:\Windows\winsxs\amd64_microsoft-windows-

Server cleanmgr.resources_31bf3856ad364e35_6.1.7600.16385_en-
2008 R2 us_b9cb6194b257cc63\cleanmgr.exe.mui

Locate cleanmgr.exe and move the file to %systemroot%\System32.

Locate cleanmgr.exe.mui and move the files to %systemroot%\System32\en-US.

You can launch the Disk Cleanup tool by running Cleanmgr.exe from a Command Prompt
window, or by selecting Start and entering Cleanmgr in the search field.

To set up the Disk Cleanup button to appear on a disk's Properties dialog, you need to install
the Desktop Experience feature, as shown in the previous section.

Related links
Free up drive space in Windows 10
cleanmgr
Advanced Troubleshooting Server
Message Block (SMB)
Article • 12/13/2022

Try our Virtual Agent - It can help you quickly identify and fix common SMB issues.

Server Message Block (SMB) is a network transport protocol for file systems operations
to enable a client to access resources on a server. The primary purpose of the SMB
protocol is to enable remote file system access between two systems over TCP/IP.

SMB troubleshooting can be extremely complex. This article isn't an exhaustive

troubleshooting guide Instead, it's a short primer to understand the basics of how to
effectively troubleshoot SMB.

Tools and data collection

One key aspect of quality SMB troubleshooting is communicating the correct
terminology. Therefore, this article introduces basic SMB terminology to ensure accuracy
of data collection and analysis.

７ Note

The SMB Server (SRV) refers to the system that is hosting the file system, also
known as the file server. The SMB Client (CLI) refers to the system that is trying to
access the file system, regardless of the OS version or edition.

For example, if you use Windows Server 2016 to reach an SMB share that is hosted on
Windows 10, Windows Server 2016 is the SMB Client and Windows 10 the SMB Server.

Collect data
Before you troubleshoot SMB issues, we recommend that you first collect a network
trace on both the client and server sides. The following guidelines apply:

On Windows systems, you can use netshell (netsh), Network Monitor, Message
Analyzer, or Wireshark to collect a network trace.

Third-party devices generally have an in-box packet capture tool, such as tcpdump
(Linux/FreeBSD/Unix), or pktt (NetApp). For example, if the SMB client or SMB
 server is a Unix host, you can collect data by running the following command:

Windows Command Prompt

# tcpdump -s0 -n -i any -w /tmp/$(hostname)-smbtrace.pcap

Stop collecting data by using Ctrl+C from keyboard.

To discover the source of the issue, you can check the two-sided traces: CLI, SRV, or
somewhere in between.

Using netshell to collect data

This section provides the steps for using netshell to collect network trace.

） Important

The Microsoft Message Analyzer tool has been retired and we recommend
Wireshark to analyze ETL files. For those who have downloaded the tool
previously and are looking for more information, see Installing and upgrading
Message Analyzer.

７ Note

A Netsh trace creates an ETL file. ETL files can be opened in Message Analyzer (MA),
Network Monitor 3.4 (set the parser to Network Monitor Parsers > Windows), and
Wireshark .

1. On both the SMB server and SMB client, create a Temp folder on drive C. Then, run
the following command:

Windows Command Prompt

netsh trace start capture=yes report=yes scenario=NetConnection level=5

maxsize=1024 tracefile=c:\\Temp\\%computername%\_nettrace.etl**

If you are using PowerShell, run the following cmdlets:

PowerShell

New-NetEventSession -Name trace -LocalFilePath

"C:\Temp\$env:computername`_netCap.etl" -MaxFileSize 1024
 Add-NetEventPacketCaptureProvider -SessionName trace -TruncationLength
1500

Start-NetEventSession trace

2. Reproduce the issue.

3. Stop the trace by running the following command:

Windows Command Prompt

netsh trace stop

If you are using PowerShell, run the following cmdlets:

PowerShell

Stop-NetEventSession trace
Remove-NetEventSession trace

７ Note

You should trace only a minimum amount of the data that's transferred. For
performance issues, always take both a good and bad trace, if the situation allows
it.

Analyze the traffic

SMB is an application-level protocol that uses TCP/IP as the network transport protocol.
Therefore, an SMB issue can also be caused by TCP/IP issues.

Check whether TCP/IP experiences any of these issues:

1. The TCP three-way handshake does not finish. This typically indicates that there is
a firewall block, or that the Server service is not running.

2. Retransmits are occurring. These can cause slow file transfers because of
compound TCP congestion throttling.

3. Five retransmits followed by a TCP reset could mean that the connection between
systems was lost, or that one of the SMB services crashed or stopped responding.
 4. The TCP receive window is diminishing. This can be caused by slow storage or
some other issue that prevents data from being retrieved from the Ancillary
Function Driver (AFD) Winsock buffer.

If there is no noticeable TCP/IP issue, look for SMB errors. To do this, follow these steps:

1. Always check SMB errors against the MS-SMB2 protocol specification. Many SMB
errors are benign (not harmful). Refer to the following information to determine
why SMB returned the error before you conclude that the error is related to any of
the following issues:

The MS-SMB2 Message Syntax article details each SMB command and its
options.

The MS-SMB2 Client Processing article details how the SMB client creates
requests and responds to server messages.

The MS-SMB2 Server Processing article details how the SMB server creates
requests and responds to client requests.

2. Check whether a TCP reset command is sent immediately after an

FSCTL_VALIDATE_NEGOTIATE_INFO (validate negotiate) command. If so, refer to
the following information:

The SMB session must be terminated (TCP reset) when the Validate Negotiate
process fails on either the client or the server.

This process might fail because a WAN optimizer is modifying the SMB
Negotiate packet.

If the connection ended prematurely, identify the last exchange

communication between the client and server.

Analyze the protocol

Look at the actual SMB protocol details in the network trace to understand the exact
commands and options that are used.

Remember that SMB does only what it is told to do.

You can learn a lot about what the application is trying to do by examining the
SMB commands.

Compare the commands and operations to the protocol specification to make sure that
everything is operating correctly. If it is not, collect data that is closer to or at a lower
level to look for more information about the root cause. To do this, follow these steps:

1. Collect a standard packet capture.

2. Run the netsh command to trace and gather details about whether there are issues
in the network stack or drops in Windows Filtering Platform (WFP) applications,
such as firewall or antivirus program.

3. If all other options fail, collect a t.cmd if you suspect that the issue occurs within
SMB itself, or if none of the other data is sufficient to identify a root cause.

For example:

You experience slow file transfers to a single file server.

The two-sided traces show that the SRV responds slowly to a READ request.

Removing an antivirus program resolves the slow file transfers.

You contact the antivirus program manufactory to resolve the issue.

７ Note

Optionally, you might also temporarily uninstall the antivirus program during
troubleshooting.

Event logs
Both SMB Client and SMB Server have a detailed event log structure, as shown in the
following screenshot. Collect the event logs to help find the root cause of the issue.

SMB-related system files

This section lists the SMB-related system files. To keep the system files updated, make
sure that the latest update rollup is installed.

SMB Client binaries that are listed under %windir%\system32\Drivers:

RDBSS.sys

MRXSMB.sys

MRXSMB10.sys

MRXSMB20.sys

MUP.sys

SMBdirect.sys

SMB Server binaries that are listed under %windir%\system32\Drivers:

SRVNET.sys

SRV.sys

SRV2.sys

SMBdirect.sys

Under %windir%\system32

srvsvc.dll

Update suggestions
We recommend that you update the following components before you troubleshoot
SMB issues:

A file server requires file storage. If your storage has iSCSI component, update
those components.

Update the network components.

For better performance and stability, update Windows Core.

Reference
Microsoft SMB Protocol Packet Exchange Scenario
How to detect, enable and disable
SMBv1, SMBv2, and SMBv3 in Windows
Article • 05/18/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows 11, Windows 10,
Windows 8.1, Windows 8

This article describes how to enable and disable Server Message Block (SMB) version 1
(SMBv1), SMB version 2 (SMBv2), and SMB version 3 (SMBv3) on the SMB client and
server components.

While disabling or removing SMBv1 might cause some compatibility issues with old
computers or software, SMBv1 has significant security vulnerabilities, and we strongly
encourage you not to use it . SMB 1.0 isn't installed by default in any edition of
Windows 11 or Windows Server 2019 and later. SMB 1.0 also isn't installed by default in
Windows 10, except Home and Pro editions. We recommend that instead of reinstalling
SMB 1.0, you update the SMB server that still requires it. For a list of third parties that
require SMB 1.0 and their updates that remove the requirement, review the SMB1
Product Clearinghouse .

Disabling SMBv2 or SMBv3 for troubleshooting

We recommend keeping SMBv2 and SMBv3 enabled, but you might find it useful to
disable one temporarily for troubleshooting. For more information, see How to detect
status, enable, and disable SMB protocols on the SMB Server.

In Windows 10, Windows 8.1, Windows Server 2019, Windows Server 2016, Windows
Server 2012 R2, and Windows Server 2012, disabling SMBv3 deactivates the following
functionality:

Transparent Failover - clients reconnect without interruption to cluster nodes

during maintenance or failover
Scale Out - concurrent access to shared data on all file cluster nodes
Multichannel - aggregation of network bandwidth and fault tolerance if multiple
paths are available between client and server
SMB Direct - adds RDMA networking support for high performance, with low
latency and low CPU use
 Encryption - Provides end-to-end encryption and protects from eavesdropping on
untrustworthy networks
Directory Leasing - Improves application response times in branch offices through
caching
Performance Optimizations - optimizations for small random read/write I/O

In Windows 7 and Windows Server 2008 R2, disabling SMBv2 deactivates the following
functionality:

Request compounding - allows for sending multiple SMBv2 requests as a single

network request
Larger reads and writes - better use of faster networks
Caching of folder and file properties - clients keep local copies of folders and files
Durable handles - allow for connection to transparently reconnect to the server if
there's a temporary disconnection
Improved message signing - HMAC SHA-256 replaces MD5 as hashing algorithm
Improved scalability for file sharing - number of users, shares, and open files per
server greatly increased
Support for symbolic links
Client oplock leasing model - limits the data transferred between the client and
server, improving performance on high-latency networks and increasing SMB
server scalability
Large MTU support - for full use of 10 Gigabit Ethernet (GbE)
Improved energy efficiency - clients that have open files to a server can sleep

The SMBv2 protocol was introduced in Windows Vista and Windows Server 2008, while
the SMBv3 protocol was introduced in Windows 8 and Windows Server 2012. For more
information about SMBv2 and SMBv3 capabilities, see the following articles:

Server Message Block overview

What's New in SMB

How to remove SMBv1 via PowerShell

Here are the steps to detect, disable and enable SMBv1 client and server by using
PowerShell commands with elevation.

７ Note

The computer will restart after you run the PowerShell commands to disable or
enable SMBv1.
 Detect:

PowerShell

Get-WindowsOptionalFeature -Online -FeatureName SMB1Protocol

Disable:

PowerShell

Disable-WindowsOptionalFeature -Online -FeatureName SMB1Protocol

Enable:

PowerShell

Enable-WindowsOptionalFeature -Online -FeatureName SMB1Protocol

 Tip

You can detect SMBv1 status, without elevation, by running: Get-

SmbServerConfiguration | Format-List EnableSMB1Protocol .

Windows Server 2012 Windows Server 2012 R2, Windows

Server 2016, Windows Server 2019: Server Manager
method
To remove SMBv1 from Windows Server:

1. On the Server Manager Dashboard of the server where you want to remove
SMBv1, under Configure this local server, select Add roles and features.
2. On the Before you begin page, select Start the Remove Roles and Features
Wizard, and then on the following page, select Next.
3. On the Select destination server page under Server Pool, ensure that the server
you want to remove the feature from is selected, and then select Next.
4. On the Remove server roles page, select Next.
5. On the Remove features page, clear the check box for SMB 1.0/CIFS File Sharing
Support and select Next.
6. On the Confirm removal selections page, confirm that the feature is listed, and
then select Remove.

Windows 8.1, Windows 10, and Windows 11: Add or

Remove Programs method
To disable SMBv1 for the mentioned operating systems:

1. In Control Panel, select Programs and Features.

2. Under Control Panel Home, select Turn Windows features on or off to open the
Windows Features box.
3. In the Windows Features box, scroll down the list, clear the check box for SMB
1.0/CIFS File Sharing Support and select OK.
4. After Windows applies the change, on the confirmation page, select Restart now.

How to detect status, enable, and disable SMB

protocols

７ Note

When you enable or disable SMBv2 in Windows 8 or Windows Server 2012, SMBv3
is also enabled or disabled. This behavior occurs because these protocols share the
same stack.

Server

Windows 8 and Windows Server 2012 introduced the new Set-

SMBServerConfiguration Windows PowerShell cmdlet. The cmdlet enables you to
enable or disable the SMBv1, SMBv2, and SMBv3 protocols on the server
component.
You don't have to restart the computer after you run the Set-
SMBServerConfiguration cmdlet.

SMBv1
Detect:

PowerShell

Get-SmbServerConfiguration | Select EnableSMB1Protocol

Disable:

PowerShell

Set-SmbServerConfiguration -EnableSMB1Protocol $false

Enable:

PowerShell

Set-SmbServerConfiguration -EnableSMB1Protocol $true

For more information, see Server storage at Microsoft .

SMB v2/v3
Detect:

PowerShell

Get-SmbServerConfiguration | Select EnableSMB2Protocol

Disable:

PowerShell

Set-SmbServerConfiguration -EnableSMB2Protocol $false

Enable:

PowerShell
 Set-SmbServerConfiguration -EnableSMB2Protocol $true

For Windows 7, Windows Server 2008 R2, Windows

Vista, and Windows Server 2008
To enable or disable SMB protocols on an SMB Server that is running Windows 7,
Windows Server 2008 R2, Windows Vista, or Windows Server 2008, use Windows
PowerShell or Registry Editor.

Additional PowerShell methods

７ Note

This method requires PowerShell 2.0 or later.

SMBv1 on SMB Server

Detect:

PowerShell

Get-Item HKLM:\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters
| ForEach-Object {Get-ItemProperty $_.pspath}

Default configuration = Enabled (No registry named value is created), so no SMB1

value will be returned

Disable:

PowerShell

Set-ItemProperty -Path
"HKLM:\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters" SMB1 -
Type DWORD -Value 0 -Force

Enable:

PowerShell

Set-ItemProperty -Path
"HKLM:\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters" SMB1 -
 Type DWORD -Value 1 -Force

Note You must restart the computer after you make these changes. For more
information, see Server storage at Microsoft .

SMBv2/v3 on SMB Server

Detect:

PowerShell

Get-ItemProperty
HKLM:\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters |
ForEach-Object {Get-ItemProperty $_.pspath}

Disable:

PowerShell

Set-ItemProperty -Path
"HKLM:\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters" SMB2 -
Type DWORD -Value 0 -Force

Enable:

PowerShell

Set-ItemProperty -Path
"HKLM:\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters" SMB2 -
Type DWORD -Value 1 -Force

７ Note

You must restart the computer after you make these changes.

Registry Editor

） Important

Follow the steps in this section carefully. Serious problems might occur if you
modify the registry incorrectly. Before you modify it, back up the registry for
restoration in case problems occur.
 To enable or disable SMBv1 on the SMB server, configure the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Para
meters

Registry entry: SMB1

REG_DWORD: 0 = Disabled
REG_DWORD: 1 = Enabled
Default: 1 = Enabled (No registry key is created)

To enable or disable SMBv2 on the SMB server, configure the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Para
meters

Registry entry: SMB2

REG_DWORD: 0 = Disabled
REG_DWORD: 1 = Enabled
Default: 1 = Enabled (No registry key is created)

７ Note

You must restart the computer after you make these changes.

Disable SMBv1 by using Group Policy

This section introduces how to use Group Policy to disable SMBv1. You can use this
method on different versions of Windows.

Server

SMBv1
This procedure configures the following new item in the registry:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Para
meters

Registry entry: SMB1

REG_DWORD: 0 = Disabled

To use Group Policy to configure this, follow these steps:

1. Open the Group Policy Management Console. Right-click the Group Policy
object (GPO) that should contain the new preference item, and then click Edit.

2. In the console tree under Computer Configuration, expand the Preferences

folder, and then expand the Windows Settings folder.

3. Right-click the Registry node, point to New, and select Registry Item.

In the New Registry Properties dialog box, select the following:

Action: Create
Hive: HKEY_LOCAL_MACHINE
Key Path: SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters
Value name: SMB1
Value type: REG_DWORD
Value data: 0
 This procedure disables the SMBv1 Server components. This Group Policy must be
applied to all necessary workstations, servers, and domain controllers in the
domain.

７ Note

WMI filters can also be set to exclude unsupported operating systems or

selected exclusions, such as Windows XP.

） Important

Be careful when you make these changes on domain controllers on which

legacy Windows XP or older Linux and third-party systems (that don't support
SMBv2 or SMBv3) require access to SYSVOL or other file shares where SMB v1
is being disabled.

Auditing SMBv1 usage

To determine which clients are attempting to connect to an SMB server with SMBv1, you
can enable auditing on Windows Server 2016, Windows 10, and Windows Server 2019.
You can also audit on Windows 7 and Windows Server 2008 R2 if the May 2018 monthly
update is installed, and on Windows 8.1 and Windows Server 2012 R2 if the July 2017
monthly update is installed.

Enable:

PowerShell

Set-SmbServerConfiguration -AuditSmb1Access $true

Disable:

PowerShell

Set-SmbServerConfiguration -AuditSmb1Access $false

Detect:

PowerShell

Get-SmbServerConfiguration | Select AuditSmb1Access

When SMBv1 auditing is enabled, event 3000 appears in the "Microsoft-Windows-

SMBServer\Audit" event log, identifying each client that attempts to connect with
SMBv1.

Summary
If all the settings are in the same GPO, Group Policy Management displays the following
settings.
Testing and validation
After completing the configuration steps in this article, allow the policy to replicate and
update. As necessary for testing, run gpupdate /force at a command prompt, and then
review the target computers to make sure that the registry settings are applied correctly.
Make sure SMBv2 and SMBv3 are functioning for all other systems in the environment.

７ Note

Don't forget to restart the target systems.

SMBv1 is not installed by default in
Windows 10 version 1709, Windows
Server version 1709 and later versions
Article • 05/18/2023

Summary
Since Windows 10 Fall Creators Update and Windows Server, version 1709 (RS3), the
Server Message Block version 1 (SMBv1) network protocol is no longer installed by
default. It was superseded by SMBv2 and later protocols starting in 2007. Microsoft
publicly deprecated the SMBv1 protocol in 2014.

SMBv1 has the following behavior in Windows 10 and Windows Server 2019 and later
versions:

SMBv1 now has both client and server sub-features that can be uninstalled
separately.
Windows 10 Enterprise, Windows 10 Education, and Windows 10 Pro for
Workstations no longer contain the SMBv1 client or server by default after a clean
installation.
Windows Server 2019 and later no longer contains the SMBv1 client or server by
default after a clean installation.
Windows 10 Home and Windows 10 Pro no longer contain the SMBv1 server by
default after a clean installation.
Windows 11 doesn't contain the SMBv1 server or client by default after a clean
installation.
Windows 10 Home and Windows 10 Pro still contain the SMBv1 client by default
after a clean installation. If the SMBv1 client isn't used for 15 days in total
(excluding the computer being turned off), it automatically uninstalls itself.
In-place upgrades and Insider flights of Windows 10 Home and Windows 10 Pro
don't automatically remove SMBv1 initially. Windows evaluate the usage of SMBv1
client and server, and if either of them isn't used for 15 days in total (excluding the
time during which the computer is off), Windows will automatically uninstall it.
In-place upgrades and Insider flights of the Windows 10 Enterprise, Windows 10
Education, and Windows 10 Pro for Workstations editions don't automatically
remove SMBv1. An administrator must decide to uninstall SMBv1 in these
managed environments.
 Automatic removal of SMBv1 after 15 days is a one-time operation. If an
administrator re-installs SMBv1, no further attempts will be made to uninstall it.
The SMB version 2.02, 2.1, 3.0, 3.02, and 3.1.1 features are still fully supported and
included by default as part of the SMBv2 binaries.
Because the Computer Browser service relies on SMBv1, the service is uninstalled if
the SMBv1 client or server is uninstalled. This means that Explorer Network can no
longer display Windows computers through the legacy NetBIOS datagram
browsing method.
SMBv1 can still be reinstalled in all editions of Windows 10 and Windows Server
2016.
Windows Server virtual machines created by Microsoft for the Azure Marketplace
don't contain the SMB1 binaries & you can't enable SMB1. Third-party Azure
Marketplace VMs may contain SMB1, contact their vendor for information.

Starting in Windows 10, version 1809 (RS5), Windows 10 Pro no longer contains the
SMBv1 client by default after a clean installation. All other behaviors from version 1709
still apply.

７ Note

Windows 10, version 1803 (RS4) Pro handles SMBv1 in the same manner
as Windows 10, version 1703 (RS2) and Windows 10, version 1607 (RS1). This issue
was fixed in Windows 10, version 1809 (RS5). You can still uninstall SMBv1
manually. However, Windows will not automatically uninstall SMBv1 after 15 days in
the following scenarios:

You do a clean install of Windows 10, version 1803.

You upgrade Windows 10, version 1607 or Windows 10, version 1703 to
Windows 10, version 1803 directly without first upgrading to Windows 10,
version 1709.

If you try to connect to devices that support only SMBv1, or if these devices try to
connect to you, you may receive one of the following error messages:

Output

You can't connect to the file share because it's not secure. This share
requires the obsolete SMB1 protocol, which is unsafe and could expose your
system to attack.
Your system requires SMB2 or higher. For more info on resolving this issue,
see: https://go.microsoft.com/fwlink/?linkid=852747
 Output

The specified network name is no longer available.

Output

Unspecified error 0x80004005

Output

System Error 64

Output

The specified server cannot perform the requested operation.

Output

Error 58

When a remote server required an SMBv1 connection from this client, and the SMBv1
client is installed, the following event is logged. This mechanism audits the use of
SMBv1 and is also used by the automatic uninstaller to set the 15-day timer of removing
SMBv1 because of lack of use.

Output

Log Name: Microsoft-Windows-SmbClient/Security

Source: Microsoft-Windows-SMBClient
Date: Date/Time
Event ID: 32002
Task Category: None
Level: Info
Keywords: (128)
User: NETWORK SERVICE
Computer: junkle.contoso.com
Description:
The local computer received an SMB1 negotiate response.

Dialect:
SecurityMode
Server name:

Guidance:
SMB1 is deprecated and should not be installed nor enabled. For more
information, see https://go.microsoft.com/fwlink/?linkid=852747.
When a remote server required an SMBv1 connection from this client, and the SMBv1
client isn't installed, the following event is logged. This event is to show why the
connection fails.

Output

Log Name: Microsoft-Windows-SmbClient/Security

Source: Microsoft-Windows-SMBClient
Date: Date/Time
Event ID: 32000
Task Category: None
Level: Info
Keywords: (128)
User: NETWORK SERVICE
Computer: junkle.contoso.com
Description:
SMB1 negotiate response received from remote device when SMB1 cannot be
negotiated by the local computer.
Dialect:
Server name:

Guidance:
The client has SMB1 disabled or uninstalled. For more information:
https://go.microsoft.com/fwlink/?linkid=852747.

These devices aren't likely running Windows. They are more likely running older versions
of Linux, Samba, or other types of third-party software to provide SMB services. Often,
these versions of Linux and Samba are, themselves, no longer supported.

７ Note

Windows 10, version 1709 is also known as "Fall Creators Update."

More Information
To work around this issue, contact the manufacturer of the product that supports only
SMBv1, and request a software or firmware update that support SMBv2.02 or a later
version. For a current list of known vendors and their SMBv1 requirements, see the
following Windows and Windows Server Storage Engineering Team Blog article:

SMBv1 Product Clearinghouse

Leasing mode
If SMBv1 is required to provide application compatibility for legacy software behavior,
such as a requirement to disable oplocks, Windows provides a new SMB share flag that's
known as Leasing mode. This flag specifies whether a share disables modern SMB
semantics such as leases and oplocks.

You can specify a share without using oplocks or leasing to allow a legacy application to
work with SMBv2 or a later version. To do this, use the New-SmbShare or Set-SmbShare
PowerShell cmdlets together with the -LeasingMode None parameter.

７ Note

You should use this option only on shares that are required by a third-party
application for legacy support if the vendor states that it is required. Do not specify
Leasing mode on user data shares or CA shares that are used by Scale-Out File
Servers. This is because the removal of oplocks and leases causes instability and
data corruption in most applications. Leasing mode works only in Share mode. It
can be used by any client operating system.

Explorer Network Browsing

The Computer Browser service relies on the SMBv1 protocol to populate the Windows
Explorer Network node (also known as "Network Neighborhood"). This legacy protocol
is long deprecated, doesn't route, and has limited security. Because the service can't
function without SMBv1, it's removed at the same time.

However, if you still have to use the Explorer Network in home and small business
workgroup environments to locate Windows-based computers, you can follow these
steps on your Windows-based computers that no longer use SMBv1:

1. Start the "Function Discovery Provider Host" and "Function Discovery Resource
Publication" services, and then set them to Automatic (Delayed Start).

2. When you open Explorer Network, enable network discovery when you're
prompted.

All Windows devices within that subnet that have these settings will now appear in
Network for browsing. This uses the WS-DISCOVERY protocol. Contact your other
vendors and manufacturers if their devices still don't appear in this browse list after the
Windows devices appear. It's possible they have this protocol disabled or that they
support only SMBv1.
 ７ Note

We recommend that you map drives and printers instead of enabling this feature,
which still requires searching and browsing for their devices. Mapped resources are
easier to locate, require less training, and are safer to use. This is especially true if
these resources are provided automatically through Group Policy. An administrator
can configure printers for location by methods other than the legacy Computer
Browser service by using IP addresses, Active Directory Domain Services (AD DS),
Bonjour, mDNS, uPnP, and so on.

If you can't use any of these workarounds, or if the application manufacturer

can't provide supported versions of SMB, you can re-enable SMBv1 manually by
following the steps in How to detect, enable and disable SMBv1, SMBv2, and SMBv3 in
Windows.

） Important

We strongly recommend that you don't reinstall SMBv1. This is because this older
protocol has known security issues regarding ransomware and other malware.

Windows Server best practices analyzer messaging

Windows Server 2012 and later server operation systems contain a best practices
analyzer (BPA) for file servers. If you've followed the correct online guidance to uninstall
SMB1, running this BPA will return a contradictory warning message:

Output

Title: The SMB 1.0 file sharing protocol should be enabled

Severity: Warning
Date: 3/25/2020 12:38:47 PM
Category: Configuration
Problem: The Server Message Block 1.0 (SMB 1.0) file sharing protocol is
disabled on this file server.
Impact: SMB not in a default configuration, which could lead to less than
optimal behavior.
Resolution: Use Registry Editor to enable the SMB 1.0 protocol.

） Important
 You should ignore this specific BPA rule's guidance, it's deprecated. The false error
was first corrected in Windows Server 2022 and Windows Server 2019 in the 2022
April Cumulative Update. We repeat: don't enable SMB 1.0.

Additional references
Stop using SMB1
SMB known issues
Article • 05/22/2020

The following topics describe some common troubleshooting issues that can occur
when you use Server Message Block (SMB). These topics also provide possible solutions
to those issues.

TCP three-way handshake failure

Negotiate, Session Setup, and Tree Connect Failures

TCP connection is aborted during Validate Negotiate

Slow files transfer speed

High CPU usage issue on the SMB server

Troubleshoot the Event ID 50 Error Message

SMB Multichannel troubleshooting

TCP three-way handshake failure during
SMB connection
Article • 05/22/2020

When you analyze a network trace, you notice that there is a Transmission Control
Protocol (TCP) three-way handshake failure that causes the SMB issue to occur. This
article describes how to troubleshoot this situation.

Troubleshooting
Generally, the cause is a local or infrastructure firewall that blocks the traffic. This issue
can occur in either of the following scenarios.

Scenario 1
The TCP SYN packet arrives on the SMB server, but the SMB server does not return a
TCP SYN-ACK packet.

To troubleshoot this scenario, follow these steps.

Step 1
Run netstat or Get-NetTcpConnection to make sure that there is a listener on TCP port
445 that should be owned by the SYSTEM process.

Windows Command Prompt

netstat -ano | findstr :445

PowerShell

Get-NetTcpConnection -LocalPort 445

Step 2
Make sure that the Server service is started and running.

Step 3
Take a Windows Filtering Platform (WFP) capture to determine which rule or program is
dropping the traffic. To do this, run the following command in a Command Prompt
window:

Windows Command Prompt

netsh wfp capture start

Reproduce the issue, and then, run the following command:

Windows Command Prompt

netsh wfp capture stop

Run a scenario trace, and look for WFP drops in SMB traffic (on TCP port 445).

Optionally, you could remove the anti-virus programs because they are not always WFP-
based.

Step 4
If Windows Firewall is enabled, enable firewall logging to determine whether it records a
drop in traffic.

Make sure that the appropriate "File and Printer Sharing (SMB-In)" rules are enabled in
Windows Firewall with Advanced Security > Inbound Rules.

７ Note

Depending on how your computer is set up, "Windows Firewall" might be called
"Windows Defender Firewall."

Scenario 2
The TCP SYN packet never arrives at the SMB server.

In this scenario, you have to investigate the devices along the network path. You may
analyze network traces that are captured on each device to determine which device is
blocking the traffic.
Negotiate, Session Setup, and Tree
Connect Failures
Article • 12/07/2020

This article describes how to troubleshoot the failures that occur during an SMB
Negotiate, Session Setup, and Tree Connect request.

Negotiate fails
The SMB server receives an SMB NEGOTIATE request from an SMB client. The
connection times out and is reset after 60 seconds. There may be an ACK message after
about 200 microseconds.

This problem is most often caused by antivirus program.

If you are using Windows Server 2008 R2, there are hotfixes for this problem. Make sure
that the SMB client and the SMB server are up to date.

Session Setup fails

The SMB server receives an SMB SESSION_SETUP request from a SMB client but failed to
response.

If the fully qualified domain name (FQDN) or Network Basic Input/Output System
(NetBIOS) name of the server is 'sed in the Universal Naming Convention (UNC) path,
Windows will use Kerberos for authentication.

After the Negotiate response, there will be an attempt to get a Kerberos ticket for the
Common Internet File System (CIFS) service principal name (SPN) of the server. Look at
the Kerberos traffic on TCP port 88 to make sure that there are no Kerberos errors when
the SMB client is gaining the token.

７ Note

The errors that occur during the Kerberos Pre-Authentication are OK. The errors
that occur after the Kerberos Pre-Authentication (instances in which authentication
does not work), are the errors that caused the SMB problem.

Additionally, make the following checks:

 Look at the security blob in the SMB SESSION_SETUP request to make sure the
correct credentials are sent.

Try to disable SMB server name hardening (SmbServerNameHardeningLevel = 0).

Make sure that the SMB server has an SPN when it is accessed through a CNAME
DNS record.

Make sure that SMB signing is working. (This is especially important for older,
third-party devices.)

Tree Connect fails

Make sure that the user account credentials have both share and NT file system (NTFS)
permissions to the folder.

The cause of common Tree Connect errors can be found in 3.3.5.7 Receiving an SMB2
TREE_CONNECT Request. The following are the solutions for two common status codes.

[STATUS_BAD_NETWORK_NAME]

Make sure that the share exists on the server, and that it is spelled correctly in the SMB
client request.

[STATUS_ACCESS_DENIED]

Verify that the disk and folder that are used by the share exists and is accessible.

If you are using SMBv3 or later, check whether the server and the share require
encryption, but the client doesn't support encryption. To do this, take the following
actions:

Check the server by running the following command.

PowerShell

Get-SmbServerConfiguration | select Encrypt*

If EncryptData and RejectUnencryptedAccess are true, the server requires

encryption.

Check the share by running the following command:

PowerShell
 Get-SmbShare | select name, EncryptData

If EncryptData is true on the share, and RejectUnencryptedAccess is true on the

server, encryption is required by the share

Follow these guidelines as you troubleshoot:

Windows 8, Windows Server 2012, and later versions of Windows support client-
side encryption (SMBv3 and later).

Windows 7, Windows Server 2008 R2 and earlier versions of Windows do not

support client-side encryption.

Samba and third-party device may not support encryption. You may have to
consult product documentation for more information.

References
For more information, see the following articles.

3.3.5.4 Receiving an SMB2 NEGOTIATE Request

3.3.5.5 Receiving an SMB2 SESSION_SETUP Request

3.3.5.7 Receiving an SMB2 TREE_CONNECT Request

TCP connection is aborted during
Validate Negotiate
Article • 07/22/2020

In the network trace for the SMB issue, you notice that a TCP Reset abort occurred
during the Validate Negotiate process. This article describes how to troubleshoot the
situation.

Cause
This issue can be caused by a failed negotiation validation. This typically occurs because
a WAN accelerator modifies the original SMB NEGOTIATE packet.

Microsoft no longer allows modification of the Validate Negotiate packet for any reason.
This is because this behavior creates a serious security risk.

The following requirements apply to the Validate Negotiate packet:

The Validate Negotiate process uses the FSCTL_VALIDATE_NEGOTIATE_INFO

command.

The Validate Negotiate response must be signed. Otherwise, the connection is

aborted.

You should compare the FSCTL_VALIDATE_NEGOTIATE_INFO messages to the

Negotiate messages to make sure that nothing was changed.

Workaround
You can temporarily disable the Validate Negotiate process. To do this, locate the
following registry subkey:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Par
ameters

Under the Parameters key, set RequireSecureNegotiate to 0.

In Windows PowerShell, you can run the following command to set this value:

PowerShell
 Set-ItemProperty -Path
"HKLM:\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters"
RequireSecureNegotiate -Value 0 -Force

７ Note

The Validate Negotiate process cannot be disabled in Windows 10, Windows Server
2016, or later versions of Windows.

If either the client or server cannot support the Validate Negotiate command, you can
work around this issue by setting SMB signing to be required. SMB signing is considered
more secure than Validate Negotiate. However, there can also be performance
degradation if signing is required.

Reference
For more information, see the following articles:

3.3.5.15.12 Handling a Validate Negotiate Info Request

3.2.5.14.12 Handling a Validate Negotiate Info Response

Slow SMB files transfer speed
Article • 03/28/2023

Server Message Block (SMB) file transfer speeds can slow down depending on the size
and quantity of your files, your connection type, and the version of apps you use. This
article provides troubleshooting procedures for slow file transfer speeds through SMB.

Slow transfer
You can troubleshoot slow file transfers by checking your current storage use. If you
observe slow transfers of files, consider the following steps:

Try the file copy command for unbuffered IO:

xcopy /J
robocopy /J

Test the storage speed. Copy speeds are limited by storage speed.

File copies sometimes start fast and then slow down. A change in copy speed
occurs when the initial copy is cached or buffered, in memory or in the RAID
controller's memory cache, and the cache runs out. This change forces data to be
written directly to disk (write-through).

To verify this situation, use storage performance monitor counters to determine

whether storage performance degrades over time. For more information, see
Performance tuning for SMB file servers.

Use RAMMap (SysInternals) to determine whether "Mapped File" usage in memory

stops growing because of free memory exhaustion.

Look for packet loss in the trace. Packet loss can cause throttling by the TCP
congestion provider.

For SMBv3 and later versions, verify that SMB Multichannel is enabled and
working.

On the SMB client, enable large MTU in SMB, and disable bandwidth throttling by
running the following command:

PowerShell

Set-SmbClientConfiguration -EnableBandwidthThrottling 0 -EnableLargeMtu

1
Slow transfer of small files
A slow transfer of small files occurs most commonly when there are many files. This
occurrence is an expected behavior.

During file transfer, file creation causes both high protocol overhead and high file
system overhead. For large file transfers, these costs occur only one time. When a large
number of small files are transferred, the cost is repetitive and causes slow transfers.

Issue details
Network latency, create commands, and antivirus programs contribute to a slower
transfer of small files. The following are technical details about this problem:

SMB calls a create command to request that the file is created. Code checks
whether the file exists and then creates the file. Otherwise, some variation of the
create command creates the actual file.

Each create command generates activity on the file system.

After the data is written, the file is closed.

The process can suffer from network latency and SMB server latency. This latency
occurs because the SMB request is first translated to a file system command and
then to the actual file system to complete the operation.

The transfer continues to slow while an antivirus program is running. This change
happens because the data is typically scanned once by the packet sniffer and a
second time when the data is written to disk. In some scenarios, these actions are
repeated thousands of times. You can potentially observe speeds of less than 1
MB/s.

Slow open of Office documents

Office documents can open slowly, which generally occurs on a WAN connection. The
manner in which Office apps (Microsoft Excel, in particular) access and read data is
typically what causes the documents to open slowly.

You should verify that the Office and SMB binaries are up-to-date, and then test by
having leasing disabled on the SMB server. To verify both conditions have resolved,
follow these steps:
 1. Run the following PowerShell command in Windows 8 and Windows Server 2012
or later versions of Windows:

PowerShell

Set-SmbServerConfiguration -EnableLeasing $false

You can also run the following command in an elevated Command Prompt
window:

Windows Command Prompt

REG ADD
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanserver\param
eters /v DisableLeasing /t REG\_DWORD /d 1 /f

７ Note

After you set this registry key, SMB2 leases are no longer granted, but oplocks
are still available. This setting is used primarily for troubleshooting.

2. Restart the file server or restart the Server service. To restart the service, run the
following commands:

Windows Command Prompt

NET STOP SERVER

NET START SERVER

To avoid this issue, you can also replicate the file to a local file server. For more
information, see saving Office documents to a network server is slow when using EFS.
High CPU usage issue on the SMB
server
Article • 05/22/2020

This article discusses how to troubleshoot the high CPU usage issue on the SMB server.

High CPU usage because of storage

performance issues
Storage performance issues can cause high CPU usage on SMB servers. Before you
troubleshoot, make sure that the latest update rollup is installed on the SMB server to
eliminate any known issues in srv2.sys.

In most cases, you will notice the issue of high CPU usage in the system process. Before
you proceed, use Process Explorer to make sure that srv2.sys or ntfs.sys is consuming
excessive CPU resources.

Storage area network (SAN) scenario

In aggregate levels, the overall SAN performance may appear to be fine. However, when
you work with SMB issues, the individual request response time is what matters the
most.

Generally, this issue can be caused by some form of command queuing in the SAN. You
can use Perfmon to capture a Microsoft-Windows-StorPort tracing, and analyze it to
accurately determine storage responsiveness.

Disk IO latency
Disk IO latency is a measure of the delay between the time that a disk IO request is
created and completed.

The IO latency that is measured in Perfmon includes all the time that is spent in the
hardware layers plus the time that is spent in the Microsoft Port Driver queue
(Storport.sys for SCSI). If the running processes generate a large StorPort queue, the
measured latency increases. This is because IO must wait before it is dispatched to the
hardware layers.

In Perfmon, the following counters show physical disk latency:

 "Physical disk performance object" -> "Avg. Disk sec/Read counter" – This shows
the average read latency.

"Physical disk performance object" -> "Avg. Disk sec/Write counter" – This shows
the average write latency.

"Physical disk performance object" -> "Avg. Disk sec/Transfer counter" – This
shows the combined averages for both reads and writes.

The "_Total" instance is an average of the latencies for all physical disks in the computer.
Each of other instances represents an individual Physical Disk.

７ Note

Do not confuse these counters with Avg. Disk Transfers/sec. These are completely
different counters.

Windows Storage Stack follows

This section gives a brief explanation on the Windows Storage Stack follows.

When an application creates an IO request, it sends the request to the Windows IO

subsystem at the top of the stack. The IO then travels all the way down the stack to the
hardware "Disk" subsystem. Then, the response travels all the way back up. During this
process, each layer performs its function and then hands the IO to the next layer.
Perfmon does not create any performance data per second. Instead, it consumes data
that is provided by other subsystems within Windows.

For the "physical disk performance object," the data is captured at the "Partition
manager" level in the storage stack.

When we measure the counters that are mentioned in the previous section, we are
measuring all the time that is spent by the request below the "Partition manager" level.
When the IO request is sent by the partition manager down the stack, we time stamp it.
When it returns, we time stamp it again and calculate the time difference. The time
difference is the latency.

By doing this, we are accounting for the time that is spent in the following components:

Class Driver - This manages the device type, such as disks, tapes, and so on.

Port Driver - This manages the transport protocol, such as SCSI, FC, SATA, and so
on.

Device Miniport Driver - This is the device driver for the Storage Adapter. It is
supplied by the manufacturer of the devices, such as Raid Controller, and FC HBA.
 Disk Subsystem - This includes everything that is below the Device Miniport Driver.
This could be as simple as a cable that is connected to a single physical hard disk,
or as complex as a Storage Area Network. If the issue is determined to be caused
by this component, you can contact the hardware vendor for more information
about troubleshooting.

Disk queuing
There is a limited amount of IO that a disk subsystem can accept at a given time. The
excess IO gets queued until the disk can accept IO again. The time that IO spends in the
queues below the "Partition manager" level is accounted for in the Perfmon physical disk
latency measurements. As queues grow larger and IO must wait longer, the measured
latency also grows.

There are multiple queues below the "Partition manager" level, as follows:

Microsoft Port Driver Queue - SCSIport or Storport queue

Manufacturer Supplied Device Driver Queue - OEM Device driver

Hardware Queues – such as disk controller queue, SAN switches queue, array
controller queue, and hard disk queue

We also account for the time that the hard disk spends actively servicing the IO and the
travel time that is taken for the request to return to the "Partition manager" level to be
marked as completed.

Finally, we have to pay special attention to the Port Driver Queue (for SCSI Storport.sys).
The Port Driver is the last Microsoft component to touch an IO before we hand it off to
the manufacturer-supplied Device Miniport Driver.

If the Device Miniport Driver can't accept any more IO because its queue or the
hardware queues below it are saturated, we will start accumulating IO on the Port Driver
Queue. The size of the Microsoft Port Driver queue is limited only by the available
system memory (RAM), and it can grow very large. This causes large measured latency.

High CPU caused by enumerating folders

To troubleshoot this issue, disable the Access Based Enumeration (ABE) feature.

To determine which SMB shares have ABE enabled, run the following PowerShell
command,
 PowerShell

Get-SmbShare | Select Name, FolderEnumerationMode

Unrestricted = ABE disabled.

AccessBase = ABE enabled.

You can enable ABE in Server Manager. Navigatie to File and Storage Services >
Shares, right-click the share, select Properties, go to Settings and then select Enable
access-based enumeration.

Also, you can reduce ABELevel to a lower level (1 or 2) to improve performance.

You can check disk performance when enumeration is slow by opening the folder locally
through a console or an RDP session.
Troubleshoot the Event ID 50 Error
Message
Article • 11/03/2020

Symptoms
When information is being written to the physical disk, the following two event
messages may be logged in the system event log:

Event ID: 50
Event Type: Warning
Event Source: Ftdisk
Description: {Lost Delayed-Write Data} The system was attempting to transfer
file data from buffers to \Device\HarddiskVolume4. The write operation
failed, and only some of the data may have been written to the file.
Data:
0000: 00 00 04 00 02 00 56 00
0008: 00 00 00 00 32 00 04 80
0010: 00 00 00 00 00 00 00 00
0018: 00 00 00 00 00 00 00 00
0020: 00 00 00 00 00 00 00 00
0028: 11 00 00 80

Event ID: 26
Event Type: Information
Event Source: Application Popup
Description: Windows - Delayed Write Failed : Windows was unable to save all
the data for the file \Device\HarddiskVolume4\Program Files\Microsoft SQL
Server\MSSQL$INSTANCETWO\LOG\ERRORLOG. The data has been lost. This error
may be caused by a failure of your computer hardware or network connection.

Please try to save this file elsewhere.

These event ID messages mean exactly the same thing and are generated for the same
reasons. For the purposes of this article, only the event ID 50 message is described.

７ Note

The device and path in the description and the specific hexadecimal data will vary.
More Information
An event ID 50 message is logged if a generic error occurs when Windows is trying to
write information to the disk. This error occurs when Windows is trying to commit data
from the file system Cache Manager (not hardware level cache) to the physical disk. This
behavior is part of the memory management of Windows. For example, if a program
sends a write request, the write request is cached by Cache Manager and the program is
told the write is completed successfully. At a later point in time, Cache Manager tries to
lazy write the data to the physical disk. When Cache Manager tries to commit the data
to disk, an error occurs writing the data, and the data is flushed from the cache and
discarded. Write-back caching improves system performance, but data loss and volume
integrity loss can occur as a result of lost delayed-write failures.

It is important to remember that not all I/O is buffered I/O by Cache Manager. Programs
can set a FILE_FLAG_NO_BUFFERING flag that bypasses Cache Manager. When SQL
performs critical writes to a database, this flag is set to guarantee that the transaction is
completed directly to disk. For example, non-critical writes to log files perform buffered
I/O to improve overall performance. An event ID 50 message never results from non-
buffered I/O.

There are several different sources for an event ID 50 message. For example, an event ID
50 message logged from a MRxSmb source occurs if there is a network connectivity
problem with the redirector. To avoid performing incorrect troubleshooting steps, make
sure to review the event ID 50 message to confirm that it refers to a disk I/O issue and
that this article applies.

An event ID 50 message is similar to an event ID 9 and an event ID 11 message.

Although the error is not as serious as the error indicated by the event ID 9 and an event
ID 11 message, you can use the same troubleshooting techniques for a event ID 50
message as you do for an event ID 9 and an event ID 11 message. However, remember
that anything in the stack can cause lost-delay writes, such as filter drivers and mini-port
drivers.

You can use the binary data that is associated with any accompanying "DISK" error
(indicated by an event ID 9, 11, 51 error message or other messages) to help you in
identifying the problem.

How to Decode the Data Section of an Event ID 50 Event

Message
When you decode the data section in the example of an event ID 50 message that is
included in the "Summary" section, you see that the attempt to perform a write
operation failed because the device was busy and the data was lost. This section
describes how to decode this event ID 50 message.

The following table describes what each offset of this message represents:

OffsetLengthValues Length Values

0x00 2 Not Used

0x02 2 Dump Data Size = 0x0004

0x04 2 Number of Strings = 0x0002

0x06 2 Offset to the strings

0x08 2 Event Category

0x0c 4 NTSTATUS Error Code = 0x80040032 =

IO_LOST_DELAYED_WRITE

0x10 8 Not Used

0x18 8 Not Used

0x20 8 Not Used

0x28 4 NT Status error code

Key Sections to Decode

The Error Code

In the example in the "Summary" section, the error code is listed in the second line. This
line starts with "0008:" and it includes the last four bytes in this line:0008: 00 00 00 00 32
00 04 80 In this case, the error code is 0x80040032. The following code is the code for
error 50, and it is the same for all event ID 50 messages:
IO_LOST_DELAYED_WRITEWARNINGNote When you are converting the hexadecimal
data in the event ID message to the status code, remember that the values are
represented in the little-endian format.

The Target Disk

You can identify the disk that the write was being tried to by using the symbolic link that
is listed to the drive in the "Description" section of the event ID message, for example:
\Device\HarddiskVolume4.

The Final Status Code

The final status code is the most important piece of information in an event ID 50
message. This is the error code that is return when the I/O request was made, and it is
the key source of information. In the example in the "Summary" section, the final status
code is listed at 0x28, the sixth line, that starts with "0028:" and includes the only four
octets in this line:

0028: 11 00 00 80

In this case, the final status equals 0x80000011.This status code maps to
STATUS_DEVICE_BUSY and implies that the device is currently busy.

７ Note

When you are converting the hexadecimal data in the event ID 50 message to the
status code, remember that the values are represented in the little-endian format.
Because the status code is the only piece of information that you are interested in,
it may be easier to view the data in WORDS format instead of BYTES. If you do so,
the bytes will be in the correct format and the data may be easier to interpret
quickly.

To do so, click Words in the Event Properties window. In the Data Words view, the
example in the "Symptoms" section would read as follows: Data:

() Bytes (.)
Words 0000: 00040000 00560002 00000000 80040032 0010: 00000000 00000000
00000000 00000000 0020: 00000000 00000000 80000011

To obtain a list of Windows NT status codes, see NTSTATUS.H in the Windows Software
Developers Kit (SDK).
SMB Multichannel troubleshooting
Article • 07/22/2020

This article describes how to troubleshoot issues that are related to SMB Multichannel.

Check the network interface status

Make sure that the binding for the network interface is set to True on the SMB client
(MS_client) and SMB server (MS_server). When you run the following command, the
output should show True under Enabled for both network interfaces:

PowerShell

Get-NetAdapterBinding -ComponentID ms_server,ms_msclient

After that, make sure the network interface is listed in the output of the following
commands:

PowerShell

Get-SmbServerNetworkInterface

PowerShell

Get-SmbClientNetworkInterface

You can also run the Get-NetAdapter command to view the interface index to verify the
result. The interface index shows all the active SMB adapters that are actively bound to
the appropriate interface.

Check the firewall

If there is only a link-local IP address, and no publicly routable address, the network
profile is likely set to Public. This means that SMB is blocked at the firewall by default.

The following command reveals which connection profile is being used. You can also use
the Network and Sharing Center to retrieve this information.

Get-NetConnectionProfile
Under the File and Printer Sharing group, check the firewall inbound rules to make sure
that "SMB-In" is enabled for the correct profile.

You can also enable File and Printer Sharing in the Network and Sharing Center
window. To do this, select Change advanced sharing settings in the menu on the left,
and then select Turn on file and printer sharing for the profile. This option enables the
File and Printer Sharing firewall rules.

Capture client and server sided traffic for

troubleshooting
You need the SMB connection tracing information that starts from the TCP three-way
handshake. We recommend that you close all applications (especially Windows Explorer)
before you start the capture. Restart the Workstation service on the SMB client, start the
packet capture, and then reproduce the issue.

Make sure that the SMBv3.x connection is being negotiated, and that nothing in
between the server and the client is affecting dialect negotiation. SMBv2 and earlier
versions don't support multichannel.
Look for the NETWORK_INTERFACE_INFO packets. This is where the SMB client requests
a list of adapters from the SMB server. If these packets aren't exchanged, multichannel
doesn't work.

The server responds by returning a list of valid network interfaces. Then, the SMB client
adds those to the list of available adapters for multichannel. At this point, multichannel
should start and, at least, try to start the connection.

For more information, see the following articles:

3.2.4.20.10 Application Requests Querying Server's Network Interfaces

2.2.32.5 NETWORK_INTERFACE_INFO Response

3.2.5.14.11 Handling a Network Interfaces Response

In the following scenarios, an adapter cannot be used:

There is a routing issue on the client. This is typically caused by an incorrect

routing table that forces traffic over the wrong interface.

Multichannel constraints have been set. For more information, see New-
SmbMultichannelConstraint.

Something blocked the network interface request and response packets.

The client and server can't communicate over the extra network interface. For
example, the TCP three-way handshake failed, the connection is blocked by a
firewall, session setup failed, and so on.

If the adapter and its IPv6 address are on the list that is sent by the server, the next step
is to see whether communications are tried over that interface. Filter the trace by the
link-local address and SMB traffic, and look for a connection attempt. If this is a
NetConnection trace, you can also examine Windows Filtering Platform (WFP) events to
see whether the connection is being blocked.
File Server Resource Manager (FSRM)
overview
Article • 03/21/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

File Server Resource Manager (FSRM) is a role service in Windows Server that enables
you to manage and classify data stored on file servers. You can use FSRM to
automatically classify files, perform tasks based on these classifications, set quotas on
folders, and create reports monitoring storage usage. In Windows Server version 1803,
FSRM adds the ability to prevent the creation of change journals.

７ Note

For new features on older versions of Windows Server, see What's New in File
Server Resource Manager.

Features
FSRM includes the following features:

Quota management: Limit the space that is allowed for a volume or folder. These
limits can be automatically applied to new folders that are created on a volume.
You can also define quota templates that can be applied to new volumes or
folders.
File Classification Infrastructure: Gain insight into your data by automating
classification processes so that you can manage your data more effectively. You
can classify files and apply policies based on this classification. Example policies
include dynamic access control for restricting access to files, file encryption, and
file expiration. Files can be classified automatically by using file classification rules
or manually by modifying the properties of a selected file or folder.
File Management Tasks: Gain the ability to apply a conditional policy or action to
files based on their classification. The conditions of a file management task include
the file location, the classification properties, the date the file was created, the last
modified date of the file, or the last time the file was accessed. The actions that a
file management task can take include the ability to expire files, encrypt files, or
run a custom command.
 File screening management: Control the types of files that the user can store on a
file server. You can limit the extension that can be stored on your shared files. For
example, you can create a file screen that doesn't allow files with an MP3 extension
to be stored in personal shared folders on a file server.
Storage reports: Use these reports to help you identify trends in disk usage and
how your data is classified. You can also monitor a selected group of users for
attempts to save unauthorized files.

You can configure and manage the FSRM features by using the FSRM app or by using
Windows PowerShell.

） Important

FSRM supports volumes formatted with the NTFS file system only. The Resilient File
System isn't supported.

Practical applications
The following list outlines some practical applications for FSRM:

Use File Classification Infrastructure with the Dynamic Access Control scenario.
Create a policy that grants access to files and folders based on the way files are
classified on the file server.

Create a file classification rule that tags any file that contains at least 10 social
security numbers as having customer content.

Expire any file that hasn't been modified in the last 10 years.

Create a 200-MB quota for each user's home directory and notify them when
they're using 180 MB.

Disallow any music files to be stored in personal shared folders.

Schedule a report that runs every Sunday night at midnight that generates a list of
the most recently accessed files from the previous two days. This report can help
you determine the weekend storage activity and plan your server downtime
accordingly.

What's new - prevent FSRM from creating

change journals
Starting with Windows Server, version 1803, you can now prevent the FSRM service from
creating a change journal (also known as a USN journal) on volumes when the service
starts. This feature can conserve some space on each volume, but disables real-time file
classification.

To prevent FSRM from creating a change journal on some or all volumes when the
service starts, complete the following steps:

1. Stop the SRMSVC service. Open a PowerShell session as an administrator and enter
Stop-Service SrmSvc .

2. Delete the USN journal for the volumes you want to conserve space on by using
the fsutil command:

PowerShell

fsutil usn deletejournal /d <VolumeName>

For example: fsutil usn deletejournal /d c:

3. Open Registry Editor by typing regedit in the same PowerShell session.

4. Go to the following key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SrmSvc\Settings.

5. To prevent change journal creation for the entire server, complete the following
steps:

） Important

If you want to disable journal creation for specific volumes only, continue to
the next step.

a. Right-click the Settings key, and then select New > DWORD (32-bit) Value.
b. Name the value SkipUSNCreationForSystem .
c. Set the value to 1 (in hexadecimal).

6. To prevent change journal creation for specific volumes, complete the following
steps:

a. Identify the volume paths you want to skip. You can use the fsutil volume list
command or the following PowerShell command:

PowerShell
 Get-Volume | Format-Table DriveLetter,FileSystemLabel,Path

Here's an example output:

Console

DriveLetter FileSystemLabel Path

----------- --------------- ----
System Reserved \\?\Volume{8d3c9e8a-0000-0000-0000-
100000000000}\
C \\?\Volume{8d3c9e8a-0000-0000-0000-
501f00000000}\

b. Return to your Registry Editor session. Right-click the

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SrmSvc\Settings key,
and then select New > Multi-String Value.

c. Name the value SkipUSNCreationForVolumes .

d. Enter the path for each volume that you want to skip. Place each path on a
separate line. For example:

PowerShell

\\?\Volume{8d3c9e8a-0000-0000-0000-100000000000}\
\\?\Volume{8d3c9e8a-0000-0000-0000-501f00000000}\

７ Note

If Registry Editor displays a warning about removed empty strings, you can
safely disregard the message. Here’s an example of the message you might
see: Data of type REG_MULTI_SZ cannot contain empty strings. Registry
Editor will remove all empty strings found.

7. Start the SRMSVC service. For example, in a PowerShell session enter Start-
Service SrmSvc .

Related links
Dynamic Access Control overview
Checklist: Apply a Quota to a volume or
folder
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

1. Configure e-mail settings if you plan to send threshold notifications or storage

reports by e-mail. Configure E-Mail Notifications

2. Assess storage requirements on the volume or folder. You can use reports on the
Storage Reports Management node to provide data. (For example, run a Files by
Owner report on demand to identify users who use large amounts of disk space.)
Generate Reports on Demand

3. Review available pre-configured quota templates. (In Quota Management, click

the Quota Templates node.) Edit Quota Template Properties
-Or-
Create a new quota template to enforce a storage policy in your organization.
Create a Quota Template

4. Create a quota based on the template on the volume or folder. Create a Quota
-Or-
Create an auto apply quota to automatically generate quotas for subfolders on the
volume or folder. Create an Auto Apply Quota

5. Schedule a report task that contains a Quota Usage report to monitor quota usage
periodically. Schedule a Set of Reports

７ Note

If you want to screen files on a volume or folder, see Checklist: Apply a File Screen
to a Volume or Folder.
Checklist - Apply a file screen to a
volume or folder
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

To apply a file screen to a volume or folder, use the following list:

1. Configure e-mail settings if you plan to send file screening notifications or storage
reports by e-mail by following the instructions in Configure E-Mail Notifications.

2. Enable recording of file screening events in the auditing database if you plan to
generate File Screening Audit reports. Configure File Screen Audit

3. Assess stored file types that are candidates for screening rules. You can use reports
at the Storage Reports Management node to provide data. (For example, run a
Files by File Group report or a Large Files report on demand to identify files that
occupy large amounts of disk space.) Generate Reports on Demand

4. Review the preconfigured file groups, or create a new file group to enforce a
specific screening policy in your organization. Define File Groups for Screening

5. Review the properties of available file screen templates. (In File Screening
Management, click the File Screen Templates node.) Edit File Screen Template
Properties
-Or-
Create a new file screen template to enforce a storage policy in your organization.
Create a File Screen Template

6. Create a file screen based on the template on a volume or folder. Create a File
Screen

7. Configure file screen exceptions in subfolders of the volume or folder. Create a File
Screen Exception

8. Schedule a report task containing a File Screening Audit report to monitor

screening activity periodically. Schedule a Set of Reports

７ Note
To limit storage on a volume or folder, see Checklist: Apply a Quota to a Volume
or Folder
Setting File Server Resource Manager
Options
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

The general File Server Resource Manager options can be set in the File Server Resource
Manager Options dialog box. These settings are used throughout the nodes, and some
of them can be modified when you work with quotas, screen files, or generate storage
reports.

This section includes the following topics:

Configure E-Mail Notifications

Configure Notification Limits
Configure Storage Reports
Configure File Screen Audit
Configure E-Mail Notifications
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

When you create quotas and file screens, you have the option of sending e-mail
notifications to users when their quota limit is approaching or after they have attempted
to save files that have been blocked. When you generate storage reports, you have the
option of sending the reports to specific recipients by e-mail. If you want to routinely
notify certain administrators about quota and file screening events, or send storage
reports, you can configure one or more default recipients.

To send these notifications and storage reports, you must specify the SMTP server to be
used for forwarding the e-mail messages.

To configure e-mail options

1. In the console tree, right-click File Server Resource Manager, and then click
Configure Options. The File Server Resource Manager Options dialog box opens.

2. On the E-mail Notifications tab, under SMTP server name or IP address, type the
host name or the IP address of the SMTP server that will forward e-mail
notifications and storage reports.

3. If you want to routinely notify certain administrators about quota or file screening
events or e-mail storage reports, under Default administrator recipients, type
each e-mail address.

Use the format account@domain. Use semicolons to separate multiple accounts.

4. To specify a different "From" address for e-mail notifications and storage reports
sent from File Server Resource Manager, under the Default "From" e-mail address,
type the e-mail address that you want to appear in your message.

5. To test your settings, click Send Test E-mail.

6. Click OK.

To configure e-mail options using PowerShell

You can use the Set-FsrmSetting cmdlet to set the e-mail configuration and the Send-
FsrmTestEmail cmdlet to send a test email as shown in the following example:

PowerShell

# Setting FSRM email options

$MHT = @{
SmtpServer = 'SMTP.Contoso.Com'
FromEmailAddress = '[email protected]'
AdminEmailAddress = '[email protected]'
}
Set-FsrmSetting @MHT

# Sending a test email to check the setup

$MHT = @{
ToEmailAddress = '[email protected]'
Confirm = $false
}
Send-FsrmTestEmail @MHT

Additional References
Setting File Server Resource Manager Options
Configure Notification Limits
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

To reduce the number of notifications that accumulate for repeatedly exceeding a quota
threshold or attempting to save an unauthorized file, File Server Resource Manager
applies time limits to the following notification types:

E-mail
Event log
Command
Report

Each limit specifies a period of time before another configured notification of the same
type is generated for an identical issue.

A default 60-minute limit is set for each notification type, but you can change these
limits. The limit applies to all the notifications of a given type, whether they are
generated by quota thresholds or by file screening events.

To specify a standard notification limit for each

notification type
1. In the console tree, right-click File Server Resource Manager, and then click
Configure Options. The File Server Resource Manager Options dialog box opens.

2. On the Notification Limits tab, enter a value in minutes for each notification type
that is shown.

3. Click OK.

７ Note

To customize time limits that are associated with notifications for a specific quota
or file screen, you can use the File Server Resource Manager command-line tools
Dirquota.exe and Filescrn.exe, or use the File Server Resource Manager cmdlets.
Additional References
Setting File Server Resource Manager Options
Command-Line Tools
Configure Storage Reports
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

You can configure the default parameters for storage reports. These default parameters
are used for the incident reports that are generated when a quota or file screening event
occurs. They are also used for scheduled and on-demand reports, and you can override
the default parameters when you define the specific properties of these reports.

） Important

When you change the default parameters for a type of report, changes affect all
incident reports and any existing scheduled report tasks that use the defaults.

To configure the default parameters for

Storage Reports
1. In the console tree, right-click File Server Resource Manager, and then click
Configure Options. The File Server Resource Manager Options dialog box opens.

2. On the Storage Reports tab, under Configure default parameters, select the type
of report that you want to modify.

3. Click Edit Parameters.

4. Depending on the type of report that you select, different report parameters will
be available for editing. Perform all necessary modifications, and then click OK to
save them as the default parameters for that type of report.

5. Repeat steps 2 through 4 for each type of report that you want to edit.

6. To see a list of the default parameters for all reports, click Review Reports. Then
click Close.

7. Click OK.

Additional References
Setting File Server Resource Manager Options
Storage Reports Management
Configure File Screen Audit
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

By using File Server Resource Manager, you can record file screening activity in an
auditing database. The information saved in this database is used to generate the File
Screening Audit report.

） Important

If the Record file screening activity in the auditing database check box is cleared,
the File Screening Audit Reports will not contain any information.

To configure file screen audit

1. In the console tree, right-click File Server Resource Manager, and then click
Configure Options. The File Server Resource Manager Options dialog box opens.

2. On the File Screen Audit tab, select the Record file screening activity in the
auditing database check box.

3. Click OK. All file screening activity will now be stored in the auditing database, and
it can be viewed by running a File Screening Audit report.

Additional References
Setting File Server Resource Manager Options
Storage Reports Management
Quota Management
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

On the Quota Management node of the File Server Resource Manager Microsoft®
Management Console (MMC) snap-in, you can perform the following tasks:

Create quotas to limit the space allowed for a volume or folder, and generate
notifications when the quota limits are approached or exceeded.
Generate auto apply quotas that apply to all existing subfolders in a volume or
folder and to any subfolders that are created in the future.
Define quota templates that can be easily applied to new volumes or folders and
then used across an organization.

For example, you can:

Place a 200 megabyte (MB) limit on users' personal server folders, with an email
notification sent to you and the user when 180 MB of storage has been exceeded.
Set a flexible 500 MB quota on a group's shared folder. When this storage limit is
reached, all users in the group are notified by e-mail that the storage quota has
been temporarily extended to 520 MB so that they can delete unnecessary files
and comply with the preset 500 MB quota policy.
Receive a notification when a temporary folder reaches 2 gigabytes (GB) of usage,
yet not limit that folder's quota because it is necessary for a service running on
your server.

This section includes the following topics:

Create a Quota
Create an Auto Apply Quota
Create a Quota Template
Edit Quota Template Properties
Edit Auto Apply Quota Properties

７ Note

To set e-mail notifications and reporting capabilities, you must first configure the
general File Server Resource Manager options. If you're just looking to free up
space on a volume, consider using Azure File Sync with cloud tiering enabled. This
 allows you to cache your most frequently accessed files locally and tier your least
frequently accessed files to the cloud, saving local storage space while maintaining
performance. For details, see Planning for an Azure File Sync deployment.

Additional References
Setting File Server Resource Manager Options
Create a Quota
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

Quotas can be created from a template or with custom properties. The following
procedure describes how to create a quota that is based on a template (recommended).
If you need to create a quota with custom properties, you can save these properties as a
template to re-use at a later date.

When you create a quota, you choose a quota path, which is a volume or folder that the
storage limit applies to. On a given quota path, you can use a template to create one of
the following types of quota:

A single quota that limits the space for an entire volume or folder.
An auto apply quota, which assigns the quota template to a folder or volume.
Quotas based on this template are automatically generated and applied to all
subfolders. For more information about creating auto apply quotas, see Create an
Auto Apply Quota.

７ Note

By creating quotas exclusively from templates, you can centrally manage your
quotas by updating the templates instead of the individual quotas. Then, you can
apply changes to all quotas based on the modified template. This feature simplifies
the implementation of storage policy changes by providing one central point where
all updates can be made.

To create a quota that is based on a template

1. In Quota Management, click the Quota Templates node.

2. In the Results pane, select the template on which you will base your new quota.

3. Right-click the template and click Create Quota from Template (or select Create
Quota from Template from the Actions pane). This opens the Create Quota dialog
box with the summary properties of the quota template displayed.

4. Under Quota path, type or browse to the folder that the quota will apply to.
 5. Click the Create quota on path option. Note that the quota properties will apply to
the entire folder.

７ Note

To create an auto apply quota, click the Auto apply template and create
quotas on existing and new subfolders option. For more information about
auto apply quotas, see Create an Auto Apply Quota

6. Under Derive properties from this quota template, the template you used in step
2 to create your new quota is pre-selected (or you can select another template
from the list). Note that the template's properties are displayed under Summary of
quota properties.

7. Click Create.

Additional References
Quota Management
Create an Auto Apply Quota
Create a Quota Template
Create an Auto Apply Quota
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

By using auto apply quotas, you can assign a quota template to a parent volume or
folder. Then File Server Resource Manager automatically generates quotas that are
based on that template. Quotas are generated for each of the existing subfolders and
for subfolders that you create in the future.

For example, you can define an auto apply quota for subfolders that are created on
demand, for roaming profile users, or for new users. Every time a subfolder is created, a
new quota entry is automatically generated by using the template from the parent
folder. These automatically generated quota entries can then be viewed as individual
quotas under the Quotas node. Each quota entry can be maintained separately.

To create an Auto Apply Quota

1. In Quota Management, click the Quotas node.

2. Right-click Quotas, and then click Create Quota (or select Create Quota from the
Actions pane). This opens the Create Quota dialog box.

3. Under Quota path, type the name of or browse to the parent folder that the quota
profile will apply to. The auto apply quota will be applied to each of the subfolders
(current and future) in this folder.

4. Click Auto apply template and create quotas on existing and new subfolders.

5. Under Derive properties from this quota template, select the quota template that
you want to apply from the drop-down list. Note that each template's properties
are displayed under Summary of quota properties.

6. Click Create.

７ Note

You can verify all automatically generated quotas by selecting the Quotas node and
then selecting Refresh. An individual quota for each subfolder and the auto apply
quota profile in the parent volume or folder are listed.
Additional References
Quota Management
Edit Auto Apply Quota Properties
Create a Quota Template
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

A quota template defines a space limit, the type of quota (hard or soft) and optionally, a
set of notifications that will be generated automatically when quota usage reaches
defined threshold levels.

By creating quotas exclusively from templates, you can centrally manage your quotas by
updating the templates instead of replicating changes in each quota. This feature
simplifies the implementation of storage policy changes by providing one central point
where you can make all updates.

To create a Quota Template

1. In Quota Management, click the Quota Templates node.

2. Right-click Quota Templates, and then click Create Quota Template (or select
Create Quota Template from the Actions pane). This opens the Create Quota
Template dialog box.

3. If you want to copy the properties of an existing template to use as a base for your
new template, select a template from the Copy properties from quota template
drop-down list. Then click Copy.

Whether you have chosen to use the properties of an existing template or you are
creating a new template, modify or set the following values on the Settings tab:

4. In the Template Name text box, enter a name for the new template.

5. In the Label text box, enter an optional descriptive label that will appear next to
any quotas derived from the template.

6. Under Space Limit:

In the Limit text box, enter a number and choose a unit (KB, MB, GB, or TB) to
specify the space limit for the quota.
Click the Hard quota or Soft quota option. (A hard quota prevents users from
saving files after the space limit is reached and generates notifications when
 the volume of data reaches each configured threshold. A soft quota does not
enforce the quota limit, but it generates all configured notifications.)

7. You can configure one or more optional threshold notifications for your quota
template, as described in the procedure that follows. After you have selected all
the quota template properties that you want to use, click OK to save the template.

Setting optional notification thresholds

When storage in a volume or folder reaches a threshold level that you define, File Server
Resource Manager can send e-mail messages to administrators or specific users, log an
event, execute a command or a script, or generate reports. You can configure more than
one type of notification for each threshold, and you can define multiple thresholds for
any quota (or quota template). By default, no notifications are generated.

For example, you could configure thresholds to send an e-mail message to the
administrator and the users who would be interested to know when a folder reaches 85
percent of its quota limit, and then send another notification when the quota limit is
reached. Additionally, you might want to run a script that uses the dirquota.exe
command to raise the quota limit automatically when a threshold is reached.

） Important

To send e-mail notifications and configure the storage reports with parameters that
are appropriate for your server environment, you must first set the general File
Server Resource Manager options. For more information, see Setting File Server
Resource Manager Options

To configure notifications that File Server Resource Manager will generate at a quota
threshold

1. In the Create Quota Template dialog box, under Notification thresholds, click
Add. The Add Threshold dialog box appears.

2. To set a quota limit percentage that will generate a notification:

In the Generate notifications when usage reaches (%) text box, enter a
percentage of the quota limit for the notification threshold. (The default
percentage for the first notification threshold is 85 percent.)

3. To configure e-mail notifications:

On the E-mail Message tab, set the following options:

 To notify administrators when a threshold is reached, select the Send e-mail
to the following administrators check box, and then enter the names of the
administrative accounts that will receive the notifications. Use the format
account@domain, and use semicolons to separate multiple accounts.
To send e-mail to the person who saved the file that reached the quota
threshold, select the Send e-mail to the user who exceeded the threshold
check box.
To configure the message, edit the default subject line and message body
that are provided. The text that is in brackets inserts variable information
about the quota event that caused the notification. For example, the [Source
Io Owner] variable inserts the name of the user who saved the file that
reached the quota threshold. To insert additional variables in the text, click
Insert Variable.
To configure additional headers (including From, Cc, Bcc, and Reply-to), click
Additional E-mail Headers.

4. To log an event:

On the Event Log tab, select the Send warning to event log check box, and edit
the default log entry.

5. To run a command or script:

On the Command tab, select the Run this command or script check box. Then
type the command, or click Browse to search for the location where the script is
stored. You can also enter command arguments, select a working directory for the
command or script, or modify the command security setting.

6. To generate one or more storage reports:

On the Report tab, select the Generate reports check box, and then select which
reports to generate. (You can choose one or more administrative e-mail recipients
for the report or e-mail the report to the user who reached the threshold.)

The report is saved in the default location for incident reports, which you can
modify in the File Server Resource Manager Options dialog box.

7. Click OK to save your notification threshold.

8. Repeat these steps if you want to configure additional notification thresholds for
the quota template.

Additional References
Quota Management
Setting File Server Resource Manager Options
Edit Quota Template Properties
Command-Line Tools
Edit Quota Template Properties
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

When you make changes to a quota template, you have the option of extending those
changes to quotas that were created from the original quota template. You can choose
to modify only those quotas that still match the original template or all quotas that were
derived from the original template, regardless of any modifications that were made to
the quotas since they were created. This feature simplifies the process of updating the
properties of your quotas by providing one central point where you can make all the
changes.

７ Note

If you choose to apply the changes to all quotas that were derived from the original
template, you will overwrite any custom quota properties that you created.

To edit Quota Template Properties

1. In Quota Templates, select the template that you want to modify.

2. Right-click the quota template, and then click Edit Template Properties (or in the
Actions pane, under Selected Quota Templates, select Edit Template Properties).
This opens the Quota Template Properties dialog box.

3. Perform all necessary changes. The settings and notification options are identical
to those that you can set when you create a quota template. Optionally, you can
copy the properties from a different template and modify them for this one.

4. When you are finished editing the template properties, click OK. This will open the
Update Quotas Derived from Template dialog box.

5. Select the type of update that you want to apply:

If you have quotas that have been modified since they were created with the
original template, and you do not want to change them, select Apply
template only to derived quotas that match the original template. This
 option will update only those quotas that have not been edited since they
were created with the original template.
If you want to modify all existing quotas that were created from the original
template, select Apply template to all derived quotas.
If you want to keep the existing quotas unchanged, select Do not apply
template to derived quotas.

6. Click OK.

Additional References
Quota Management
Create a Quota Template
Edit Auto Apply Quota Properties
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

When you make changes to an auto apply quota, you have the option of extending
those changes to existing quotas in the auto apply quota path. You can choose to
modify only those quotas that still match the original auto apply quota or all quotas in
the auto apply quota path, regardless of any modifications that were made to the
quotas since they were created. This feature simplifies the process of updating the
properties of quotas that were derived from an auto apply quota by providing one
central point where you can make all the changes.

７ Note

If you choose to apply the changes to all quotas in the auto apply quota path, you
will overwrite any custom quota properties that you have created.

To edit an Auto Apply Quota

1. In Quotas, select the auto apply quota that you want to modify. You can filter the
quotas to show only auto apply quotas.

2. Right-click the quota entry, and then click Edit Quota Properties (or in the Actions
pane, under Selected Quotas, select Edit Quota Properties). This opens the Edit
Auto Apply Quota dialog box.

3. Under Derive properties from this quota template, select the quota template that
you want to apply. You can review the properties of each quota template in the
summary list box.

4. Click OK. This will open the Update Quotas Derived from Auto Apply Quota
dialog box.

5. Select the type of update that you want to apply:

If you have quotas that have been modified since they were automatically
generated, and you do not want to change them, select Apply auto apply
quota only to derived quotas that match the original auto apply quota. This
 option will update only those quotas in the auto apply quota path that have
not been edited since they were automatically generated.
If you want to modify all existing quotas in the auto apply quota path, select
Apply auto apply quota to all derived quotas.
If you want to keep the existing quotas unchanged but make the modified
auto apply quota effective for new subfolders in the auto apply quota path,
select Do not apply auto apply quota to derived quotas.

6. Click OK.

Additional References
Quota Management
Create an Auto Apply Quota
File Screening Management
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

On the File Screening Management node of the File Server Resource Manager MMC
snap-in, you can perform the following tasks:

Create file screens to control the types of files that users can save, and generate
notifications when users attempt to save unauthorized files.
Define file screening templates that can be applied to new volumes or folders and
that can be used across an organization.
Create file screening exceptions that extend the flexibility of the file screening
rules.

For example, you can:

Ensure that no music files are stored on personal folders on a server, yet you could
allow storage of specific types of media files that support legal rights management
or comply with company policies. In the same scenario, you might want to give a
vice president in the company special privileges to store any type of files in his
personal folder.
Implement a screening process to notify you by e-mail when an executable file is
stored on a shared folder, including information about the user who stored the file
and the file's exact location, so that you can take the appropriate precautionary
steps.

This section includes the following topics:

Define File Groups for Screening

Create a File Screen
Create a File Screen Exception
Create a File Screen Template
Edit File Screen Template Properties

７ Note

To set e-mail notifications and certain reporting capabilities, you must first
configure the general File Server Resource Manager options.
Additional References
Setting File Server Resource Manager Options
Define File Groups for Screening
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

A file group is used to define a namespace for a file screen, file screen exception, or Files
by File Group storage report. It consists of a set of file name patterns, which are
grouped by the following:

Files to include: files that belong in the group

Files to exclude: files that do not belong in the group

７ Note

For convenience, you can create and edit file groups while you edit the properties
of file screens, file screen exceptions, file screen templates, and Files by File Group
reports. Any file group changes that you make from these property sheets are not
limited to the current item that you are working on.

To create a File Group

1. In File Screening Management, click the File Groups node.

2. On the Actions pane, click Create File Group. This opens the Create File Group
Properties dialog box.

(Alternatively, while you edit the properties of a file screen, file screen exception,
file screen template, or Files by File Group report, under Maintain file groups, click
Create.)

3. In the Create File Group Properties dialog box, type a name for the file group.

4. Add files to include and files to exclude:

For each set of files that you want to include in the file group, in the Files to
include box, enter a file name pattern, and then click Add.
For each set of files that you want to exclude from the file group, in the Files
to exclude box, enter a file name pattern, and then click Add. Note that
standard wildcard rules apply, for example, *.exe selects all executable files.
 5. Click OK.

Additional References
File Screening Management
Create a File Screen
Create a File Screen Exception
Create a File Screen Template
Storage Reports Management
Create a File Screen
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

When creating a new file screen, you can choose to save a file screen template that is
based on the custom file screen properties that you define. The advantage of this is that
a link is maintained between file screens and the template that is used to create them,
so that in the future, changes to the template can be applied to all file screens that
derive from it. This is a feature that simplifies the implementation of storage policy
changes by providing one central point where you can make all updates.

To create a File Screen with custom properties

1. In File Screening Management, click the File Screens node.

2. Right-click File Screens, and click Create File Screen (or select Create File Screen
from the Actions pane). This opens the Create File Screen dialog box.

3. Under File screen path, type the name of or browse to the folder that the file
screen will apply to. The file screen will apply to the selected folder and all of its
subfolders.

4. Under How do you want to configure file screen properties, click Define custom
file screen properties, and then click Custom Properties. This opens the File
Screen Properties dialog box.

5. If you want to copy the properties of an existing template to use as a base for your
file screen, select a template from the Copy properties from template drop-down
list. Then click Copy.

In the File Screen Properties dialog box, modify or set the following values on the
Settings tab:

6. Under Screening type, click the Active screening or Passive screening option.
(Active screening prevents users from saving files that are members of blocked file
groups and generates notifications when users try to save unauthorized files.
Passive screening sends configured notifications, but it does not prevent users
from saving files.)
 7. Under File groups, select each file group that you want to include in your file
screen. (To select the check box for the file group, double-click the file group
label.)

If you want to view the file types that a file group includes and excludes, click the
file group label, and then click Edit. To create a new file group, click Create.

8. Additionally, you can configure File Server Resource Manager to generate one or
more notifications by setting options on the E-mail Message, Event Log,
Command, and Report tabs. For more information about file screen notification
options, see Create a File Screen Template.

9. After you have selected all the file screen properties that you want to use, click OK
to close the File Screen Properties dialog box.

10. In the Create File Screen dialog box, click Create to save the file screen. This opens
the Save Custom Properties as a Template dialog box.

11. Select the type of custom file screen you want to create:

To save a template that is based on these customized properties

(recommended), click Save the custom properties as a template and enter a
name for the template. This option will apply the template to the new file
screen, and you can use the template to create additional file screens in the
future. This will enable you to later update the file screens automatically by
updating the template.
If you do not want to save a template when you save the file screen, click
Save the custom file screen without creating a template.

12. Click OK.

Additional References
File Screening Management
Define File Groups for Screening
Create a File Screen Template
Edit File Screen Template Properties
Create a File Screen Exception
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

Occasionally, you need to allow exceptions to file screening. For example, you might
want to block video files from a file server, but you need to allow your training group to
save the video files for their computer-based training. To allow files that other file
screens are blocking, create a file screen exception.

A file screen exception is a special type of file screen that over-rides any file screening
that would otherwise apply to a folder and all its subfolders in a designated exception
path. That is, it creates an exception to any rules derived from a parent folder.

７ Note

You cannot create a file screen exception on a parent folder where a file screen is
already defined. You must assign the exception to a subfolder or make changes to
the existing file screen.

You assign file groups to determine which file types will be allowed in the file screen
exception.

To create a File Screen Exception

1. In File Screening Management, click the File Screens node.

2. Right-click File Screens, and click Create File Screen Exception (or select Create
File Screen Exception from the Actions pane). This opens the Create File Screen
Exception dialog box.

3. In the Exception path text box, type or select the path that the exception will apply
to. The exception will apply to the selected folder and all of its subfolders.

4. To specify which files to exclude from file screening:

Under File groups, select each file group that you want to exclude from file
screening. (To select the check box for the file group, double-click the file
group label.)
 If you want to view the file types that a file group includes and excludes, click
the file group label, and click Edit.
To create a new file group, click Create.

5. Click OK.

Additional References
File Screening Management
Define File Groups for Screening
Create a File Screen Template
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

A file screen template defines a set of file groups to screen, the type of screening to
perform (active or passive), and optionally, a set of notifications that will be generated
automatically when a user saves, or attempts to save, an unauthorized file.

File Server Resource Manager can send e-mail messages to administrators or specific
users, log an event, execute a command or a script, or generate reports. You can
configure more than one type of notification for a file screen event.

By creating file screens exclusively from templates, you can centrally manage your file
screens by updating the templates instead of replicating changes in each file screen.
This feature simplifies the implementation of storage policy changes by providing one
central point where you can make all updates.

） Important

To send e-mail notifications and to configure the storage reports with parameters
that are appropriate for your server environment, you must first set the general File
Server Resource Manager options. For more information, see Setting File Server
Resource Manager Options.

To create a File Screen Template

1. In File Screening Management, click the File Screen Templates node.

2. Right-click File Screen Templates, and then click Create File Screen Template (or
select Create File Screen Template from the Actions pane). This opens the Create
File Screen Template dialog box.

3. If you want to copy the properties of an existing template to use as a base for your
new template, select a template from the Copy properties from template drop-
down list and then click Copy.

Whether you have chosen to use the properties of an existing template or you are
creating a new template, modify or set the following values on the Settings tab:
4. In the Template name text box, enter a name for the new template.

5. Under Screening type, click the Active screening or Passive screening option.
(Active screening prevents users from saving files that are members of blocked file
groups and generates notifications when users try to save unauthorized files.
Passive screening sends configured notifications, but it does not prevent users
from saving files).

6. To specify which file groups to screen:

Under File groups, select each file group that you want to include. (To select the
check box for the file group, double-click the file group label.)

If you want to view the file types that a file group includes and excludes, click the
file group label, and then click Edit. To create a new file group, click Create.

Additionally, you can configure File Server Resource Manager to generate one or
more notifications by setting the following options on the E-mail Message, Event
Log, Command, and Report tabs.

7. To configure e-mail notifications:

On the E-mail Message tab, set the following options:

To notify administrators when a user or application attempts to save an

unauthorized file, select the Send e-mail to the following administrators
check box, and then enter the names of the administrative accounts that will
receive the notifications. Use the format account@domain, and use
semicolons to separate multiple accounts.
To send e-mail to the user who attempted to save the file, select the Send e-
mail to the user who attempted to save an unauthorized file check box.
To configure the message, edit the default subject line and message body
that are provided. The text that is in brackets inserts variable information
about the file screen event that caused the notification. For example, the
[Source Io Owner] variable inserts the name of the user who attempted to
save an unauthorized file. To insert additional variables in the text, click Insert
Variable.
To configure additional headers (including From, Cc, Bcc, and Reply-to), click
Additional E-mail Headers.

8. To log an error to the event log when a user tries to save an unauthorized file:

On the Event Log tab, select the Send warning to event log check box, and edit
the default log entry.
 9. To run a command or script when a user tries to save an unauthorized file:

On the Command tab, select the Run this command or script check box. Then
type the command, or click Browse to search for the location where the script is
stored. You can also enter command arguments, select a working directory for the
command or script, or modify the command security setting.

10. To generate one or more storage reports when a user tries to save an unauthorized
file:

On the Report tab, select the Generate reports check box, and then select which
reports to generate. (You can choose one or more administrative e-mail recipients
for the report or e-mail the report to the user who attempted to save the file.)

The report is saved in the default location for incident reports, which you can
modify in the File Server Resource Manager Options dialog box.

11. After you have selected all the file template properties that you want to use, click
OK to save the template.

Additional References
File Screening Management
Setting File Server Resource Manager Options
Edit File Screen Template Properties
Edit File Screen Template Properties
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

When you make changes to a file screen template, you have the option of extending
those changes to file screens that were created with the original file screen template.
You can choose to modify only those file screens that match the original template or all
file screens that derive from the original template, regardless of any modifications that
you made to the file screens since they were created. This feature simplifies the process
of updating the properties of file screens by providing one central point where you
make all the changes.

７ Note

If you apply the changes to all file screens that derive from the original template,
you will overwrite any custom file screen properties that you created.

To edit File Screen Template Properties

1. In File Screen Templates, select the template that you want to modify.

2. Right-click the file screen template and click Edit Template Properties (or in the
Actions pane, under Selected File Screen Templates, select Edit Template
Properties.) This opens the File Screen Template Properties dialog box.

3. If you want to copy the properties of another template as a base for your modified
template, select a template from the Copy properties from template drop-down
list. Then click Copy.

4. Perform all necessary changes. The settings and notification options are identical
to those that are available when you create a file screen template.

5. When you are finished editing the template properties, click OK. This will open the
Update File Screens Derived from Template dialog box.

6. Select the type of update that you want to apply:

 If you have file screens that have been modified since they were created
using the original template, and you do not want to change them, click Apply
template only to derived file screens that match the original template. This
option will update only those file screens that have not been edited since
they were created with the original template properties.
If you want to modify all existing file screens that were created using the
original template, click Apply template to all derived file screens.
If you want to keep the existing file screens unchanged, click Do not apply
template to derived file screens.

7. Click OK.

Additional References
File Screening Management
Create a File Screen Template
Storage Reports Management
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

On the Storage Reports Management node of the File Server Resource Manager
Microsoft® Management Console (MMC) snap-in, you can perform the following tasks:

Schedule periodic storage reports that allow you to identify trends in disk usage.
Monitor attempts to save unauthorized files for all users or a selected group of
users.
Generate storage reports instantly.

For example, you can:

Schedule a report that will run every Sunday at midnight, generating a list that
includes the most recently accessed files from the previous two days. With this
information, you can monitor weekend storage activity and plan server down-time
that will have less impact on users who are connecting from home over the
weekend.
Run a report at any time to identify all duplicate files in a volume on a server so
that disk space can be quickly reclaimed without losing any data.
Run a Files by File Group report to identify how storage resources are segmented
across different file groups
Run a Files by Owner report to analyze how individual users are using shared
storage resources.

This section includes the following topics:

Schedule a Set of Reports

Generate Reports on Demand

７ Note

To set e-mail notifications and certain reporting capabilities, you must first
configure the general File Server Resource Manager options.

Additional References
Setting File Server Resource Manager Options
Schedule a Set of Reports
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

To generate a set of reports on a regular schedule, you schedule a report task. The
report task specifies which reports to generate and what parameters to use; which
volumes and folders to report on; how often to generate the reports and which file
formats to save them in.

Scheduled reports are saved in a default location, which you can specify in the File
Server Resource Manager Options dialog box. You also have the option of delivering
the reports by e-mail to a group of administrators.

７ Note

To minimize the impact of report processing on performance, generate multiple

reports on the same schedule so that the data is only gathered once. To quickly add
reports to existing report tasks, you can use the Add or Remove Reports for a
Report Task action. This allows you to add or remove reports from multiple report
tasks and to edit the report parameters. To change schedules or delivery addresses,
you must edit individual report tasks.

To schedule a Report Task

1. Click the Storage Reports Management node.

2. Right-click Storage Reports Management, and then click Schedule a New Report
Task (or select Schedule a New Report Task from the Actions pane). This opens
the Storage Reports Task Properties dialog box.

3. To select volumes or folders on which to generate reports:

Under Scope, click Add.

Browse to the volume or folder on which you want to generate the reports,
select it, and then click OK to add the path to the list.
Add as many volumes or folders as you want to include in the reports. (To
remove a volume or folder, click the path and then click Remove).
4. To specify which reports to generate:

Under Report data, select each report that you want to include. By default, all
reports are generated for a scheduled report task.

To edit the parameters of a report:

Click the report label, and then click Edit Parameters.

In the Report Parameters dialog box, edit the parameters as needed, and
then click OK.

To see a list of parameters for all the selected reports, click Review Selected
Reports. Then click Close.

5. To specify the formats for saving the reports:

Under Report formats, select one or more formats for the scheduled reports.
By default, reports are generated in Dynamic HTML (DHTML). You can also
select HTML, XML, CSV, and text formats. The reports are saved to the default
location for scheduled reports.

6. To deliver copies of the reports to administrators by e-mail:

On the Delivery tab, select the Send reports to the following administrators
check box, and then enter the names of the administrative accounts that will
receive reports.
Use the format account@domain, and use semicolons to separate multiple
accounts.

7. To schedule the reports:

On the Schedule tab, click Create Schedule, and then in the Schedule dialog box,
click New. This displays a default schedule set for 9:00 A.M. daily, but you can
modify the default schedule.

To specify a frequency for generating the reports, select an interval from the
Schedule Task drop-down list. You can schedule daily, weekly, or monthly
reports, or generate the reports only once. You can also generate reports at
system startup or logon, or when the computer has been idle for a specified
time.
To provide additional scheduling information for the chosen interval, modify
or set the values in the Schedule Task options. These options change based
on the interval that you choose. For example, for a weekly report, you can
 specify the number of weeks between reports and on which days of the week
to generate reports.
To specify the time of day when you want to generate the report, type or
select the value in the Start time box.
To access additional scheduling options (including a start date and end date
for the task), click Advanced.
To save the schedule, click OK.
To create an additional schedule for a task (or modify an existing schedule),
on the Schedule tab, click Edit Schedule.

8. To save the report task, click OK.

The report task is added to the Storage Reports Management node. Tasks are identified
by the reports to be generated, the namespace to be reported on, and the report
schedule.

In addition, you can view the current status of the report (whether or not the report is
running), the last run time and the result of that run, and the next scheduled run time.

Additional References
Storage Reports Management
Setting File Server Resource Manager Options
Generate reports on demand
Article • 05/15/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

You can generate reports on demand using the File Server Resource Manager (FSRM).
With these reports, you can analyze the different aspects of current disk usage on the
server. Current data is gathered before the reports are generated.

When you generate reports on demand, the reports are saved in a default location
unless specified in FSRM > Action > Configure Options > Report Locations. No report
task is created for later use. You can view the reports immediately after they're
generated or e-mail the reports to a group of administrators.

７ Note

If you choose to open the reports immediately, you must wait while the reports are
generated. Processing time varies depending on the types of reports and the scope
of the data.

Prerequisites
The following must be installed to use this feature:

A Windows Server with the File and Storage Services role service installed. To learn
more, see Install or Uninstall Roles, Role Services, or Features.
An account with Administrative privileges.

How to generate reports

1. Click Start > Windows Administrative Tools > select File Server Resource
Management.
Alternatively, click Start > type fsrm.msc > hit Enter .
2. Select Storage Reports Management, right-click Storage Reports Management,
then select Generate Reports Now (or select Generate Reports Now from the
Actions pane).
3. In the Storage Reports Task Properties dialog box, select the data you want to
include from the Report data selection box.
 4. Under Report formats, select one or more formats for your report. The default is
set to Dynamic HTML (DHTML). Other available formats are HTML, XML, CSV, and
Text. The default save location for the reports are stored in C:\StorageReports
unless changed.
5. Click the Scope tab > click Add > browse to the volume or folder that you want to
generate the reports from and click OK to add it to your report scope.

You can add as many volumes or folders as you want to include in the
reports.
To remove a volume or folder, select it from the list > click Remove.
You can also include the following data types by selecting them within the
Scope tab:
Application Files
Backup and Archival Files
Group Files
User Files

6. Click OK > select one of two available options to generate your report > then click
OK to generate your report.

Generate reports in the background

Wait for reports to be generated and then display them

The following animation demonstrates the steps to generate a report using the
Generate Reports Now feature.
Generate custom reports
To customize your reports, perform the following:

1. In the Storage Reports Task Properties dialog box, select the data you want to
include from the Report data selection box.

2. Click Edit Parameters, then in the Report Parameters dialog box, edit the
parameters as needed, then click OK.

 Tip

Each report data label has its own set of parameters. When customizing
reports, it's recommended to select one report label at a time in making
modifications.

To see a list of parameters set for all the selected reports, click Review
Selected Reports, then click Close when complete.

3. After customizing your report, select the report output type from Report format,
click OK, select how you want your report generated, and then click OK.
Deliver reports via email
To deliver copies of the reports to administrators by e-mail:

1. In the Storage Reports Task Properties dialog box, complete your selection of the
report data you want to generate along with the Report format of choice.

2. Click the Delivery tab, select the Send reports to the following administrators
check box, then enter the names of the administrative accounts that receive
reports in account@domain formatting using the semicolon (;) to indicate multiple
accounts.

3. Click OK, select how you want your report generated, and then click OK.

７ Note

In order to use this feature, an SMTP server must be configured to prevent e-

mail delivery failure.

Additional references
Storage Reports Management
Setting File Server Resource Manager Options
Classification Management
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

Classification properties are used to categorize files and can be used to select files for
scheduled file management tasks.

There are many ways to classify a file. One way is to create a classification property that
assigns a value to all files within a specified directory. Another way is to create rules to
decide what value to set for a given property.

This section includes the following topics:

Create a Classification Property

Create an Automatic Classification Rule

７ Note

To set e-mail notifications and certain reporting capabilities, you must first
configure the general File Server Resource Manager options.

Additional References
Setting File Server Resource Manager Options
Create an Automatic Classification Rule
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

The following procedure guides you through the process of creating a classification rule.
Each rule sets the value for a single property. By default, a rule is run only once and
ignores files that already have a property value assigned. However, you can configure a
rule to evaluate files regardless of whether a value is already assigned to the property.

To create a Classification Rule

1. In Classification Management, click the Classification Rules node.

2. Right-click Classification Rules, and then click Create a New Rule (or click Create a
New Rule in the Actions pane). This opens the Classification Rule Definitions
dialog box.

3. On the Rule Settings tab, enter the following information:

Rule name. Type a name for the rule.

Enabled. This rule will only be applied if the Enabled check box is selected. To
disable the rule, clear this box.
Description. Type an optional description for this rule.
Scope. Click Add to select a location where this rule will apply. You can add
multiple locations, or remove a location by clicking Remove. The classification
rule will apply to all folders and their subfolders in this list.

4. On the Classification tab, enter the following information:

Classification mechanism. Choose a method for assigning the property value.

Property name. Select the property that this rule will assign.
Property value. Select the property value that this rule will assign.

5. Optionally, click the Advanced button to select further options. On the Evaluation
Type tab, the check box to Re-evaluate files is unchecked by default. The options
that can be selected here are as follows:

Re-evaluate files unchecked: A rule is applied to a file if, and only if, the
property specified by the rule has not been set to any value on the file.
 Re-evaluate files checked and the Overwrite the existing value option
selected: the rule will be applied to the files every time the automatic
classification process runs. For example, if a file has a Boolean property that is
set to Yes, a rule using the folder classifier to set all files to No with this
option set will leave the property set to No.
Re-evaluate files checked and the Aggregate the values option selected: The
rule will be applied to the files every time the automatic classification process
runs. However, when the rule has decided what value to set the property file
to, it aggregates that value with the one already in the file. For example, if a
file has a Boolean property that is set to Yes, a rule using the folder classifier
to set all files to No with this option set will leave the property set to Yes.

On the Additional Classification Parameters tab, you can specify additional

parameters that are recognized by the selected classification method by entering
the name and value and clicking the Insert button.

6. Click OK or Cancel to close the Advanced dialog box.

7. Click OK.

Additional References
Create a Classification Property
Classification Management
Create a Classification Property
Article • 05/03/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

Classification properties are used to assign values to files within a specified folder or
volume. There are many property types that you can choose from, depending on your
needs. The following table defines the available property types.

Property Description

Yes/No A Boolean property that can be Yes or No. When combining multiple values during
classification or from file content, a No value is overridden by a Yes value.

Date- A simple date/time property. When combining multiple values during classification or
time from file content, conflicting values prevent reclassification.

Number A simple number property. When combining multiple values during classification or
from file content, conflicting values prevent reclassification.

Ordered A list of fixed values. Only one value can be assigned to a property at a time. When
List combining multiple values during classification or from file content, the value highest
in the list is used.

String A simple string property. When combining multiple values during classification or
from file content conflicting values prevent reclassification.

Multi- A list of values that can be assigned to a property. More than one value can be
choice assigned to a property at a time. When combining multiple values during
classification or from file content, each value in the list is used.

Multi- A list of strings that can be assigned to a property. More than one value can be
string assigned to a property at a time. When combining multiple values during
classification or from file content, each value in the list is used.

To create a Classification property

The following procedure guides you through the process of creating a classification
property.

1. In Classification Management, select the Classification Properties node.

2. Right-select Classification Properties, and then select Create property (or select
Create property in the Actions pane). This opens the Classification Property
Definitions dialog box.
 3. In the Property Name text box, type a name for the property.

4. In the Description text box, add an optional description for the property.

5. In the Property Type drop-down menu, select a property type from the list.

6. Select OK.

Related links
Create an Automatic Classification Rule
Classification Management
File Management Tasks
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

File management tasks automate the process of finding subsets of files on a server and
applying simple commands. These tasks can be scheduled to occur periodically to
reduce repetitive costs. Files that will be processed by a file management task can be
defined through any of the following properties:

Location
Classification properties
Creation time
Modification time
Last accessed time

File management tasks can also be configured to notify file owners of any impending
policy that will be applied to their files.

７ Note

Individual File Management tasks are run on independent schedules.

This section includes the following topics:

Create a File Expiration Task

Create a Custom File Management Task

７ Note

To set e-mail notifications and certain reporting capabilities, you must first
configure the general File Server Resource Manager options.

Additional References
Setting File Server Resource Manager Options
Create a File Expiration Task
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

The following procedure guides you through the process of creating a file management
task for expiring files. File expiration tasks are used to automatically move all files that
match certain criteria to a specified expiration directory, where an administrator can
then back those files up and delete them.

When a file expiration task is run, a new directory is created within the expiration
directory, grouped by the server name on which the task was run.

The new directory name is based on the name of the file management task and the time
it was run. When an expired file is found it is moved into the new directory, while
preserving its original directory structure.

To create a file expiration task

1. Click the File Management Tasks node.

2. Right-click File Management Tasks, and then click Create File Management Task
(or click Create File Management Task in the Actions pane). This opens the Create
File Management Task dialog box.

3. On the General tab, enter the following information:

Name. Enter a name for the new task.

Description. Enter an optional descriptive label for this task.

Scope. Add the directories that this task should operate on by using the Add
button. Optionally, directories can be removed from the list using the
Remove button. The file management task will apply to all folders and their
subfolders in this list.

4. On the Action tab, enter the following information:

Type. Select File Expiration from the drop-down box.

Expiration Directory. Select a directory where files will be expired to.

 ２ Warning

Do not select a directory that is within the scope of the task, as defined
in the previous step. Doing so could cause an iterative loop that could
lead to system instability and data loss.

5. Optionally, on the Notification tab, click Add to send e-mail notifications, log an
event, or run a command or script a specified minimum number of days before the
task performs an action on a file.

In the Number of days before task is executed to send notification combo

box, type or select a value to specify the minimum number of days prior to a
file being acted on that a notification will be sent.

７ Note

Notifications are sent only when a task is run. If the specified minimum
number of days to send a notification does not coincide with a
scheduled task, the notification will be sent on the day of the previous
scheduled task.

To configure e-mail notifications, click the E-mail Message tab and enter the
following information:

To notify administrators when a threshold is reached, select the Send e-

mail to the following administrators check box, and then enter the names
of the administrative accounts that will receive the notifications. Use the
format account@domain, and use semicolons to separate multiple
accounts.

To send e-mail to the person whose files are about to expire, select the
Send e-mail to the user whose files are about to expire check box.

To configure the message, edit the default subject line and message body
that are provided. The text that is in brackets inserts variable information
about the quota event that caused the notification. For example, the
[Source File Owner] variable inserts the name of the user whose file is
about to expire. To insert additional variables in the text, click Insert
Variable.

To attach a list of the files that are about to expire, click Attach to the e-
mail list of files on which action will be performed, and type or select a
 value for Maximum number of files in the list.

To configure additional headers (including From, Cc, Bcc, and Reply-to),

click Additional E-mail Headers.

To log an event, click the Event Log tab and select the Send warning to event
log check box, and then edit the default log entry.

To run a command or script, click the Command tab and select the Run this
command or script check box. Then type the command, or click Browse to
search for the location where the script is stored. You can also enter
command arguments, select a working directory for the command or script,
or modify the command security setting.

6. Optionally, use the Report tab to generate one or more logs or storage reports.

To generate logs, select the Generate log check box and then select one or
more available logs.

To generate reports, select the Generate a report check box and then select
one or more available report formats.

To create e-mail generated logs or storage reports, select the Send reports to
the following administrators check box and type one or more administrative
e-mail recipients using the format account@domain. Use a semicolon to
separate multiple addresses.

７ Note

The report is saved in the default location for incident reports, which you
can modify in the File Server Resource Manager Options dialog box.

7. Optionally, use the Condition tab to run this task only on files that match a defined
set of conditions. The following settings are available:

Property conditions. Click Add to create a new condition based on the file's
classification. This will open the Property Condition dialog box, which allows
you to select a property, an operator to perform on the property, and the
value to compare the property against. After clicking OK, you can then create
additional conditions, or edit or remove an existing condition.

Days since file was last modified. Click the check box and then enter a
number of days into the spin box. This will result in the file management task
 only being applied to files that have not been modified for more than the
specified number of days.

Days since file was last accessed. Click the check box and then enter a
number of days into the spin box. If the server is configured to track
timestamps for when files were last accessed, this will result in the file
management task only being applied to files that have not been accessed for
more than the specified number of days. If the server is not configured to
track accessed times, this condition will be ineffective.

Days since file was created. Click the check box and then enter a number of
days into the spin box. This will result in the task only being applied to files
that were created at least the specified number of days ago.

Effective starting. Set a date when this file management task should start
processing files. This option is useful for delaying the task until you have had
a chance to notify users or make other preparations in advance.

8. On the Schedule tab, click Create Schedule, and then in the Schedule dialog box,
click New. This displays a default schedule set for 9:00 A.M. daily, but you can
modify the default schedule. When you have finished configuring the schedule,
click OK and then click OK again.

Additional References
Classification Management
File Management Tasks
Create a Custom File Management Task
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

Expiration is not always a desired action to be performed on files. File management

tasks allow you to run custom commands as well.

７ Note

This procedure assumes that you are familiar with file management tasks, and
therefore only covers the Action tab where custom settings are configured.

To create a custom task

1. Click the File Management Tasks node.

2. Right-click File Management Tasks, and then click Create File Management Task
(or click Create File Management Task in the Actions pane). This opens the Create
File Management Task dialog box.

3. On the Action tab, enter the following information:

Type. Select Custom from the drop-down menu.

Executable. Type or browse to a command to run when the file management
task processes files. This executable must be set to be writable by
Administrators and System only. If any other users have write access to the
executable, it will not run correctly.
Command settings. To configure the arguments passed to the executable
when a file management job processes files, edit the text box labeled
Arguments. To insert additional variables in the text, place the cursor in the
location in the text box where you want to insert the variable, select the
variable that you want to insert, and then click Insert Variable. The text that is
in brackets inserts variable information that the executable can receive. For
example, the [Source File Path] variable inserts the name of the file that
should be processed by the executable. Optionally, click the Working
directory button to specify the location of the custom executable.
 Command Security. Configure the security settings for this executable. By
default, the command is run as Local Service, which is the most restrictive
account available.

4. Click OK.

Additional References
Classification Management
File Management Tasks
Managing Remote Storage Resources
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

To manage storage resources on a remote computer, you have two options:

Connect to the remote computer from the File Server Resource Manager
Microsoft® Management Console (MMC) snap-in (which you can then use to
manage the remote resources).
Use the command-line tools that are installed with File Server Resource Manager.

Either option allows you to work with quotas, screen files, manage classifications,
schedule file management tasks, and generate reports with those remote resources.

７ Note

File Server Resource Manager can manage resources on either the local computer
or a remote computer, but not both at the same time.

For example, you can:

Connect to another computer in the domain using the File Server Resource
Manager MMC snap-in and review storage utilization on a volume or folder
located on the remote computer.
Create quota and file screen templates on a local server and then use the
command-line tools to import those templates into a file server located in a
branch office.

This section includes the following topics:

Connect to a Remote Computer

Command-Line Tools
Connect to a Remote Computer
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

To manage storage resources on a remote computer, you can connect to the computer
from File Server Resource Manager. While you are connected, File Server Resource
Manager allows you to manage quotas, screen files, manage classifications, schedule file
management tasks, and generate reports with those remote resources.

７ Note

File Server Resource Manager can manage resources on either the local computer
or a remote computer, but not both at the same time.

To connect to a remote computer from File

Server Resource Manager
1. In Administrative Tools, click File Server Resource Manager.

2. In the console tree, right-click File Server Resource Manager, and then click
Connect to Another Computer.

3. In the Connect to Another Computer dialog box, click Another computer. Then
type the name of the server you want to connect to (or click Browse to search for a
remote computer).

4. Click OK.

） Important

The Connect to Another Computer command is available only when you open File
Server Resource Manager from Administrative Tools. When you access File Server
Resource Manager from Server Manager, the command is not available.

Additional considerations
To manage remote resources with File Server Resource Manager:

You must be logged on to the local computer with a domain account that is a
member of the Administrators group on the remote computer.
The remote computer must be running Windows Server, and File Server Resource
Manager must be installed.
The Remote File Server Resource Manager Management exception on the remote
computer must be enabled. Enable this exception by using Windows Firewall in
Control Panel.

Additional References
Managing Remote Storage Resources
File Server Resource Manager
command-line tools
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

File Server Resource Manager installs the FileServerResourceManager PowerShell

cmdlets as well as the following command-line tools:

Dirquota.exe. Use to create and manage quotas, auto apply quotas, and quota
templates.
Filescrn.exe. Use to create and manage file screens, file screen templates, file
screen exceptions, and file groups.
Storrept.exe. Use to configure report parameters and generate storage reports on
demand. Also use to create report tasks, which you can schedule by using
schtasks.exe.

You can use these tools to manage storage resources on a local computer or a remote
computer. For more information about these command-line tools, see the following
references:

Dirquota: https://go.microsoft.com/fwlink/?LinkId=92741
Filescrn: https://go.microsoft.com/fwlink/?LinkId=92742
Storrept: https://go.microsoft.com/fwlink/?LinkId=92743

７ Note

To see command syntax and the available parameters for a command, run the
command with the /? parameter.

Remote management using the command-line

tools
Each tool has several options for performing actions similar to those that are available in
the File Server Resource Manager MMC snap-in. To have a command perform an action
on a remote computer instead of the local computer, use the /remote:ComputerName
parameter.
For example,Dirquota.exe includes a template export parameter to write quota
template settings to an XML file, and a template import parameter to import template
settings from the XML file. Adding the /remote:ComputerName parameter to the
Dirquota.exe template import command will import the templates from the XML file on
the local computer to the remote computer.

７ Note

When you run the command-line tools with the /remote:ComputerName parameter
to perform a template export (or import) on a remote computer, the templates are
written to (or copied from) an XML file on the local computer.

Additional considerations
To manage remote resources with the command-line tools:

You must be logged on with a domain account that is a member of the

Administrators group on the local computer and on the remote computer.
You must run the command-line tools from an elevated Command Prompt
window. To open an elevated Command Prompt window, click Start, point to All
Programs, click Accessories, right-click Command Prompt, and then click Run as
administrator.
The remote computer must be running Windows Server, and File Server Resource
Manager must be installed.
The Remote File Server Resource Manager Management exception on the remote
computer must be enabled. Enable this exception by using Windows Firewall in
Control Panel.

Additional References
Managing Remote Storage Resources
Troubleshooting File Server Resource
Manager
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2

This section lists common issues that you might encounter when using File Server
Resource Manager.

７ Note

A good troubleshooting option is to look for event logs that have been generated
by File Server Resource Manager. All event log entries for File Server Resource
Manager can be found in theApplication event log under the source SRMSVC

I am not receiving e-mail notifications.

Cause: The e-mail options have not been configured or have been configured
incorrectly.

Solution: On the E-mail Notifications tab, in the File Server Resource Manager
Options dialog box, verify that the SMTP server and default e-mail recipients have
been specified and are valid. Send a test e-mail to confirm that the information is
correct and that the SMTP server that is used to send the notifications is working
properly. For more information see Configure E-Mail Notifications.

I am only receiving one e-mail notification,

even though the event that triggered that
notification happened several times in a row.
Cause: When a user attempts several times to save a file that is blocked or to save
a file that exceeds a quota threshold, and there is an e-mail notification configured
for that file screening or quota event, only one e-mail is sent to the administrator
over a 60-minute period by default. This prevents an abundance of redundant
messages in the administrator's e-mail account.
 Solution: On the Notification Limits tab, in the File Server Resource Manager
Options dialog box, you can set a time limit for each of the notification types: e-
mail, event log, command, and report. Each limit specifies a time period that must
pass before another notification of the same type is generated for the same issue.
For more information see Configure Notification Limits.

My storage reports keep failing and little or no

information is available in the Event Log
regarding the source of the failure.
Cause: The volume where the reports are being saved may be corrupted.
Solution: Run chkdsk on the volume and try generating the reports again.

My File Screening Audit reports do not contain

any information.
Cause: One or more of the following may be the cause:
The auditing database is not configured to record file screening activity.
The auditing database is empty (that is, no file screening activity has been
recorded).
The parameters for the File Screening Audit report are not selecting data from
the auditing database.

Solution: On the File Screen Audit tab, in the File Server Resource Manager
Options dialog box, verify that the Record file screening activity in the auditing
database check box is selected.
For more information about recording file screening activity, see Configure File
Screen Audit.
To configure default parameters for the File Screening Audit report, see
Configure Storage Reports.
To edit report parameters for scheduled report tasks or on-demand reports, see
Storage Reports Management.

The "Used" and "Available" values for some of

the quotas I have created do not correspond to
the actual "Limit" setting.
Cause: You may have a nested quota, where the quota of a subfolder derives a
more restrictive limit from the quota of its parent folder. For example, you might
have a quota limit of 100 megabytes (MB) applied to a parent folder and a quota
of 200 MB applied separately to each of its subfolders. If the parent folder has a
total of 50 MB of data stored in it (the sum of the data stored in its subfolders),
each of the subfolders will list only 50 MB of available space.

Solution: Under Quota Management, click Quotas. In the Results pane, select the
quota entry that you are troubleshooting. In the Actions pane, click View Quotas
Affecting Folder, and then look for quotas that are applied to the parent folders.
This will allow you to identify which parent folder quotas have a lower storage limit
setting than the quota you have selected.
Folder Redirection, Offline Files, and
Roaming User Profiles overview
Article • 03/22/2023

Applies to: Windows 11, Windows 10, Windows Server 2022, Windows Server 2019,
Windows Server 2016

This topic discusses the Folder Redirection, Offline Files (client-side caching or CSC), and
Roaming User Profiles (sometimes known as RUP) technologies, including what's new
and where to find more information.

Technology description
Folder Redirection and Offline Files are used together to redirect the path of local
folders, such as the Documents folder, to a network location, while caching the contents
locally for increased speed and availability. Roaming User Profiles is used to redirect a
user profile to a network location. These features used to be referred to as Intellimirror.

Folder Redirection enables users and administrators to redirect the path of a

known folder to a new location, manually or by using Group Policy. The new
location can be a folder on the local computer or a directory on a file share. Users
interact with files in the redirected folder as if it still existed on the local drive. For
example, you can redirect the Documents folder, which is usually stored on a local
drive, to a network location. The files in the folder are then available to the user
from any computer on the network.

Offline Files makes network files available to a user, even if the network
connection to the server is unavailable or slow. When working online, file access
performance is at the speed of the network and server. When working offline, files
are retrieved from the Offline Files folder at local access speeds. A computer
switches to Offline mode in the following situations:
Always Offline mode has been enabled.
The server is unavailable.
The network connection is slower than a configurable threshold.
The user manually switches to Offline mode by using the Work offline button in
Windows Explorer.

Roaming User Profiles redirects user profiles to a file share so that users receive
the same operating system and application settings on multiple computers. When
 a user signs in to a computer by using an account that's set up with a file share as
the profile path, the user's profile loads to the local computer and merges with the
local profile, if present. When the user signs out of the computer, the local copy of
their profile, including any changes, merges with the server copy of the profile.
Typically, a network administrator enables Roaming User Profiles on domain
accounts.

Practical applications
Administrators can use Folder Redirection, Offline Files, and Roaming User Profiles to
centralize storage for user data and settings. These features let users access their data
while offline or in the event of a network or server outage. These features can also
accomplish these specific applications:

Centralize data from client computers for administrative tasks, such as using a
server-based backup tool to back up user folders and settings.
Enable users to continue accessing network files, even if there is a network or
server outage.
Optimize bandwidth usage and enhance the experience of users in branch offices
who access files and folders that are hosted by corporate servers located offsite.
Enable mobile users to access network files while working offline or over slow
networks.

New and changed functionality

The following table describes some of the major changes in Folder Redirection, Offline
Files, and Roaming User Profiles that are available in this release.

Feature/functionality New or Description

updated?

Always Offline mode New Provides faster access to files and lower bandwidth usage
by always working offline, even when connected through a
high-speed network connection.

Cost-aware New Helps users avoid high data usage costs from
synchronization synchronization while using metered connections that have
usage limits, or while roaming on another provider's
network.

Primary Computer New Enables you to limit the use of Folder Redirection, Roaming
support User Profiles, or both to only a user's primary computers.
Always Offline mode
Starting with Windows 8 and Windows Server 2012, administrators can configure the
experience for users of Offline Files to always work offline, even when they are
connected through a high-speed network connection. Windows updates files in the
Offline Files cache by synchronizing hourly in the background, by default.

What value does Always Offline mode add?

The Always Offline mode provides the following benefits:

Users experience faster access to files in redirected folders, such as the Documents
folder.
Network bandwidth is reduced, decreasing costs on expensive WAN connections
or metered connections such as a 4G mobile network.

How has Always Offline mode changed things?

Prior to Windows 8, Windows Server 2012, users would transition between the Online
and Offline modes, depending on network availability and conditions, even when the
Slow-Link mode (also known as the Slow Connection mode) was enabled and set to a
1 millisecond latency threshold.

With Always Offline mode, computers never transition to Online mode when the
Configure slow-link mode Group Policy setting is configured and the Latency threshold
parameter is set to 1 millisecond. Changes are synced in the background every 120
minutes, by default, but synchronization is configurable by using the Configure
Background Sync Group Policy setting.

For more information, see Enable Always Offline mode for faster access to files.

Cost-aware synchronization
With cost-aware synchronization, Windows disables background synchronization when
the user is using a metered network connection, such as a 4G mobile network, and the
subscriber is near or over their bandwidth limit, or roaming on another provider's
network.

７ Note
 Metered network connections usually have round-trip network latencies that are
slower than the default 35 millisecond latency value for transitioning to Offline
(Slow Connection) mode in Windows 8, Windows Server 2019, Windows Server
2016, and Windows Server 2012. Therefore, these connections usually transition to
Offline (Slow Connection) mode automatically.

What value does cost-aware synchronization add?

Cost-aware synchronization helps users avoid unexpectedly high data usage costs while
using metered connections that have usage limits, or while roaming on another
provider's network.

How has cost-aware synchronization changed things?

Prior to Windows 8 and Windows Server 2012, users who wanted to minimize fees while
using Offline Files on metered network connections had to track their data usage by
using tools from the mobile network provider. The users could then manually switch to
Offline mode when they were roaming, near their bandwidth limit, or over their limit.

With cost-aware sync, Windows automatically tracks roaming and bandwidth usage
limits while on metered connections. When the user is roaming, near their bandwidth
limit, or over their limit, Windows switches to Offline mode and prevents all
synchronization. Users can still manually initiate synchronization, and administrators can
override cost-aware synchronization for specific users, such as executives.

For more information, see Enable Background File Synchronization on metered

networks.

Primary computers for Folder Redirection and

Roaming User Profiles
You can now designate a set of computers, known as primary computers, for each
domain user, which enables you to control which computers use Folder Redirection,
Roaming User Profiles, or both. Designating primary computers is a simple and powerful
method to associate user data and settings with particular computers or devices,
simplify administrator oversight, improve data security, and help protect user profiles
from corruption.

What value do primary computers add?

There are four major benefits to designating primary computers for users:

The administrator can specify which computers users can use to access their
redirected data and settings. For example, the administrator can choose to roam
user data and settings between a user's desktop and laptop, and to not roam the
information when that user logs on to any other computer, such as a conference
room computer.
Designating primary computers reduces the security and privacy risk of leaving
residual personal or corporate data on computers where the user has logged on.
For example, a general manager who logs on to an employee's computer for
temporary access does not leave behind any personal or corporate data.
Primary computers enable the administrator to mitigate the risk of an improperly
configured or otherwise corrupt profile, which could result from roaming between
differently configured systems, such as between x86-based and x64-based
computers.
The amount of time required for a user's first sign-in on a non-primary computer,
such as a server, is faster because the user's roaming user profile and/or redirected
folders are not downloaded. Sign-out times are also reduced, because changes to
the user profile do not need to be uploaded to the file share.

How have primary computers changed things?

To limit downloading private user data to primary computers, the Folder Redirection and
Roaming User Profiles technologies perform the following logic checks when a user
signs in to a computer:

1. The Windows operating system checks the new Group Policy settings (Download
roaming profiles on primary computers only and Redirect folders on primary
computers only) to determine if the msDS-Primary-Computer attribute in Active
Directory Domain Services (AD DS) should influence the decision to roam the
user's profile or apply Folder Redirection.

2. If the policy setting enables primary computer support, Windows verifies that the
AD DS schema supports the msDS-Primary-Computer attribute. If it does,
Windows determines if the computer that the user is logging on to is designated
as a primary computer for the user as follows:
a. If the computer is one of the user's primary computers, Windows applies the
Roaming User Profiles and Folder Redirection settings.
b. If the computer isn't one of the user's primary computers, Windows loads the
user's cached local profile, if present, or it creates a new local profile. Windows
also removes any existing redirected folders according to the removal action
 that was specified by the previously applied Group Policy setting, which is
retained in the local Folder Redirection configuration.

For more information, see Deploy primary computers for Folder Redirection and
Roaming User Profiles

Hardware requirements
Folder Redirection, Offline Files, and Roaming User Profiles require an x64-based or x86-
based computer, and they are not supported by Windows on ARM (WOA)-based
computers.

Software requirements
To designate primary computers, your environment must meet the following
requirements:

The Active Directory Domain Services (AD DS) schema must be updated to include
Windows Server 2012 schema and conditions. Installing a Windows Server 2012 or
later domain controller automatically updates the schema. For more information
about upgrading the AD DS schema, see Upgrade domain controllers to a newer
version of Windows Server.
Client computers must run Windows 11, Windows 10, Windows Server 2022,
Windows Server 2019, or Windows Server 2016 and be joined to the Active
Directory domain that you're managing.

Related links
Deploy Folder Redirection with offline files
Deploy Roaming User Profiles
Deploy primary computers for Folder Redirection and Roaming User Profiles
Disable Offline Files on individual redirected folders
Enable Always Offline mode for faster access to files
Enable optimized moves of redirected folders
Windows Server Storage Forum
Deploy roaming user profiles
Article • 03/29/2023

Applies To: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, Windows
11, Windows 10, Windows 8.1, Windows 8, Windows 7

This article describes how to use Windows Server to deploy roaming user profiles to
Windows client computers. A roaming user profile redirects user profiles to a file share
so that users receive the same operating system and application settings on multiple
computers.

For a list of recent changes to this article, see the Change history section.

） Important

User customizations to the Start menu are lost after an OS in-place upgrade in the
following configuration:

Users are configured for a roaming profile

Users are allowed to make changes to Start

As a result, the Start menu is reset to the default of the new OS version after the OS
in-place upgrade. For workarounds, see Appendix C: Work around reset Start
menu layouts after upgrades.

Prerequisites

Hardware requirements
An x64-based or x86-based computer is required. Windows RT doesn't support roaming
user profiles.

Software requirements
If you deploy roaming user profiles with folder redirection in an environment with
existing local user profiles, deploy folder redirection before roaming user profiles
to minimize the size of roaming profiles. After the existing user folders have been
successfully redirected, you can deploy roaming user profiles.
 To administer roaming user profiles, you must be signed in as a member of the
Domain Administrators security group, Enterprise Administrators security group, or
Group Policy Creator Owners security group.
Client computers must run Windows 10, Windows 8.1, Windows 8, Windows 7,
Windows Vista, Windows Server 2012 R2, Windows Server 2012, Windows Server
2008 R2, or Windows Server 2008.
Client computers must be joined to the Active Directory Domain Services (AD DS)
that you're managing.
A computer must be available with Group Policy Management and Active Directory
Administration Center installed.
A file server must be available to host roaming user profiles.
If the file share uses DFS Namespaces, the DFS folders (links) must have a single
target to prevent users from making conflicting edits on different servers.
If the file share uses DFS Replication to replicate the contents with another
server, users must be able to access only the source server to prevent users
from making conflicting edits on different servers.
If the file share is clustered, disable continuous availability on the file share to
avoid performance issues.
To use primary computer support in roaming user profiles, there are other client
computer and Active Directory schema requirements. For more information, see
Deploy primary computers for folder redirection and roaming user profiles.
The layout of a user's Start menu doesn't roam on Windows 10, Windows Server
2019, or Windows Server 2016 if they're using more than one PC, Remote Desktop
Session Host, or Virtualized Desktop Infrastructure (VDI) server. As a workaround,
you can specify a Start layout as described in this article. Or you can make use of
user profile disks, which properly roam Start menu settings when used with
Remote Desktop Session Host servers or VDI servers. For more info, see Easier user
data management with user profile disks in Windows Server 2012 .

Considerations when using multiple versions of Windows

To use roaming user profiles across multiple versions of Windows, you should take the
following actions:

Configure Windows to maintain separate profile versions for each operating

system version. This helps prevent undesirable and unpredictable issues such as
profile corruption.
Use folder redirection to store user files such as documents and pictures outside of
user profiles. This enables the same files to be available to users across operating
system versions. It also keeps profiles small and sign-ins quick.
 Allocate sufficient storage for roaming user profiles. If you support two operating
system versions, profiles double in number, and thus total space consumed,
because a separate profile is maintained for each operating system version.
Don't use roaming user profiles across computers running Windows
Vista/Windows Server 2008 and Windows 7/Windows Server 2008 R2. Roaming
between these operating system versions isn't supported due to incompatibilities
in their profile versions.
Inform your users that changes made on one operating system version don't roam
to another operating system version.
When moving your environment to a version of Windows that uses a different
profile version (such as from Windows 10 to Windows 10 version 1607, see
Appendix B: Profile version reference information for a list), users receive a new,
empty roaming user profile. You can minimize the effect of getting a new profile by
using folder redirection to redirect common folders. There isn't a supported
method of migrating roaming user profiles from one profile version to another.

Step 1: Enable the use of separate profile

versions
If you deploy roaming user profiles on computers that run Windows 8.1, Windows 8,
Windows Server 2012 R2, or Windows Server 2012, you should first make a couple of
changes to your Windows environment. These changes help ensure that future
operating system upgrades go smoothly, and facilitate the ability to simultaneously run
multiple versions of Windows with roaming user profiles.

To make these changes, use the following procedure.

1. Download and install the appropriate software update on all computers on which
you're going to use roaming, mandatory, super-mandatory, or domain default
profiles:

Windows 8.1, or Windows Server 2012 R2: Install the software update
described in article 2887595 in the Microsoft Knowledge Base, when
released.
Windows 8 or Windows Server 2012: Install the software update described in
article 2887239 in the Microsoft Knowledge Base.

2. On all computers that run Windows 8.1, Windows 8, Windows Server 2012 R2, or
Windows Server 2012 on which you use roaming user profiles, use registry editor
or group policy to create the following registry key DWORD Value and set it to 1 .
 For information about creating registry keys by using group policy, see Configure a
registry item.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ProfSvc\Parameters
\UseProfilePathExtensionVersion

２ Warning

Incorrect editing of the registry may severely damage your system. Before
making changes to the registry, you should back up any valued data on the
computer.

3. Restart the computers.

Step 2: Create a roaming user profiles security

group
If your environment isn't already set up with roaming user profiles, the first step is to
create a security group that contains all users and/or computers to which you want to
apply roaming user profiles policy settings.

Administrators of general-purpose roaming user profiles deployments typically

create a security group for users.
Administrators of Remote Desktop Services or virtualized desktop deployments
typically use a security group for users and the shared computers.

Here's how to create a security group for roaming user profiles:

1. Open Server Manager on a computer with Active Directory Administration Center

installed.

2. On the Tools menu, select Active Directory Administration Center. Active

Directory Administration Center appears.

3. Right-click the appropriate domain or OU, select New, and then select Group.

4. In the Create Group window, in the Group section, specify the following settings:

In Group name, type the name of the security group, for example: Roaming
user profiles users and computers.
 In Group type, select Security, and then select Global in Group scope.

5. In the Members section, select Add. The Select Users, Contacts, Computers,
Service Accounts or Groups dialog box appears.

6. If you want to include computer accounts in the security group, select Object
Types, select the Computers check box and then select OK.

7. Type the names of the users, groups, and/or computers to which you want to
deploy roaming user profiles, select OK, and then select OK again.

Step 3: Create a file share for roaming user

profiles
If you don't already have a separate file share for roaming user profiles, independent
from any shares for redirected folders to prevent inadvertent caching of the roaming
profile folder, use the following procedure to create a file share on a server running
Windows Server.

７ Note

Some functionality might differ or be unavailable depending on the version of

Windows Server that you use.

Here's how to create a file share on Windows Server:

1. In the Server Manager navigation pane, select File and Storage Services, and then
select Shares to display the Shares page.

2. In the Shares tile, select Tasks, and then select New Share. The New Share Wizard
appears.

3. On the Select Profile page, select SMB Share – Quick. If you have File Server
Resource Manager installed and are using folder management properties, instead
select SMB Share - Advanced.

4. On the Share Location page, select the server and volume on which you want to
create the share.

5. On the Share Name page, type a name for the share (for example, User Profiles$)
in the Share name box.
  Tip

When creating the file share, hide it from casual browsers by putting a $ after
the share name.

6. On the Other Settings page, clear the Enable continuous availability checkbox, if
present, and optionally select the Enable access-based enumeration and Encrypt
data access checkboxes.

7. On the Permissions page, select Customize permissions. The Advanced Security

Settings dialog box appears.

8. Select Disable inheritance, and then select Convert inherited permissions into
explicit permission on this object.

9. Set the permissions as described in Required permissions for roaming user profiles
and shown in the following screenshot. Remove permissions for unlisted groups
and accounts, and add special permissions to the Roaming user profiles users and
computers group that you created in Step 2.

10. If you chose the SMB Share - Advanced profile, on the Management Properties
page, select the User Files folder usage value.
 11. If you chose the SMB Share - Advanced profile, on the Quota page, optionally
select a quota to apply to users of the share.

12. On the Confirmation page, select Create.

Required permissions for roaming user profiles

The following table lists the required file share hosting permissions for roaming user
profiles.

User account Access Applies to

System Full control This folder,

subfolders and
files

Administrators Full control This folder only

Creator/Owner Full control Subfolders and

files only

Security group of users needing to put data on List folder / read data This folder only
share (Roaming user profiles users and computers) (Advanced permissions)
Create folders / append
data (Advanced
permissions)

Other groups and accounts None (remove)

Step 4: Optionally create a GPO for roaming

user profiles
If you don't already have a Group Policy Object (GPO) created for roaming user profiles
settings, use the following procedure to create an empty GPO. This GPO allows you to
configure settings such as primary computer support, which is discussed separately, and
can also be used to enable roaming user profiles on computers, as is typically done
when deploying in virtualized desktop environments or with Remote Desktop Services.

Here's how to create a GPO for roaming user profiles:

1. Open Server Manager on a computer with Group Policy Management installed.

2. From the Tools menu, select Group Policy Management. Group Policy
Management appears.
 3. Right-click the domain or OU in which you want to set up roaming user profiles,
then select Create a GPO in this domain, and Link it here.

4. In the New GPO dialog box, type a name for the GPO (for example, Roaming user
profile settings), and then select OK.

5. Right-click the newly created GPO and then clear the Link Enabled checkbox. This
prevents the GPO from being applied until you finish configuring it.

6. Select the GPO. In the Security Filtering section of the Scope tab, select
Authenticated Users, and then select Remove to prevent the GPO from being
applied to everyone.

7. In the Security Filtering section, select Add.

8. In the Select User, Computer, or Group dialog box, type the name of the security
group you created in Step 2 (for example, Roaming user profiles users and
computers), and then select OK.

9. Select the Delegation tab, select Add, type Authenticated Users, select OK, and
then select OK again to accept the default Read permissions.

This step is necessary due to security changes made in MS16-072 .

Step 5: Optionally set up roaming user profiles

on user accounts
If you deploy roaming user profiles to user accounts, use the following procedure to
specify roaming user profiles for user accounts in Active Directory Domain Services. If
you deploy roaming user profiles to computers, as is typically done for Remote Desktop
Services or virtualized desktop deployments, instead use the procedure documented in
Step 6: Optionally set up roaming user profiles on computers.

７ Note

If you set up roaming user profiles on user accounts by using Active Directory and
on computers by using Group Policy, the computer-based policy setting takes
precedence.

Here's how to set up roaming user profiles on user accounts:

 1. In Active Directory Administration Center, navigate to the Users container (or OU)
in the appropriate domain.

2. Select all users to which you want to assign a roaming user profile, right-click the
users and then select Properties.

3. In the Profile section, select the Profile path: checkbox and then enter the path to
the file share where you want to store the user's roaming user profile, followed by
%username% (which is automatically replaced with the user name the first time the
user signs in). For example:

\\fs1.corp.contoso.com\User Profiles$\%username%

To specify a mandatory roaming user profile, specify the path to the NTuser.man
file that you created previously, for example, fs1.corp.contoso.comUser
Profiles$default . For more information, see Create mandatory user profiles.

4. Select OK.

７ Note

By default, deployment of all Windows Runtime-based (Windows Store) apps is

allowed when using roaming user profiles. However, when using a special profile,
apps are not deployed by default. Special profiles are user profiles where changes
are discarded after the user signs out:

To remove restrictions on app deployment for special profiles, enable the Allow
deployment operations in special profiles policy setting (located in Computer
Configuration\Policies\Administrative Templates\Windows Components\App Package
Deployment). However, deployed apps in this scenario leave some data stored on
the computer, which could accumulate, for example, if there are hundreds of users
of a single computer. To clean up apps, locate or develop a tool that uses the
CleanupPackageForUserAsync API to clean up app packages for users who no
longer have a profile on the computer.

For additional background information about Windows Store apps, see Manage
client access to the Windows Store.

Step 6: Optionally set up roaming user profiles

on computers
If you deploy roaming user profiles to computers, as is typically done for Remote
Desktop Services or virtualized desktop deployments, use the following procedure. If
you deploy roaming user profiles to user accounts, instead use the procedure described
in Step 5: Optionally set up roaming user profiles on user accounts.

７ Note

If you set up roaming user profiles on computers by using Group Policy and on
user accounts by using Active Directory, the computer-based policy setting takes
precedence.

Here's how to set up roaming user profiles on computers:

1. Open Server Manager on a computer with Group Policy Management installed.

2. From the Tools menu, select Group Policy Management. Group Policy
Management appears.

3. In Group Policy Management, right-click the GPO you created in Step 4 (for
example, Roaming User Profiles Settings), and then select Edit.

4. In the Group Policy Management Editor window, navigate to Computer

Configuration, then Policies, then Administrative Templates, then System, and
then User Profiles.

5. Right-click Set roaming profile path for all users logging onto this computer and
then select Edit.

 Tip

A user's home folder, if configured, is the default folder used by some

programs such as Windows PowerShell. You can configure an alternative local
or network location on a per-user basis by using the Home folder section of
the user account properties in AD DS. To configure the home folder location
for all users of a computer running Windows 8.1, Windows 8, Windows Server
2019, Windows Server 2016, Windows Server 2012 R2, or Windows Server
2012 in a virtual desktop environment, enable the Set user home folder policy
setting, and then specify the file share and drive letter to map (or specify a
local folder). Don't use environment variables or ellipses. The user's alias is
appended to the end of the path specified during user sign in.

6. In the Properties dialog box, select Enabled

 7. In the Users logging onto this computer should use this roaming profile path
box, enter the path to the file share where you want to store the user's roaming
user profile, followed by %username% (which is automatically replaced with the user
name the first time the user signs in). For example:

\\fs1.corp.contoso.com\User Profiles$\%username%

To specify a mandatory roaming user profile, which is a preconfigured profile to

which users can't make permanent changes (changes are reset when the user signs
out), specify the path to the NTuser.man file that you created previously, for
example, \\fs1.corp.contoso.com\User Profiles$\default . For more information,
see Creating a mandatory user profile.

8. Select OK.

Step 7: Optionally specify a Start layout for

Windows 10 PCs
You can use Group Policy to apply a specific Start menu layout so that users see the
same Start layout on all PCs. If users sign in to more than one PC and you want them to
have a consistent Start layout across PCs, make sure that the GPO applies to all of their
PCs.

To specify a Start layout, do the following:

1. Update your Windows 10 PCs to Windows 10 version 1607 (also known as the
Anniversary Update) or newer, and install the March 14, 2017 cumulative update
(KB4013429 ) or newer.

2. Create a full or partial Start menu layout XML file. To do so, see Customize and
export Start layout.

If you specify a full Start layout, a user can't customize any part of the Start
menu. If you specify a partial Start layout, users can customize everything but
the locked groups of tiles you specify. However, with a partial Start layout,
user customizations to the Start menu don't roam to other PCs.

3. Use Group Policy to apply the customized Start layout to the GPO you created for
roaming user profiles. To do so, see Use Group Policy to apply a customized Start
layout in a domain.

4. Use Group Policy to set the following registry value on your Windows 10 PCs. To
do so, see Configure a registry item.
 Action Update

Hive HKEY_LOCAL_MACHINE

Key path Software\Microsoft\Windows\CurrentVersion\Explorer

Value name SpecialRoamingOverrideAllowed

Value type REG_DWORD

Value data 1 (or 0 to disable)

Base Decimal

5. (Optional) Enable first-time sign-in optimizations to make signing in faster for

users. To do so, see Apply policies to improve sign-in time.

6. (Optional) Further decrease sign-in times by removing unnecessary apps from the
Windows 10 base image you use to deploy client PCs. Windows Server 2019 and
Windows Server 2016 don't have any pre-provisioned apps, so you can skip this
step on server images.

To remove apps, use the Remove-AppxProvisionedPackage cmdlet in

Windows PowerShell to uninstall the following applications. If your PCs are
already deployed, you can script the removal of these apps by using Remove-
AppxPackage.
Microsoft.windowscommunicationsapps_8wekyb3d8bbwe
Microsoft.BingWeather_8wekyb3d8bbwe
Microsoft.DesktopAppInstaller_8wekyb3d8bbwe
Microsoft.Getstarted_8wekyb3d8bbwe
Microsoft.Windows.Photos_8wekyb3d8bbwe
Microsoft.WindowsCamera_8wekyb3d8bbwe
Microsoft.WindowsFeedbackHub_8wekyb3d8bbwe
Microsoft.WindowsStore_8wekyb3d8bbwe
Microsoft.XboxApp_8wekyb3d8bbwe
Microsoft.XboxIdentityProvider_8wekyb3d8bbwe
Microsoft.ZuneMusic_8wekyb3d8bbwe

７ Note

Uninstalling these apps decreases sign-in times, but you can leave them installed if
your deployment needs any of them.
Step 8: Enable the roaming user profiles GPO
If you set up roaming user profiles on computers by using Group Policy, or if you
customized other roaming user profiles settings by using Group Policy, the next step is
to enable the GPO, permitting it to be applied to affected users.

 Tip

If you plan to implement primary computer support, do so before you enable the
GPO. This prevents user data from being copied to non-primary computers before
primary computer support is enabled. For the specific policy settings, see Deploy
primary computers for folder redirection and roaming user profiles.

Here's how to enable the roaming user profiles GPO:

1. Open Group Policy Management.

2. Right-click the GPO that you created and then select Link Enabled. A checkbox
appears next to the menu item.

Step 9: Test roaming user profiles

To test roaming user profiles, sign in to a computer with a user account configured for
roaming user profiles, or sign in to a computer configured for roaming user profiles.
Then confirm that the profile is redirected.

Here's how to test roaming user profiles:

1. Sign in to a primary computer (if you enabled primary computer support) with a
user account for which you've enabled roaming user profiles. If you enabled
roaming user profiles on specific computers, sign in to one of these computers.

2. If the user has previously signed in to the computer, open an elevated command
prompt, and then type the following command to ensure that the latest Group
Policy settings are applied to the client computer:

Windows Command Prompt

gpupdate /force

3. To confirm that the user profile is roaming, open Control Panel, select System and
Security, select System, select Advanced System Settings, select Settings in the
User Profiles section and then look for Roaming in the Type column.
Appendix A: Checklist for deploying roaming
user profiles
Status Action

1. Prepare domain
☐ - Join computers to domain
☐ - Enable the use of separate profile versions
☐ - Create user accounts
☐ - (Optional) Deploy folder redirection

2. Create security group for roaming user profiles

☐ - Group name
☐ - Members

3. Create a file share for roaming user profiles

☐ - File share name

4. Create a GPO for roaming user profiles

☐ - GPO name

☐ 5. Configure roaming user profiles policy settings

6. Enable roaming user profiles

☐ - Enabled in AD DS on user accounts?
☐ - Enabled in Group Policy on computer accounts?

☐ 7. (Optional) Specify a mandatory Start layout for Windows 10 PCs

8. (Optional) Enable primary computer support

☐ - Designate primary computers for users
☐ - Location of user and primary computer mappings
☐ - (Optional) Enable primary computer support for folder redirection
☐ - Computer-based or user-based?
☐ - (Optional) Enable primary computer support for roaming user profiles

☐ 9. Enable the roaming user profiles GPO

☐ 10. Test roaming user profiles

Appendix B: Profile version reference

information
Each profile has a profile version that corresponds roughly to the version of Windows on
which the profile is used. For example, Windows 10 version 1703 and version 1607 both
use the .V6 profile version. Microsoft creates a new profile version only when necessary
to maintain compatibility, which is why not every version of Windows includes a new
profile version.

The following table lists the location of roaming user profiles on various versions of
Windows.

Operating system version Roaming user profiles location

Windows XP and Windows \\<servername>\<fileshare>\<username>

Server 2003

Windows Vista and \\<servername>\<fileshare>\<username>.V2

Windows Server 2008

Windows 7 and Windows \\<servername>\<fileshare>\<username>.V2

Server 2008 R2

Windows 8 and Windows \\<servername>\<fileshare>\<username>.V3 (after the software

Server 2012 update and registry key are applied)
\\<servername>\<fileshare>\<username>.V2 (before the software
update and registry key are applied)

Windows 8.1 and Windows \\<servername>\<fileshare>\<username>.V4 (after the software

Server 2012 R2 update and registry key are applied)
\\<servername>\<fileshare>\<username>.V2 (before the software
update and registry key are applied)

Windows 10 \\<servername>\<fileshare>\<username>.V5

Windows 10, version 1703 \\<servername>\<fileshare>\<username>.V6

and version 1607

Appendix C: Work around reset Start menu

layouts after upgrades
Here are some ways to work around Start menu layouts getting reset after an in-place
upgrade:

If only one user ever uses the device and the IT Admin uses a managed OS
deployment strategy such as Configuration Manager, they can do the following:

1. Export the Start menu layout with Export-Startlayout before the upgrade.

2. Import the Start menu layout with Import-StartLayout after OOBE but before
the user signs in.
 ７ Note

Importing a StartLayout modifies the Default User profile. All user

profiles created after the import has occurred will get the imported
StartLayout.

IT Admins can opt to manage Start's layout with Group Policy. Using Group Policy
provides a centralized management solution to apply a standardized Start layout
to users. There are two modes to using Group Policy for Start management: full
lockdown and partial lockdown. The full lockdown scenario prevents the user from
making any changes to Start's layout. The partial lockdown scenario allows user to
make changes to a specific area of Start. For more info, see Customize and export
Start layout.

７ Note

User-made changes in the partial lockdown scenario are still lost during
upgrade.

Let the Start layout reset occur and allow end users to reconfigure Start. A
notification email or other notification can be sent to end users to expect their
Start layouts to be reset after the OS upgrade to minimized impact.

Change history
The following table summarizes some of the most important changes to this article.

Date Description Reason

May 1, Added updates for Windows Server 2019

2019

April 10, Added discussion of when user customizations to Callout known issue.
2018 Start are lost after an OS in-place upgrade

March 13, Updated for Windows Server 2016 Moved out of Previous Versions
2018 library and updated for current
version of Windows Server.
Date Description Reason

April 13, Added profile information for Windows 10, version Customer feedback.
2017 1703, and clarified how roaming profile versions
work when upgrading operating systems. See
Considerations when using multiple versions of
Windows.

March 14, Added optional step for specifying a mandatory Feature changes in latest
2017 Start layout for Windows 10 PCs in Appendix A: Windows update.
Checklist for deploying roaming user profiles.

January Added a step to Step 4: Optionally create a GPO Security changes to Group
23, 2017 for roaming user profiles to delegate Read Policy processing.
permissions to Authenticated Users, which is now
required because of a Group Policy security
update.

December Added a link in Step 8: Enable the roaming user Customer feedback.
29, 2016 profiles GPO to make it easier to get info on how
to set Group Policy for primary computers. Also
fixed a couple references to steps 5 and 6 that had
the numbers wrong.

December Added info explaining a Start menu settings Customer feedback.

5, 2016 roaming issue.

July 6, Added Windows 10 profile version suffixes in Updates for the new versions
2016 Appendix B: Profile version reference information. of Windows, and removed info
Also removed Windows XP and Windows Server about versions of Windows that
2003 from the list of supported operating systems. are no longer supported.

July 7, Added requirement and step to disable Clustered file shares have
2015 continuous availability when using a clustered file better performance for small
server. writes (which are typical with
roaming user profiles) when
continuous availability is
disabled.

March 19, Capitalized profile version suffixes (.V2, .V3, .V4) in Although Windows is case
2014 Appendix B: Profile version reference information. insensitive, if you use NFS with
the file share, it's important to
have the correct (uppercase)
capitalization for the profile
suffix.
Date Description Reason

October Revised for Windows Server 2012 R2 and Windows Updates for new version;
9, 2013 8.1, clarified a few things, and added the customer feedback.
Considerations when using roaming user profiles
on multiple versions of Windows and Appendix B:
Profile version reference information sections.

Related links
Deploy folder redirection with offline files
Deploy primary computers for folder redirection and roaming user profiles
Implementing user state management
Microsoft's support statement around replicated user profile data
Sideload apps with Deployment Image Servicing and Management (DISM)
Troubleshooting packaging, deployment, and query of Windows apps
Deploy Folder Redirection with Offline
Files
Article • 03/23/2023

Applies To: Windows 11, Windows 10, Windows Server 2022, Windows Server 2019,
Windows Server 2016

This article describes the requirements for deploying Folder Redirection and Offline Files
together, including the steps that you need to follow to control access to the redirected
files.

Prerequisites
Before you begin, make sure your environment meets the following requirements.

Administration requirements
To administer Folder Redirection, you must be signed in as a member of the
Domain Administrators security group, the Enterprise Administrators security
group, or the Group Policy Creator Owners security group.
A computer must be available that has Group Policy Management and Active
Directory Administration Center installed.

File server requirements

The file server is the computer that hosts the redirected folders. Make sure your file
server meets the following requirements.

Interoperability with Remote Desktop Services

Your remote access configuration affects how you configure the file server, file shares,
and policies. If your file server also hosts Remote Desktop Services, there are a few
deployment steps that differ.

You don't have to create a security group for folder redirection users.
You have to configure different permissions on the file share that hosts the
redirected folders.
 You have to precreate folders for new users, and set specific permissions on those
folders.

） Important

Most of the procedures in the rest of this section apply to both remote access
configurations. The procedures or steps that are specific to one configuration or
the other are labeled as such.

Restricting access
Apply the following changes to the file server, as appropriate for your configuration:

All configurations: Make sure only required IT administrators have administrative

access to the file server. The procedure in the next step configures access for the
individual file shares.
Servers that don't also host Remote Desktop Services: Disable the Remote
Desktop Services service ( termserv ) on your file server if it's not also hosting
Remote Desktop Services.

Interoperability with other storage features

To make sure Folder Redirection and Offline Files interact correctly with other storage
features, check the following configurations.

If the file share uses DFS Namespaces, the DFS folders (links) must have a single
target to prevent users from making conflicting edits on different servers.
If the file share uses DFS Replication to replicate the contents with another server,
users must be able to access only the source server to prevent users from making
conflicting edits on different servers.
When using a clustered file share, disable continuous availability on the file share
to avoid performance issues with Folder Redirection and Offline Files. When
continuous availability is enabled, Offline Files might not transition to offline mode
for 3-6 minutes after a user loses access to the file share. The delay could frustrate
users who aren’t yet using the Always Offline mode of Offline Files.

Client requirements
Client computers must run Windows 11, Windows 10, Windows Server 2022,
Windows Server 2019, or Windows Server 2016.
 Client computers must be joined to the Active Directory Domain Services (AD DS)
domain that you're managing.
Client computers must run x64-based or x86-based processors. Folder Redirection
isn't supported on PCs powered by ARM processors.

） Important

Some newer features in Folder Redirection have additional client computer and
Active Directory schema requirements. For more information, see Deploy primary
computers for Folder Redirection and Roaming User Profiles, Disable Offline Files
on individual redirected folders, Enable Always Offline mode for faster access to
files, and Enable optimized moves of redirected folders.

Step 1: Create a folder redirection security

group
If you're running Remote Desktop Services on the file server, skip this step. Instead,
assign permissions to the users when you precreate folders for new users.

This procedure creates a security group that contains all users to which you want to
apply Folder Redirection policy settings.

1. On a computer that has Active Directory Administration Center installed, open

Server Manager.

2. Select Tools > Active Directory Administration Center. Active Directory

Administration Center appears.

3. Right-click the appropriate domain or OU, and then select New > Group.

4. In the Create Group window, in the Group section, specify the following settings:

In Group name, enter the name of the security group, for example: Folder
Redirection Users.
In Group scope, select Security > Global.

5. In the Members section, select Add. The Select Users, Contacts, Computers,
Service Accounts or Groups dialog box appears.

6. Enter the names of the users or groups to which you want to deploy Folder
Redirection, select OK, and then select OK again.
Step 2: Create a file share for redirected folders
If you don't already have a file share for redirected folders, use the following procedure
to create a file share on a server that runs Windows Server 2016 or a later version.

７ Note

Some functionality might differ or be unavailable if you create the file share on a
server that runs a different version of Windows Server.

1. In the Server Manager navigation pane, select File and Storage Services > Shares
to display the Shares page.

2. In the Shares page, select Tasks > New Share. The New Share Wizard appears.

3. On the Select Profile page, choose the option that corresponds to your File Server
Resource Manager configuration:

If you have File Server Resource Manager installed and are using folder
management properties, select SMB Share - Advanced.
If you don't have File Server Resource Manager installed or you aren't using
folder management properties, select SMB Share – Quick.

4. On the Share Location page, select the server and volume on which you want to
create the share.

5. On the Share Name page, enter a name for the share (for example, Users$ ) in the
Share name box.

 Tip

When you create the share, hide the share by putting a $ (dollar sign) after
the share name. This change hides the share from casual browsers.

6. On the Other Settings page, clear the Enable continuous availability checkbox, if
present. Optionally, select the Enable access-based enumeration and Encrypt data
access checkboxes.

7. On the Permissions page, select Customize permissions to open the Advanced

Security Settings dialog box.
8. Select Disable inheritance, and then select Convert inherited permissions into
explicit permission on this object.

9. Set the permissions as described in the following tables and figures.

） Important

The permissions that you use depend on your remote access configuration, so
make sure you use the correct table.

Permissions for file servers without Remote Desktop Services

User Account Permission Applies to

System Full Control This folder,

subfolders, and
files

Administrators Full Control This folder only

Creator/Owner Full Control Subfolders and

files only

Security group of users who need to List folder/read data1 This folder only
put data on the share (Folder Create folders/append
Redirection Users) data1
Read attributes1
Read extended
attributes1
Read permissions1
Traverse folder/execute
file1

Other groups and accounts None (Remove any

accounts that this table
doesn't list)

1
Advanced permissions
Permissions for file servers with Remote Desktop Services

User Account or Permission Applies to

Role

System Full Control This folder,

subfolders, and files

Administrators Full Control This folder,

subfolders, and files

Creator/Owner Full Control Subfolders and files

only

Other groups and None (remove any other accounts from

accounts the access control list)
 10. If you chose the SMB Share - Advanced profile earlier in this procedure, follow
these extra steps:

On the Management Properties page, select the User Files Folder Usage
value.
Optionally, select a quota to apply to users of the share.

11. On the Confirmation page, select Create.

Step 3: Precreate folders for new users on

servers that also host Remote Desktop Services
If the file server also hosts Remote Desktop Services, use the following procedure to
precreate folders for new users and assign the appropriate permissions to the folders.

1. In the file share that you created in the previous procedure, navigate to the file
share's root folder.

2. Use one of the following methods to create a new folder.

Right-click the root folder, and then select New > Folder. For the name of the
folder, enter the user name of the new user.

Alternatively, to use Windows PowerShell to create the new folder, open a

PowerShell Command Prompt window and run the following cmdlet:

PowerShell

New-Item -Path 'c:\shares\frdeploy\<newuser>' -ItemType Directory

In this command, <newuser> represents the user name of the new user.

3. Right-click the new folder, and then select Properties > Security > Advanced >
Owner. Verify that the folder owner is the Administrators group.

4. Set the permissions as described in the following table and figure. Remove
permissions for any groups and accounts that aren't listed here.

User Account Permission Applies to

System Full control This folder, subfolders,

and files
 User Account Permission Applies to

Administrators Full Control This folder, subfolders,

and files

Creator/Owner Full Control Subfolders and

files only

newuser1 Full Control This folder, subfolders,

and files

Other groups and None (remove any other accounts from the
accounts access control list)

1
newuser represents the user name of the new user's account.

Step 4: Create a GPO for Folder Redirection

If you don't already have a Group Policy object (GPO) that manages the Folder
Redirection and Offline Files functionality, use the following procedure to create one.

1. On a computer that has Group Policy Management installed, open Server Manager.

2. Select Tools > Group Policy Management.

3. In Group Policy Management, right-click the domain or OU in which you want to

set up Folder Redirection, and then select Create a GPO in this domain, and Link it
here.

4. In the New GPO dialog box, enter a name for the GPO (for example, Folder
Redirection Settings), and then select OK.
 5. Right-click the newly created GPO, and then clear the Link Enabled checkbox. This
change prevents the GPO from being applied until you finish configuring it.

6. Select the GPO. Select Scope > Security Filtering > Authenticated Users, and then
select Remove to prevent the GPO from being applied to everyone.

7. In the Security Filtering section, select Add.

8. In the Select User, Computer, or Group dialog box, configure the option that
corresponds to your configuration:

File servers without Remote Desktop Services: Enter the name of the
security group that you created in Step 1: Create a folder redirection security
group (for example, Folder Redirection Users), and then select OK.
File servers with Remote Desktop Services: Enter the user name that you
used for the user folder in Step 3: Precreate folders for new users on servers
that also host Remote Desktop Services and then select OK.

9. Select Delegation > Add, and then enter Authenticated Users. Select OK, and then
select OK again to accept the default Read permission.

） Important

This step is necessary because of security changes made in MS16-072 . You

must give the Authenticated Users group delegated Read permissions to the
Folder Redirection GPO. If you don't, the GPO isn't applied to users, or if it's
already applied, the GPO is removed, redirecting folders back to the local PC.
For more information, see Deploying Group Policy Security Update MS16-
072 .

Step 5: Configure the Group Policy settings for

Folder Redirection and Offline Files
After you create a GPO for Folder Redirection settings, follow these steps to edit the
Group Policy settings that enable and configure Folder Redirection.

７ Note

By default, the Offline Files feature is enabled for redirected folders on Windows
client computers, and disabled on Windows Server computers. Users can enable
 this feature, or you can use Group Policy to control it. The policy is Allow or
disallow use of the Offline Files feature.

For information about some of the other Offline Files Group Policy settings, see
Enable Advanced Offline Files Functionality, and Configuring Group Policy for
Offline Files.

1. In Group Policy Management, right-click the GPO you created (for example, Folder
Redirection Settings), and then select Edit.

2. In the Group Policy Management Editor window, navigate to User Configuration >
Policies > Windows Settings > Folder Redirection.

3. Right-click a folder that you want to redirect (for example, Documents), and then
select Properties.

4. In the Properties dialog box, from the Setting box, select Basic - Redirect
everyone’s folder to the same location.

5. In the Target folder location section, select Create a folder for each user under
the root path.

6. In the Root Path box, enter the path to the file share that stores the redirected
folders, such as \\fs1.corp.contoso.com\users$ .

7. (Optional) Select the Settings tab, and in the Policy Removal section, select
Redirect the folder back to the local userprofile location when the policy is
removed. This setting can help make Folder Redirection behave more predictably
for administrators and users.

8. Select OK, and then select Yes in the Warning dialog box.

Step 6: Enable the Folder Redirection GPO

After you finish configuring the Folder Redirection Group Policy settings, the next step is
to enable the GPO. This change allows the GPO to be applied to affected users.

 Tip

If you plan to implement primary computer support or other policy settings, do so

now, before you enable the GPO. Implementing these settings prevents user data
from being copied to non-primary computers before primary computer support is
enabled.
 1. Open Group Policy Management.
2. Right-click the GPO that you created, and then select Link Enabled. A checkbox
appears next to the menu item.

Step 7: Test Folder Redirection

To test Folder Redirection, sign in to a computer by using a user account that is
configured to use redirected folders. Then confirm that the folders and profiles are
redirected.

1. Sign in to a primary computer (if you enabled primary computer support) by using
a user account for which you have enabled Folder Redirection.

2. If the user has previously signed in to the computer, open an elevated command
prompt, and then enter the following command to ensure that the latest Group
Policy settings are applied to the client computer:

Console

gpupdate /force

3. Open File Explorer.

4. Right-click a redirected folder (for example, the My Documents folder in the

Documents library), and then select Properties.

5. Select the Location tab, and confirm that the path displays the file share you
specified instead of a local path.

Appendix A: Checklist for deploying Folder

Redirection
Complete Task or item

Prepare domain and other prerequisites

- Join computers to domain

- Create user accounts

- Check file server prerequisites and compatibility with other services

Complete Task or item

- Does the file server also host Remote Desktop Services?

- Restrict access to the file server

Step 1: Create a folder redirection security group

- Group name:

- Members:

Step 2: Create a file share for redirected folders

- File share name:

Step 3: Precreate folders for new users on servers that also host Remote Desktop
Services

Step 4: Create a GPO for Folder Redirection

- GPO name:

Step 5: Configure the Group Policy settings for Folder Redirection and Offline Files

- Redirected folders:

- Windows 2000, Windows XP, and Windows Server 2003 support enabled?

- Offline Files enabled? (enabled by default on Windows client computers)

- Always Offline Mode enabled?

- Background file synchronization enabled?

- Optimized Move of redirected folders enabled?

(Optional) Enable primary computer support:

- Computer-based or User-based?

- Designate primary computers for users

- Location of user and primary computer mappings:

- (Optional) Enable primary computer support for Folder Redirection

- (Optional) Enable primary computer support for Roaming User Profiles

Step 6: Enable the Folder Redirection GPO

Step 7: Test Folder Redirection

Related links
Folder Redirection, Offline Files, and Roaming User Profiles overview
Deploy primary computers for Folder Redirection and Roaming User Profiles
Enable Always Offline mode for faster access to files
Information about Microsoft support policy for a DFS-R and DFS-N deployment
scenario
Sideload Apps with DISM
Troubleshooting packaging, deployment, and query of Windows apps
Deploy primary computers for Folder
Redirection and Roaming User Profiles
Article • 12/23/2021

Applies to: Windows Server 2022, Windows 10, Windows 8, Windows 8.1, Windows
Server 2019, Windows Server 2016, Windows Server 2012, Windows Server 2012 R2

This topic describes how to enable primary computer support and designate primary
computers for users. Doing so enables you to control which computers use Folder
Redirection and Roaming User Profiles.

） Important

When enabling primary computer support for Roaming User Profiles, always enable
primary computer support for Folder Redirection as well. This keeps documents
and other user files out of the user profiles, which helps profiles remain small and
sign on times stay fast.

Prerequisites

Software requirements
Primary computer support has the following requirements:

The Active Directory Domain Services (AD DS) schema must be updated to include
Windows Server 2012 schema additions (installing a Windows Server 2012 domain
controller automatically updates the schema). For information about updating the
AD DS schema, see Adprep.exe integration and Running Adprep.exe.
Client computers must run Windows 10, Windows 8.1, Windows 8, Windows Server
2019, Windows Server 2016, Windows Server 2012 R2, or Windows Server 2012.

 Tip

Although primary computer support requires Folder Redirection and/or Roaming

User Profiles, if you are deploying these technologies for the first time, it is best to
set up primary computer support before enabling the GPOs that configure Folder
Redirection and Roaming User Profiles. This prevents user data from being copied
 to non-primary computers before primary computer support is enabled. For
configuration information, see Deploy Folder Redirection and Deploy Roaming
User Profiles.

Step 1: Designate primary computers for users

The first step in deploying primary computers support is designating the primary
computers for each user. To do so, use Active Directory Administration Center to obtain
the distinguished name of the relevant computers and then set the msDs-
PrimaryComputer attribute.

 Tip

To use Windows PowerShell to work with primary computers, see the blog post
Digging a little deeper into Windows 8 Primary Computer.

Here's how to specify the primary computers for users:

1. Open Server Manager on a computer with Active Directory Administration Tools

installed.
2. On the Tools menu, select Active Directory Administration Center. Active
Directory Administration Center appears.
3. Navigate to the Computers container in the appropriate domain.
4. Right-click a computer that you want to designate as a primary computer and then
select Properties.
5. In the Navigation pane, select Extensions.
6. Select the Attribute Editor tab, scroll to distinguishedName, select View, right-
click the value listed, select Copy, select OK, and then select Cancel.
7. Navigate to the Users container in the appropriate domain, right-click the user to
which you want to assign the computer, and then select Properties.
8. In the Navigation pane, select Extensions.
9. Select the Attribute Editor tab, select msDs-PrimaryComputer and then select
Edit. The Multi-valued String Editor dialog box appears.
10. Right-click the text box, select Paste, select Add, select OK, and then select OK
again.

Step 2: Optionally enable primary computers

for Folder Redirection in Group Policy
The next step is to optionally configure Group Policy to enable primary computer
support for Folder Redirection. Doing so enables a user's folders to be redirected on
computers designated as the user's primary computers, but not on any other
computers. You can control primary computers for Folder Redirection on a per-
computer basis, or a per-user basis.

Here's how to enable primary computers for Folder Redirection:

1. In Group Policy Management, right-click the GPO you created when doing the
initial configuration of Folder Redirection and/or Roaming User Profiles (for
example, Folder Redirection Settings or Roaming User Profiles Settings), and then
select Edit.
2. To enable primary computers support using computer-based Group Policy,
navigate to Computer Configuration. For user-based Group Policy, navigate to
User Configuration.

Computer-based Group Policy applies primary computer processing to all

computers to which the GPO applies, affecting all users of the computers.
User-based Group Policy to applies primary computer processing to all user
accounts to which the GPO applies, affecting all computers to which the users
sign on.

3. Under Computer Configuration or User Configuration, navigate to Policies, then

Administrative Templates, then System, then Folder Redirection.
4. Right-click Redirect folders on primary computers only, and then select Edit.
5. Select Enabled, and then select OK.

Step 3: Optionally enable primary computers

for Roaming User Profiles in Group Policy
The next step is to optionally configure Group Policy to enable primary computer
support for Roaming User Profiles. Doing so enables a user's profile to roam on
computers designated as the user's primary computers, but not on any other
computers.

Here's how to enable primary computers for Roaming User Profiles:

1. Enable primary computer support for Folder Redirection, if you haven't already.
This keeps documents and other user files out of the user profiles, which helps
profiles remain small and sign on times stay fast.
2. In Group Policy Management, right-click the GPO you created (for example, Folder
Redirection and Roaming User Profiles Settings), and then select Edit.
 3. Navigate to Computer Configuration, then Policies, then Administrative
Templates, then System, and then User Profiles.
4. Right-click Download roaming profiles on primary computers only, and then
select Edit.
5. Select Enabled, and then select OK.

Step 4: Enable the GPO

Once you have completed configuring Folder Redirection and Roaming User Profiles,
enable the GPO if you have not already. Doing so permits it to be applied to affected
users and computers.

Here's how to enable the Folder Redirection and/or Roaming User Profiles GPOs:

1. Open Group Policy Management

2. Right-click the GPOs that you created, and then select Link Enabled. A checkbox
should appear next to the menu item.

Step 5: Test primary computer function

To test primary computer support, sign in to a primary computer, confirm that the
folders and profiles are redirected, then sign in to a non-primary computer and confirm
that the folders and profiles are not redirected.

Here's how to test primary computer functionality:

1. Sign in to a designated primary computer with a user account for which you have
enabled Folder Redirection and/or Roaming User Profiles.

2. If the user account has signed on to the computer previously, open a Windows
PowerShell session or Command Prompt window as an administrator, type the
following command and then sign off when prompted to ensure that the latest
Group Policy settings are applied to the client computer:

PowerShell

Gpupdate /force

3. Open File Explorer.

4. Right-click a redirected folder (for example, the My Documents folder in the

Documents library), and then select Properties.
 5. Select the Location tab, and confirm that the path displays the file share you
specified instead of a local path. To confirm that the user profile is roaming, open
Control Panel, select System and Security, select System, select Advanced System
Settings, select Settings in the User Profiles section and then look for Roaming in
the Type column.

6. Sign in with the same user account to a computer that is not designated as the
user's primary computer.

7. Repeat steps 2–5, instead looking for local paths and a Local profile type.

７ Note

If folders were redirected on a computer before you enabled primary computer

support, the folders will remain redirected unless the following setting is configured
in each folder's folder redirection policy setting: Redirect the folder back to the
local userprofile location when the policy is removed. Similarly, profiles that were
previously roaming on a particular computer will show Roaming in the Type
columns; however, the Status column will show Local.

More information
Deploy Folder Redirection with Offline Files
Deploy Roaming User Profiles
Folder Redirection, Offline Files, and Roaming User Profiles overview
Digging a little deeper into Windows 8 Primary Computer
Disable Offline Files on individual
redirected folders
Article • 07/11/2022

Applies to: Windows Server 2022, Windows 10, Windows 8, Windows 8.1, Windows
Server 2019, Windows Server 2016, Windows Server 2012, Windows Server 2012 R2,
Windows (Semi-annual Channel)

This topic describes how to disable Offline Files caching on individual folders that are
redirected to network shares by using Folder Redirection. This provides the ability to
specify which folders to exclude from caching locally, reducing the Offline Files cache
size and time required to synchronize Offline Files.

７ Note

This topic includes sample Windows PowerShell cmdlets that you can use to
automate some of the procedures described. For more information, see Windows
PowerShell Basics.

Prerequisites
To disable Offline Files caching of specific redirected folders, your environment must
meet the following prerequisites.

An Active Directory Domain Services (AD DS) domain, with client computers joined
to the domain. There are no forest or domain functional-level requirements or
schema requirements.
Client computers running Windows 10, Windows 8.1, Windows 8, Windows Server
2019, Windows Server 2016, Windows Server 2012 R2, Windows Server 2012 or
Windows (Semi-annual Channel).
A computer with Group Policy Management installed.

Disabling Offline Files on individual redirected

folders
To disable Offline Files caching of specific redirected folders, use Group Policy to enable
the Do not automatically make specific redirected folders available offline policy
setting for the appropriate Group Policy Object (GPO). Configuring this policy setting to
Disabled or Not Configured makes all redirected folders available offline.

７ Note

Only domain administrators, enterprise administrators, and members of the Group

Policy creator owners group can create GPOs.

To disable Offline Files on specific redirected folders

1. Open Group Policy Management.
2. To optionally create a new GPO that specifies which users should have redirected
folders excluded from being made available offline, right-click the appropriate
domain or organizational unit (OU) and then select Create a GPO in this domain,
and Link it here.
3. In the console tree, right-click the GPO for which you want to configure the Folder
Redirection settings and then select Edit. The Group Policy Management Editor
appears.
4. In the console tree, under User Configuration, expand Policies, expand
Administrative Templates, expand System, and expand Folder Redirection.
5. Right-click Do not automatically make specific redirected folders available offline
and then select Edit. The Do not automatically make specific redirected folders
available offline window appears.
6. Select Enabled. In the Options pane select the folders that should not be made
available offline by selecting the appropriate check boxes. Select OK.

Windows PowerShell equivalent commands

The following Windows PowerShell cmdlet or cmdlets perform the same function as the
procedure described in Disabling Offline Files on individual redirected folders. Enter
each cmdlet on a single line, even though they may appear word-wrapped across
several lines here because of formatting constraints.

This example creates a new GPO named Offline Files Settings in the MyOu organizational
unit in the contoso.com domain (the LDAP distinguished name is
"ou=MyOU,dc=contoso,dc=com"). It then disables Offline Files for the Videos redirected
folder.

PowerShell
 New-GPO -Name "Offline Files Settings" | New-Gplink -Target
"ou=MyOu,dc=contoso,dc=com" -LinkEnabled Yes

Set-GPRegistryValue –Name "Offline Files Settings" –Key

"HKCU\Software\Policies\Microsoft\Windows\NetCache\{18989B1D-99B5-455B-841C-
AB7C74E4DDFC}" -ValueName DisableFRAdminPinByFolder –Type DWORD –Value 1

See the following table for a listing of registry key names (folder GUIDs) to use for each
redirected folder.

Redirected folder Registry key name (folder GUID)

AppData(Roaming) {3EB685DB-65F9-4CF6-A03A-E3EF65729F3D}

Desktop {B4BFCC3A-DB2C-424C-B029-7FE99A87C641}

Start Menu {625B53C3-AB48-4EC1-BA1F-A1EF4146FC19}

Documents {FDD39AD0-238F-46AF-ADB4-6C85480369C7}

Pictures {33E28130-4E1E-4676-835A-98395C3BC3BB}

Music {4BD8D571-6D19-48D3-BE97-422220080E43}

Videos {18989B1D-99B5-455B-841C-AB7C74E4DDFC}

Favorites {1777F761-68AD-4D8A-87BD-30B759FA33DD}

Contacts {56784854-C6CB-462b-8169-88E350ACB882}

Downloads {374DE290-123F-4565-9164-39C4925E467B}

Links {BFB9D5E0-C6A9-404C-B2B2-AE6DB6AF4968}

Searches {7D1D3A04-DEBB-4115-95CF-2F29DA2920DA}

Saved Games {4C5C32FF-BB9D-43B0-B5B4-2D72E54EAAA4}

More information
Folder Redirection, Offline Files, and Roaming User Profiles overview
Deploy Folder Redirection with Offline Files
Enable Always Offline mode for faster
access to files
Article • 12/23/2021

Applies to: Windows Server 2022, Windows 10, Windows 8, Windows 8.1, Windows
Server 2019, Windows Server 2016, Windows Server 2012, Windows Server 2012 R2,
and Windows (Semi-annual Channel)

This document describes how to use the Always Offline mode of Offline Files to provide
faster access to cached files and redirected folders. Always Offline also provides lower
bandwidth usage because users are always working offline, even when they are
connected through a high-speed network connection.

Prerequisites
To enable Always Offline mode, your environment must meet the following
prerequisites.

An Active Directory Domain Services (AD DS) domain with client computers joined
to the domain. There are no forest or domain functional-level requirements or
schema requirements.
Client computers running Windows 10, Windows 8.1, Windows 8, Windows Server
2016, Windows Server 2012 R2, or Windows Server 2012. (Client computers
running earlier versions of Windows might continue to transition to Online mode
on very high-speed network connections.)
A computer with Group Policy Management installed.

Enable Always Offline mode

To enable Always Offline mode, use Group Policy to enable the Configure slow-link
mode policy setting and set the latency to 1 (millisecond). Doing so causes client
computers running Windows 8 or Windows Server 2012 to automatically use Always
Offline mode.

７ Note

Computers running Windows 7, Windows Vista, Windows Server 2008 R2, or

Windows Server 2008 might continue to transition to the Online mode if the
 latency of the network connection drops below one millisecond.

1. Open Group Policy Management.

2. To optionally create a new Group Policy Object (GPO) for Offline Files settings,
right-click the appropriate domain or organizational unit (OU), and then select
Create a GPO in this domain, and link it here.
3. In the console tree, right-click the GPO for which you want to configure the Offline
Files settings and then select Edit. The Group Policy Management Editor appears.
4. In the console tree, under Computer Configuration, expand Policies, expand
Administrative Templates, expand Network, and expand Offline Files.
5. Right-click Configure slow-link mode, and then select Edit. The Configure slow-
link mode window will appear.
6. Select Enabled.
7. In the Options box, select Show. The Show Contents window will appear.
8. In the Value name box, specify the file share for which you want to enable Always
Offline mode.
9. To enable Always Offline mode on all file shares, enter *.
10. In the Value box, enter Latency=1 to set the latency threshold to one millisecond,
and then select OK.

７ Note

By default, when in Always Offline mode, Windows synchronizes files in the Offline
Files cache in the background every two hours. To change this value, use the
Configure Background Sync policy setting.

More information
Folder Redirection, Offline Files, and Roaming User Profiles overview
Deploy Folder Redirection with Offline Files
Enable optimized moves of redirected
folders
Article • 12/23/2021

Applies to: Windows Server 2022, Windows 10, Windows 8, Windows 8.1, Windows
Server 2019, Windows Server 2016, Windows Server 2012 R2, Windows Server 2012

This topic describes how to perform an optimized move of redirected folders (Folder
Redirection) to a new file share. If you enable this policy setting, when an administrator
moves the file share hosting redirected folders and updates the target path of the
redirected folders in Group Policy, the cached content is simply renamed in the local
Offline Files cache without any delays or potential data loss for the user.

Previously, administrators could change the target path of the redirected folders in
Group Policy and let the client computers copy the files at the affected user's next sign
in, causing a delayed sign in. Alternatively, administrators could move the file share and
update the target path of the redirected folders in Group Policy. However, any changes
made locally on the client computers between the start of the move and the first sync
after the move would be lost.

Prerequisites
Optimized move has the following requirements:

Folder Redirection must be setup. For more information see Deploy Folder
Redirection with Offline Files.
Client computers must run Windows 10, Windows 8.1, Windows 8, Windows Server
2019, Windows Server 2016, Windows Server 2012 R2, Windows Server 2012 or
Windows Server (Semi-annual Channel).

Step 1: Enable optimized move in Group Policy

To optimize the relocation of Folder Redirection data, use Group Policy to enable the
Enable optimized move of contents in Offline Files cache on Folder Redirection server
path change policy setting for the appropriate Group Policy Object (GPO). Configuring
this policy setting to Disabled or Not Configured causes the client to copy all the Folder
Redirection content to the new location and then delete the content from the old
location if the server path changes.
Here's how to enable optimized moving of redirected folders:

1. In Group Policy Management, right-click the GPO you created for Folder
Redirection settings (for example, Folder Redirection and Roaming User Profiles
Settings), and then select Edit.
2. Under User Configuration, navigate to Policies, then Administrative Templates,
then System, then Folder Redirection.
3. Right-click Enable optimized move of contents in Offline Files cache on Folder
Redirection server path change, and then select Edit.
4. Select Enabled, and then select OK.

Step 2: Relocate the file share for redirected

folders
When moving the file share that contains users' redirected folders, it is important to take
precautions to ensure that the folders are relocated properly.

） Important

If the users' files are in use or if the full file state is not preserved in the move, users
might experience poor performance as the files are copied over the network,
synchronization conflicts generated by Offline Files, or even data loss.

1. Notify users in advance that the server hosting their redirected folders will change
and recommend that they perform the following actions:

Synchronize the contents of their Offline Files cache and resolve any conflicts.

Open an elevated command prompt, enter GpUpdate /Target:User /Force,

and then sign out and sign back in to ensure that the latest Group Policy
settings are applied to the client computer

７ Note

By default, client computers update Group Policy every 90 minutes, so if

you allow sufficient time for client computers to receive updated policy,
you do not need to ask users to use GpUpdate.

2. Remove the file share from the server to ensure that no files in the file share are in
use. To do so in Server Manager, on the Shares page of File and Storage Services,
 right-click the appropriate file share, then select Remove.

Users will work offline using Offline Files until the move is complete and they
receive the updated Folder Redirection settings from Group Policy.

3. Using an account with backup privileges, copy the contents of the file share to the
new location using a method that preserves file timestamps, such as a backup and
restore utility. To use the Robocopy command, open an elevated command
prompt, and then type the following command, where <Source> is the current
location of the file share, and <Destination> is the new location:

PowerShell

Robocopy /B <Source> <Destination> /Copyall /MIR /EFSRAW

７ Note

The Xcopy command does not preserve all of the file state.

4. Edit the Folder Redirection policy settings, updating the target folder location for
each redirected folder that you want to relocate. For more information, see Step 4
of Deploy Folder Redirection with Offline Files.

5. Notify users that the server hosting their redirected folders has changed, and that
they should use the GpUpdate /Target:User /Force command, and then sign out
and sign back in to get the updated configuration and resume proper file
synchronization.

Users should sign on to all machines at least once to ensure that the data gets
properly relocated in each Offline Files cache.

More information
Deploy Folder Redirection with Offline Files
Deploy Roaming User Profiles
Folder Redirection, Offline Files, and Roaming User Profiles overview
Troubleshoot user profiles with events
Article • 12/23/2021

Applies to: Windows Server 2022, Windows 10, Windows 8, Windows 8.1, Windows
Server 2019, Windows Server 2016, Windows Server 2012 R2, Windows Server 2012.

This topic discusses how to troubleshoot problems loading and unloading user profiles
by using events and trace logs. The following sections describe how to use the three
event logs that record user profile information.

Step 1: Checking events in the Application log

The first step in troubleshooting issues with loading and unloading user profiles
(including roaming user profiles) is to use Event Viewer to examine any Warning and
Error events that User Profile Service records in the Application log.

Here's how to view User Profile Services events in the Application log:

1. Start Event Viewer. To do so, open Control Panel, select System and Security, and
then, in the Administrative Tools section, select View event logs. The Event Viewer
window opens.
2. In the console tree, first navigate to Windows Logs, then Application.
3. In the Actions pane, select Filter Current Log. The Filter Current Log dialog box
opens.
4. In the Event sources box, select the User Profiles Service checkbox, and then
select OK.
5. Review the listing of events, paying particular attention to Error events.
6. When you find noteworthy events, select the Event Log Online Help link to display
additional information and troubleshooting procedures.
7. To perform further troubleshooting, note the date and time of noteworthy events
and then examine the Operational log (as described in Step 2) to view details
about what the User Profile Service was doing around the time of the Error or
Warning events.

７ Note

You can safely ignore User Profile Service event 1530 "Windows detected your
registry file is still in use by other applications or services."
Step 2: View the Operational log for the User
Profile Service
If you cannot resolve the issue using the Application log alone, use the following
procedure to view User Profile Service events in the Operational log. This log shows
some of the inner workings of the service, and can help pinpoint where in the profile
load or unload process the problem is occurring.

Both the Windows Application log and the User Profile Service Operational log are
enabled by default in all Windows installations.

Here's how to view the Operational log for the User Profile Service:

1. In the Event Viewer console tree, navigate to Applications and Services Logs, then
Microsoft, then Windows, then User Profile Service, and then Operational.
2. Examine the events that occurred around the time of the Error or Warning events
that you noted in the Application log.

Step 3: Enable and view analytic and debug

logs
If you require more detail than the Operational log provides, you can enable analytic
and debug logs on the affected computer. This level of logging is much more detailed
and should be disabled except when trying to troubleshoot an issue.

Here's how to enable and view analytic and debug logs:

1. In the Actions pane of Event Viewer, select View, and then select Show Analytic
and Debug Logs.
2. Navigate to Applications and Services Logs, then Microsoft, then Windows, then
User Profile Service, and then Diagnostic.
3. Select Enable Log and then select Yes. This enables the Diagnostic log, which will
start logging.
4. If you require even more detailed information, see Step 4: Creating and decoding a
trace for more information about how to create a trace log.
5. When you are finished troubleshooting the issue, navigate to the Diagnostic log,
select Disable Log, select View and then clear the Show Analytic and Debug Logs
checkbox to hide analytic and debug logging.

Step 4: Creating and decoding a trace

If you cannot resolve the issue using events, you can create a trace log (an ETL file) while
reproducing the issue, and then decode it using public symbols from the Microsoft
symbol server. Trace logs provide very specific information about what the User Profile
Service is doing, and can help pinpoint where the failure occurred.

The best strategy when using ETL tracing is to first capture the smallest log possible.
Once the log is decoded, search the log for failures.

Here's how to create and decode a trace for the User Profile Service:

1. Sign on to the computer where the user is experiencing problems, using an

account that is a member of the local Administrators group.

2. From an elevated command prompt enter the following commands, where <Path>
is the path to a local folder that you have previously created, for example C:\logs:

PowerShell

logman create trace -bs 1024 -nb 16 16 -n RUP -o <Path>\RUP.etl -ets

logman update RUP -p {eb7428f5-ab1f-4322-a4cc-1f1a9b2c5e98} 0x7FFFFFFF
0x7 -ets
logman update RUP -p {9891e0a7-f966-547f-eb21-d98616bf72ee} 0xFFFFFFFF
0xFF -ets
logman update RUP -p {9959adbd-b5ac-5758-3ffa-ee0da5b8fe4b} 0xFFFFFFFF
0xFF -ets
logman update RUP -p {7f1bd045-965d-4f47-b3a7-acdbcfb11ca6} 0xFFFFFFFF
0xFF -ets
logman update RUP -p {40654520-7460-5c90-3c10-e8b6c8b430c1} 0xFFFFFFFF
0xFF -ets
logman update RUP -p {d5ee9312-a511-4c0e-8b35-b6d980f6ba25} 0xFFFFFFFF
0xFF -ets
logman update RUP -p {04a241e7-cea7-466d-95a1-87dcf755f1b0} 0xFFFFFFFF
0xFF -ets
logman update RUP -p {9aed307f-a41d-40e7-9539-b8d2742578f6} 0xFFFFFFFF
0xFF -ets

3. From the Start screen, select the user name, and then select Switch account, being
careful not to log off the administrator. If you are using Remote Desktop, close the
Administrator session to establish the user session.

4. Reproduce the problem. The procedure to reproduce the problem is typically to

sign on as the user experiencing the issue, sign the user off, or both.

5. After reproducing the problem, sign on as the local administrator again.

6. From an elevated command prompt run the following command to save the log
into an ETL file:
 PowerShell

logman stop -n RUP -ets

7. Type the following command to export the ETL file into a human-readable file in
the current directory (likely your home folder or the %WINDIR%\System32 folder):

PowerShell

Tracerpt <path>\RUP.etl

8. Open the Summary.txt file and Dumpfile.xml file (you can open them in Microsoft
Excel to more easily view the complete details of the log). Look for events that
include fail or failed; you can safely ignore lines that include the Unknown event
name.

More information
Deploy Roaming User Profiles
iSCSI Target Server overview
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012

This topic provides a brief overview of iSCSI Target Server, a role service in Windows
Server that enables you to make storage available via the iSCSI protocol. This is useful
for providing access to storage on your Windows server for clients that can't
communicate over the native Windows file sharing protocol, SMB.

iSCSI Target Server is ideal for the following:

Network and diskless boot By using boot-capable network adapters or a

software loader, you can deploy hundreds of diskless servers. With iSCSI Target
Server, the deployment is fast. In Microsoft internal testing, 256 computers
deployed in 34 minutes. By using differencing virtual hard disks, you can save up to
90% of the storage space that was used for operating system images. This is ideal
for large deployments of identical operating system images, such as on virtual
machines running Hyper-V or in high-performance computing (HPC) clusters.

Server application storage Some applications require block storage. iSCSI Target
Server can provide these applications with continuously available block storage.
Because the storage is remotely accessible, it can also consolidate block storage
for central or branch office locations.

Heterogeneous storage iSCSI Target Server supports non-Microsoft iSCSI

initiators, making it easy to share storage on servers in a mixed software
environment.

Development, test, demonstration, and lab environments When iSCSI Target

Server is enabled, a computer running the Windows Server operating system
becomes a network-accessible block storage device. This is useful for testing
applications prior to deployment in a storage area network (SAN).

Block storage requirements

Enabling iSCSI Target Server to provide block storage leverages your existing Ethernet
network. No additional hardware is needed. If high availability is an important criterion,
consider setting up a high-availability cluster. You need shared storage for a high-
availability cluster—either hardware for Fibre Channel storage or a serial attached SCSI
(SAS) storage array.

If you enable guest clustering, you need to provide block storage. Any servers running
Windows Server software with iSCSI Target Server can provide block storage.

See Also
iSCSI Target Block Storage, How To What's New in iSCSI Target Server in Windows Server
iSCSI Target Server Scalability Limits
Article • 10/08/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012

This topic provides the supported and tested Microsoft iSCSI Target Server limits on
Windows Server. The following tables display the tested support limits and, where
applicable, whether the limits are enforced.

General limits
Item Support Enforced? Comment
Limit

iSCSI target 256 No

instances per iSCSI
Target Server

iSCSI logical units 512 No Testing configurations included: 8 LUs per target
(LUs) or virtual disks instance with an average over 64 targets, and
per iSCSI Target 256 target instances with one LU per target.
Server

iSCSI LUs or virtual 256 (128 Yes

disks per iSCSI target on
instance Windows
Server
2012)

Sessions that can 544 (512 Yes

simultaneously on
connect to an iSCSI Windows
target instance Server
2012)

Snapshots per LU 512 Yes There is a limit of 512 snapshots per

independent iSCSI application volume.

Locally mounted 32 Yes Locally mounted virtual disks don't offer any
virtual disks or iSCSI-specific functionality, and are deprecated -
snapshots per for more info, see Features Removed or
storage appliance Deprecated in Windows Server 2012 R2.
Fault Tolerance limits
Item Support Enforced? Comment
Limit

Failover cluster nodes 8 (5 on No

Windows
Server
2012)

Multiple active cluster nodes Supported N/A Each active node in the failover cluster
owns a different iSCSI Target Server
clustered instance with other nodes
acting as possible owner nodes.

Error recovery level (ERL) 0 Yes

Connections per session 1 Yes

Sessions that can 544 (512 No

simultaneously connect to an on
iSCSI target instance Windows
Server
2012)

Multipath Input/Output Supported N/A

(MPIO)

MPIO paths 4 No

Converting a stand-alone Not No The iSCSI Target instance and virtual

iSCSI Target Server to a supported disk configuration data, including
clustered iSCSI Target Server snapshot metadata, is lost during
or vice versa conversion.

Network limits
Item Support Limit Enforced? Comment

Maximum 8 No Applies to network adapters that are

number of dedicated to iSCSI traffic, rather than
active network the total number of network adapters
adapters in the appliance.

Portal (IP 64 Yes

addresses)
supported
Item Support Limit Enforced? Comment

Network port 1Gbps, 10 Gbps, 40Gbps, No

speed 56 Gbps (Windows Server
2012 R2 and newer only)

IPv4 Supported N/A

IPv6 Supported N/A

TCP offload Supported N/A Leverage Large Send (segmentation),

checksum, interrupt moderation, and
RSS offload

iSCSI offload Not supported

N/A

Jumbo frames Supported N/A

IPSec Supported N/A

CRC offload Supported N/A

iSCSI virtual disk limits

Item Support Enforced? Comment
limit

From an iSCSI initiator Yes No

converting the virtual
disk from a basic disk to
a dynamic disk

Virtual hard disk format .vhdx

(Windows
Server 2012
R2 and
newer only)

.vhd

VHD minimum format .vhdx: 3 MB Yes Applies to all supported VHD types:
size parent, differencing, and fixed.
.vhd: 8 MB

Parent VHD max size .vhdx: 64 TB Yes

.vhd: 2 TB
Item Support Enforced? Comment
limit

Fixed VHD max size .vhdx: 64 TB Yes

.vhd: 16 TB

Differencing VHD max .vhdx: 64 TB Yes

size
.vhd: 2 TB

VHD fixed format Supported No

VHD differencing format Supported No Snapshots cannot be taken of

differencing VHD-based iSCSI virtual
disks.

Number of differencing 256 No (Yes Two levels of depth (grandchildren .vhdx

VHDs per parent VHD on files) is the maximum for .vhdx files; one
Windows level of depth (child .vhd files) is the
Server maximum for .vhd files.
2012)

VHD dynamic format .vhdx: Yes Yes Unmap isn't supported.

.vhd: Yes (No

on Windows
Server 2012)

exFAT/FAT32/FAT Not Yes

(hosting volume of the supported
VHD)

CSV v2 Not Yes

supported

ReFS Supported N/A

NTFS Supported N/A

Non-Microsoft CFS Not Yes

supported

Thin provisioning No N/A Dynamic VHDs are supported, but

Unmap isn't supported.

Logical Unit shrink Yes N/A Use Resize-iSCSIVirtualDisk to shrink a

(Windows LUN.
Server 2012
R2 and
newer only)
 Item Support Enforced? Comment
limit

Logical Unit cloning Not N/A You can rapidly clone disk data by using
supported differencing VHDs.

Snapshot limits
Item Support Comment
limit

Snapshot Supported
create

Snapshot Supported
restore

Writable Not
snapshots supported

Snapshot – Not
convert to supported
full

Snapshot – Not
online supported
rollback

Snapshot – Not
convert to supported
writable

Snapshot - Not
redirection supported

Snapshot - Not
pinning supported

Local Supported Locally mounted iSCSI virtual disks are deprecated - for more info, see
mount Features Removed or Deprecated in Windows Server 2012 R2.
Dynamic disk snapshots cannot be locally mounted.

iSCSI Target Server manageability and backup

If you want to create volume shadow copies (VSS open-file snapshots) of data on iSCSI
virtual disks from an application server, or you want to manage iSCSI virtual disks with
an older app (such as the Diskraid command) that requires a Virtual Disk Service (VDS)
hardware provider, install the iSCSI Target Storage Provider on the server from which
you want to take a snapshot or use a VDS management app.

The iSCSI Target Storage Provider is a role service in Windows Server 2016, Windows
Server 2012 R2, and Windows Server 2012; you can also download and install iSCSI
Target Storage Providers (VDS/VSS) for down-level application servers on the
following operating systems as long as the iSCSI Target Server is running on Windows
Server 2012:

Windows Storage Server 2008 R2

Windows Server 2008 R2

Windows HPC Server 2008 R2

Windows HPC Server 2008

Note that if the iSCSI Target Server is hosted by a server running Windows Server 2012
R2 or newer and you want to use VSS or VDS from a remote server, the remote server
has to also run the same version of Windows Server and have the iSCSI Target Storage
Provider role service installed. Also note that on all versions of Windows you should
install only one version of the iSCSI Target Storage Provider role service.

For more info about the iSCSI Target Storage Provider, see iSCSI Target Storage
(VDS/VSS) Provider.

Tested compatibility with iSCSI initiators

We've tested the iSCSI Target Server software with the following iSCSI initiators:

Initiator Windows Windows Comments

Server Server
2012 R2 2012

Windows Server 2012 R2 Validated

Windows Server 2012, Windows Server Validated Validated

2008 R2, Windows Server 2008, Windows
Server 2003

VMWare vSphere 5 Validated

VMWare ESXi 5.0 Validated

VMWare ESX 4.1 Validated

CentOS 6.x Validated Must log out a session and

 log back in to detect a
resized virtual disk.

Red Hat Enterprise Linux 6 Validated

RedHat Enterprise Linux 5 and 5 Validated Validated

SUSE Linux Enterprise Server 10 Validated

Oracle Solaris 11.x Validated

We've also tested the following iSCSI initiators performing a diskless boot from virtual
disks hosted by iSCSI Target Server:

Windows Server 2012 R2

Windows Server 2012

PCIe NIC with iPXE

CD or USB disk with iPXE

Additional References
The following list provides additional resources about iSCSI Target Server and related
technologies.

iSCSI Target Block Storage Overview

iSCSI Target Boot Overview

Storage in Windows Server

iSCSI target boot overview
Article • 06/17/2021

Applies to: Windows Server 2016

iSCSI Target Server in Windows Server can boot hundreds of computers from a single
operating system image that is stored in a centralized location. This improves efficiency,
manageability, availability, and security.

Feature description
By using differencing virtual hard disks (VHDs), you can use a single operating system
image (the "master image") to boot up to 256 computers. As an example, let's assume
that you deployed Windows Server with an operating system image of approximately
20 GB, and you used two mirrored disk drives to act as the boot volume. You would
have needed approximately 10 TB of storage for only the operating system image to
boot 256 computers. With iSCSI Target Server, you will use 40 GB for the operating
system base image, and 2 GB for differencing virtual hard disks per server instance,
totaling 552 GB for the operating system images. This provides a savings of over 90% on
storage for the operating system images alone.

Practical applications
Using a controlled operating system image provides the following benefits:

More secure and easier to manage. Some enterprises require that data be secured by
physically locking storage in a centralized location. In this scenario, servers access the
data remotely, including the operating system image. With iSCSI Target Server,
administrators can centrally manage the operating system boot images, and control
which applications to use for the master image.

Rapid deployment. Because the master image is prepared by using Sysprep, when
computers boot from the master image, they skip the file copying and installation phase
that occurs during Windows Setup, and they go straight to the customization phase. In
Microsoft internal testing, 256 computers were deployed in 34 minutes.

Fast recovery. Because the operating system images are hosted on the computer
running iSCSI Target Server, if a disk-less client needs to be replaced, the new computer
can point to the operating system image, and boot up immediately.
 ７ Note

Various vendors provide a storage area network (SAN) boot solution, which can be
used by the iSCSI Target Server in Windows Server on commodity hardware.

Hardware requirements
iSCSI Target Server does not require special hardware for functional verification. In data
centers with large-scale deployments, the design should be validated against specific
hardware. For reference, Microsoft internal testing indicated that a 256 computer
deployment required 24x15k-RPM disks in a RAID 10 configuration for storage. A
network bandwidth of 10 Gigabit is optimal. A general estimate is 60 iSCSI boot servers
per 1 Gigabit network adapter.

A specialized network adapter is not required for this scenario, and a software boot
loader can be used (such as iPXE open source boot firmware).

Software requirements
iSCSI Target Server can be installed as part of the File and iSCSI Services role service in
Server Manager.

７ Note

Booting Nano Server from iSCSI (either from the Windows iSCSI Target Server, or a
3rd party target implementation) is not supported.

Additional References
iSCSI Target Server
iSCSI initiator cmdlets
iSCSI Target Server cmdlets
Resilient File System (ReFS) overview
Article • 02/15/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012

The Resilient File System (ReFS) is Microsoft's newest file system, designed to maximize
data availability, scale efficiently to large data sets across diverse workloads, and provide
data integrity with resiliency to corruption. It seeks to address an expanding set of
storage scenarios and establish a foundation for future innovations.

Key benefits

Resiliency
ReFS introduces new features that can precisely detect corruptions and also fix those
corruptions while remaining online, helping provide increased integrity and availability
for your data:

Integrity-streams - ReFS uses checksums for metadata and optionally for file data,
giving ReFS the ability to reliably detect corruptions.
Storage Spaces integration - when used with a mirror or parity space, ReFS can
automatically repair detected corruptions using the alternate copy of the data
provided by Storage Spaces. Repair processes are both localized to the area of
corruption and performed online, requiring no volume downtime.
Salvaging data - if a volume becomes corrupted and an alternate copy of the
corrupted data doesn't exist, ReFS removes the corrupt data from the namespace.
ReFS keeps the volume online while it handles most non-correctable corruptions,
but there are rare cases that require ReFS to take the volume offline.
Proactive error correction - in addition to validating data before reads and writes,
ReFS introduces a data integrity scanner, known as a scrubber. This scrubber
periodically scans the volume, identifying latent corruptions and proactively
triggering a repair of corrupt data.

Performance
In addition to providing resiliency improvements, ReFS introduces new features for
performance-sensitive and virtualized workloads. Real-time tier optimization, block
cloning, and sparse VDL are good examples of the evolving capabilities of ReFS, which
are designed to support dynamic and diverse workloads:

Mirror-accelerated parity - Mirror-accelerated parity delivers both high

performance and also capacity efficient storage for your data.

To deliver both high performance and capacity efficient storage, ReFS divides a
volume into two logical storage groups, known as tiers. These tiers can have their
own drive and resiliency types, allowing each tier to optimize for either
performance or capacity. Some example configurations include:

Performance tier Capacity tier

Mirrored SSD Mirrored HDD

Mirrored SSD Parity SSD

Mirrored SSD Parity HDD

Once these tiers are configured, ReFS uses them to deliver fast storage for hot data
and capacity-efficient storage for cold data:
All writes will occur in the performance tier, and large chunks of data that
remain in the performance tier will be efficiently moved to the capacity tier in
real time.
If using a hybrid deployment (mixing flash and HDD drives), the cache in
Storage Spaces Direct helps accelerate reads, reducing the effect of data
fragmentation characteristic of virtualized workloads. Otherwise, if using an all-
flash deployment, reads also occur in the performance tier.

７ Note

For Windows Server deployments, mirror-accelerated parity is only supported

on Storage Spaces Direct. We recommend using mirror-accelerated parity
with archival and backup workloads only. For virtualized and other high
performance random workloads, we recommend using three-way mirrors for
better performance.

Accelerated VM operations - ReFS introduces new functionality specifically

targeted to improve the performance of virtualized workloads:
Block cloning - block cloning accelerates copy operations, enabling quick, low-
impact VM checkpoint merge operations.
Sparse VDL - sparse VDL allows ReFS to zero files rapidly, reducing the time
needed to create fixed VHDs from 10s of minutes to mere seconds.
 Variable cluster sizes - ReFS supports both 4K and 64K cluster sizes. 4K is the
recommended cluster size for most deployments, but 64K clusters are appropriate
for large, sequential IO workloads.

Scalability
ReFS is designed to support extremely large data sets - millions of terabytes - without
negatively impacting performance, achieving greater scale than prior file systems.

Supported deployments
Microsoft has developed NTFS specifically for general-purpose use with a wide range of
configurations and workloads. For customers specially requiring the availability,
resiliency, and/or scale that ReFS provides, Microsoft supports ReFS for use with the
following configurations and scenarios:

７ Note

All ReFS supported configurations must use Windows Server Catalog certified
hardware and meet application requirements.

） Important

If you plan to use ReFS for Cluster Shared Volumes (CSVs), please see Use Cluster
Shared Volumes in a failover cluster for important information.

Storage Spaces Direct

Deploying ReFS on Storage Spaces Direct is recommended for virtualized workloads or
network-attached storage:

Mirror-accelerated parity and the cache in Storage Spaces Direct deliver high
performance and capacity-efficient storage.
The introduction of block clone and sparse VDL dramatically accelerates .vhdx file
operations, such as creation, merge, and expansion.
Integrity-streams, online repair, and alternate data copies enable ReFS and Storage
Spaces Direct to jointly detect and correct storage controller and storage media
corruptions within both metadata and data.
ReFS provides the functionality to scale and support large data sets.
Storage Spaces
Deploying ReFS on Storage Spaces with shared SAS enclosures is suitable for hosting
archival data and storing user documents:

Integrity-streams, online repair, and alternate data copies enable ReFS and Storage
Spaces to jointly detect and correct storage controller and storage media
corruptions within both metadata and data.
Storage Spaces deployments can also utilize block-cloning and the scalability
offered in ReFS.

７ Note

Storage Spaces supports local non-removable direct-attached via BusTypes SATA,

SAS, NVME, or attached via HBA (also known as RAID controller in pass-through
mode).

Basic disks
Deploying ReFS on basic disks is best suited for applications that implement their own
software resiliency and availability solutions:

Applications that introduce their own resiliency and availability software solutions
can use integrity-streams, block-cloning, and the ability to scale and support large
data sets.

７ Note

Basic disks include local non-removable direct-attached via BusTypes SATA, SAS,
NVME, or RAID. Basic disks do not include Storage Spaces.

Backup target
Deploying ReFS as a backup target is best suited for applications and hardware that
implements its own resiliency and availability solutions:

Applications that introduce their own resiliency and availability software solutions
can use integrity-streams, block-cloning, and the ability to scale and support large
data sets.
 ７ Note

Backup targets include the above supported configurations. Please contact

application and storage array vendors for support details on Fiber Channel and
iSCSI SANs. For SANs, if features such as thin provisioning, TRIM/UNMAP, or
Offloaded Data Transfer (ODX) are required, NTFS must be used.

Feature comparison

Limits

Feature ReFS NTFS

Maximum file name length 255 Unicode characters 255 Unicode characters

Maximum path name length 32K Unicode characters 32K Unicode characters

Maximum file size 35 PB (petabytes) 256 TB

Maximum volume size 35 PB 256 TB

Functionality

The following features are available with ReFS and NTFS:

Feature ReFS NTFS

BitLocker encryption Yes Yes

Data Deduplication Yes1 Yes

Cluster Shared Volume (CSV) support Yes2 3 Yes

Junctions/Soft links Yes Yes

Hard links Yes4 Yes

Failover cluster support Yes Yes

Access-control lists Yes Yes

USN journal Yes Yes

Changes notifications Yes Yes

Feature ReFS NTFS

Junction points Yes Yes

Mount points Yes Yes

Reparse points Yes Yes

Volume snapshots Yes Yes

File IDs Yes Yes

Oplocks Yes Yes

Sparse files Yes Yes

Named streams Yes Yes

Thin Provisioning Yes5 Yes

Trim/Unmap Yes5 Yes

Page file support Yes6 Yes

1. Available on Windows Server, version 1709 and later, Windows Server 2019 (1809)
LTSC or later.
2. Available on Windows Server 2012 R2 and later.
3. CSV will not use Direct I/O with Storage Spaces, Storage Spaces Direct (S2D) or
SAN.
4. Version ReFS 3.5 formatted by Windows 10 Enterprise Insider Preview build 19536
and later. Hard links support is added for newly formatted volumes only. Hard
links can't be used on volumes that have been upgraded from previous versions
5. Storage Spaces only.
6. Available on ReFS 3.7 and later.

The following features are only available with ReFS:

Functionality ReFS NTFS

Block clone Yes No

Sparse VDL Yes No

Mirror-accelerated parity Yes (on Storage Spaces Direct) No

File-level snapshots Yes1 No

1. Available on Windows Server 2022 and later.

The following features are unavailable on ReFS at this time:

Functionality ReFS NTFS

File system compression No Yes

File system encryption No Yes

Transactions No Yes

Object IDs No Yes

Offloaded Data Transfer (ODX) No Yes

Short names No Yes

Extended attributes No Yes

Disk quotas No Yes

Bootable No Yes

Supported on removable media No Yes

Additional References
Cluster size recommendations for ReFS and NTFS
Storage Spaces Direct overview
ReFS block cloning
ReFS integrity streams
Troubleshoot ReFS with ReFSUtil
Use of ReFS with Cluster-Shared Volumes
ReFS versions and compatibility matrix
Mirror-accelerated parity
Article • 09/21/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

Storage Spaces can provide fault tolerance for data using two fundamental techniques:
mirror and parity. In Storage Spaces Direct, ReFS introduces mirror-accelerated parity,
which enables you to create volumes that use both mirror and parity resiliencies. Mirror-
accelerated parity offers inexpensive, space-efficient storage without sacrificing
performance.

Background
Mirror and parity resiliency schemes have fundamentally different storage and
performance characteristics:

Mirror resiliency allows users to attain fast write performance, but replicating the
data for each copy isn't space efficient.
Parity, on the other hand, must re-compute parity for every write, causing random
write performance to suffer. Parity does, however, allow users to store their data
with greater space efficiency. For more info, see Storage Spaces fault tolerance.

Thus, mirror is predisposed to deliver performance-sensitive storage while parity offers

improved storage capacity utilization. In mirror-accelerated parity, ReFS leverages the
benefits of each resiliency type to deliver both capacity-efficient and performance-
sensitive storage by combining both resiliency schemes within a single volume.

Data rotation on mirror-accelerated parity

ReFS actively rotates data between mirror and parity, in real-time. This allows incoming
writes to be quickly written to mirror then rotated to parity to be stored efficiently. In
doing so, incoming IO is serviced quickly in mirror while cold data is stored efficiently in
parity, delivering both optimal performance and lost-cost storage within the same
volume.

To rotate data between mirror and parity, ReFS logically divides the volume into regions
of 64 MiB, which are the unit of rotation. The image below depicts a mirror-accelerated
parity volume divided into regions.

ReFS begins rotating full regions from mirror to parity once the mirror tier has reached a
specified capacity level. Instead of immediately moving data from mirror to parity, ReFS
waits and retains data in mirror as long as possible, allowing ReFS to continue delivering
optimal performance for the data (see “IO performance” below).

When data is moved from mirror to parity, the data is read, parity encodings are
computed, and then that data is written to parity. The animation below illustrates this
using a three-way mirrored region that is converted into an erasure coded region during
rotation:

IO on mirror-accelerated parity

IO behavior
Writes: ReFS services incoming writes in three distinct ways:

1. Writes to Mirror:

1a. If the incoming write modifies existing data in mirror, ReFS will modify the
data in place.
 1b. If the incoming write is a new write, and ReFS can successfully find
enough free space in mirror to service this write, ReFS will write to mirror.

2. Writes to Mirror, Reallocated from Parity:

If the incoming write modifies data that's in parity, and ReFS can successfully find
enough free space in mirror to service the incoming write, ReFS will first invalidate
the previous data in parity and then write to mirror. This invalidation is a quick and
inexpensive metadata operation that helps meaningfully improve write
performance made to parity.

3. Writes to Parity:

If ReFS cannot successfully find enough free space in mirror, ReFS will write new
data to parity or modify existing data in parity directly. The “Performance
optimizations” section below provides guidance that helps minimize writes to
parity.

Reads: ReFS will read directly from the tier containing the relevant data. If parity is
constructed with HDDs, the cache in Storage Spaces Direct will cache this data to
accelerate future reads.

７ Note

Reads never cause ReFS to rotate data back into the mirror tier.
IO performance
Writes: Each type of write described above has its own performance characteristics.
Roughly speaking, writes to the mirror tier are much faster than reallocated writes, and
reallocated writes are significantly faster than writes made directly to the parity tier. This
relationship is illustrated by the inequality below:

Mirror Tier > Reallocated Writes >> Parity Tier

Reads: There is no meaningful, negative performance impact when reading from parity:

If mirror and parity are constructed with the same media type, read performance
will be equivalent.
If mirror and parity are constructed with different media types—Mirrored SSDs,
Parity HDDs, for example—the cache in Storage Spaces Direct will help cache hot
data to accelerate any reads from parity.

ReFS compaction
Compaction for ReFS is available with Windows Server 2019 and later versions, which
substantially improves performance for mirror-accelerated parity volumes that are 90+%
full.

Background: Previously, as mirror-accelerated parity volumes became full, the

performance of these volumes could degrade. The performance degrades because hot
and cold data become mixed throughout the volume overtime. This means less hot data
can be stored in mirror since cold data occupies space in mirror that could otherwise be
used by hot data. Storing hot data in mirror is critical to maintaining high performance
because writes directly to mirror are much faster than reallocated writes and orders of
magnitude faster than writes directly to parity. Thus, having cold data in mirror is bad
for performance, as it reduces the likelihood that ReFS can make writes directly to
mirror.

ReFS compaction addresses these performance issues by freeing up space in mirror for
hot data. Compaction first consolidates all data—from both mirror and parity—into
parity. This reduces fragmentation within the volume and increases the amount of
addressable space in mirror. More importantly, this process enables ReFS to consolidate
hot data back into mirror:

When new writes come in, they will be serviced in mirror. Thus, newly written, hot
data resides in mirror.
 When a modifying write is made to data in parity, ReFS makes a reallocated write,
so this write is serviced in mirror as well. Consequently, hot data that was moved
into parity during compaction will be reallocated back into mirror.

Performance optimizations

） Important

We recommend placing write-heavy VHDs in different subdirectories. This is

because ReFS writes metadata changes at the level of a directory and its files. So if
you distribute write-heavy files across directories, metadata operations are smaller
and run in parallel, reducing latency for apps.

Performance counters
ReFS maintains performance counters to help evaluate the performance of mirror-
accelerated parity.

As described above in the Write to Parity section, ReFS will write directly to parity
when it can't find free space in mirror. Generally, this occurs when the mirrored tier
fills up faster than ReFS can rotate data to parity. In other words, ReFS rotation is
not able to keep up with the ingestion rate. The performance counters below
identify when ReFS writes directly to parity:

# Windows Server 2016

ReFS\Data allocations slow tier/sec
ReFS\Metadata allocations slow tier/sec

# Windows Server 2019

ReFS\Allocation of Data Clusters on Slow Tier/sec
ReFS\Allocation of Metadata Clusters on Slow Tier/sec

If these counters are non-zero, this indicates ReFS is not rotating data fast enough
out of mirror. To help alleviate this, one can either change the rotation
aggressiveness or increase the size of the mirrored tier.

Rotation aggressiveness
ReFS begins rotating data once mirror has reached a specified capacity threshold.
 Higher values of this rotation threshold cause ReFS to retain data in the mirror tier
longer. Leaving hot data in the mirror tier is optimal for performance, but ReFS will
not be able to effectively service large amounts of incoming IO.
Lower values enable ReFS to proactively destage data and better ingest incoming
IO. This is applicable to ingest-heavy workloads, such as archival storage. Lower
values, however, could degrade performance for general purpose workloads.
Unnecessarily rotating data out of the mirror tier carries a performance penalty.

ReFS introduces a tunable parameter to adjust this threshold, which is configurable

using a registry key. This registry key must be configured on each node in a Storage
Spaces Direct deployment, and a restart is required for any changes to take effect.

Key: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Policies
ValueName (DWORD): DataDestageSsdFillRatioThreshold
ValueType: Percentage

If this registry key is not set, ReFS will use a default value of 85%. This default value is
recommended for most deployments, and values below 50% are not recommended. The
PowerShell command below demonstrates how to set this registry key with a value of
75%:

PowerShell

Set-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Policies -Name

DataDestageSsdFillRatioThreshold -Value 75

To configure this registry key across each node in a Storage Spaces Direct deployment,
you can use the PowerShell command below:

PowerShell

$Nodes = 'S2D-01', 'S2D-02', 'S2D-03', 'S2D-04'

Invoke-Command $Nodes {Set-ItemProperty -Path
HKLM:\SYSTEM\CurrentControlSet\Policies -Name
DataDestageSsdFillRatioThreshold -Value 75}

Increasing the size of the mirrored tier

Increasing the size of the mirrored tier enables ReFS to retain a larger portion of the
working set in mirror. This improves the likelihood that ReFS can write directly to mirror,
which will help achieve better performance. The PowerShell cmdlets below demonstrate
how to increase the size of the mirrored tier:
 PowerShell

Resize-StorageTier -FriendlyName "Performance" -Size 20GB

Resize-StorageTier -InputObject (Get-StorageTier -FriendlyName
"Performance") -Size 20GB

 Tip

Make sure to resize the Partition and Volume after you resize the StorageTier. For
more information and examples, see Extend volumes.

Creating a mirror-accelerated parity volume

The PowerShell cmdlet below creates a mirror-accelerated parity volume with a
Mirror:Parity ratio of 20:80, which is the recommended configuration for most
workloads. For more information and examples, see Creating volumes in Storage Spaces
Direct.

PowerShell

New-Volume -FriendlyName "TestVolume" -FileSystem CSVFS_ReFS -

StoragePoolFriendlyName "StoragePoolName" -StorageTierFriendlyNames
Performance, Capacity -StorageTierSizes 200GB, 800GB

Additional References
ReFS overview
ReFS block cloning
ReFS integrity streams
Storage Spaces Direct overview
Block cloning on ReFS
Article • 03/29/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

Block cloning instructs the file system to copy a range of file bytes on behalf of an
application, where the destination file may be the same as, or different from, the source
file. Copy operations, unfortunately, are expensive, since they trigger expensive read and
writes to the underlying, physical data.

Block cloning in ReFS, however, performs copies as a low-cost metadata operation

rather than reading from and writing to file data. Because ReFS enables multiple files to
share the same logical clusters (physical locations on a volume), copy operations only
need to remap a region of a file to a separate physical location, converting an expensive,
physical operation to a quick, logical one. This allows copies to complete faster and
generate less I/O to the underlying storage. This improvement also benefits
virtualization workloads, as .vhdx checkpoint merge operations are dramatically
accelerated when using block clone operations. Additionally, because multiple files can
share the same logical clusters, identical data isn't physically stored multiple times,
improving storage capacity.

How it works
Block cloning on ReFS converts a file data operation into a metadata operation. In order
to make this optimization, ReFS introduces reference counts into its metadata for copied
regions. This reference count records the number of distinct file regions that reference
the same physical regions. This allows multiple files to share the same physical data:
By keeping a reference count for each logical cluster, ReFS doesn't break the isolation
between files: writes to shared regions trigger an allocate-on-write mechanism, where
ReFS allocates a new region for the incoming write. This mechanism preserves the
integrity of the shared logical clusters.

Example
Suppose there are two files, X and Y, where each file is composed of three regions, and
each region maps to separate logical clusters.

Now suppose an application issues a block clone operation from File X to File Y, for
regions A and B to be copied at the offset of region E. The following file system state
would result:

This file system state reveals a successful duplication of the block cloned region.
Because ReFS performs this copy operation by only updating VCN to LCN mappings, no
physical data was read, nor was the physical data in File Y overwritten. File X and Y now
share logical clusters, reflected by the reference counts in the table. Because no data
was physically copied, ReFS reduces capacity consumption on the volume.

Now suppose the application attempts to overwrite region A in File X. ReFS will
duplicate the shared region, update the reference counts appropriately, and perform the
incoming write to the newly duplicated region. This ensures that isolation between the
files is preserved.

After the modifying write, region B is still shared by both files. Note that if region A were
larger than a cluster, only the modified cluster would have been duplicated, and the
remaining portion would have remained shared.

Functionality restrictions and remarks

The source and destination region must begin and end at a cluster boundary.
The cloned region must be less than 4GB in length.
The maximum number of file regions that can map to the same physical region is
8175.
The destination region must not extend past the end of the file. If the application
wishes to extend the destination with cloned data, it must first call SetEndOfFile.
If the source and destination regions are in the same file, they must not overlap.
(The application may be able to proceed by splitting up the block clone operation
into multiple block clones that no longer overlap).
The source and destination files must be on the same ReFS volume.
The source and destination files must have the same Integrity Streams setting.
If the source file is sparse, the destination file must also be sparse.
The block clone operation will break Shared Opportunistic Locks (also know as
Level 2 Opportunistic Locks).
The ReFS volume must have been formatted with Windows Server 2016, and if
Failover Clustering is in use, the Clustering Functional Level must have been
Windows Server 2016 or later at format time.

Additional References
ReFS overview
ReFS integrity streams
Storage Spaces Direct overview
DUPLICATE_EXTENTS_DATA
FSCTL_DUPLICATE_EXTENTS_TO_FILE
ReFS integrity streams
Article • 03/29/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows 10

Integrity streams is an optional feature in ReFS that validates and maintains data
integrity using checksums. While ReFS always uses checksums for metadata, ReFS
doesn't, by default, generate or validate checksums for file data. Integrity streams is an
optional feature that allows users to utilize checksums for file data. When integrity
streams are enabled, ReFS can clearly determine if data is valid or corrupt. Additionally,
ReFS and Storage Spaces can jointly correct corrupt metadata and data automatically.

How it works
Integrity streams can be enabled for individual files, directories, or the entire volume,
and integrity stream settings can be toggled at any time. Additionally, integrity stream
settings for files and directories are inherited from their parent directories.

Once integrity streams is enabled, ReFS will create and maintain a checksum for the
specified file(s) in that file's metadata. This checksum allows ReFS to validate the
integrity of the data before accessing it. Before returning any data that has integrity
streams enabled, ReFS will first calculate its checksum:

Then, this checksum is compared to the checksum contained in the file metadata. If the
checksums match, then the data is marked as valid and returned to the user. If the
checksums don't match, then the data is corrupt. The resiliency of the volume
determines how ReFS responds to corruptions:

If ReFS is mounted on a non-resilient simple space or a bare drive, ReFS will return
an error to the user without returning the corrupted data.
If ReFS is mounted on a resilient mirror or parity space, ReFS will attempt to correct
the corruption.
If the attempt is successful, ReFS will apply a corrective write to restore the
integrity of the data, and it will return the valid data to the application. The
application remains unaware of any corruptions.
If the attempt is unsuccessful, ReFS will return an error.

ReFS will record all corruptions in the System Event Log, and the log will reflect whether
the corruptions were fixed.

Performance
Though integrity streams provides greater data integrity for the system, it also incurs a
performance cost. There are a couple different reasons for this:

If integrity streams are enabled, all write operations become allocate-on-write

operations. Though this avoids any read-modify-write bottlenecks since ReFS
doesn't need to read or modify any existing data, file data frequently becomes
fragmented, which delays reads.
Depending on the workload and underlying storage of the system, the
computational cost of computing and validating the checksum can cause IO
latency to increase.
Because integrity streams carries a performance cost, we recommend leaving integrity
streams disabled on performance sensitive systems.

Integrity scrubber
As described above, ReFS will automatically validate data integrity before accessing any
data. ReFS also uses a background scrubber, which enables ReFS to validate infrequently
accessed data. This scrubber periodically scans the volume, identifies latent corruptions,
and proactively triggers a repair of any corrupt data.

７ Note

The data integrity scrubber can only validate file data for files where integrity
streams is enabled.

By default, the scrubber runs every four weeks, though this interval can be configured
within Task Scheduler under Microsoft\Windows\Data Integrity Scan.

Examples
To monitor and change the file data integrity settings, ReFS uses the Get-FileIntegrity
and Set-FileIntegrity cmdlets.

Get-FileIntegrity
To see if integrity streams is enabled for file data, use the Get-FileIntegrity cmdlet.

PowerShell

PS C:\> Get-FileIntegrity -FileName 'C:\Docs\TextDocument.txt'

You can also use the Get-Item cmdlet to get the integrity stream settings for all the files
in a specified directory.

PowerShell

PS C:\> Get-Item -Path 'C:\Docs\*' | Get-FileIntegrity

Set-FileIntegrity
To enable/disable integrity streams for file data, use the Set-FileIntegrity cmdlet.

PowerShell

PS C:\> Set-FileIntegrity -FileName 'H:\Docs\TextDocument.txt' -Enable $True

You can also use the Get-Item cmdlet to set the integrity stream settings for all the files
in a specified folder.

PowerShell

PS C:\> Get-Item -Path 'H\Docs\*' | Set-FileIntegrity -Enable $True

The Set-FileIntegrity cmdlet can also be used on volumes and directories directly.

PowerShell

PS C:\> Set-FileIntegrity H:\ -Enable $True

PS C:\> Set-FileIntegrity H:\Docs -Enable $True

Additional References
ReFS overview
ReFS block cloning
Storage Spaces Direct overview
ReFSUtil
Article • 02/14/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows 10

ReFSUtil is a tool included in Windows and Windows Server that attempts to diagnose
heavily damaged ReFS volumes, identify remaining files, and copy those files to another
volume. This tool comes in the %SystemRoot%\Windows\System32 folder.

ReFS salvage is the primary function of ReFSUtil, and is useful for recovering data from
volumes that show as RAW in Disk Management. ReFS Salvage has two phases: Scan
Phase and a Copy Phase. In automatic mode, the Scan Phase and Copy Phase will run
sequentially. In manual mode, each phase can be run separately. Progress and logs are
saved in a working directory to allow phases to be run separately as well as Scan Phase
to be paused and resumed. You shouldn't need to use the ReFSutil tool unless the
volume is RAW. If read-only, then data is still accessible.

Parameters
Parameter Description

<source Specifies the ReFS volume to process. The drive letter must be formatted as "L:", or
volume> you must provide a path to the volume mount point.

<working Specifies the location to store temporary information and logs. It must not be
directory> located on the <source volume> .

<target Specifies the location where identified files are copied to. It must not be located on
directory> the <source volume> .

-m Recovers all possible files including deleted ones.

WARNING: Not only does this parameter cause the process to take longer to run,
but it can also lead to unexpected results.

-v Specifies to use verbose mode.

-x Forces the volume to dismount first, if necessary. All opened handles to the volume
are then invalid. For example, refsutil salvage -QA R: N:\WORKING N:\DATA -x .

Usage and available options

Quick automatic mode
Performs a Quick Scan Phase followed by a Copy Phase. This mode runs quicker as it
assumes some critical structures of the volume aren't corrupted and so there's no need
to scan the entire volume to locate them. This also reduces the recovery of stale
files/directories/volumes.

refsutil salvage -QA <source volume> <working directory> <target directory>

<options>

Full automatic mode

Performs a Full Scan Phase followed by a Copy Phase. This mode may take a long time
as it will scan the entire volume for any recoverable files/directories/volumes.

refsutil salvage -FA <source volume> <working directory> <target directory>

<options>

Diagnose phase (manual mode)

First, try to determine if the <source volume> is an ReFS volume and determine if the
volume is mountable. If a volume isn't mountable, the reason(s) will be provided. This is
a standalone phase.

refsutil salvage -D <source volume> <working directory> <options>

Quick Scan phase

Performs a Quick Scan of the <source volume> for any recoverable files. This mode runs
quicker as it assumes some critical structures of the volume aren't corrupted and so
there's no need to scan the entire volume to locate them. This also reduces the recovery
of stale files/directories/volumes. Discovered files are logged to the foundfiles.<volume
signature>.txt file, located in your <working directory> . If the Scan Phase was

previously stopped, running with the -QS flag again resumes the scan from where it left
off.
 refsutil salvage -QS <source volume> <working directory> <options>

Full Scan phase

Scans the entire <source volume> for any recoverable files. This mode may take a long
time as it will scan the entire volume for any recoverable files. Discovered files will be
logged to the foundfiles.<volume signature>.txt file, located in your <working
directory> . If the Scan Phase was previously stopped, running with the -FS flag again
resumes the scan from where it left off.

refsutil salvage -FS <source volume> <working directory> <options>

Copy phase
Copies all files described in the foundfiles.<volume signature>.txt file to your <target
directory> . If you stop the Scan Phase too early, it's possible that the foundfiles.

<volume signature>.txt file might not yet exist, so no file is copied to the <target
directory> .

refsutil salvage -C <source volume> <working directory> <target directory>

<options>

Copy phase with list

Copies all the files in the <file list> from the <source volume> to your <target
directory> . The files in the <file list> must have first been identified by the Scan
Phase, though the scan need not have been run to completion. The <file list> can be
generated by copying foundfiles.<volume signature>.txt to a new file, removing lines
referencing files that shouldn't be restored, and preserving files that should be restored.
The PowerShell cmdlet Select-String may be helpful in filtering foundfiles.<volume
signature>.txt to only include desired paths, extensions, or file names.
 refsutil salvage -SL <source volume> <working directory> <target directory>
<file list> <options>

Copy phase with interactive console

Advanced users can salvage files using an interactive console. This mode also requires
files generated from either of the Scan Phases.

refsutil salvage -IC <source volume> <working directory> <options>

Related links
Command-Line Syntax Key
Storage Migration Service overview
Article • 03/21/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2

Storage Migration Service makes it easier to migrate storage to Windows Server or to

Azure. It provides a graphical tool that inventories data on Windows, Linux, and NetApp
CIFS servers and then transfers the data to newer servers or to Azure virtual machines.
Storage Migration Service also provides the option to transfer the identity of a server to
the destination server so that apps and users can access their data without changing
links or paths.

This article discusses the following Storage Migration Service aspects:

Why you'd want to use Storage Migration Service.

How the migration process works.
What the requirements are for source and destination servers.
What's new in Storage Migration Service.

Why use Storage Migration Service

Use Storage Migration Service because you've got a server or multiple servers that you
want to migrate to newer hardware or virtual machines. Storage Migration Service is
designed to help by doing the following tasks:

Inventory multiple servers and their data

Rapidly transfer files, file shares, and security configuration from the source servers
Optionally take over the identity of the source servers (also known as cutting over)
so that users and apps don't have to change anything to access existing data
Manage one or multiple migrations from the Windows Admin Center user
interface
How the migration process works
Migration is a three-step process:

1. Inventory servers to gather info about their files and configuration, shown in the
following figure.

2. Transfer (copy) data from the source servers to the destination servers.

3. Cut over to the new servers (optional).

The destination servers assume the source servers' former identities so that apps
and users don't have to change anything.
The source servers enter a maintenance state where they still contain the same files
 they always have (we never remove files from the source servers) but are
unavailable to users and apps. You can then decommission the servers at your
convenience.

The following video shows how to use Storage Migration Service to take a server, such
as a Windows Server 2008 R2 server that's out of support, and move the storage to a
newer server.
https://www.youtube-nocookie.com/embed/h-Xc9j1w144

Requirements
To use Storage Migration Service, you need the following items:

A source server or failover cluster to migrate files and data from.

A destination server running Windows Server 2019 or Windows Server 2022
(clustered or standalone) to migrate to. Windows Server 2016 and Windows Server
2012 R2 work as well but are around 50% slower.
An orchestrator server running Windows Server 2019 or Windows Server 2022 to
manage the migration
If you're migrating only a few servers, and one of the servers is running Windows
Server 2019 or Windows Server 2022, you can use that as the orchestrator. If you're
migrating more servers, use a separate orchestrator server.
A PC or server running the latest Windows Admin Center to run the Storage
Migration Service user interface, along with the latest Storage Migration Service
tool (extension) available from the feed. The Windows Admin Center must be at
least version 2103.

We strongly recommend that the orchestrator and destination computers have at least
two cores or two vCPUs and at least 2 GB of memory. Inventory and transfer operations
are faster with more processors and memory.

Security requirements, the Storage Migration Service

proxy service, and firewall ports
A migration account that is an administrator on the source computers and the
orchestrator computer. This account can be a domain or local account except in
the case of a computer that isn't joined to the domain, in which case it must be a
local user.

A migration account that is an administrator on the destination computers and the

orchestrator computer. This account can be a domain or local account except in
 the case of a computer that isn't joined to the domain, in which case it must be a
local user.

The orchestrator computer must have the File and Printer Sharing (SMB-In) firewall
rule enabled inbound.

The source and destination computers must have the following firewall rules
enabled inbound (although you might already have them enabled):
File and Printer Sharing (SMB-In)
Netlogon Service (NP-In)
Windows Management Instrumentation (DCOM-In)
Windows Management Instrumentation (WMI-In)

 Tip

Installing the Storage Migration Service Proxy service on a Windows Server

2019 or Windows Server 2022 computer automatically opens the necessary
firewall ports on that computer. To do so, connect to the destination server in
Windows Admin Center and then go to Server Manager (in Windows Admin
Center) > Roles and features, select Storage Migration Service Proxy, and
then choose Install.

If the computers belong to an Active Directory Domain Services domain, they

should all belong to the same forest. The destination server must also be in the
same domain as the source server if you want to transfer the source's domain
name to the destination when cutting over. Cutover technically works across
domains, but the fully qualified domain name of the destination is different from
the source.

Requirements for source servers

The source server must run one of the following operating systems:

Windows Server, Semi-Annual Channel

Windows Server 2022
Windows Server 2019
Windows Server 2016
Windows Server 2012 R2
Windows Server 2012
Windows Server 2008 R2
Windows Server 2008
 Windows Server 2003 R2
Windows Server 2003
Windows Small Business Server 2003 R2
Windows Small Business Server 2008
Windows Small Business Server 2011
Windows Server 2012 Essentials
Windows Server 2012 R2 Essentials
Windows Server 2016 Essentials
Windows Server 2019 Essentials
Windows Storage Server 2008
Windows Storage Server 2008 R2
Windows Storage Server 2012
Windows Storage Server 2012 R2
Windows Storage Server 2016

７ Note

Windows Small Business Server and Windows Server Essentials are domain
controllers. Storage Migration Service can't yet cut over from domain controllers,
but it can inventory and transfer files from them.

You can migrate the following other source types if the orchestrator is running Windows
Server 2019 with KB5001384 installed or Windows Server 2022:

Failover clusters running Windows Server 2022, Windows Server 2019, Windows
Server 2016, Windows Server 2012 R2, Windows Server 2012, or Windows Server
2008 R2. Windows Server 2008 R2 only supports inventory and transfer, not
cutover.
Linux servers that use Samba. We've tested the following distributions:
CentOS 7
Debian GNU/Linux 8
RedHat Enterprise Linux 7.6
SUSE Linux Enterprise Server (SLES) 11 SP4
Ubuntu 16.04 LTS and 12.04.5 LTS
Samba 4.8, 4.7, 4.3, 4.2, and 3.6
NetApp FAS arrays hosting NetApp CIFS server, running NetApp ONTAP 9.

Requirements for destination servers

The destination server must run one of the following operating systems:
 Windows Server, Semi-Annual Channel
Windows Server 2022
Windows Server 2019
Windows Server 2016
Windows Server 2012 R2

The destination servers can be standalone servers or part of a Windows failover cluster.
They can't run Azure Stack HCI or use a non-Microsoft clustering add-on. While the
Storage Migration Service doesn't support Azure Files as a destination, it fully supports
servers running the Azure File Sync agent with cloud tiering.

 Tip

Destination servers running Windows Server 2022 or Windows Server 2019 have
double the transfer performance of earlier versions of Windows Server. This
performance boost is due to the inclusion of a built-in Storage Migration Service
proxy service.

Azure VM migration
Windows Admin Center integrates Azure IaaS deployment into Storage Migration
Service. It lets you avoid building new servers and VMs in the Azure portal by hand prior
to deploying your workload. It also lets you avoid possibly missing required steps and
configuration. Windows Admin Center can deploy the Azure IaaS VM, configure its
storage, join it to your domain, install roles, and then set up your distributed system.

The following video shows how to use Storage Migration Service to migrate to Azure
VMs.
https://www.youtube-nocookie.com/embed/k8Z9LuVL0xQ

If you want to lift and shift virtual machines to Azure without migrating to a later
operating system, consider using Azure Migrate. For more information, see About Azure
Migrate.

What's new in Storage Migration Service

Windows Admin Center version 1910 added the ability to deploy Azure virtual machines.
This update integrates Azure VM deployment into Storage Migration Service. For more
information, see Azure VM migration.
You can access the following post-RTM features when running the Storage Migration
Server orchestrator on Windows Server 2019 with KB5001384 installed or on Windows
Server 2022:

Migrate local users and groups to the new server.

Migrate storage from failover clusters, migrate to failover clusters, and migrate
between standalone servers and failover clusters.
Migrate storage from a Linux server that uses Samba.
Sync migrated shares more easily into Azure by using Azure File Sync.
Migrate to new networks such as Azure.
Migrate NetApp CIFS servers from NetApp FAS arrays to Windows servers and
clusters.

Related links
Use Storage Migration Service to migrate a server
Storage Migration Service frequently asked questions (FAQ)
Storage Migration Service known issues
Use Storage Migration Service to
migrate a server
Article • 03/28/2023

You can use Storage Migration Service and Windows Admin Center to migrate one
server to another, including their files and configuration. This article describes how to
migrate a Windows server, Windows Failover Cluster, Samba server, or a NetApp Fabric
Attached Storage (FAS) array to another Windows server or Windows Failover Cluster.

The migration process begins with a server inventory to identify the content to migrate,
and a firewall check to ensure successful migration. The process transfers data from your
source servers to the destination servers, and then cuts over to your new servers. After
migration, you can process the decommissioned source servers, and reissue certificates
on your new destination server.

Step 1: Install migration service and check

firewall
Before you get started, install Storage Migration Service and make sure the necessary
firewall ports are open.

1. Check the Storage Migration Service requirements and install the latest version of
Windows Admin Center on your PC or a management server if you haven't already.
You also need the latest version of the Storage Migration Service extension, which
is installed automatically by Windows Admin Center if Automatically update
extensions is enabled in Settings > Extensions. If migrating domain-joined source
computers, you must install and run the Storage Migration Service on a server
joined to the same domain or forest as the source computers.

2. In Windows Admin Center, connect to the orchestrator server running Windows

Server 2019 or Windows Server 2022. This orchestrator is the server that you install
Storage Migration Service on and use to manage the migration.

If you're migrating only one server, you can use the destination server as long
as it's running Windows Server 2019 or Windows Server 2022.
We recommend you use a separate orchestration server for any multi-server
migrations.

3. In Windows Admin Center, go to Server Manager > Storage Migration Service.

Select Install to install Storage Migration Service and its required components, as
 shown in the following figure.

4. Install the Storage Migration Service proxy on all destination servers running
Windows Server 2019 or Windows Server 2022. This setup doubles the transfer
speed when installed on destination servers.
a. Connect to the destination server in Windows Admin Center.
b. Go to Server Manager (in Windows Admin Center) > Roles and features >
Features.
c. Select Storage Migration Service Proxy, and then select Install.

5. If you intend to migrate to or from Windows Failover Clusters, install the Failover
Clustering tools on the orchestrator server. This installation happens automatically
in the latest version of Windows Admin Center when you select Migrate from
failover clusters in the Job Settings option of Inventory.
a. To install outside of the Storage Migration Service Inventory phase, connect to
the orchestrator server in Windows Admin Center.
 b. Go to Server Manager (in Windows Admin Center) > Roles and features, >
Features, > Remote Server Administration Tools, > Feature Administration
Tools.
c. Select Failover Clustering Tools, and then select Install.

７ Note

If you're migrating from a NetApp FAS array, you must manually install the
latest version of the NetApp PowerShell Toolkit onto the orchestrator. This
toolkit is available to all licensed NetApp customers with an active NetApp
support agreement from mysupport.netapp.com .

6. On all source servers and on any destination servers running Windows Server 2016
or Windows Server 2012 R2, in Windows Admin Center, connect to each server, go
to Server Manager (in Windows Admin Center) > Firewall > Incoming rules, and
then confirm the following rules are enabled:

File and Printer Sharing (SMB-In)

Netlogon Service (NP-In)
Windows Management Instrumentation (DCOM-In)
Windows Management Instrumentation (WMI-In)

If you're using third party firewalls, the inbound port ranges to open are as follows:

TCP/445 (SMB)
TCP/135 (RPC/DCOM endpoint mapper)
TCP 1025-65535 (RPC/DCOM ephemeral ports)

The Storage Migration Service ports are as follows:

TCP/28940 (Orchestrator)
TCP/28941 (Proxy).

7. If you're using an orchestrator server, and you want to download events or a data
transfer log, confirm the File and Printer Sharing (SMB-In) firewall rule is enabled
on the server.

Step 2: Create job and inventory server data

In this step, specify what servers to migrate and then scan them to collect info on their
files and configurations.
1. Select New job, name the job, and then select whether to migrate Windows
servers and clusters, Linux servers that use Samba, or NetApp FAS array. Then
select OK.

2. On the Check prerequisites page, review the prerequisites. Then select Next.

3. If you're migrating from a NetApp FAS Array, on the Select the NetApp FAS array
page, enter your NetApp FAS Array IP address, admin credential, and password.
Then select Next.

4. If you're migrating from a Windows server or cluster, on the Enter credentials

page, enter admin credentials for the servers you want to migrate from, and then
select Next.

If you're migrating from Linux servers, enter credentials on the Samba

credentials and Linux credentials pages, including an SSH password or
private key.

If you're migrating from a NetApp FAS Array, complete the following steps:
a. Use the Enter credentials and prescan NetApp page to enter admin
credentials for the NetApp Common Internet File System (CIFS) servers you
want to migrate from.
b. Select Start scan to list all the NetApp CIFS servers running on the NetApp
FAS array. You can uncheck any CIFS servers you don't want to migrate.
c. Select Next.

5. On the Install required tools page, confirm the required tools have installed
without error. Then select Next.

6. If you're migrating from a Windows server or cluster, or from Linux Samba, on the
Add and scan devices page, select Add a device, then search Active Directory for a
source server cluster. You can use an asterisk to perform wildcard partial searches.
You can also enter an exact source server name or the name of a clustered file
server. Select OK.
Repeat this step for any other servers you want to inventory. If you're migrating
from a NetApp FAS array, the Select and scan servers page already lists the source
server.

7. Select Validate and ensure validation passes for all servers.

７ Note

An error for backup privileges is expected from NetApp CIFS servers. You can
safely ignore this error.
 8. Select Start scan. The page updates to show when the scan is complete.

9. Select each server to review the inventoried shares, configuration, network

adapters, and volumes.

Storage Migration Service doesn't transfer files or folders that could interfere with
Windows operation, so you see warnings for any shares located in the Windows
system folder. You have to skip these shares during the transfer phase. For more
information, see What files and folders are excluded from transfers.

10. Select Next to move on to transferring data.

Step 3: Transfer data to destination servers

In this step you transfer data after specifying where to put it on the destination servers.

1. On the Transfer data > Enter credentials page, enter admin credentials that work
on the destination servers you want to migrate to, and then select Next.

2. On the Add a destination device and mappings page, the first source server is
listed. Enter the name of the server or clustered file server to which you want to
migrate and then select Scan device. If migrating from a domain-joined source
computer, the destination server must be joined to the same domain. You can also
select Create a new Azure VM then use the wizard to deploy a new destination
server in Azure. This function automatically sizes your VM, provisions storage,
formats disks, joins the domain, and adds the Storage Migration Service proxy to a
Windows Server 2019 or Windows Server 2022 destination. You can choose from
 Windows Server 2022 (recommended), Windows Server 2019, Windows Server
2016, and Windows Server 2012 R2 VMs of any size and use managed disks.

７ Note

Using Create a new Azure VM requires you to have the following items:

A valid Azure subscription.

An existing Azure Compute resource group where you have Create
rights.
An existing Azure Virtual Network and subnet.
An Azure ExpressRoute circuit or Azure VPN solution tied to the
Virtual Network and subnet that allows connectivity from this Azure IaaS
VM to your on-premises clients, domain controllers, the Storage
Migration Service orchestrator computer, the Windows Admin Center
computer, and the source computer to be migrated.

The following video shows how to use Storage Migration Service to migrate to
Azure VMs.
https://www.youtube-nocookie.com/embed/k8Z9LuVL0xQ

3. Map the source volumes to destination volumes, clear the Include checkbox for
any shares you don't want to transfer (including any administrative shares located
in the Windows system folder) and ensuring the Azure File Sync checkbox is set for
any volumes or shares cloud tiering with Azure File Sync, and then select Next.

７ Note

When migrating NetApp CIFS servers, source volumes don't show drive
letters. You can map these volumes to any destination volumes, and you can
map multiple NetApp CIFS volumes to the same destination volume. New
root folder paths are created to avoid any folder overwrites or collisions, and
then shares are created at the correct level. The Shares detail pane shows the
folder structure you're about to create.
 

4. Add a destination server and mappings for any more source servers, and then
select Next.

5. On the Adjust transfer settings page, specify whether to migrate local users and
groups on the source servers and then select Next. This option lets you recreate
any local users and groups on the destination servers so file or share permissions
set to local users and groups aren't lost. Here are the options when migrating local
users and groups:

） Important

If migrating NetApp CIFS servers, you cannot migrate local users and groups.

Rename accounts with the same name is selected by default and migrates all
local users and groups on the source server. If local users or groups on the
source are found with the same name on the destination, these items receive
new names on the destination. However, a built-in user or group uses the
same name on the source and destination, such as the Administrator user or
the Administrators group.
Reuse accounts with the same name maps identically named users and
groups on the source and destination. Don't use this setting if your source or
destination server is a domain controller.
Don't transfer users and groups skips migrating local users and groups,
which is required when your source or destination is a domain controller, or
when seeding data for DFS Replication (DFS Replication doesn't support local
groups and users).
 ７ Note

Migrated user accounts are disabled on the destination and assigned a 127-
character password that's both complex and random, so you'll have to enable
them and assign a new password when you're finished to keep using them.
This helps ensure any old accounts with forgotten and weak passwords on the
source don't continue to be a security problem on the destination. You might
also want to check out Local Administrator Password Solution (LAPS) as a
way to manage local Administrator passwords.

6. Select Validate and then select Next.

7. Select Start transfer to start transferring data.

The first time you transfer, we move any existing files in a destination to a backup
folder. For destination servers running Azure File Sync with cloud tiering, this
backup option isn't supported. We otherwise fully support Azure File Sync with
cloud tiering and include updated transfer information details in Windows Admin
Center. On subsequent transfers, by default we refresh the destination without
backing it up first.
Also, Storage Migration Service is smart enough to deal with overlapping shares—
we don't copy the same folders twice in the same job.

8. After the transfer completes, check out the destination server to make sure
everything transferred properly. Select Error log only if you want to download a
log of any files that didn't transfer.

７ Note

If you want to keep an audit trail of transfers or are planning to perform more
than one transfer in a job, click Transfer log or the other log save options to
save a CSV copy. Every subsequent transfer overwrites the database
information of a previous run. If you're migrating a large number of files, you
might need to adjust the timeout for saving this CSV file. For details, see
Storage Migration Service times out downloading the transfer error CSV

At this point, you have three options:

Go to the next step, cutting over so the destination servers adopt the identities of
the source servers.
Consider the migration complete without taking over the source servers'
identities.
 Transfer again, copying only files updated since the last transfer.

If your goal is to sync the files with Azure, you could set up the destination servers with
Azure File Sync after transferring files, or after cutting over to the destination servers.
See Planning for an Azure File Sync deployment.

Step 4: Cut over to new servers

In this step you cut over from the source servers to the destination servers, moving the
IP addresses and computer names to the destination servers. After this step is finished,
apps and users access the new servers via the names and addresses of the servers you
migrated from.

1. If you've navigated away from the migration job, in Windows Admin Center, go to
Server Manager > Storage Migration Service and then select the job you want to
complete.

2. On the Cut over to the new servers > Enter credentials page, select Next to use
the credentials you entered previously.

3. On the Configure cutover page, specify which network adapter on the destination
should take over the settings from each adapter on the source. This option moves
the IP address from the source to the destination as part of the cutover, giving the
source server a new DHCP or static IP address. You can skip all network migrations
or certain interfaces.

4. Specify what IP address to use for the source server after cutover moves its
address to the destination. If migrating a Windows server or cluster, or to Linux
Samba, you can use DHCP or specify a new static IP address. If using a static
address, the new subnet must be the same as the old subnet or cutover fails. If
you're migrating a NetApp FAS array, use NetApp subnets instead of DHCP.
 

5. Specify how to rename the source server after the destination server takes over its
name. You can use a randomly generated name or enter one yourself. Then select
Next.

6. On the Adjust settings page, you might need to provide new AD user credentials
with permissions to remove the source computer or clustered file server from the
domain and then add them back with a new name if your source migration
credentials don't have that permission.

7. Select Validate on the Validate source and destination device page, and then
select Next.

8. When you're ready to perform the cutover, select Start cutover.

Users and apps might experience an interruption while the address and names are
moved and the servers restarted several times. Note that users and apps aren’t
otherwise affected by the migration. The amount of time to complete the cutover
depends on how quickly the servers restart, and Active Directory and DNS
replication times.

Post-migration operations
After migrating a server or cluster, evaluate the environment for possible post migration
operations.

Create a plan for the now-decommissioned source server: The Storage Migration
Service uses the cutover process to enable a destination server to assume the
identity of a source server. The process changes the names and IP addresses of the
source server to prevent access by users and applications. It doesn't, however, turn
 off or otherwise alter the contents of the source server. You should create a plan
for decommissioning the source server. We recommend keeping the source online
for at least two weeks to allow for migration of any in-use data. The waiting period
ensures that all files can easily be retrieved without a need for offline backup
restoration. After that period, we recommend turning off the server for another
four weeks so it's still available for data retrieval but is no longer consuming
operational or power resources. After that period, perform one final full backup of
the server, then evaluate repurposing if it's a physical server, or deleting if it's a
virtual machine.
Reissue certificates on the new destination server: During the time the
destination server was online but not yet cut over, certificates might have been
issued to it through autoenrollment or other processes. Renaming a Windows
Server doesn't automatically change or reissue existing certificates, so the existing
certificates might contain the name of the server prior to it cutover. You should
examine the existing certificates on the server and reissue new ones as necessary.

Related links
Storage Migration Service overview
Storage Migration Services frequently asked questions (FAQ)
Planning for an Azure File Sync deployment
How cutover works in Storage Migration
Service
Article • 05/13/2021

Cutover is the phase of migration that moves the network identity of the source
computer to the destination computer. After cutover, the source computer will still
contain the same files as before, but it won't be available to users and apps.

Summary

Figure 1: Storage Migration Service cutover configuration

Before cutover starts, you provide the network configuration information needed to cut
over from the source computer to the destination computer. You can also choose a new
unique name for the source computer or let Storage Migration Service create a random
one.

Then, Storage Migration Service takes the following steps to cut over the source
computer to the destination computer:

1. We connect to the source and destination computers. They should both already
have the following firewall rules enabled inbound:

File and Printer Sharing (SMB-In), TCP Port 445

Netlogon Service (NP-In), TCP Port 445
Windows Management Instrumentation (DCOM-In), TCP Port 135
 Windows Management Instrumentation (WMI-In), TCP, Any Port

2. We set security permissions on the destination computer in Active Directory

Domain Services to match the source computer's permissions.

3. We create a temporary local user account on the source computer. If the computer
is domain-joined, the account username is "MsftSmsStorMigratSvc". We disable
the local account token filter policy on the source computer to allow the account
through, and then connect to the source computer. We make this temporary
account so that when we restart and remove the source computer from the
domain later, we can still access the source computer.

4. We repeat the previous step on the destination computer.

5. We remove the source computer from the domain to free up its Active Directory
account, which the destination computer will later take over.

6. We map network interfaces on the source computer and rename the source
computer.

7. We add the source computer back to the domain. The source computer now has a
new identity and is available to admins, but not to users and apps.

8. On the source computer, we remove any lingering alternate computer names,

remove the temporary local account we created, and re-enable the local account
token filter policy.

9. We remove the destination computer from the domain.

10. We replace the IP addresses on the destination computer with the IP information
provided by the source, and then rename the destination computer to the source
computer's original name.

11. We join the destination computer back to the domain. When joined, it uses the
source computer's original Active Directory computer account. This preserves
group memberships and security ACLs. The destination computer now has the
identity of the source computer.

12. On the destination computer, we remove any lingering alternate computer names,
remove the temporary local account we created, and re-enable the local account
token filter policy, completing cutover.

After cutover finishes, the destination computer has taken on the identity of the source
computer, and you can then decommission the source computer.
Detailed stages

Figure 2: Storage Migration Service showing a cutover stage description

You can keep track of cutover progress through descriptions of each stage that appear
as shown in the figure above. The following table shows each possible stage along with
its progress, description, and any clarifying notes.

Progress Description Notes

0% The cutover is idle.

2% Connecting to the Please ensure that the requirements for both source and
source computer... destination computers are fulfilled.

5% Connecting to the
destination
computer...

6% Setting security Replicates the source computer's Active Directory object

permissions on the security permissions on the destination computer.
computer object in
Active Directory...

8% Making sure that Makes sure that we can create a temporary account with the
the temporary same name.
account that we
created was
successfully deleted
on the source
computer...
Progress Description Notes

11% Creating a If the source computer is domain-joined, the temporary

temporary local account username is "MsftSmsStorMigratSvc". The password
user account on the consists of 127 random unicode wide characters with letters,
source computer... numbers, symbols, and case changes. If the source computer is
in a workgroup, we use the original source credentials.

13% Setting the local Disables the policy so that we can connect to the source when
account token filter it's not joined to the domain. Learn more about the local
policy on the account token filter policy here .
source computer...

16% Connecting to the

source computer
using the
temporary local
user account...

19% Making sure that

the temporary
account that we
created was
successfully deleted
on the destination
computer...

22% Creating a If the destination computer is domain-joined, the temporary

temporary local account username is "MsftSmsStorMigratSvc". The password
user account on the consists of 127 random unicode wide characters with letters,
destination numbers, symbols, and case changes. If the destination
computer... computer is in a workgroup, we use the original destination
credentials.

25% Setting the local Disables the policy so that we can connect to the destination
account token filter when it's not joined to the domain. Learn more about the local
policy on the account token filter policy here .
destination
computer...

27% Connecting to the

destination
computer using the
temporary local
user account...

30% Removing the

source computer
from the domain...
Progress Description Notes

31% Collecting the Applies only to Linux source computers.

source computer IP
addresses.

33% Restarting the

source computer...
(1st restart)

36% Waiting for the Likely to become unresponsive if the source computer isn't
source computer to covered by a DHCP subnet, but you selected DHCP during
respond after the network configuration.
1st restart...

38% Mapping network

interfaces on the
source computer...

41% Renaming the

source computer...

42% Restarting the Applies only to Linux source computers.

source computer...
(1st restart)

43% Restarting the Applies only to domain-joined Windows Server 2003 source
source computer... computers.
(2nd restart)

43% Waiting for the

source computer to
respond after the
1st restart...

43% Waiting for the

source computer to
respond after the
2nd restart...

44% Adding the source

computer to the
domain...

47% Restarting the

source computer...
(1st restart)

50% Restarting the

source computer...
(2nd restart)
Progress Description Notes

51% Restarting the Applies only to Windows Server 2003 source computers.
source computer...
(3rd restart)

52% Waiting for the

source computer to
respond...

52% Waiting for the

source computer to
respond after the
1st restart...

55% Waiting for the

source computer to
respond after the
2nd restart...

56% Waiting for the

source computer to
respond after the
3rd restart...

57% Removing alternate Ensures that the source is unreachable to other users and apps.
computer names For more info, see Netdom computername.
on the source...

58% Removing a
temporary local
account we created
on the source
computer...

61% Resetting the local Enables the policy.

account token filter
policy on the
source computer...

63% Removing the

destination
computer from the
domain...

66% Restarting the

destination
computer... (1st
restart)
Progress Description Notes

69% Waiting for the

destination
computer to
respond after the
1st restart...

72% Mapping network Maps each network adapter and IP address from the source
interfaces on computer onto the destination computer, replacing the
destination destination's network information.
computer...

75% Renaming the

destination
computer...

77% Adding the The destination computer takes over the old source computer's
destination Active Directory object. This can fail if the destination user isn't
computer to the a member of Domain Admins or doesn't have admin rights to
domain... the source computer Active Directory object. You can specify
alternate destination credentials in the "Enter credentials" step
before cutover starts.

80% Restarting the

destination
computer... (1st
restart)

83% Restarting the

destination
computer... (2nd
restart)

84% Waiting for the

destination
computer to
respond...

86% Waiting for the

destination
computer to
respond after the
1st restart...

88% Waiting for the

destination
computer to
respond after the
2nd restart...
 Progress Description Notes

91% Waiting for the May take a long time due to Active Directory and DNS
destination replication.
computer to
respond with the
new name...

93% Removing alternate Ensures that the destination name has been replaced.
computer names
on the destination...

94% Removing a
temporary local
account we created
on the destination
computer...

97% Resetting the local Enables the policy.

account token filter
policy on the
destination
computer...

(100%) Succeeded

FAQ

Is domain controller migration supported?

Not currently, but see the FAQ page for a workaround.

Known issues
Ensure that you have fulfilled the requirements from the Storage Migration Service
overview and installed the latest Windows update on the computer running Storage
Migration Service.

See the known issues page for more information on the following issues.

Storage Migration Service cutover validation fails with error "Access is denied
for the token filter policy on destination computer"
 Error "CLUSCTL_RESOURCE_NETNAME_REPAIR_VCO failed against netName
resource" and Windows Server 2008 R2 cluster cutover fails

Cutover hangs on "38% Mapping network interfaces on the source computer..."

when using static IPs

Cutover hangs on "38% Mapping network interfaces on the source computer..."

Additional References
Storage Migration Service overview
Migrate a file server by using Storage Migration Service
Storage Migration Services frequently asked questions (FAQ)
Storage Migration Service known issues
Storage Migration Service
frequently asked questions (FAQ)
FAQ

This topic contains answers to frequently asked questions (FAQs) about using Storage
Migration Service to migrate servers.

What files and folders are excluded

from transfers?
Storage Migration Service won't transfer files or folders that we know could interfere
with Windows operation. Specifically, here's what we won't transfer or move into the
PreExistingData folder on the destination:

Windows, Program Files, Program Files (x86), Program Data, Users

$Recycle.bin, Recycler, Recycled, System Volume Information, $UpgDrv$, $SysReset,
$Windows.~BT, $Windows.~LS, Windows.old, boot, Recovery, Documents and
Settings
pagefile.sys, hiberfil.sys, swapfile.sys, winpepge.sys, config.sys, bootsect.bak,
bootmgr, bootnxt
Any files or folders on the source server that conflicts with excluded folders on the
destination.
For example, if there's a N:\Windows folder on the source and it gets mapped to
the C:\ volume on the destination, it won't get transferred—regardless of what it
contains—because it would interfere with the C:\Windows system folder on the
destination.

Are locked files migrated?

The Storage Migration Service doesn't migrate files that applications exclusively lock.
The service does automatically retry three times with a sixty second delay between tries,
and you can control the number of attempts and the delay. You can also re-run transfers
to copy just the files that were previously skipped due to sharing violations.

Are domain migrations supported?

The Storage Migration Service doesn't allow migrating between Active Directory
domains. Migrations between servers will always join the destination server to the same
domain. You can use migration credentials from different domains in the Active
Directory forest. The Storage Migration Service does support migrating between
workgroups. You cannot migrate NetAPP CIFS instances that are not domain joined.

Are clusters supported as sources or

destinations?
The Storage Migration Service supports migrating from and to clusters after installation
of cumulative update KB4513534 or subsequent updates on Windows Server 2019
and with Windows Server 2022 out of the box. This includes migrating from a source
cluster to a destination cluster as well as migrating from a standalone source server to a
destination cluster for device consolidation purposes. You cannot, however, migrate a
cluster to a standalone server. You can migrate from Samba and NetApp CIFS servers to
clusters.

Are destinations other than Windows

Server supported?
The Storage Migration Service supports migrating to Windows Server 2022, Windows
Server 2019, and Windows Failover Clusters running those operating systems. It does
not support migration to Samba, NetApp, or Azure Files. The Storage Migration Services
does support migration to a Windows Server or cluster running Azure File Sync with
cloud tiering when using the latest version of Windows Admin Center and Windows
Server 2022, or Windows Server 2019 after installation of cumulative update
KB5006746

Do local groups and local users

migrate?
The Storage Migration Service supports migrating local users and groups after
installation of cumulative update KB4513534 or subsequent updates. It doesn't
support migrating local users and groups from NetApp CIFS servers.
Is domain controller migration
supported?
The Storage Migration Service doesn't currently migrate domain controllers in Windows
Server 2019 or Windows Server 2022. As a workaround, as long as you have more than
one domain controller in the Active Directory domain, demote the domain controller
before migrating it, then promote the destination after cut over completes. If you do
choose to migrate a domain controller source or destination, you won't be able to cut
over. You must never migrate users and groups when migrating from or to a domain
controller.

What attributes are migrated by the

Storage Migration Service?
Storage Migration Service migrates all flags, settings, and security of SMB shares. That
list of flags that Storage Migration Service migrates includes:

Share State
Availability Type
Share Type
Folder Enumeration Mode (also known as Access-Based Enumeration or ABE)
Caching Mode
Leasing Mode
Smb Instance
CA Timeout
Concurrent User Limit
Continuously Available
Description
Encrypt Data
Identity Remoting
Infrastructure
Name
Path
Scoped
Scope Name
Security Descriptor
Shadow Copy
Special
Temporary
Can I consolidate multiple servers into
one server?
The Storage Migration Service version shipped in Windows Server 2019 and Windows
Server 2022 doesn't support consolidating multiple servers into one server. An example
of consolidation would be migrating three separate source servers - which may have the
same share names and local file paths - onto a single new server that virtualized those
paths and shares to prevent any overlap or collision, then answered all three previous
servers names and IP address. You can migrate standalone servers onto multiple file
server resources on a single cluster, however.

Can I migrate from sources other than

Windows Server?
The Storage Migration Service supports migrating from Samba Linux servers after
installation of cumulative update KB4513534 or subsequent updates. See the
requirements for a list of supported Samba versions and Linux distros. The Storage
Migration Service supports migrating from NetApp FAS arrays after installation of
cumulative update KB5001384 .

Can I migrate previous file versions?

The Storage Migration Service version shipped in Windows Server 2019 and Windows
Server 2022 doesn't support migrating Previous Versions (made with the volume
shadow copy service) of files. Only the current version will migrate.

Optimizing inventory and transfer

performance
The Storage Migration Service contains a multi-threaded read and copy engine called
the Storage Migration Service Proxy service which we designed to be both fast as well as
bring along perfect data fidelity lacking in many file copy tools. While the default
configuration will be optimal for many customers, there are ways to improve SMS
performance during inventory and transfer.

Use Windows Server 2019 or Windows Server 2022 for the destination operating
system. Windows Server 2019 and Windows Server 2022 contain the Storage
Migration Service Proxy service. When you install this feature and migrate to
Windows Server 2019 or Windows Server 2022 destinations, all transfers operate as
direct line of sight between source and destination. This service runs on the
orchestrator during transfer if the destination computers are Windows Server 2012
R2 or Windows Server 2016, which means the transfers double-hop and will be
much slower. If there are multiple jobs running with Windows Server 2012 R2 or
Windows Server 2016 destinations, the orchestrator will become a bottleneck. The
latest version of Windows Admin Center automatically configures the proxy service
if not installed.

Install latest monthly Cumulative Update. We have improved the Storage

Migration Service Proxy service in several updates for better transfer and re-
transfer performance, as well as Inventory performance. Install KB4580390 October
2020 Cumulative Update or later to gain significant speed improvements, or
migrate using Windows Server 2022.

Alter default transfer threads. The Storage Migration Service Proxy service copies
8 files simultaneously in a given job. You can increase the number of simultaneous
copy threads by adjusting the following registry REG_DWORD value name in
decimal on every node running the Storage Migration Service Proxy:

HKEY_Local_Machine\Software\Microsoft\SMSProxy

FileTransferThreadCount

The valid range is 1 to 512 in Windows Server 2019 and Windows Server 2022. You
don't need to restart the service to start using this setting as long as you create a
new job. Use caution with this setting; setting it higher may require additional
cores, storage performance, and network bandwidth. Setting it too high may lead
to reduced performance compared to default settings.

Alter default parallel share threads. The Storage Migration Service Proxy service
copies from 8 shares simultaneously in a given job. You can increase the number of
simultaneous share threads by adjusting the following registry REG_DWORD value
name in decimal on the Storage Migration Service orchestrator server:

HKEY_Local_Machine\Software\Microsoft\SMS

EndpointFileTransferTaskCount

The valid range is 1 to 512 in Windows Server 2019 and Windows Server 2022. You
don't need to restart the service to start using this setting as long as you create a
new job. Use caution with this setting; setting it higher may require additional
cores, storage performance, and network bandwidth. Setting it too high may lead
to reduced performance compared to default settings.

The sum of FileTransferThreadCount and EndpointFileTransferTaskCount is how

many files the Storage Migration Service can simultaneously copy from one source
node in a job. To add more parallel source nodes, create and run more
simultaneous jobs.

Add cores and memory. We strongly recommend that the source, orchestrator,
and destination computers have at least two processor cores or two vCPUs, and
more can significantly aid inventory and transfer performance, especially when
combined with FileTransferThreadCount (above). When transferring files that are
larger than the usual Office formats (gigabytes or greater) transfer performance
will benefit from more memory than the default 2GB minimum.

Create multiple jobs. When creating a job with multiple server sources, each server
is contacted in serial fashion for inventory, transfer, and cutover. This means that
each server must complete its phase before another server starts. To run more
servers in parallel, simply create multiple jobs, with each job containing only one
server. SMS supports up to 100 simultaneously running jobs, meaning a single
orchestrator can parallelize many Windows Server 2019 and Windows Server 2022
destination computers. We do not recommend running multiple parallel jobs if
your destination computers are Windows Server 2016 or Windows Server 2012 R2
as without the SMS proxy service running on the destination, the orchestrator must
perform all transfers itself and could become a bottleneck. The ability for servers to
run in parallel inside a single job is a feature we plan to add in a later version of
SMS.

Use SMB 3 with RDMA networks. If transferring from a Windows Server 2012 or
later source computer, SMB 3.x supports SMB Direct mode and RDMA networking.
RDMA moves most CPU cost of transfer from the motherboard CPUs to onboard
NIC processors, reducing latency and server CPU utilization. In addition, RDMA
networks like ROCE and iWARP typically have substantially higher bandwidth than
typical TCP/ethernet, including 25, 50, and 100Gb speeds per interface. Using SMB
Direct typically moves the transfer speed limit from the network down to the
storage itself.

Use SMB 3 multichannel. If transferring from a Windows Server 2012 or later

source computer, SMB 3.x supports multichannel copies that can greatly improve
file copy performance. This feature works automatically as long as the source and
destination both have:
Multiple network adapters
 One or more network adapters that support Receive Side Scaling (RSS)
One of more network adapters that are configured by using NIC Teaming
One or more network adapters that support RDMA

Update drivers. As appropriate, install latest vendor storage and enclosure

firmware and drivers, latest vendor HBA drivers, latest vendor BIOS/UEFI firmware,
latest vendor network drivers, and latest motherboard chipset drivers on source,
destination, and orchestrator servers. Restart nodes as needed. Consult your
hardware vendor documentation for configuring shared storage and networking
hardware.

Enable high-performance processing. Ensure that BIOS/UEFI settings for servers

enable high performance, such as disabling C-State, setting QPI speed, enabling
NUMA, and setting highest memory frequency. Ensure power management in
Windows Server is set to High Performance. Restart as required. Don't forget to
return these to appropriate states after completing migration.

Tune hardware Review the Performance Tuning Guidelines for Windows Server
2016 for tuning the orchestrator and destination computers running Windows
Server 2022, Windows Server 2019, or Windows Server 2016. The Network
Subsystem Performance Tuning section contains especially valuable information.
There is an updated guide for Windows Server 2022 named (Performance Tuning
Guidelines for Windows Server 2022)[/windows-
server/administration/performance-tuning/].

Use faster storage. While it may be difficult to upgrade the source computer
storage speed, you should ensure the destination storage is at least as fast at write
IO performance as the source is at read IO performance in order to ensure there is
no unnecessary bottleneck in transfers. If the destination is a VM, ensure that, at
least for the purposes of migration, it runs in the fastest storage layer of your
hypervisor hosts, such as on the flash tier or with Storage Spaces Direct HCI
clusters utilizing mirrored all-flash or hybrid spaces. When the SMS migration is
complete the VM can be live migrated to a slower tier or host.

Use SMB compression. If your source and destination servers are Windows Server
2022, you can enable SMB compression to gain significant performance gains on
larger files. Review (SMB compression)[/windows-server/storage/file-server/smb-
compression].

Update antivirus. Always ensure your source and destination are running the latest
patched version of antivirus software to ensure minimal performance overhead. As
a test, you can temporarily exclude scanning of folders you're inventorying or
migrating on the source and destination servers. If your transfer performance is
 improved, contact your antivirus software vendor for instructions or for an updated
version of the antivirus software or an explanation of expected performance
degradation.

Can I migrate from NTFS to ReFS?

The Storage Migration Service version shipped in Windows Server 2019 and Windows
Server 2022 doesn't support migrating from the NTFS to ReFS file systems. You can
migrate from NTFS to NTFS, and ReFS to ReFS. This is by design, due to the many
differences in functionality, metadata, and other aspects that ReFS doesn't duplicate
from NTFS. ReFS is intended as an application workload file system, not a general file
system. For more information, see Resilient File System (ReFS) overview

Can I move the Storage Migration

Service database?
The Storage Migration Service uses an extensible storage engine (ESE) database that is
installed by default in the hidden c:\programdata\microsoft\storagemigrationservice
folder. This database will grow as jobs are added and transfers are completed, and can
consume significant drive space after migrating millions of files if you do not delete
jobs. If the database needs to move, perform the following steps:

1. Stop the "Storage Migration Service" service on the orchestrator computer.

2. Take ownership of the %programdata%/Microsoft/StorageMigrationService folder

3. Add your user account to have full control over that share and all of its files and
subfolders.

4. Move the folder to another drive on the orchestrator computer.

5. Set the following registry REG_SZ value:

HKEY_Local_Machine\Software\Microsoft\SMS DatabasePath = path to the new

database folder on a different volume

6. Ensure that SYSTEM has full control to all files and subfolders of that folder

7. Remove your own accounts permissions.

8. Start the "Storage Migration Service" service.

Does the Storage Migration Service
migrate locally installed applications
from the source computer?
No, the Storage Migration Service doesn't migrate locally installed applications. After
you complete migration, re-install any applications onto the destination computer that
were running on the source computer. There's no need to reconfigure any users or their
applications; the Storage Migration Service is designed to make the server change
invisible to clients.

What happens with existing files on the

destination server?
When performing a transfer, the Storage Migration Service seeks to mirror data from the
source server. The destination server should not contain any production data or
connected users, as that data could be overwritten. By default, the first transfer makes a
backup copy of any data on the destination server as a safeguard. On all subsequent
transfers, by default, the Storage Migration Service will mirror data onto the destination;
this means not only adding new files, but also arbitrarily overwriting any existing files
and deleting any files not present on the source. This behavior is intentional and
provides perfect fidelity with the source computer.

What do the error numbers mean in the

transfer CSV?
Most errors found in the transfer CSV file are Windows System Error Codes. You can find
out what each error means by reviewing the Win32 error codes documentation.

Are existing certificates updated on the

destination server during cutover?
A destination server may contain certificates - issued prior to cutover - in its local
certificate store, with the name of the server being part of the subject, subject
alternative name, or other fields. When cutover occurs and the server is renamed, these
certificates are not updated. You must reissue certificates to your newly renamed servers
using your current deployment methods, such as Group Policy or web enrollment.
What are my options to give feedback,
file bugs, or get support?
To give feedback on the Storage Migration Service:

Use the Feedback Hub tool included in Windows 10, clicking "Suggest a Feature",
and specifying the category of "Windows Server" and subcategory of "Storage
Migration"
Email [email protected]

To file bugs:

Use the Feedback Hub tool included in Windows 10, clicking "Report a Problem",
and specifying the category of "Windows Server" and subcategory of "Storage
Migration"
Open a support case via Microsoft Support

To get support:

Post a question on the Windows Server Tech Community

Post on the Windows Server 2019 forum
Open a support case via Microsoft Support

Additional references
Storage Migration Service overview
Storage Migration Service known issues
Article • 04/24/2023

This topic contains answers to known issues when using Storage Migration Service to
migrate servers.

Storage Migration Service is released in two parts: the service in Windows Server, and
the user interface in Windows Admin Center. The service is available in Windows Server,
Long-Term Servicing Channel, as well as Windows Server, Semi-Annual Channel; while
Windows Admin Center is available as a separate download. We also periodically include
changes in cumulative updates for Windows Server, released via Windows Update.

For example, Windows Server, version 1903 includes new features and fixes for Storage
Migration Service, which are also available for Windows Server 2019 and Windows
Server, version 1809 by installing KB4512534 .

How to collect log files when working with

Microsoft Support
The Storage Migration Service contains event logs for the Orchestrator service and the
Proxy Service. The orchestrator server always contains both event logs, and destination
servers with the proxy service installed contain the proxy logs. These logs are located
under:

Application and Services Logs \ Microsoft \ Windows \ StorageMigrationService

Application and Services Logs \ Microsoft \ Windows \ StorageMigrationService-
Proxy

If you need to gather these logs for offline viewing or to send to Microsoft Support,
there's an open-source PowerShell script available on GitHub:

Storage Migration Service Helper

Review the README for usage.

Storage Migration Service doesn't show up in

Windows Admin Center unless managing
Windows Server 2019
When using the 1809 version of Windows Admin Center to manage a Windows Server
2019 orchestrator, you don't see the tool option for Storage Migration Service.

The Windows Admin Center Storage Migration Service extension is version-bound to

only manage Windows Server 2019 version 1809 or later operating systems. If you use it
to manage older Windows Server operating systems or insider previews, the tool will not
appear. This behavior is by design.

To resolve, use or upgrade to Windows Server 2019 build 1809 or later.

Storage Migration Service cutover validation

fails with error "Access is denied for the token
filter policy on destination computer"
When running cutover validation, you receive error "Fail: Access is denied for the token
filter policy on destination computer." This occurs even if you provided correct local
administrator credentials for both the source and destination computers.

This issue was fixed in the KB4512534 update.

Storage Migration Service isn't included in

Windows Server 2019 Evaluation or Windows
Server 2019 Essentials edition
When using Windows Admin Center to connect to a Windows Server 2019 Evaluation
release or Windows Server 2019 Essentials edition, there isn't an option to manage
the Storage Migration Service. Storage Migration Service also isn't included in Roles and
Features.

This issue is caused by a servicing issue in the Evaluation media of Windows Server 2019
and Windows Server 2019 Essentials.

To work around this issue for evaluation, install a retail, MSDN, OEM, or Volume License
version of Windows Server 2019 and don't activate it. Without activation, all editions of
Windows Server operate in evaluation mode for 180 days.

We've fixed this issue in a later release of Windows Server.

Storage Migration Service times out
downloading the transfer error CSV
When using Windows Admin Center or PowerShell to download the transfer operations
detailed errors-only CSV log, you receive error:

Transfer Log - Please check file sharing is allowed in your firewall. : This
request operation sent to net.tcp://localhost:28940/sms/service/1/transfer
did not receive a reply within the configured timeout (00:01:00). The time
allotted to this operation may have been a portion of a longer timeout. This
may be because the service is still processing the operation or because the
service was unable to send a reply message. Please consider increasing the
operation timeout (by casting the channel/proxy to IContextChannel and
setting the OperationTimeout property) and ensure that the service is able
to connect to the client.

This issue is caused by an extremely large number of transferred files that can't be
filtered in the default one minute timeout allowed by Storage Migration Service.

To work around this issue:

1. On the orchestrator computer, edit the

%SYSTEMROOT%\SMS\Microsoft.StorageMigration.Service.exe.config file using
Notepad.exe to change the "sendTimeout" from its 1 minute default to 10 minutes

<bindings>
<netTcpBinding>
<binding name="NetTcpBindingSms"
sendTimeout="00:10:00"

2. Restart the "Storage Migration Service" service on the orchestrator computer.

3. On the orchestrator computer, start Regedit.exe

4. Create the following registry subkey if it doesn't already exist:

HKEY_LOCAL_MACHINE\Software\Microsoft\SMSPowershell

5. On the Edit menu, point to New, and then click DWORD Value.

6. Type "WcfOperationTimeoutInMinutes" for the name of the DWORD, and then

press ENTER.
 7. Right-click "WcfOperationTimeoutInMinutes", and then click Modify.

8. In the Base data box, click "Decimal"

9. In the Value data box, type "10", and then click OK.

10. Exit Registry Editor.

11. Attempt to download the errors-only CSV file again.

You may need to increase this timeout to more than 10 minutes if you are migrating an
extremely large number of files.

Validation warnings for destination proxy and

credential administrative privileges
When validating a transfer job, you see the following warnings:

The credential has administrative privileges.

Warning: Action isn't available remotely.
The destination proxy is registered.
Warning: The destination proxy wasn't found.

If you have not installed the Storage Migration Service Proxy service on the Windows
Server 2019 destination computer, or the destination computer is Windows Server 2016
or Windows Server 2012 R2, this behavior is by design. We recommend migrating to a
Windows Server 2019 computer with the proxy installed for significantly improved
transfer performance.

Certain files don't inventory or transfer, error 5

"Access is denied"
When inventorying or transferring files from source to destination computers, files from
which a user has removed permissions for the Administrators group fail to migrate.
Examining the Storage Migration Service-Proxy debug shows:

Log Name: Microsoft-Windows-StorageMigrationService-Proxy/Debug

Source: Microsoft-Windows-StorageMigrationService-Proxy
Date: 2/26/2019 9:00:04 AM
 Event ID: 10000
Task Category: None
Level: Error
Keywords:
User: NETWORK SERVICE
Computer: srv1.contoso.com
Description:

02/26/2019-09:00:04.860 [Error] Transfer error for

\\srv1.contoso.com\public\indy.png: (5) Access is denied.
Stack Trace:
at
Microsoft.StorageMigration.Proxy.Service.Transfer.FileDirUtils.OpenFile(Stri
ng fileName, DesiredAccess desiredAccess, ShareMode shareMode,
CreationDisposition creationDisposition, FlagsAndAttributes
flagsAndAttributes)
at
Microsoft.StorageMigration.Proxy.Service.Transfer.FileDirUtils.GetTargetFile
(String path)
at
Microsoft.StorageMigration.Proxy.Service.Transfer.FileDirUtils.GetTargetFile
(FileInfo file)
at
Microsoft.StorageMigration.Proxy.Service.Transfer.FileTransfer.InitializeSou
rceFileInfo()
at
Microsoft.StorageMigration.Proxy.Service.Transfer.FileTransfer.Transfer()
at
Microsoft.StorageMigration.Proxy.Service.Transfer.FileTransfer.TryTransfer()

This issue is caused by a code defect in the Storage Migration Service where the backup
privilege was not being invoked.

To resolve this issue, install Windows Update April 2, 2019—KB4490481 (OS Build
17763.404) on the orchestrator computer and the destination computer if the proxy
service is installed there. Ensure that the source migration user account is a local
administrator on the source computer and the Storage Migration Service orchestrator.
Ensure that the destination migration user account is a local administrator on the
destination computer and the Storage Migration Service orchestrator.

DFSR hashes mismatch when using Storage

Migration Service to preseed data
When using the Storage Migration Service to transfer files to a new destination, then
configuring DFS Replication to replicate that data with an existing server through
preseeded replication or DFS Replication database cloning, all files experience a hash
mismatch and are re-replicated. The data streams, security streams, sizes, and attributes
all appear to be perfectly matched after using Storage Migration Service to transfer
them. Examining the files with ICACLS or the DFS Replication database cloning debug
log reveals:

Source file

icacls d:\test\Source:

icacls d:\test\thatcher.png /save out.txt /t thatcher.png

D:AI(A;;FA;;;BA)(A;;0x1200a9;;;DD)(A;;0x1301bf;;;DU)(A;ID;FA;;;BA)
(A;ID;FA;;;SY)(A;ID;0x1200a9;;;BU)

Destination file

icacls d:\test\thatcher.png /save out.txt /t thatcher.png

D:AI(A;;FA;;;BA)(A;;0x1301bf;;;DU)(A;;0x1200a9;;;DD)(A;ID;FA;;;BA)
(A;ID;FA;;;SY)(A;ID;0x1200a9;;;BU)**S:PAINO_ACCESS_CONTROL**

DFSR Debug Log

20190308 10:18:53.116 3948 DBCL 4045 [WARN] DBClone::IDTableImportUpdate

Mismatch record was found.

Local ACL hash:1BCDFE03-A18BCE01-D1AE9859-23A0A5F6

LastWriteTime:20190308 18:09:44.876
FileSizeLow:1131654
FileSizeHigh:0
Attributes:32

Clone ACL hash:**DDC4FCE4-DDF329C4-977CED6D-F4D72A5B**

LastWriteTime:20190308 18:09:44.876
FileSizeLow:1131654
FileSizeHigh:0
Attributes:32

This issue is fixed by the KB4512534 update.

Error "Couldn't transfer storage on any of the
endpoints" when transferring from Windows
Server 2008 R2
When attempting to transfer data from a Windows Server 2008 R2 source computer, no
data transfers and you receive error:

Couldn't transfer storage on any of the endpoints.

0x9044

This error is expected if your Windows Server 2008 R2 computer isn't fully patched with
all Critical and Important updates from Windows Update. It's especially important to
keep a Windows Server 2008 R2 computer updated for security purposes, as that
operating system doesn't contain the security improvements of newer versions of
Windows Server.

Error "Couldn't transfer storage on any of the

endpoints" and "Check if the source device is
online - we couldn't access it."
When attempting to transfer data from a source computer, some or all shares don't
transfer, with the error:

Couldn't transfer storage on any of the endpoints.

0x9044

Examining the SMB transfer details shows error:

Check if the source device is online - we couldn't access it.

Examining the StorageMigrationService/Admin event log shows:

 Couldn't transfer storage.

Job: Job1
ID:
State: Failed
Error: 36931
Error Message:

Guidance: Check the detailed error and make sure the transfer requirements
are met. The transfer job couldn't transfer any source and destination
computers. This could be because the orchestrator computer couldn't reach
any source or destination computers, possibly due to a firewall rule, or
missing permissions.

Examining the StorageMigrationService-Proxy/Debug log shows:

07/02/2019-13:35:57.231 [Error] Transfer validation failed. ErrorCode:

40961, Source endpoint is not reachable, or doesn't exist, or source
credentials are invalid, or authenticated user doesn't have sufficient
permissions to access it.
at
Microsoft.StorageMigration.Proxy.Service.Transfer.TransferOperation.Validate
()
at
Microsoft.StorageMigration.Proxy.Service.Transfer.TransferRequestHandler.Pro
cessRequest(FileTransferRequest fileTransferRequest, Guid operationId)

This was a code defect that would manifest if your migration account does not have at
least Read permissions to the SMB shares. This issue was first fixed in cumulative update
4520062 .

Another possible cause might be insufficient access rights to the source file server.
While examining the "Microsoft.StorageMigration.Proxy.Service.exe" process with
Process Monitor you might see the below result:

Date: 6/04/2022 15:36:09,1943419

Thread: 1688
Class: File System
Operation: CreateFile
Result: PRIVILEGE_NOT_HELD
Path: \\srv1.contoso.com\F$\\public
Duration: 0.0002573

Desired Access: Read Attributes, Read Control, Synchronize, Access System

Security
 Disposition: Open
Options: Synchronous IO Non-Alert, Open For Backup
Attributes: N
ShareMode: Read, Write
AllocationSize: n/a
Impersonating: CONTOSO\ServiceAccount
OpenResult: PRIVILEGE_NOT_HELD

The actual operation being performed needs the "Open For Backup"-privileges on the
source file server. Verify that your user account used to access the source file server is
granted the necessary permissions via the following Local Security Policy on this server
or using a Group Policy Object: Security Settings > Local Policies > User Rights
Assignment > Back up files and directories

Error 0x80005000 when running inventory

After installing KB4512534 and attempting to run inventory, inventory fails with errors:

EXCEPTION FROM HRESULT: 0x80005000

Log Name: Microsoft-Windows-StorageMigrationService/Admin

Source: Microsoft-Windows-StorageMigrationService
Date: 9/9/2019 5:21:42 PM
Event ID: 2503
Task Category: None
Level: Error
Keywords:
User: NETWORK SERVICE
Computer: FS02.TailwindTraders.net
Description:
Couldn't inventory the computers.
Job: foo2
ID: 20ac3f75-4945-41d1-9a79-d11dbb57798b
State: Failed
Error: 36934
Error Message: Inventory failed for all devices
Guidance: Check the detailed error and make sure the inventory requirements
are met. The job couldn't inventory any of the specified source computers.
This could be because the orchestrator computer couldn't reach it over the
network, possibly due to a firewall rule or missing permissions.

Log Name: Microsoft-Windows-StorageMigrationService/Admin

Source: Microsoft-Windows-StorageMigrationService
Date: 9/9/2019 5:21:42 PM
Event ID: 2509
Task Category: None
Level: Error
Keywords:
 User: NETWORK SERVICE
Computer: FS02.TailwindTraders.net
Description:
Couldn't inventory a computer.
Job: foo2
Computer: FS01.TailwindTraders.net
State: Failed
Error: -2147463168
Error Message:
Guidance: Check the detailed error and make sure the inventory requirements
are met. The inventory couldn't determine any aspects of the specified
source computer. This could be because of missing permissions or privileges
on the source or a blocked firewall port.

Log Name: Microsoft-Windows-StorageMigrationService-Proxy/Debug

Source: Microsoft-Windows-StorageMigrationService-Proxy
Date: 2/14/2020 1:18:21 PM
Event ID: 10000
Task Category: None
Level: Error
Keywords:
User: NETWORK SERVICE
Computer: 2019-rtm-orc.ned.contoso.com
Description:
02/14/2020-13:18:21.097 [Erro] Failed device discovery stage SystemInfo with
error: (0x80005000) Unknown error (0x80005000)

This error is caused by a code defect in Storage Migration Service when you provide
migration credentials in the form of a User Principal Name (UPN), such as
'[email protected]'. The Storage Migration Service orchestrator service fails to
parse this format correctly, which leads to a failure in a domain lookup that was added
for cluster migration support in KB4512534 and 19H1.

To work around this issue, provide credentials in the domain\user format, such as
'Contoso\Meghan'.

Error "ServiceError0x9006" or "The proxy isn't

currently available." when migrating to a
Windows Server failover cluster
When attempting to transfer data against a clustered File Server, you receive errors such
as:

Make sure the proxy service is installed and running, and then try again.
The proxy isn't currently available.
 0x9006
ServiceError0x9006,Microsoft.StorageMigration.Commands.UnregisterSmsProxyCom
mand

This error is expected if the File Server resource moved from its original Windows Server
2019 cluster owner node to a new node and the Storage Migration Service Proxy feature
wasn't installed on that node.

As a workaround, move the destination File Server resource back to the original owner
cluster node that was in use when you first configured transfer pairings.

As an alternative workaround:

1. Install the Storage Migration Service Proxy feature on all nodes in a cluster.

2. Run the following Storage Migration Service PowerShell command on the

orchestrator computer:

PowerShell

Register-SMSProxy -ComputerName <destination server> -Force

Error "Dll was not found" when running

inventory from a cluster node
When attempting to run inventory with the Storage Migration Service and targeting a
Windows Server failover cluster general use file server source, you receive the following
errors:

DLL not found

[Error] Failed device discovery stage VolumeInfo with error: (0x80131524)
Unable to load DLL 'Microsoft.FailoverClusters.FrameworkSupport.dll': The
specified module could not be found. (Exception from HRESULT: 0x8007007E)

To work around this issue, install the "Failover Cluster Management Tools" (RSAT-
Clustering-Mgmt) on the server running the Storage Migration Service orchestrator.

Error "There are no more endpoints available

from the endpoint mapper" when running
inventory against a Windows Server 2003
source computer
When attempting to run inventory with the Storage Migration Service orchestrator
against a Windows Server 2003 source computer, you receive the following error:

There are no more endpoints available from the endpoint mapper

This issue is resolved by the KB4537818 update.

Uninstalling a cumulative update prevents

Storage Migration Service from starting
Uninstalling Windows Server cumulative updates may prevent the Storage Migration
Service from starting. To resolve this issue, you can back up and delete the Storage
Migration Service database:

1. Open an elevated cmd prompt, where you are a member of Administrators on the
Storage Migration Service orchestrator server, and run:

DOS

TAKEOWN /d y /a /r /f c:\ProgramData\Microsoft\StorageMigrationService

MD c:\ProgramData\Microsoft\StorageMigrationService\backup

ICACLS c:\ProgramData\Microsoft\StorageMigrationService\* /grant

Administrators:(GA)

XCOPY c:\ProgramData\Microsoft\StorageMigrationService\* .\backup\*

DEL c:\ProgramData\Microsoft\StorageMigrationService\* /q

ICACLS c:\ProgramData\Microsoft\StorageMigrationService /GRANT

networkservice:F /T /C

ICACLS c:\ProgramData\Microsoft\StorageMigrationService /GRANT

networkservice:(GA) /T /C

2. Start the Storage Migration Service service, which will create a new database.
Error
"CLUSCTL_RESOURCE_NETNAME_REPAIR_VCO
failed against netName resource" and Windows
Server 2008 R2 cluster cutover fails
When attempting to run cut over of a Windows Server 2008 R2 cluster source, the cut
over gets stuck at phase "Renaming the source computer..." and you receive the
following error:

Log Name: Microsoft-Windows-StorageMigrationService-Proxy/Debug

Source: Microsoft-Windows-StorageMigrationService-Proxy
Date: 10/17/2019 6:44:48 PM
Event ID: 10000
Task Category: None
Level: Error
Keywords:
User: NETWORK SERVICE
Computer: WIN-RNS0D0PMPJH.contoso.com
Description:
10/17/2019-18:44:48.727 [Erro] Exception error: 0x1. Message: Control code
CLUSCTL_RESOURCE_NETNAME_REPAIR_VCO failed against netName resource
2008r2FS., stackTrace: at
Microsoft.FailoverClusters.Framework.ClusterUtils.NetnameRepairVCO(SafeClust
erResourceHandle netNameResourceHandle, String netName)
at
Microsoft.FailoverClusters.Framework.ClusterUtils.RenameFSNetName(SafeCluste
rHandle ClusterHandle, String clusterName, String FsResourceId, String
NetNameResourceId, String newDnsName, CancellationToken ct)
at
Microsoft.StorageMigration.Proxy.Cutover.CutoverUtils.RenameFSNetName(Networ
kCredential networkCredential, Boolean isLocal, String clusterName, String
fsResourceId, String nnResourceId, String newDnsName, CancellationToken ct)
[d:\os\src\base\dms\proxy\cutover\cutoverproxy\CutoverUtils.cs::RenameFSNetN
ame::1510]

This issue is caused by a missing API in older versions of Windows Server. Currently
there's no way to migrate Windows Server 2008 and Windows Server 2003 clusters. You
can perform inventory and transfer without issue on Windows Server 2008 R2 clusters,
then manually perform cutover by manually changing the cluster's source file server
resource netname and IP address, then changing the destination cluster netname and IP
address to match the original source.
Cutover hangs on "38% Mapping network
interfaces on the source computer..." when
using static IPs
When attempting to run cut over of a source computer, having set the source computer
to use a new static (not DHCP) IP address on one or more network interfaces, the cut
over gets stuck at phase "38% Mapping network interfaces on the source computer..."
and you receive the following error in the Storage Migration Service event log:

Log Name: Microsoft-Windows-StorageMigrationService-Proxy/Admin

Source: Microsoft-Windows-StorageMigrationService-Proxy
Date: 11/13/2019 3:47:06 PM
Event ID: 20494
Task Category: None
Level: Error
Keywords:
User: NETWORK SERVICE
Computer: orc2019-rtm.corp.contoso.com
Description:
Couldn't set the IP address on the network adapter.

Computer: fs12.corp.contoso.com
Adapter: microsoft hyper-v network adapter
IP address: 10.0.0.99
Network mask: 16
Error: 40970
Error Message: Unknown error (0xa00a)

Guidance: Confirm that the Netlogon service on the computer is reachable

through RPC and that the credentials provided are correct.

Examining the source computer shows that the original IP address fails to change.

This issue does not happen if you selected "Use DHCP" on the Windows Admin Center
"configure cutover" screen, only if you specify a new static IP address.

There are two solutions for this issue:

1. This issue was first resolved by the KB4537818 update. That earlier code defect
prevented all use of static IP addresses.

2. If you have not specified a default gateway IP address on the source computer's
network interfaces, this issue will occur even with the KB4537818 update. To work
around this issue, set a valid default IP address on the network interfaces using the
Network Connections applet (NCPA.CPL) or Set-NetRoute Powershell cmdlet.
Slower than expected re-transfer performance
After completing a transfer, then running a subsequent re-transfer of the same data, you
may not see much improvement in transfer time even when little data has changed in
the meantime on the source server.

This issue is resolved by kb4580390 . To tune performance further, review Optimizing

Inventory and Transfer Performance.

Slower than expected inventory performance

While inventorying a source server, you find the file inventory taking a very long time
when there are many files or nested folders. Millions of files and folders may lead to
inventories taking many hours even on fast storage configurations.

This issue is resolved by kb4580390 .

Data does not transfer, user renamed when

migrating to or from a domain controller
After starting the transfer from or to a domain controller:

1. No data is migrated and no shares are created on the destination.

2. There's a red error symbol shown in Windows Admin Center with no error message

3. One or more AD users and Domain Local groups have their name and/or pre-
Windows 2000 logon attribute changed

4. You see event 3509 on the Storage Migration Service orchestrator:

Log Name: Microsoft-Windows-StorageMigrationService/Admin

Source: Microsoft-Windows-StorageMigrationService
Date: 1/10/2020 2:53:48 PM
Event ID: 3509
Task Category: None
Level: Error
Keywords:
User: NETWORK SERVICE
Computer: orc2019-rtm.corp.contoso.com
Description:
Couldn't transfer storage for a computer.
 Job: dctest3
Computer: dc02-2019.corp.contoso.com
Destination Computer: dc03-2019.corp.contoso.com
State: Failed
Error: 53251
Error Message: Local accounts migration failed with error
System.Exception: -2147467259
at
Microsoft.StorageMigration.Service.DeviceHelper.MigrateSecurity(IDevice
Record sourceDeviceRecord, IDeviceRecord destinationDeviceRecord,
TransferConfiguration config, Guid proxyId, CancellationToken
cancelToken)

This is expected behavior if you attempted to migrate from or to a domain

controller with Storage Migration Service and used the "migrate users and groups"
option to rename or reuse accounts. instead of selecting "Don't transfer users and
groups". DC migration is not supported with Storage Migration Service. Because a
DC doesn't have true local users and groups, Storage Migration Service treats
these security principals as it would when migrating between two member servers
and attempts to adjust ACLs as instructed, leading to the errors and mangled or
copied accounts.

If you have already run transfer one ore more times:

1. Use the following AD PowerShell command against a DC to locate any modified

users or groups (changing SearchBase to match your domain distinguished name):

PowerShell

Get-ADObject -Filter 'Description -like "*storage migration service

renamed*"' -SearchBase 'DC=<domain>,DC=<TLD>' | ft
name,distinguishedname

2. For any users returned with their original name, edit their "User Logon Name (pre-
Windows 2000)" to remove the random character suffix added by Storage
Migration Service, so that this user can log on.

3. For any groups returned with their original name, edit their "Group Name (pre-
Windows 2000)" to remove the random character suffix added by Storage
Migration Service.

4. For any disabled users or groups with names that now contain a suffix added by
Storage Migration Service, you can delete these accounts. You can confirm that
user accounts were added later because they will only contain the Domain Users
group and will have a created date/time matching the Storage Migration Service
transfer start time.
 If you wish to use Storage Migration Service with domain controllers for transfer
purposes, ensure you always select "Don't transfer users and groups" on the
transfer settings page in Windows Admin Center.

Error 53, "failed to inventory all specified

devices" when running inventory,
When attempting to run inventory, you receive:

Failed to inventory all specified devices

Log Name: Microsoft-Windows-StorageMigrationService/Admin

Source: Microsoft-Windows-StorageMigrationService
Date: 1/16/2020 8:31:17 AM
Event ID: 2516
Task Category: None
Level: Error
Keywords:
User: NETWORK SERVICE
Computer: ned.corp.contoso.com
Description:
Couldn't inventory files on the specified endpoint.
Job: ned1
Computer: ned.corp.contoso.com
Endpoint: hithere
State: Failed
File Count: 0
File Size in KB: 0
Error: 53
Error Message: Endpoint scan failed
Guidance: Check the detailed error and make sure the inventory requirements
are met. This could be because of missing permissions on the source
computer.

Log Name: Microsoft-Windows-StorageMigrationService-Proxy/Debug

Source: Microsoft-Windows-StorageMigrationService-Proxy
Date: 1/16/2020 8:31:17 AM
Event ID: 10004
Task Category: None
Level: Critical
Keywords:
User: NETWORK SERVICE
Computer: ned.corp.contoso.com
Description:
01/16/2020-08:31:17.031 [Crit] Consumer Task failed with error:The network
path was not found.
. StackTrace= at Microsoft.Win32.RegistryKey.Win32ErrorStatic(Int32
errorCode, String str)
 at Microsoft.Win32.RegistryKey.OpenRemoteBaseKey(RegistryHive hKey,
String machineName, RegistryView view)
at
Microsoft.StorageMigration.Proxy.Service.Transfer.FileDirUtils.GetEnvironmen
tPathFolders(String ServerName, Boolean IsServerLocal)
at Microsoft.StorageMigration.Proxy.Service.Discovery.ScanUtils.
<ScanSMBEndpoint>d__3.MoveNext()
at Microsoft.StorageMigration.Proxy.EndpointScanOperation.Run()
at
Microsoft.StorageMigration.Proxy.Service.Discovery.EndpointScanRequestHandle
r.ProcessRequest(EndpointScanRequest scanRequest, Guid operationId)
at
Microsoft.StorageMigration.Proxy.Service.Discovery.EndpointScanRequestHandle
r.ProcessRequest(Object request)
at
Microsoft.StorageMigration.Proxy.Common.ProducerConsumerManager`3.Consume(Ca
ncellationToken token)

01/16/2020-08:31:10.015 [Erro] Endpoint Scan failed. Error: (53) The network

path was not found.
Stack trace:
at Microsoft.Win32.RegistryKey.Win32ErrorStatic(Int32 errorCode, String
str)
at Microsoft.Win32.RegistryKey.OpenRemoteBaseKey(RegistryHive hKey,
String machineName, RegistryView view)

At this stage, Storage Migration Service orchestrator is attempting remote registry reads
to determine source machine configuration, but is being rejected by the source server
saying the registry path does not exist. This can be caused by:

The Remote Registry service isn't running on the source computer.

firewall does not allow remote registry connections to the source server from the
Orchestrator.
The source migration account does not have remote registry permissions to
connect to the source computer.
The source migration account does not have read permissions within the registry
of the source computer, under
"HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion" or
under
"HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer"

Cutover hangs on "38% Mapping network

interfaces on the source computer..."
When attempting to run cut over of a source computer, the cut over gets stuck at phase
"38% Mapping network interfaces on the source computer..." and you receive the
following error in the Storage Migration Service event log:

Log Name: Microsoft-Windows-StorageMigrationService-Proxy/Admin

Source: Microsoft-Windows-StorageMigrationService-Proxy
Date: 1/11/2020 8:51:14 AM
Event ID: 20505
Task Category: None
Level: Error
Keywords:
User: NETWORK SERVICE
Computer: nedwardo.contosocom
Description:
Couldn't establish a CIM session with the computer.

Computer: 172.16.10.37
User Name: nedwardo\MsftSmsStorMigratSvc
Error: 40970
Error Message: Unknown error (0xa00a)

Guidance: Confirm that the Netlogon service on the computer is reachable

through RPC and that the credentials provided are correct.

This issue is caused by Group Policy that sets the following registry value on the source
computer:
"HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\Syste
m\LocalAccountTokenFilterPolicy = 0"

This setting isn't part of standard Group Policy, it's an add-on configured using the
Microsoft Security Compliance Toolkit :

Windows Server 2012 R2: "Computer Configuration\Administrative

Templates\SCM: Pass the Hash Mitigations\Apply UAC restrictions to local accounts
on network logons"

Widows Server 2016: "Computer Configuration\Administrative Templates\MS

Security Guide\Apply UAC restrictions to local accounts on network logons"

It can also be set using Group Policy Preferences with a custom registry setting. You can
use the GPRESULT tool to determine which policy is applying this setting to the source
computer.

The Storage Migration Service temporarily enables the LocalAccountTokenFilterPolicy

as part of the cut over process, then removes it when done. When Group Policy applies
a conflicting Group Policy Object (GPO), it overrides the Storage Migration Service and
prevents cut over.
To work around this issue, use one of the following options:

1. Temporarily move the source computer from the Active Directory OU that applies
this conflicting GPO.
2. Temporarily disable the GPO that applies this conflicting policy.
3. Temporarily create a new GPO that sets this setting to Disabled and applies to
specific OU of source servers, with a higher precedence than any other GPOs.

Inventory or transfer fail when using

credentials from a different domain
When attempting to run inventory or transfer with the Storage Migration Service and
targeting a Windows Server while using migration credentials from a different domain
than the targeted server, you receive the following errors

Exception from HRESULT:0x80131505

The server was unable to process the request due to an internal error

04/28/2020-11:31:01.169 [Error] Failed device discovery stage SystemInfo

with error: (0x490) Could not find computer object 'myserver' in Active
Directory
[d:\os\src\base\dms\proxy\discovery\discoveryproxy\DeviceDiscoveryOperation.
cs::TryStage::1042]

Examining the logs further shows that the migration account and the server being
migrated from or two are in different domains:

06/25/2020-10:11:16.543 [Info] Creating new job=NedJob user=**CONTOSO**\ned

[d:\os\src\base\dms\service\StorageMigrationService.IInventory.cs::CreateJob
::133]

GetOsVersion(fileserver75.**corp**.contoso.com)
[d:\os\src\base\dms\proxy\common\proxycommon\CimSessionHelper.cs::GetOsVersi
on::66] 06/25/2020-10:20:45.368 [Info] Computer
'fileserver75.corp.contoso.com': OS version
This issue is caused by a code defect in the Storage Migration Service. To work around
this issue, use migration credentials from the same domain that the source and
destination computer belong to. For instance, if the source and destination computer
belong to the "corp.contoso.com" domain in the "contoso.com" forest, use
'corp\myaccount' to perform the migration, not a 'contoso\myaccount' credential.

Inventory fails with "Element not found"

Consider the following scenario:

You have a source server with a DNS Host Name and Active Directory name more than
15 unicode characters, such as "iamaverylongcomputername". By design, Windows did
not let you set the legacy NetBIOS name to be set this long and warned when the server
was named that the NetBIOS name would be truncated to 15 unicode wide characters
(example: "iamaverylongcom"). When you attempt to inventory this computer, you
receive in Windows Admin Center and the event log:

DOS

"Element not found"

========================

Log Name: Microsoft-Windows-StorageMigrationService/Admin

Source: Microsoft-Windows-StorageMigrationService
Date: 4/10/2020 10:49:19 AM
Event ID: 2509
Task Category: None
Level: Error
Keywords:
User: NETWORK SERVICE
Computer: WIN-6PJAG3DHPLF.corp.contoso.com
Description:
Couldn't inventory a computer.

Job: longnametest
Computer: iamaverylongcomputername.corp.contoso.com
State: Failed
Error: 1168
Error Message:

Guidance: Check the detailed error and make sure the inventory requirements
are met. The inventory couldn't determine any aspects of the specified
source computer. This could be because of missing permissions or privileges
on the source or a blocked firewall port.

This issue is caused by a code defect in the Storage Migration Service. The only
workaround currently is to rename the computer to have the same name as the NetBIOS
name, then use NETDOM COMPUTERNAME /ADD to add an alternate computer name
that contains the longer name that was in use prior to starting Inventory. Storage
Migration Service supports migrating alternate computer names.

Storage Migration Service inventory fails with

"a parameter cannot be found that matches
parameter name 'IncludeDFSN'"
When using the 2009 version of Windows Admin Center to manage a Windows Server
2019 orchestrator, you receive the following error when you attempt to inventory a
source computer:

Remote exception : a parameter cannot be found that matches parameter name

'IncludeDFSN'"

To resolve, update the Storage Migration Service extension to at least version 1.113.0 in
Windows Admin Center. The update should automatically appear in the feed and
prompt for installation.

Storage Migration Service transfer validation

returns 'Error HRESULT E_FAIL has been
returned from a call to a COM component'
After installing the Windows Server 2019 November cumulative update KB4586793 ,
some transfer validations may fail with:

Error HRESULT E_FAIL has been returned from a call to a COM component

It doesn't necessarily happen for all source computers. We are working to diagnose this
issue. As a workaround, install the 1.115 or later Storage Migration Service tool in
Windows Admin Center. The update should automatically appear in the Windows Admin
Center feed and prompt for installation, and will allow you to ignore this error. To work
around it:

1. Navigate to the "Adjust Settings" step of the Transfer phase.

 2. Enable "Override Transfer Validation".
3. Proceed with your transfer, either without running "Validate" or running it and
ignoring the E_FAIL error.

） Important

Don't uninstall KB4586793 . This update upgrades the Storage Migration Service
database and removing the update will require you to delete your database.

Transfer fails with "Failed to get file handle"

and one or no shares transfer from a particular
volume
When you attempt to transfer data from a source computer, you find that no files for a
particular volume transfer even though they do transfer for other volumes. You receive
the following errors in Windows Admin Center and the event log:

"Couldn't transfer storage on any of the endpoints"

========================

SMS Admin log:

06/11/2021 08:44:17 3515 Error Couldn't transfer all of the files in the
endpoint on the computer.

Job: test1
Computer: nedsrv1.corp.contoso.com
Destination Computer: nedsrv2.corp.contoso.com
Endpoint: foo
State: Failed
Source File Count: 0
Source File Size in KB: 0
Succeeded File Count: 0
Succeeded File Size in KB: 0
New File Count: 0
New File Size in KB: 0
Failed File Count: 0
Error: -2146233088
Error Message:

Guidance: Check the detailed error and make sure the transfer requirements
are met. This could be because the orchestrator computer couldn't reach a
source or destination computer, possibly due to a firewall rule, or missing
permissions.
 ========================

If you dump the SMS debug logs with Get-SMSLogs you also see:

SMS Debug log:

06/11/2021-08:44:17.236 [Erro] End file transfer failed with -2146233088

exception:ErrorCode: -2146233088, Transfer failed
at
Microsoft.StorageMigration.Service.EndpointHelper.TransferFiles(String
source, String destination, String sourceOSVersion, IEndpointRecord
endpointRecord, TransferConfiguration config, String sourcePath, String
destinationPath, ProxyInformation transferProxyInformation, Int64&
skippedSystemObjectCount, CancellationToken cancelToken, SourceType
sourceType, Protocol protocol, String sourceClusterSharedVolumesRoot, String
targetClusterSharedVolumesRoot, ServerType sourceServerType, ServerType
targetServerType, Boolean isTieredAFSEnabled, Int32 volumeMinimumFreeSpace,
String targetVolume, String[] mountedVolumes)
[d:\os\src\base\dms\service\OperationManager\EndpointHelper.cs::TransferFile
s::510]

SMS Proxy Debug log:

14090 06/11/2021-08:44:17.123 [Crit] Failed to create root of the share

\\nedsrv1.corp.contoso.com\D$ with error -2147467259 and message Failed to
get file handle
[d:\os\src\base\dms\proxy\transfer\transferproxy\stages\DirectoryEnumeration
Stage.cs::ProcessItem::112]
14091 06/11/2021-08:44:17.124 [Erro] Stage DirectoryEnumerationStage
cancelled. Received error: Failed to get file handle
[d:\os\src\base\dms\proxy\transfer\transferproxy\stages\StageBase.cs::DoStag
e::50]
14124 06/11/2021-08:44:17.141 [Erro] Failed pipeline execution.
System.AggregateException: One or more errors occurred. --->
System.ComponentModel.Win32Exception: Failed to get file handle
14125 at
Microsoft.StorageMigration.Proxy.Service.Transfer.DirectoryEnumerationStage.
ProcessItem(DirEnumResultWithParent input)
14126 at
Microsoft.StorageMigration.Proxy.Service.Transfer.StageBase`3.DoStage(Cancel
lationTokenSource cts)
14127 at System.Threading.Tasks.Task.Execute()
14128 --- End of inner exception stack trace ---
14129 at System.Threading.Tasks.Task.WaitAll(Task[] tasks, Int32
millisecondsTimeout, CancellationToken cancellationToken)
14130 at
Microsoft.StorageMigration.Proxy.Service.Transfer.Pipeline.Run(CancellationT
oken token)
14131 at
 Microsoft.StorageMigration.Proxy.Service.Transfer.TransferOperation.Run()
14132 at
Microsoft.StorageMigration.Proxy.Service.Transfer.TransferRequestHandler.Pro
cessRequest(FileTransferRequest fileTransferRequest, Guid operationId)
14133 ---> (Inner Exception #0) System.ComponentModel.Win32Exception
(0x80004005): Failed to get file handle
14134 at
Microsoft.StorageMigration.Proxy.Service.Transfer.DirectoryEnumerationStage.
ProcessItem(DirEnumResultWithParent input)
14135 at
Microsoft.StorageMigration.Proxy.Service.Transfer.StageBase`3.DoStage(Cancel
lationTokenSource cts)
14136 at System.Threading.Tasks.Task.Execute()<---
14137
[d:\os\src\base\dms\proxy\transfer\transferproxy\TransferRequestHandler.cs::
ProcessRequest::132]

This issue is caused by a limitation in the Storage Migration Service Proxy service when
an entire NTFS volume has been configured with the Compression flag. To work around
this issue, remove the compression flag from the destination volume:

1. Open File Explorer, right-click the destination drive letter, and click Properties.
2. Uncheck "Compress this drive to save disk space"
3. Rerun the transfer.

Alternatively, you can perform the same steps on the source computer if its volume was
compressed and if it has free space to hold the expanded files. NTFS-compressed files
are always decompressed while copying or moving, compressing them won't reduce
transfer time.

An error requires resetting the Storage

Migration Service database
Under rare circumstances you may need to reset the Storage Migration Service
database. To do this:

1. Open an elevated cmd prompt, where you are a member of Administrators on the
Storage Migration Service orchestrator server, and run:

CMD

NET STOP SMS

NET STOP SMSPROXY

TAKEOWN /d y /a /r /f c:\ProgramData\Microsoft\StorageMigrationService
 MD c:\ProgramData\Microsoft\StorageMigrationService\backup

ICACLS c:\ProgramData\Microsoft\StorageMigrationService\* /grant

Administrators:(GA)

XCOPY c:\ProgramData\Microsoft\StorageMigrationService\* .\backup\*

DEL c:\ProgramData\Microsoft\StorageMigrationService\* /q

ICACLS c:\ProgramData\Microsoft\StorageMigrationService /GRANT

networkservice:F /T /C

ICACLS c:\ProgramData\Microsoft\StorageMigrationService /GRANT

networkservice:(GA) /T /C

2. Validate that there were no errors in the above commands. Then start the Storage
Migration Service service, which will create a new database.

CMD

NET START SMS

NET START SMSPROXY

Transfers halts with error: Unable to translate

Unicode character
A running transfer halts. You receive event log error:

Log Name: Microsoft-Windows-StorageMigrationService/Admin

Source: Microsoft-Windows-StorageMigrationService
Date:
Event ID: 3515
Task Category: None
Level: Error
Keywords:
User: NETWORK SERVICE
Computer:
Description:
Couldn't transfer all of the files in the endpoint on the computer.
Job:
Computer:
Destination Computer:
Endpoint:
State: Failed
Source File Count: 833617
Source File Size in KB: 45919696
 Succeeded File Count: 833438
Succeeded File Size in KB: 45919696
New File Count: 0
New File Size in KB: 0
Failed File Count: 179
Error: -2146233087
Error Message: The socket connection was aborted. This could be caused by an
error processing your message or a receive timeout being exceeded by the
remote host, or an underlying network resource issue. Local socket timeout
was '00:00:59.9970000'.

Examining the Storage Migration Service debug log shows:

03. 07. 2023-23:28:08.647 [Erro] ExceptionMessage : (Unable to translate

Unicode character \uDB71 at index 1 to specified code page.),
ExceptionToString: (System.Text.EncoderFallbackException: Unable to
translate Unicode character \uDB71 at index 1 to specified code page.

This issue is caused by an unhandled unicode character that the Storage Migration
Service cannot translate. To locate the name of the file(s) with the invalid character, edit
the following sample PowerShell script and run it on the source computer, then examine
the results and rename or remove the files:

# Sample PowerShell script to find files with unhandled unicode characters

$FolderPath = "C:\temp"
$OutputFilePath = "C:\temp\invalid_char_results.txt"
$UnhandledChar = "\uDB71"

Get-ChildItem -path $FolderPath -Recurse | ForEach-Object {

if ($_ -is [System.IO.FileInfo]) {
if ($_.Name -match $UnhandledChar) {
Add-Content $outputFilePath "$($_.FullName)"
}
}
}

Cut over fails at 77% or 30%

When you're performing cut over, the operation hangs at "77% - adding the destination
computer to the domain" or "30% - Can't unjoin domain." The issue only happens when:
 A user who isn't a member of a built-in admin group in AD created the source or
destination computer account in Active Directory.

Or

The migration user account isn't the same user who created the source computer
account.

Windows updates released on and after October 11, 2022 contain extra protections to
address CVE-2022-38042 , these extra protections caused the issue. The protections
were further updated with the March 14, 2023 monthly cumulative update, adding a
workaround option for this issue. The protections intentionally prevent domain join
operations from reusing an existing computer account in the target domain unless:

The user attempting the operation is the creator of the existing account.

The user attempting the operation is a member of Active Directory built-in groups
Domain Administrators, Enterprise Administrators or Administrators created the
computer account.

The user attempting the operation is a member of the "Domain controller: Allow
computer account reuse during domain join." Group Policy setting for the
computer account.

To resolve this issue, use one of the following solutions.

Solution 1 - Use "Allow computer account re-use during

domain join"
1. Ensure all domain controllers, the source computer, destination computer, and
SMS migration computer have installed the March 14, 2023 cumulative update and
have been rebooted.
2. Follow the steps in detailed in the Take Action section of KB5020276 .
3. In Windows Admin Center, go to Server Manager > Storage Migration Service,
create or continue an existing job.
4. On the Cut over to the new servers > Adjust Settings page, ensure the account
used for AD Credentials is the same account that was allowed to reuse computer
accounts in step 2."

Solution 2 - Use the original account for migration

1. In Windows Admin Center, go to Server Manager > Storage Migration Service,
create or continue an existing job.
 2. On the Cut over to the new servers > Adjust Settings page, ensure the account
used for AD Credentials is the same account that created or joined the source and
destination computer to the domain.

Solution 3 (not recommended) - Use a high-privilege

group
1. In Windows Admin Center, go to Server Manager > Storage Migration Service,
create or continue an existing job.
2. On the Cut over to the new servers > Adjust Settings page, ensure the account
used for AD Credentials is a member of one of the high-privilege Active Directory
built-in groups Domain Administrators, Enterprise Administrators or
Administrators.

） Important

If you have followed Solution 1 and the unjoin operation fails "33% - can't unjoin
domain" with error 0x6D1 "The procedure is out of range", the March 14, 2024
cumulative update has not been installed on the source computer, or it was
installed but the computer was not restarted.

Cut over fails for Windows Server 2008 R2

When you're performing cut over from a source computer running Windows Server
2008 R2 or older, you receive the error "Couldn’t rename the computer from the
domain." Using the Storage Migration Service Helper Get-SmsLog command shows
error 0x6D1 and "Object reference not set to an instance of an object". The following
example is the log file output from the PowerShell Get-SmsLog command.

Log

Line 360: 04/02/2023-14:06:02.877 [Info] UnjoinDomain(isLocal=False,

server='2008R2.corp.contoso.com')
[d:\os\src\base\dms\proxy\cutover\cutoverproxy\CutoverUtils.cs::UnjoinDomain
::2151]
Line 361: 04/02/2023-14:06:02.948 [Erro] Attempt #1 failed to unjoin machine
'2008R2.corp.contoso' from the domain with credential 'corp\ned'. Error
0x6D1.
[d:\os\src\base\dms\proxy\cutover\cutoverproxy\CutoverUtils.cs::UnjoinDomain
::2184]
Line 362: 04/02/2023-14:06:02.954 [Erro] Fatal exception during cutover
stage processing. Source: 2008R2.corp.contoso.com, CutoverStage:
UnjoinSource, ErrorCode: 0x80004003, Message: Object reference not set to an
 instance of an object.
[d:\os\src\base\dms\proxy\cutover\cutoverproxy\CutoverOperation.cs::Run::111
6]

Changes introduced in KB5020276 to combat CVE-2022-38042 cause this error.

To resolve this issue, use one of the following solutions.

Solution 1 (using Windows Server 2008 R2 with valid ESU)

For a source computer running Windows Server 2008 R2 with valid Extended Support
Updates, first install the latest cumulative update. Once the cumulative update has been
successfully installed, follow the steps detailed in the article Cut over fails at 77% or 30%
to resolve the issue.

Solution 2 (using Windows Server 2008 R2 without a valid

ESU, Windows Server 2008 or Windows Server 2003)
If your source computer is running Windows Server 2008 R2 without ESU, Windows
Server 2008 or Windows Server 2003, you need to perform a manual cutover using the
steps described in How cutover works in Storage Migration Service, but with the
following changes.

1. Skip steps 3 and 4

2. For step 5, you must sign in to the computer and remove it from the domain
manually using SYSDM.CPL , NETDOM.exe , or the Remove-Compuer PowerShell
command. You can't remotely remove the computer from the domain after
KB5020276 .

Transfer validation warning "The destination

proxy wasn't found"
If you didn't already have the SMS Proxy service installed on the destination server
before starting the transfer, Windows Admin Center installs it automatically. But under
certain circumstances it will fail to register and display validation error "The destination
proxy wasn't found".

To resolve this issue, ensure the SMS Proxy service feature is installed on the destination
server, then run the following PowerShell command on the Orchestrator server:

PowerShell
 Register-SMSProxy -ComputerName <destination server FQDN> -Force

Validation will then pass.

See also
Storage Migration Service overview
Storage Replica overview
Article • 03/21/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

Storage Replica is Windows Server technology that enables replication of volumes

between servers or clusters for disaster recovery. It also enables you to create stretch
failover clusters that span two sites, with all nodes staying in sync.

Storage Replica supports synchronous and asynchronous replication:

Synchronous replication mirrors data within a low-latency network site with crash-
consistent volumes to ensure zero data loss at the file-system level during a failure.
Asynchronous replication mirrors data across sites beyond metropolitan ranges
over network links with higher latencies, but without a guarantee that both sites
have identical copies of the data at the time of a failure.

Why use Storage Replica?

Storage Replica offers disaster recovery and preparedness capabilities in Windows
Server. Windows Server offers the peace of mind of zero data loss, with the ability to
synchronously protect data on different racks, floors, buildings, campuses, counties, and
cities. After a disaster strikes, all data exists elsewhere without any possibility of loss. The
same applies before a disaster strikes; Storage Replica offers you the ability to switch
workloads to safe locations prior to catastrophes when granted a few moments warning
- again, with no data loss.

Storage Replica allows more efficient use of multiple datacenters. By stretching clusters
or replicating clusters, workloads can be run in multiple datacenters for quicker data
access by local proximity users and applications, as well as better load distribution and
use of compute resources. If a disaster takes one datacenter offline, you can move its
typical workloads to the other site temporarily.

Storage Replica may allow you to decommission existing file replication systems such as
DFS Replication that were pressed into duty as low-end disaster recovery solutions.
While DFS Replication works well over low bandwidth networks, its latency is high -
often measured in hours or days. This is caused by its requirement for files to close and
its artificial throttles meant to prevent network congestion. With those design
characteristics, the newest and hottest files in a DFS Replication replica are the least
likely to replicate. Storage Replica operates below the file level and has none of these
restrictions.

Storage Replica also supports asynchronous replication for longer ranges and higher
latency networks. Because it isn't checkpoint-based, and instead continuously replicates,
the delta of changes tends to be far lower than snapshot-based products. Storage
Replica operates at the partition layer and therefore replicates all VSS snapshots created
by Windows Server or backup software. By using VSS snapshots it allows use of
application-consistent data snapshots for point in time recovery, especially unstructured
user data replicated asynchronously.

Supported configurations
You can deploy Storage Replica in a stretch cluster, between cluster-to-cluster, and in
server-to-server configurations (see Figures 1-3).

Stretch Cluster allows configuration of computers and storage in a single cluster, where
some nodes share one set of asymmetric storage and some nodes share another, then
synchronously or asynchronously replicate with site awareness. This scenario can utilize
Storage Spaces with shared SAS storage, SAN and iSCSI-attached LUNs. It's managed
with PowerShell and the Failover Cluster Manager graphical tool, and allows for
automated workload failover.

FIGURE 1: Storage replication in a stretch cluster using Storage Replica

Cluster to Cluster allows replication between two separate clusters, where one cluster
synchronously or asynchronously replicates with another cluster. This scenario can utilize
Storage Spaces Direct, Storage Spaces with shared SAS storage, SAN and iSCSI-attached
LUNs. It's managed with Windows Admin Center and PowerShell, and requires manual
intervention for failover.
FIGURE 2: Cluster-to-cluster storage replication using Storage Replica

Server to server allows synchronous and asynchronous replication between two

standalone servers, using Storage Spaces with shared SAS storage, SAN and iSCSI-
attached LUNs, and local drives. It's managed with Windows Admin Center and
PowerShell, and requires manual intervention for failover.

FIGURE 3: Server-to-server storage replication using Storage Replica

７ Note

You can also configure server-to-self replication, using four separate volumes on
one computer. However, this guide does not cover this scenario.

Storage Replica Features

Zero data loss, block-level replication. With synchronous replication, there's no
possibility of data loss. With block-level replication, there's no possibility of file
locking.

Simple deployment and management. Storage Replica has a design mandate for
ease of use. Creation of a replication partnership between two servers can utilize
the Windows Admin Center. Deployment of stretch clusters uses intuitive wizard in
the familiar Failover Cluster Manager tool.

Guest and host. All capabilities of Storage Replica are exposed in both virtualized
guest and host-based deployments. This means guests can replicate their data
 volumes even if running on non-Windows virtualization platforms or in public
clouds, as long as using Windows Server in the guest.

SMB3-based. Storage Replica uses the proven and mature technology of SMB 3,
first released in Windows Server 2012. This means all of SMB's advanced
characteristics - such as multichannel and SMB direct support on RoCE, iWARP, and
InfiniBand RDMA network cards - are available to Storage Replica.

Security. Unlike many vendor's products, Storage Replica has industry-leading

security technology baked in. This includes packet signing, AES-128-GCM full data
encryption, support for Intel AES-NI encryption acceleration, and pre-
authentication integrity man-in-the-middle attack prevention. Storage Replica
utilizes Kerberos AES256 for all authentication between nodes.

High performance initial sync. Storage Replica supports seeded initial sync, where
a subset of data already exists on a target from older copies, backups, or shipped
drives. Initial replication only copies the differing blocks, potentially shortening
initial sync time and preventing data from using up limited bandwidth. Storage
replicas block checksum calculation and aggregation means that initial sync
performance is limited only by the speed of the storage and network.

Consistency groups. Write ordering guarantees that applications such as Microsoft

SQL Server can write to multiple replicated volumes and know the data is written
on the destination server sequentially.

User delegation. Users can be delegated permissions to manage replication

without being a member of the built-in Administrators group on the replicated
nodes, therefore limiting their access to unrelated areas.

Network Constraint. Storage Replica can be limited to individual networks by

server and by replicated volumes, in order to provide application, backup, and
management software bandwidth.

Thin provisioning. Support for thin provisioning in Storage Spaces and SAN
devices is supported in order to provide near-instantaneous initial replication times
under many circumstances. Once initial replication is initiated, the volume won't be
able to shrink or trim

Compression. Storage Replica offers compression for data transferred over the
network between the source and destination server. Storage Replica Compression
for Data Transfer is only supported in Windows Server Datacenter: Azure Edition
beginning with OS build 20348.1070 and later (KB5017381 ).

Storage Replica includes the following features:

 Feature Details

Type Host-based

Synchronous Yes

Asynchronous Yes

Storage hardware agnostic Yes

Replication unit Volume (Partition)

Windows Server stretch cluster creation Yes

Server to server replication Yes

Cluster to cluster replication Yes

Transport SMB3

Network TCP/IP or RDMA

Network constraint support Yes

Network compression Yes**

RDMA* iWARP, InfiniBand, RoCE v2

Replication network port firewall requirements Single IANA port (TCP 445 or 5445)

Multipath/Multichannel Yes (SMB3)

Kerberos support Yes (SMB3)

Over the wire encryption and signing Yes (SMB3)

Per-volume failovers allowed Yes

Thin-provisioned storage support Yes

Management UI in-box PowerShell, Failover Cluster Manager

*May require additional long haul equipment and cabling. **When using Windows
Server Datacenter: Azure Edition beginning with OS build 20348.1070

Storage Replica prerequisites

Active Directory Domain Services forest.

Storage Spaces with SAS JBODs, Storage Spaces Direct, fibre channel SAN, shared
VHDX, iSCSI Target, or local SAS/SCSI/SATA storage. SSD or faster recommended
 for replication log drives. Microsoft recommends that the log storage is faster than
the data storage. Log volumes must never be used for other workloads.

At least one ethernet/TCP connection on each server for synchronous replication,

but preferably RDMA.

At least 2 GB of RAM and two cores per server.

A network between servers with enough bandwidth to contain your IO write

workload and an average of 5 ms round trip latency or lower, for synchronous
replication. Asynchronous replication doesn't have a latency recommendation.

Windows Server, Datacenter Edition, or Windows Server, Standard Edition. Storage

Replica running on Windows Server, Standard Edition, has the following limitations:
You must use Windows Server 2019 or later
Storage Replica replicates a single volume instead of an unlimited number of
volumes.
Volumes can have a size of up to 2 TB instead of an unlimited size.

Background
This section includes information about high-level industry terms, synchronous and
asynchronous replication, and key behaviors.

High-level industry terms

Disaster Recovery (DR) refers to a contingency plan for recovering from site
catastrophes so that the business continues to operate. Data DR means multiple copies
of production data in a separate physical location. For example, a stretch cluster, where
half the nodes are in one site and half are in another. Disaster Preparedness (DP) refers
to a contingency plan for preemptively moving workloads to a different location prior to
an oncoming disaster, such as a hurricane.

Service level agreements (SLAs) define the availability of a business' applications and
their tolerance of down time and data loss during planned and unplanned outages.
Recovery Time Objective (RTO) defines how long the business can tolerate total
inaccessibility of data. Recovery Point Objective (RPO) defines how much data the
business can afford to lose.

Synchronous replication
Synchronous replication guarantees that the application writes data to two locations at
once before completion of the IO operation. This replication is more suitable for mission
critical data, as it requires network and storage investments, and risks degraded
application performance by needing to perform writes in two locations.

When application writes occur on the source data copy, the originating storage doesn't
acknowledge the IO immediately. Instead, those data changes replicate to the remote
destination copy and return an acknowledgment. Only then does the application receive
the IO acknowledgment. This ensures constant synchronization of the remote site with
the source site, in effect extending storage IOs across the network. In the event of a
source site failure, applications can fail over to the remote site and resume their
operations with assurance of zero data loss.

Mode Diagram Steps

Synchronous 1. Application writes data

2. Log data is written and the data is
Zero Data replicated to the remote site
Loss 3. Log data is written at the remote site
4. Acknowledgment from the remote site
RPO
5. Application write acknowledged

t & t1 : Data flushed to the volume, logs

always write through

Asynchronous replication
Contrarily, asynchronous replication means that when the application writes data, that
data replicates to the remote site without immediate acknowledgment guarantees. This
mode allows faster response time to the application and a DR solution that works
geographically.

When the application writes data, the replication engine captures the write and
immediately acknowledges to the application. The captured data then replicates to the
remote location. The remote node processes the copy of the data and lazily
acknowledges back to the source copy. Since replication performance is no longer in the
application IO path, the remote site's responsiveness and distance are less important
factors. There's risk of data loss if the source data is lost and the destination copy of the
data was still in buffer without leaving the source.

With its higher than zero RPO, asynchronous replication is less suitable for HA solutions
like Failover Clusters, as they're designed for continuous operation with redundancy and
no loss of data.
 Mode Diagram Steps

Asynchronous 1. Application writes data

2. Log data written
Near zero data 3. Application write acknowledged
loss 4. Data replicated to the remote site
5. Log data written at the remote site
(depends on
6. Acknowledgment from the remote
multiple factors)
site
RPO
t & t1 : Data flushed to the volume,
logs always write through

Key evaluation points and behaviors

Network bandwidth and latency with fastest storage. There are physical limitations
around synchronous replication. Because Storage Replica implements an IO
filtering mechanism using logs and requiring network round trips, synchronous
replication is likely make application writes slower. By using low latency, high-
bandwidth networks, and high-throughput disk subsystems for the logs, you
minimize performance overhead.

The destination volume isn't accessible while replicating in Windows Server 2016.
When you configure replication, the destination volume dismounts, making it
inaccessible to any reads or writes by users. Its driver letter may be visible in typical
interfaces like File Explorer, but an application can't access the volume itself. Block-
level replication technologies are incompatible with allowing access to the
destination target's mounted file system in a volume. NTFS and ReFS don't support
users writing data to the volume while blocks change underneath them.

The Test-Failover cmdlet debuted in Windows Server, version 1709, and was also
included in Windows Server 2019. This now supports temporarily mounting a read-write
snapshot of the destination volume for backups, testing, etc. See Frequently asked
questions about Storage Replica for more info.

The Microsoft implementation of asynchronous replication is different than most.

Most industry implementations of asynchronous replication rely on snapshot-
based replication, where periodic differential transfers move to the other node and
merge. Storage Replica asynchronous replication operates just like synchronous
replication, except that it removes the requirement for a serialized synchronous
acknowledgment from the destination. This means that Storage Replica
theoretically has a lower RPO as it continuously replicates. However, this also
means it relies on internal application consistency guarantees rather than using
 snapshots to force consistency in application files. Storage Replica guarantees
crash consistency in all replication modes

Many customers use DFS Replication as a disaster recovery solution even though
often impractical for that scenario - DFS Replication can't replicate open files and is
designed to minimize bandwidth usage at the expense of performance, leading to
large recovery point deltas. Storage Replica may allow you to retire DFS Replication
from some of these types of disaster recovery duties.

Storage Replica isn't a backup solution. Some IT environments deploy replication

systems as backup solutions, due to their zero data loss options when compared to
daily backups. Storage Replica replicates all changes to all blocks of data on the
volume, regardless of the change type. If a user deletes all data from a volume,
Storage Replica replicates the deletion instantly to the other volume, irrevocably
removing the data from both servers. Don't use Storage Replica as a replacement
for a point-in-time backup solution.

Storage Replica isn't Hyper-V Replica or Microsoft SQL AlwaysOn Availability

Groups. Storage Replica is a general purpose, storage-agnostic engine. By
definition, it can't tailor its behavior as ideally as application-level replication. This
may lead to specific feature gaps that encourage you to deploy or remain on
specific application replication technologies.

７ Note

This document contains a list of known issues and expected behaviors as well as
Frequently Asked Questions section.

Storage Replica terminology

This guide frequently uses the following terms:

The source is a computer's volume that allows local writes and replicates
outbound. Also known as "primary".

The destination is a computer's volume that doesn't allow local writes and
replicates inbound. Also known as "secondary".

A replication partnership is the synchronization relationship between a source and

destination computer for one or more volumes and utilizes a single log.
 A replication group is the organization of volumes and their replication
configuration within a partnership, on a per server basis. A group may contain one
or more volumes.

What's new for Storage Replica

For a list of new features in Storage Replica in Windows Server 2019, see What's new in
storage

More information
Stretch Cluster Replication Using Shared Storage
Server to Server Storage Replication
Cluster to Cluster Storage Replication
Storage Replica: Known Issues
Storage Replica: Frequently Asked Questions
Storage Spaces Direct in Windows Server 2016
Windows IT Pro Support
Stretch Cluster Replication Using Shared
Storage
Article • 01/20/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

In this evaluation example, you will configure these computers and their storage in a
single stretch cluster, where two nodes share one set of storage and two nodes share
another set of storage, then replication keeps both sets of storage mirrored in the
cluster to allow immediate failover. These nodes and their storage should be located in
separate physical sites, although it is not required. There are separate steps for creating
Hyper-V and File Server clusters as sample scenarios.

） Important

In this evaluation, servers in different sites must be able to communicate with the
other servers via a network, but not have any physical connectivity to the other
site's shared storage. This scenario does not make use of Storage Spaces Direct.

７ Note

You may also want to consider using an Azure Stack HCI solution to implement a
stretch cluster. For more information, see Stretched clusters overview in Azure
Stack HCI.

Terms
This walkthrough uses the following environment as an example:

Four servers, named SR-SRV01, SR-SRV02, SR-SRV03, and SR-SRV04 formed into
a single cluster called SR-SRVCLUS.

A pair of logical "sites" that represents two different data centers, with one called
Redmond and the other called Bellevue.

７ Note
 You can use only as few as two nodes, where one node each is in each site.
However, you will not be able to perform intra-site failover with only two servers.
You can use as many as 64 nodes.

FIGURE 1: Storage Replication in a stretch cluster

Prerequisites
Active Directory Domain Services forest (does not need to run Windows Server
2016).
2-64 servers running Windows Server 2019 or Windows Server 2016, Datacenter
Edition. If you're running Windows Server 2019, you can instead use Standard
Edition if you're OK replicating only a single volume up to 2 TB in size.
Two sets of shared storage, using SAS JBODs (such as with Storage Spaces), Fibre
Channel SAN, Shared VHDX, or iSCSI Target. The storage should contain a mix of
HDD and SSD media and must support Persistent Reservation. You will make each
storage set available to two of the servers only (asymmetric).
Each set of storage must allow creation of at least two virtual disks, one for
replicated data and one for logs. The physical storage must have the same sector
sizes on all the data disks. The physical storage must have the same sector sizes on
all the log disks.
At least one 1GbE connection on each server for synchronous replication.
At least 2GB of RAM and two cores per server. You will need more memory and
cores for more virtual machines.
Appropriate firewall and router rules to allow ICMP, SMB (port 445, plus 5445 for
SMB Direct) and WS-MAN (port 5985) bi-directional traffic between all nodes.
A network between servers with enough bandwidth to contain your IO write
workload and an average of =5ms round trip latency, for synchronous replication.
Asynchronous replication does not have a latency recommendation.
 The replicated storage cannot be located on the drive containing the Windows
operating system folder.

） Important

While it is possible to attach a storage device to a single server and use this for
replication, Windows Failover Clustering still relies on SCSI Persistent Reservations.
Therefore, the storage must still be a Shared Storage type such as a SAN
technology. Local disks or disks presented by a hypervisor might not be
compatible. In Azure, the disks must be a Premium SSD size that supports sharing,
even if only one VM is to be attached to it.

Many of these requirements can be determined by using the Test-SRTopology cmdlet.

You get access to this tool if you install Storage Replica or the Storage Replica
Management Tools features on at least one server. There is no need to configure
Storage Replica to use this tool, only to install the cmdlet. More information is included
in the following steps.

Provision operating system, features, roles,

storage, and network
1. Install Windows Server on all server nodes, using either the Server Core or Server
with Desktop Experience installation options.

） Important

From this point on, always logon as a domain user who is a member of the
built-in administrator group on all servers. Always remember to elevate your
PowerShell and CMD prompts going forward when running on a graphical
server installation or on a Windows 10 computer.

2. Add network information and join the nodes to the domain, then restart them.

７ Note

As of this point, the guide presumes you have two pairings of servers for use
in a stretch cluster. A WAN or LAN network separate the servers and the
servers belong to either physical or logical sites. The guide considers SR-
 SRV01 and SR-SRV02 to be in site Redmond and SR-SRV03 and SR-SRV04 to
be in site Bellevue.

3. Connect the first set of shared JBOD storage enclosure, Shared VHDX, iSCSI target,
or FC SAN to the servers in site Redmond.

4. Connect the second set of storage to the servers in site Bellevue.

5. As appropriate, install latest vendor storage and enclosure firmware and drivers,
latest vendor HBA drivers, latest vendor BIOS/UEFI firmware, latest vendor network
drivers, and latest motherboard chipset drivers on all four nodes. Restart nodes as
needed.

７ Note

Consult your hardware vendor documentation for configuring shared storage

and networking hardware.

6. Ensure that BIOS/UEFI settings for servers enable high performance, such as
disabling C-State, setting QPI speed, enabling NUMA, and setting highest memory
frequency. Ensure power management in Windows Server is set to high
performance. Restart as required.

7. Configure roles as follows:

Graphical method

Run ServerManager.exe and add all server nodes by clicking Manage and
Add Servers.

） Important

Install the Failover Clustering, and Storage Replica roles and features on
each of the nodes and restart them. If planning to use other roles like
Hyper-V, File Server, etc. you can install them now too.

Using Windows PowerShell method

On SR-SRV04 or a remote management computer, run the following

command in a Windows PowerShell console to install the required features
and roles for a stretch cluster on the four nodes and restart them:
 PowerShell

$Servers = 'SR-SRV01','SR-SRV02','SR-SRV03','SR-SRV04'

$Servers | foreach { Install-WindowsFeature -ComputerName $_ -Name

Storage-Replica,Failover-Clustering,FS-FileServer -
IncludeManagementTools -restart }

For more information on these steps, see Install or Uninstall Roles, Role
Services, or Features.

8. Configure storage as follows:

） Important

You must create two volumes on each enclosure: one for data and one
for logs.
Log and data disks must be initialized as GPT, not MBR.
The two data volumes must be of identical size.
The two log volumes should be of identical size.
All replicated data disks must have the same sector sizes.
All log disks must have the same sector sizes.
The log volumes should use flash-based storage and high performance
resiliency settings. Microsoft recommends that the log storage be as
faster than the data storage. Log volumes must never be used for other
workloads.
The data disks can use HDD, SSD, or a tiered combination and can use
either mirrored or parity spaces or RAID 1 or 10, or RAID 5 or RAID 50.
The log volume must be at least 9GB by default and can to be larger or
smaller based on log requirements.
The volumes must be formatted with NTFS or ReFS.
The File Server role is only necessary for Test-SRTopology to operate, as
it opens the necessary firewall ports for testing.

For JBOD enclosures:

a. Ensure that each set of paired server nodes can see that site's storage
enclosures only (i.e. asymmetric storage) and that the SAS connections are
correctly configured.
 b. Provision the storage using Storage Spaces by following Steps 1-3
provided in the Deploy Storage Spaces on a Stand-Alone Server using
Windows PowerShell or Server Manager.

For iSCSI storage:

a. Ensure that each set of paired server nodes can see that site's storage
enclosures only (i.e. asymmetric storage). You should use more than one
single network adapter if using iSCSI.

b. Provision the storage using your vendor documentation. If using

Windows-based iSCSI Targeting, consult iSCSI Target Block Storage, How
To.

For FC SAN storage:

a. Ensure that each set of paired server nodes can see that site's storage
enclosures only (i.e. asymmetric storage) and that you have properly
zoned the hosts.

b. Provision the storage using your vendor documentation.

Configure a Hyper-V Failover Cluster or a File

Server for a General Use Cluster
After you setup your server nodes, the next step is to create one of the following types
of clusters:

Hyper-V failover cluster

File Server for general use cluster

Configure a Hyper-V Failover Cluster

７ Note

Skip this section and go to the Configure a file server for general use cluster
section, if you want to create a file server cluster and not a Hyper-V cluster.

You will now create a normal failover cluster. After configuration, validation, and testing,
you will stretch it using Storage Replica. You can perform all of the steps below on the
cluster nodes directly or from a remote management computer that contains the
Windows Server Remote Server Administration Tools.
Graphical method
1. Run cluadmin.msc.

2. Validate the proposed cluster and analyze the results to ensure you can continue.

７ Note

You should expect storage errors from cluster validation, due to the use of
asymmetric storage.

3. Create the Hyper-V compute cluster. Ensure that the cluster name is 15 characters
or fewer. The example used below is SR-SRVCLUS. If the nodes are going to reside
in different subnets, you must create an IP Address for the Cluster Name for each
subnet and use the “OR” dependency. More information can be found at
Configuring IP Addresses and Dependencies for Multi-Subnet Clusters – Part III .

4. Configure a File Share Witness or Cloud Witness to provide quorum in the event of
site loss.

７ Note

Windows Server now includes an option for Cloud (Azure)-based Witness.

You can choose this quorum option instead of the file share witness.

２ Warning

For more information about quorum configuration, see the Configure and
Manage the Quorum in a Windows Server 2012 Failover Cluster guide's
Witness Configuration. For more information on the Set-ClusterQuorum
cmdlet, see Set-ClusterQuorum.

5. Review Network Recommendations for a Hyper-V Cluster in Windows Server 2012

and ensure that you have optimally configured cluster networking.

6. Add one disk in the Redmond site to the cluster CSV. To do so, right click a source
disk in the Disks node of the Storage section, and then click Add to Cluster Shared
Volumes.

7. Using the Deploy a Hyper-V Cluster guide, follow steps 7-10 within Redmond site
to create a test virtual machine only to ensure the cluster is working normally
 within the two nodes sharing the storage in the first test site.

8. If you're creating a two-node stretch cluster, you must add all storage before
continuing. To do so, open a PowerShell session with administrative permissions on
the cluster nodes, and run the following command: Get-ClusterAvailableDisk -All
| Add-ClusterDisk .

This is by-design behavior in Windows Server 2016.

9. Start Windows PowerShell and use the Test-SRTopology cmdlet to determine if you
meet all the Storage Replica requirements.

For example, to validate two of the proposed stretch cluster nodes that each have
a D: and E: volume and run the test for 30 minutes:

a. Move all available storage to SR-SRV01.

b. Click Create Empty Role in the Roles section of Failover Cluster Manager.

c. Add the online storage to that empty role named New Role.

d. Move all available storage to SR-SRV03.

e. Click Create Empty Role in the Roles section of Failover Cluster Manager.

f. Move the empty New Role (2) to SR-SRV03.

g. Add the online storage to that empty role named New Role (2).

h. Now you have mounted all your storage with drive letters, and can evaluate the
cluster with Test-SRTopology .

For example:

MD c:\temp

Test-SRTopology -SourceComputerName SR-SRV01 -SourceVolumeName D: -

SourceLogVolumeName E: -DestinationComputerName SR-SRV03 -
DestinationVolumeName D: -DestinationLogVolumeName E: -
DurationInMinutes 30 -ResultPath c:\temp

） Important
 When using a test server with no write IO load on the specified source
volume during the evaluation period, consider adding a workload or it
Test-SRTopology will not generate a useful report. You should test with
production-like workloads in order to see real numbers and recommended
log sizes. Alternatively, simply copy some files into the source volume
during the test or download and run DISKSPD to generate write IOs. For
instance, a sample with a low write IO workload for ten minutes to the D:
volume: Diskspd.exe -c1g -d600 -W5 -C5 -b4k -t2 -o2 -r -w5 -i100
d:\test.dat

10. Examine the TestSrTopologyReport-< date >.html report to ensure you meet the
Storage Replica requirements and note the initial sync time prediction and log
recommendations.

11. Return the disks to Available Storage and remove the temporary empty roles.

12. Once satisfied, remove the test virtual machine. Add any real test virtual machines
needed for further evaluation to a proposed source node.

13. Configure stretch cluster site awareness so that servers SR-SRV01 and SR-SRV02
are in site Redmond, SR-SRV03 and SR-SRV04 are in site Bellevue, and Redmond
is preferred for node ownership of the source storage and VMs:

PowerShell

New-ClusterFaultDomain -Name Redmond -Type Site -Description "Primary"

-Location "Redmond Datacenter"
 New-ClusterFaultDomain -Name Bellevue -Type Site -Description
"Secondary" -Location "Bellevue Datacenter"

Set-ClusterFaultDomain -Name sr-srv01 -Parent Redmond

Set-ClusterFaultDomain -Name sr-srv02 -Parent Redmond
Set-ClusterFaultDomain -Name sr-srv03 -Parent Bellevue
Set-ClusterFaultDomain -Name sr-srv04 -Parent Bellevue

(Get-Cluster).PreferredSite="Redmond"

７ Note

There is no option to configure site awareness using Failover Cluster Manager

in Windows Server 2016.

14. (Optional) Configure cluster networking and Active Directory for faster DNS site
failover. You can utilize Hyper-V software defined networking, stretched VLANs,
network abstraction devices, lowered DNS TTL, and other common techniques.

For more information, review the Microsoft Ignite session: Stretching Failover
Clusters and Using Storage Replica in Windows Server vNext and the Enable
Change Notifications between Sites - How and Why? blog post.

15. (Optional) Configure VM resiliency so that guests do not pause for long during
node failures. Instead, they failover to the new replication source storage within 10
seconds.

PowerShell

(Get-Cluster).ResiliencyDefaultPeriod=10

７ Note

There is no option to configure VM resiliency using Failover Cluster Manager

in Windows Server 2016.

Windows PowerShell method

1. Test the proposed cluster and analyze the results to ensure you can continue:

PowerShell
 Test-Cluster SR-SRV01, SR-SRV02, SR-SRV03, SR-SRV04

７ Note

You should expect storage errors from cluster validation, due to the use of
asymmetric storage.

2. Create the File Server for General Use storage cluster (you must specify your own
static IP address the cluster will use). Ensure that the cluster name is 15 characters
or fewer. If the nodes reside in different subnets, than an IP Address for the
additional site must be created using the “OR” dependency. More information can
be found at Configuring IP Addresses and Dependencies for Multi-Subnet Clusters
– Part III .

PowerShell

New-Cluster -Name SR-SRVCLUS -Node SR-SRV01, SR-SRV02, SR-SRV03, SR-

SRV04 -StaticAddress <your IP here>
Add-ClusterResource -Name NewIPAddress -ResourceType "IP Address" -
Group "Cluster Group"
Set-ClusterResourceDependency -Resource "Cluster Name" -Dependency "
[Cluster IP Address] or [NewIPAddress]"

3. Configure a File Share Witness or Cloud (Azure) witness in the cluster that points to
a share hosted on the domain controller or some other independent server. For
example:

PowerShell

Set-ClusterQuorum -FileShareWitness \\someserver\someshare

７ Note

Windows Server now includes an option for Cloud (Azure)-based Witness.

You can choose this quorum option instead of the file share witness.

For more information about quorum configuration, see the Configure and Manage
the Quorum in a Windows Server 2012 Failover Cluster guide's Witness
Configuration. For more information on the Set-ClusterQuorum cmdlet, see Set-
ClusterQuorum.
 4. Review Network Recommendations for a Hyper-V Cluster in Windows Server 2012
and ensure that you have optimally configured cluster networking.

5. If you're creating a two-node stretch cluster, you must add all storage before
continuing. To do so, open a PowerShell session with administrative permissions on
the cluster nodes, and run the following command: Get-ClusterAvailableDisk -All
| Add-ClusterDisk .

This is by-design behavior in Windows Server 2016.

6. Using the Deploy a Hyper-V Cluster guide, follow steps 7-10 within Redmond site
to create a test virtual machine only to ensure the cluster is working normally
within the two nodes sharing the storage in the first test site.

7. Once satisfied, remove the test VM. Add any real test virtual machines needed for
further evaluation to a proposed source node.

8. Configure stretch cluster site awareness so that servers SR-SRV01 and SR-SRV02
are in site Redmond, SR-SRV03 and SR-SRV04 are in site Bellevue, and Redmond
is preferred for node ownership of the source storage and virtual machines:

PowerShell

New-ClusterFaultDomain -Name Redmond -Type Site -Description "Primary"

-Location "Redmond Datacenter"

New-ClusterFaultDomain -Name Bellevue -Type Site -Description

"Secondary" -Location "Bellevue Datacenter"

Set-ClusterFaultDomain -Name sr-srv01 -Parent Redmond

Set-ClusterFaultDomain -Name sr-srv02 -Parent Redmond
Set-ClusterFaultDomain -Name sr-srv03 -Parent Bellevue
Set-ClusterFaultDomain -Name sr-srv04 -Parent Bellevue

(Get-Cluster).PreferredSite="Redmond"

9. (Optional) Configure cluster networking and Active Directory for faster DNS site
failover. You can utilize Hyper-V software defined networking, stretched VLANs,
network abstraction devices, lowered DNS TTL, and other common techniques.

For more information, review the Microsoft Ignite session: Stretching Failover
Clusters and Using Storage Replica in Windows Server vNext and Enable Change
Notifications between Sites - How and Why.

10. (Optional) Configure VM resiliency so that guests do not pause for long periods
during node failures. Instead, they failover to the new replication source storage
 within 10 seconds.

PowerShell

(Get-Cluster).ResiliencyDefaultPeriod=10

７ Note

There is no option to VM Resiliency using Failover Cluster Manager in

Windows Server 2016.

Configure a File Server for General Use Cluster

７ Note

Skip this section if you have already configured a Hyper-V Failover cluster as
described in Configure a Hyper-V Failover Cluster.

You will now create a normal failover cluster. After configuration, validation, and testing,
you will stretch it using Storage Replica. You can perform all of the steps below on the
cluster nodes directly or from a remote management computer that contains the
Windows Server Remote Server Administration Tools.

Graphical method

1. Run cluadmin.msc.

2. Validate the proposed cluster and analyze the results to ensure you can continue.

７ Note

You should expect storage errors from cluster validation, due to the use of
asymmetric storage.

3. Create the File Server for General Use storage cluster. Ensure that the cluster name
is 15 characters or fewer. The example used below is SR-SRVCLUS. If the nodes are
going to reside in different subnets, you must create an IP Address for the Cluster
Name for each subnet and use the “OR” dependency. More information can be
 found at Configuring IP Addresses and Dependencies for Multi-Subnet Clusters –
Part III .

4. Configure a File Share Witness or Cloud Witness to provide quorum in the event of
site loss.

７ Note

Windows Server now includes an option for Cloud (Azure)-based Witness.

You can choose this quorum option instead of the file share witness.

７ Note

For more information about quorum configuration, see the Configure and
Manage the Quorum in a Windows Server 2012 Failover Cluster guide's
Witness Configuration. For more information on the Set-ClusterQuorum
cmdlet, see Set-ClusterQuorum.

5. If you're creating a two-node stretch cluster, you must add all storage before
continuing. To do so, open a PowerShell session with administrative permissions on
the cluster nodes, and run the following command: Get-ClusterAvailableDisk -All
| Add-ClusterDisk .

This is by-design behavior in Windows Server 2016.

6. Ensure that you have optimally configured cluster networking.

７ Note

The File Server role must be installed on all nodes prior to continuing to the
next step. |

7. Under Roles, click Configure Role. Review Before you Begin and click Next.

8. Select File Server and click Next.

9. Leave File Server for general use selected and click Next.

10. Provide a Client Access Point name (15 characters or fewer) and click Next.

11. Select a disk to be your data volume and click Next.

 12. Review your settings and click Next. Click Finish.

13. Right click your new File Server role and click Add File Share. Proceed through the
wizard to configure shares.

14. Optional: Add another File Server role that uses the other storage in this site.

15. Configure stretch cluster site awareness so that servers SR-SRV01 and SR-SRV02
are in site Redmond, SR-SRV03 and SR-SRV04 are in site Bellevue, and Redmond is
preferred for node ownership of the source storage and VMs:

PowerShell

New-ClusterFaultDomain -Name Redmond -Type Site -Description "Primary"

-Location "Redmond Datacenter"

New-ClusterFaultDomain -Name Bellevue -Type Site -Description

"Secondary" -Location "Bellevue Datacenter"

Set-ClusterFaultDomain -Name sr-srv01 -Parent Redmond

Set-ClusterFaultDomain -Name sr-srv02 -Parent Redmond
Set-ClusterFaultDomain -Name sr-srv03 -Parent Bellevue
Set-ClusterFaultDomain -Name sr-srv04 -Parent Bellevue

(Get-Cluster).PreferredSite="Redmond"

７ Note

There is no option to configure site awareness using Failover Cluster Manager

in Windows Server 2016.

16. (Optional) Configure cluster networking and Active Directory for faster DNS site
failover. You can utilize stretched VLANs, network abstraction devices, lowered
DNS TTL, and other common techniques.

For more information, review the Microsoft Ignite session Stretching Failover Clusters
and Using Storage Replica in Windows Server vNext and the blog post Enable Change
Notifications between Sites - How and Why.

PowerShell Method

1. Test the proposed cluster and analyze the results to ensure you can continue:

PowerShell
 Test-Cluster SR-SRV01, SR-SRV02, SR-SRV03, SR-SRV04

７ Note

You should expect storage errors from cluster validation, due to the use of
asymmetric storage.

2. Create the Hyper-V compute cluster (you must specify your own static IP address
the cluster will use). Ensure that the cluster name is 15 characters or fewer. If the
nodes reside in different subnets, than an IP Address for the additional site must
be created using the “OR” dependency. More information can be found at
Configuring IP Addresses and Dependencies for Multi-Subnet Clusters – Part III .

PowerShell

New-Cluster -Name SR-SRVCLUS -Node SR-SRV01, SR-SRV02, SR-SRV03, SR-

SRV04 -StaticAddress <your IP here>

Add-ClusterResource -Name NewIPAddress -ResourceType "IP Address" -

Group "Cluster Group"

Set-ClusterResourceDependency -Resource "Cluster Name" -Dependency "

[Cluster IP Address] or [NewIPAddress]"

3. Configure a File Share Witness or Cloud (Azure) witness in the cluster that points to
a share hosted on the domain controller or some other independent server. For
example:

PowerShell

Set-ClusterQuorum -FileShareWitness \\someserver\someshare

７ Note

Windows Server now includes an option for cloud witness using Azure. You
can choose this quorum option instead of the file share witness.

For more information about quorum configuration, see the Understanding cluster
and pool quorum. For more information on the Set-ClusterQuorum cmdlet, see
Set-ClusterQuorum.
4. If you're creating a two-node stretch cluster, you must add all storage before
continuing. To do so, open a PowerShell session with administrative permissions on
the cluster nodes, and run the following command: Get-ClusterAvailableDisk -All
| Add-ClusterDisk .

This is by-design behavior in Windows Server 2016.

5. Ensure that you have optimally configured cluster networking.

6. Configure a File Server role. For example:

PowerShell

Get-ClusterResource
Add-ClusterFileServerRole -Name SR-CLU-FS2 -Storage "Cluster Disk 4"

MD f:\share01

New-SmbShare -Name Share01 -Path f:\share01 -ContinuouslyAvailable

$false

7. Configure stretch cluster site awareness so that servers SR-SRV01 and SR-SRV02
are in site Redmond, SR-SRV03 and SR-SRV04 are in site Bellevue, and Redmond is
preferred for node ownership of the source storage and virtual machines:

PowerShell

New-ClusterFaultDomain -Name Redmond -Type Site -Description "Primary"

-Location "Redmond Datacenter"

New-ClusterFaultDomain -Name Bellevue -Type Site -Description

"Secondary" -Location "Bellevue Datacenter"

Set-ClusterFaultDomain -Name sr-srv01 -Parent Redmond

Set-ClusterFaultDomain -Name sr-srv02 -Parent Redmond
Set-ClusterFaultDomain -Name sr-srv03 -Parent Bellevue
Set-ClusterFaultDomain -Name sr-srv04 -Parent Bellevue

(Get-Cluster).PreferredSite="Redmond"

8. (Optional) Configure cluster networking and Active Directory for faster DNS site
failover. You can utilize stretched VLANs, network abstraction devices, lowered
DNS TTL, and other common techniques.

For more information, review the Microsoft Ignite session Stretching Failover
Clusters and Using Storage Replica in Windows Server vNext and the blog post
Enable Change Notifications between Sites - How and Why.
Configure a stretch cluster
Now you will configure the stretch cluster, using either Failover Cluster Manager or
Windows PowerShell. You can perform all of the steps below on the cluster nodes
directly or from a remote management computer that contains the Windows Server
Remote Server Administration Tools.

Failover Cluster Manager Method

1. For Hyper-V workloads, on one node where you have the data you wish to
replicate out, add the source data disk from your available disks to cluster shared
volumes if not already configured. Do not add all the disks; just add the single disk.
At this point, half the disks will show offline because this is asymmetric storage. If
replicating a physical disk resource (PDR) workload like File Server for general use,
you already have a role-attached disk ready to go.

2. Right-click the CSV disk or role-attached disk, click Replication, and then click
Enable.

3. Select the appropriate destination data volume and click Next. The destination
disks shown will have a volume the same size as the selected source disk. When
moving between these wizard dialogs, the available storage will automatically
move and come online in the background as needed.
4. Select the appropriate source log disk and click Next. The source log volume
should be on a disk that uses SSD or similarly fast media, not spinning disks.

5. Select the appropriate destination log volume and click Next. The destination log
disks shown will have a volume the same size as the selected source log disk
volume.

6. Leave the Overwrite Volume value at Overwrite destination Volume if the

destination volume does not contain a previous copy of the data from the source
server. If the destination does contain similar data, from a recent backup or
previous replication, select Seeded destination disk, and then click Next.

7. Leave the Replication Mode value at Synchronous Replication if you plan to use
zero RPO replication. Change it to Asynchronous Replication if you plan to stretch
your cluster over higher latency networks or need lower IO latency on the primary
site nodes.

8. Leave the Consistency Group value at Highest Performance if you do not plan to
use write ordering later with additional disk pairs in the replication group. If you
plan to add further disks to this replication group and you require guaranteed
write ordering, select Enable Write Ordering, and then click Next.

9. Click Next to configure replication and the stretch cluster formation.

10. On the Summary screen, note the completion dialog results. You can view the
report in a web browser.

11. At this point, you have configured a Storage Replica partnership between the two
halves of the cluster but replication is ongoing. There are several ways to see the
state of replication via a graphical tool.

a. Use the Replication Role column and the Replication tab. When done with
initial synchronization, the source and destination disks will have a Replication
Status of Continuously Replicating.
b. Start eventvwr.exe.

i. On the source server, navigate to Applications and Services \ Microsoft \

Windows \ StorageReplica \ Admin and examine events 5015, 5002, 5004,
1237, 5001, and 2200.

ii. On the destination server, navigate to Applications and Services \ Microsoft

\ Windows \ StorageReplica \ Operational and wait for event 1215. This
event states the number of copied bytes and the time taken. Example:

Log Name: Microsoft-Windows-StorageReplica/Operational

Source: Microsoft-Windows-StorageReplica
Date: 4/6/2016 4:52:23 PM
Event ID: 1215
Task Category: (1)
Level: Information
Keywords: (1)
User: SYSTEM
Computer: SR-SRV03.Threshold.nttest.microsoft.com
Description:
Block copy completed for replica.

ReplicationGroupName: Replication 2
ReplicationGroupId: {c6683340-0eea-4abc-ab95-c7d0026bc054}
ReplicaName: \\?\Volume{43a5aa94-317f-47cb-a335-2a5d543ad536}\
ReplicaId: {00000000-0000-0000-0000-000000000000}
 End LSN in bitmap:
LogGeneration: {00000000-0000-0000-0000-000000000000}
LogFileId: 0
CLSFLsn: 0xFFFFFFFF
Number of Bytes Recovered: 68583161856
Elapsed Time (ms): 140

iii. On the destination server, navigate to Applications and Services \ Microsoft

\ Windows \ StorageReplica \ Admin and examine events 5009, 1237, 5001,
5015, 5005, and 2200 to understand the processing progress. There should
be no warnings of errors in this sequence. There will be many 1237 events;
these indicate progress.

２ Warning

CPU and memory usage are likely to be higher than normal until initial
synchronization completes.

Windows PowerShell method

1. Ensure your Powershell console is running with an elevated administrator account.

2. Add the source data storage only to the cluster as CSV. To get the size, partition,
and volume layout of the available disks, use the following commands:

PowerShell

Move-ClusterGroup -Name "available storage" -Node sr-srv01

$DiskResources = Get-ClusterResource | Where-Object { $_.ResourceType -

eq 'Physical Disk' -and $_.State -eq 'Online' }
$DiskResources | foreach {
$resource = $_
$DiskGuidValue = $resource | Get-ClusterParameter DiskIdGuid

Get-Disk | where { $_.Guid -eq $DiskGuidValue.Value } | Get-

Partition | Get-Volume |
Select @{N="Name"; E={$resource.Name}}, @{N="Status"; E=
{$resource.State}}, DriveLetter, FileSystemLabel, Size, SizeRemaining
} | FT -AutoSize

Move-ClusterGroup -Name "available storage" -Node sr-srv03

$DiskResources = Get-ClusterResource | Where-Object { $_.ResourceType -

eq 'Physical Disk' -and $_.State -eq 'Online' }
$DiskResources | foreach {
$resource = $_
 $DiskGuidValue = $resource | Get-ClusterParameter DiskIdGuid

Get-Disk | where { $_.Guid -eq $DiskGuidValue.Value } | Get-

Partition | Get-Volume |
Select @{N="Name"; E={$resource.Name}}, @{N="Status"; E=
{$resource.State}}, DriveLetter, FileSystemLabel, Size, SizeRemaining
} | FT -AutoSize

3. Set the correct disk to CSV with:

PowerShell

Add-ClusterSharedVolume -Name "Cluster Disk 4"

Get-ClusterSharedVolume
Move-ClusterSharedVolume -Name "Cluster Disk 4" -Node sr-srv01

4. Configure the stretch cluster, specifying the following:

Source and destination nodes (where the source data is a CSV disk and all
other disks are not).

Source and Destination replication group names.

Source and destination disks, where the partition sizes match.

Source and destination log volumes, where there is enough free space to
contain the log size on both disks and the storage is SSD or similar fast
media.

Source and destination log volumes, where there is enough free space to
contain the log size on both disks and the storage is SSD or similar fast
media.

Log size.

The source log volume should be on a disk that uses SSD or similarly fast
media, not spinning disks.

PowerShell

New-SRPartnership -SourceComputerName sr-srv01 -SourceRGName rg01 -

SourceVolumeName "C:\ClusterStorage\Volume1" -SourceLogVolumeName e: -
DestinationComputerName sr-srv03 -DestinationRGName rg02 -
DestinationVolumeName d: -DestinationLogVolumeName e:

７ Note
 You can also use New-SRGroup on one node in each site and New-
SRPartnership to create replication in stages, rather than all at once.

5. Determine the replication progress.

a. On the source server, run the following command and examine events 5015,
5002, 5004, 1237, 5001, and 2200:

PowerShell

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica -max 20

b. On the destination server, run the following command to see the Storage
Replica events that show creation of the partnership. This event states the
number of copied bytes and the time taken. Example:

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica | Where-

Object {$_.ID -eq "1215"} | fl

TimeCreated : 4/6/2016 4:52:23 PM

ProviderName : Microsoft-Windows-StorageReplica
Id : 1215
Message : Block copy completed for replica.

ReplicationGroupName: Replication 2
ReplicationGroupId: {c6683340-0eea-4abc-ab95-c7d0026bc054}
ReplicaName: ?Volume{43a5aa94-317f-47cb-a335-2a5d543ad536}
ReplicaId: {00000000-0000-0000-0000-000000000000}
End LSN in bitmap:
LogGeneration: {00000000-0000-0000-0000-000000000000}
LogFileId: 0
CLSFLsn: 0xFFFFFFFF
Number of Bytes Recovered: 68583161856
Elapsed Time (ms): 140

c. On the destination server, run the following command and examine events
5009, 1237, 5001, 5015, 5005, and 2200 to understand the processing progress.
There should be no warnings of errors in this sequence. There will be many
1237 events; these indicate progress.

PowerShell

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica | FL

 d. Alternately, the destination server group for the replica states the number of
byte remaining to copy at all times, and can be queried through PowerShell. For
example:

PowerShell

(Get-SRGroup).Replicas | Select-Object numofbytesremaining

As a progress sample (that will not terminate):

PowerShell

while($true) {

$v = (Get-SRGroup -Name "Replication 2").replicas | Select-Object

numofbytesremaining
[System.Console]::Write("Number of bytes remaining: {0}`r",
$v.numofbytesremaining)
Start-Sleep -s 5
}

6. To get replication source and destination state within the stretch cluster, use Get-
SRGroup and Get-SRPartnership to see the configured state of replication in the
stretch cluster.

PowerShell

Get-SRGroup
Get-SRPartnership
(Get-SRGroup).replicas

Manage stretched cluster replication

Now you will manage and operate your stretch cluster. You can perform all of the steps
below on the cluster nodes directly or from a remote management computer that
contains the Windows Server Remote Server Administration Tools.

Graphical Tools Method

1. Use Failover Cluster Manager to determine the current source and destination of
replication and their status.

2. To measure replication performance, run Perfmon.exe on both the source and

destination nodes.
 a. On the destination node:

i. Add the Storage Replica Statistics objects with all their performance
counters for the data volume.

ii. Examine the results.

b. On the source node:

i. Add the Storage Replica Statistics and Storage Replica Partition I/O
Statistics objects with all their performance counters for the data volume (the
latter is only available with data on the current source server).

ii. Examine the results.

3. To alter replication source and destination within the stretch cluster, use the
following methods:

a. To move the source replication between nodes in the same site: right-click the
source CSV, click Move Storage, click Select Node, and then select a node in
the same site. If using non-CSV storage for a role assigned disk, you move the
role.

b. To move the source replication from one site to another: right-click the source
CSV, click Move Storage, click Select Node, and then select a node in another
site. If you configured a preferred site, you can use best possible node to always
move the source storage to a node in the preferred site. If using non-CSV
storage for a role assigned disk, you move the role.

c. To perform planned failover the replication direction from one site to another:
shutdown both nodes in one site using ServerManager.exe or SConfig.

d. To perform unplanned failover the replication direction from one site to

another: cut power to both nodes in one site.

７ Note

In Windows Server 2016, you may need to use Failover Cluster Manager or
Move-ClusterGroup to move the destination disks back to the other site
manually after the nodes come back online.

７ Note

Storage Replica dismounts the destination volumes. This is by design.

 4. To change the log size from the default 8GB, right-click both the source and
destination log disks, click the Replication Log tab, then change the sizes on both
the disks to match.

７ Note

The default log size is 8GB. Depending on the results of the Test-SRTopology
cmdlet, you may decide to use -LogSizeInBytes with a higher or lower value.

5. To add another pair of replicated disks to the existing replication group, you must
ensure that there is at least one extra disk in available storage. You can then right-
click the Source disk and select Add replication partnership.

７ Note

This need for an additional 'dummy' disk in available storage is due to a

regression and not intentional. Failover Cluster Manager previously support
adding more disks normally and will again in a later release.

6. To remove the existing replication:

a. Start cluadmin.msc.

b. Right-click the source CSV disk and click Replication, then click Remove. Accept
the warning prompt.

c. Optionally, remove the storage from CSV to return it to available storage for
further testing.

７ Note

You may need to use DiskMgmt.msc or ServerManager.exe to add back

drive letters to volumes after return to available storage.

Windows PowerShell Method

1. Use Get-SRGroup and (Get-SRGroup).Replicas to determine the current source
and destination of replication and their status.
2. To measure replication performance, use the Get-Counter cmdlet on both the
source and destination nodes. The counter names are:

\Storage Replica Partition I/O Statistics(*)\Number of times flush paused

\Storage Replica Partition I/O Statistics(*)\Number of pending flush I/O

\Storage Replica Partition I/O Statistics(*)\Number of requests for last log

write

\Storage Replica Partition I/O Statistics(*)\Avg. Flush Queue Length

\Storage Replica Partition I/O Statistics(*)\Current Flush Queue Length

\Storage Replica Partition I/O Statistics(*)\Number of Application Write

Requests

\Storage Replica Partition I/O Statistics(*)\Avg. Number of requests per log

write

\Storage Replica Partition I/O Statistics(*)\Avg. App Write Latency

\Storage Replica Partition I/O Statistics(*)\Avg. App Read Latency

\Storage Replica Statistics(*)\Target RPO

\Storage Replica Statistics(*)\Current RPO

\Storage Replica Statistics(*)\Avg. Log Queue Length

\Storage Replica Statistics(*)\Current Log Queue Length

\Storage Replica Statistics(*)\Total Bytes Received

\Storage Replica Statistics(*)\Total Bytes Sent

\Storage Replica Statistics(*)\Avg. Network Send Latency

\Storage Replica Statistics(*)\Replication State

\Storage Replica Statistics(*)\Avg. Message Round Trip Latency

\Storage Replica Statistics(*)\Last Recovery Elapsed Time

\Storage Replica Statistics(*)\Number of Flushed Recovery Transactions

\Storage Replica Statistics(*)\Number of Recovery Transactions

\Storage Replica Statistics(*)\Number of Flushed Replication Transactions

 \Storage Replica Statistics(*)\Number of Replication Transactions

\Storage Replica Statistics(*)\Max Log Sequence Number

\Storage Replica Statistics(*)\Number of Messages Received

\Storage Replica Statistics(*)\Number of Messages Sent

For more information on performance counters in Windows PowerShell, see Get-

Counter.

3. To alter replication source and destination within the stretch cluster, use the
following methods:

a. To move the replication source from one node to another in the Redmond site,
move the CSV resource using the Move-ClusterSharedVolume cmdlet.

PowerShell

Get-ClusterSharedVolume | fl *
Move-ClusterSharedVolume -Name "cluster disk 4" -Node sr-srv02

b. To move the replication direction from one site to another "planned", move the
CSV resource using the Move-ClusterSharedVolume cmdlet.

PowerShell

Get-ClusterSharedVolume | fl *
Move-ClusterSharedVolume -Name "cluster disk 4" -Node sr-srv04

This will also move the logs and data appropriately for the other site and nodes.

c. To perform unplanned failover the replication direction from one site to

another: cut power to both nodes in one site.

７ Note

Storage Replica dismounts the destination volumes. This is by design.

4. To change the log size from the default 8GB, use Set-SRGroup on both the source
and destination Storage Replica Groups. For example, to set all logs to 2GB:

PowerShell
 Get-SRGroup | Set-SRGroup -LogSizeInBytes 2GB
Get-SRGroup

5. To add another pair of replicated disks to the existing replication group, you must
ensure that there is at least one extra disk in available storage. You can then right
click the Source disk and select add replication partnership.

７ Note

This need for an additional 'dummy' disk in available storage is due to a

regression and not intentional. Failover Cluster Manager previously support
adding more disks normally and will again in a later release.

Use the Set-SRPartnership cmdlet with the -SourceAddVolumePartnership and -

DestinationAddVolumePartnership parameters.

6. To remove replication, use Get-SRGroup , Get- SRPartnership , Remove-SRGroup , and

Remove-SRPartnership on any node.

PowerShell

Get-SRPartnership | Remove-SRPartnership
Get-SRGroup | Remove-SRGroup

７ Note

If using a remote management computer you will need to specify the cluster
name to these cmdlets and provide the two RG names.

Related Topics
Storage Replica Overview
Server to Server Storage Replication
Cluster to Cluster Storage Replication
Storage Replica: Known Issues
Storage Replica: Frequently Asked Questions

See Also
Windows Server 2016
Storage Spaces Direct in Windows Server 2016
Stretched Clusters in Azure Stack HCI
Server-to-server storage replication with
Storage Replica
Article • 08/25/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

You can use Storage Replica to configure two servers to sync data so that each has an
identical copy of the same volume. This topic provides some background of this server-
to-server replication configuration, as well as how to set it up and manage the
environment.

To manage Storage Replica you can use Windows Admin Center or PowerShell.

Here's an overview video of using Storage Replica in Windows Admin Center.

https://www.microsoft.com/en-us/videoplayer/embed/3aa09fd4-867b-45e9-953e-
064008468c4b?autoplay=false&postJsllMsg=true

Prerequisites
Active Directory Domain Services forest (doesn't need to run Windows Server
2016).
Two servers running Windows Server 2019 or Windows Server 2016, Datacenter
Edition. If you're running Windows Server 2019, you can instead use Standard
Edition if you're OK replicating only a single volume up to 2 TB in size.
Two sets of storage, using SAS JBODs, fibre channel SAN, iSCSI target, or local
SCSI/SATA storage. The storage should contain a mix of HDD and SSD media. You
will make each storage set available only to each of the servers, with no shared
access.
Each set of storage must allow creation of at least two virtual disks, one for
replicated data and one for logs. The physical storage must have the same sector
sizes on all the data disks. The physical storage must have the same sector sizes on
all the log disks.
At least one ethernet/TCP connection on each server for synchronous replication,
but preferably RDMA.
Appropriate firewall and router rules to allow ICMP, SMB (port 445, plus 5445 for
SMB Direct) and WS-MAN (port 5985) bi-directional traffic between all nodes.
A network between servers with enough bandwidth to contain your IO write
workload and an average of =5ms round trip latency, for synchronous replication.
Asynchronous replication doesn't have a latency recommendation.
 If you're replicating between on-premises servers and Azure VMs, you must create
a network link between the on-premises servers and the Azure VMs. To do so, use
Express Route, a Site-to-Site VPN gateway connection, or install VPN software in
your Azure VMs to connect them with your on-premises network.
The replicated storage cannot be located on the drive containing the Windows
operating system folder.

） Important

In this scenario, each server should be in a different physical or logical site. Each
server must be able to communicate with the other via a network.

Many of these requirements can be determined by using the Test-SRTopology cmdlet .

You get access to this tool if you install Storage Replica or the Storage Replica
Management Tools features on at least one server. There is no need to configure
Storage Replica to use this tool, only to install the cmdlet. More information is included
in the steps below.

Windows Admin Center requirements

To use Storage Replica and Windows Admin Center together, you need the following:

System Operating system Required

for

Two servers Windows Server 2019, Windows Server Storage

(any mix of on-premises hardware, 2016, or Windows Server (Semi-Annual Replica
VMs, and cloud VMs including Azure Channel)
VMs)

One PC Windows 10 Windows

Admin
Center

７ Note

Right now you can't use Windows Admin Center on a server to manage Storage
Replica.

Terms
This walkthrough uses the following environment as an example:

Two servers, named SR-SRV05 and SR-SRV06.

A pair of logical "sites" that represent two different data centers, with one called
Redmond and one called Bellevue.

Figure 1: Server to server replication

Step 1: Install and configure Windows Admin

Center on your PC
If you're using Windows Admin Center to manage Storage Replica, use the following
steps to prep your PC to manage Storage Replica.

1. Download and install Windows Admin Center.

2. Download and install the Remote Server Administration Tools .

If you're using Windows 10, version 1809 or later, install the "RSAT: Storage
Replica Module for Windows PowerShell" from Features on Demand.

3. Open a PowerShell session as administrator by selecting the Start button, typing

PowerShell, right-clicking Windows PowerShell, and then selecting Run as
administrator.

4. Enter the following command to enable the WS-Management protocol on the

local computer and set up the default configuration for remote management on
the client.

PowerShell

winrm quickconfig

5. Type Y to enable WinRM services and enable WinRM Firewall Exception.

Step 2: Provision operating system, features,
roles, storage, and network
1. Install Windows Server on both server nodes with an installation type of Windows
Server (Desktop Experience).

To use an Azure VM connected to your network via an ExpressRoute, see Adding

an Azure VM connected to your network via ExpressRoute.

７ Note

Starting in Windows Admin Center version 1910, you can configure a

destination server automatically in Azure. If you choose that option, install
Windows Server on the source server and then skip to Step 3: Set up server-
to-server replication.

2. Add network information, join the servers to the same domain as your Windows 10
management PC (if you're using one), and then restart the servers.

７ Note

From this point on, always logon as a domain user who is a member of the
built-in administrator group on all servers. Always remember to elevate your
PowerShell and CMD prompts going forward when running on a graphical
server installation or on a Windows 10 computer.

3. Connect the first set of JBOD storage enclosure, iSCSI target, FC SAN, or local fixed
disk (DAS) storage to the server in site Redmond.

4. Connect the second set of storage to the server in site Bellevue.

5. As appropriate, install latest vendor storage and enclosure firmware and drivers,
latest vendor HBA drivers, latest vendor BIOS/UEFI firmware, latest vendor network
drivers, and latest motherboard chipset drivers on both nodes. Restart nodes as
needed.

７ Note

Consult your hardware vendor documentation for configuring shared storage

and networking hardware.
6. Ensure that BIOS/UEFI settings for servers enable high performance, such as
disabling C-State, setting QPI speed, enabling NUMA, and setting highest memory
frequency. Ensure power management in Windows Server is set to High
Performance. Restart as required.

7. Configure roles as follows:

Windows Admin Center method

a. In Windows Admin Center, navigate to Server Manager, and then select
one of the servers.
b. Navigate to Roles & Features.
c. Select Features > Storage Replica, and then click Install.
d. Repeat on the other server.

Server Manager method

a. Run ServerManager.exe and create a Server Group, adding all server

nodes.

b. Install the File Server and Storage Replica roles and features on each of
the nodes and restart them.

Windows PowerShell method

On SR-SRV06 or a remote management computer, run the following

command in a Windows PowerShell console to install the required features
and roles and restart them:

PowerShell

$Servers = 'SR-SRV05','SR-SRV06'

$Servers | ForEach { Install-WindowsFeature -ComputerName $_ -Name

Storage-Replica,FS-FileServer -IncludeManagementTools -restart }

For more information on these steps, see Install or Uninstall Roles, Role
Services, or Features

8. Configure storage as follows:

） Important

You must create two volumes on each enclosure: one for data and one
for logs.
 Log and data disks must be initialized as GPT, not MBR.
The two data volumes must be of identical size.
The two log volumes should be of identical size.
All replicated data disks must have the same sector sizes.
All log disks must have the same sector sizes.
The log volumes should use flash-based storage, such as SSD. Microsoft
recommends that the log storage be faster than the data storage. Log
volumes must never be used for other workloads.
The data disks can use HDD, SSD, or a tiered combination and can use
either mirrored or parity spaces or RAID 1 or 10, or RAID 5 or RAID 50.
The log volume must be at least 9GB by default and may be larger or
smaller based on log requirements.
The File Server role is only necessary for Test-SRTopology to operate, as
it opens the necessary firewall ports for testing.

For JBOD enclosures:

a. Ensure that each server can see that site's storage enclosures only and that
the SAS connections are correctly configured.

b. Provision the storage using Storage Spaces by following Steps 1-3

provided in the Deploy Storage Spaces on a Stand-Alone Server using
Windows PowerShell or Server Manager.

For iSCSI storage:

a. Ensure that each cluster can see that site's storage enclosures only. You
should use more than one single network adapter if using iSCSI.

b. Provision the storage using your vendor documentation. If using

Windows-based iSCSI Targeting, consult iSCSI Target Block Storage, How
To.

For FC SAN storage:

a. Ensure that each cluster can see that site's storage enclosures only and
that you have properly zoned the hosts.

b. Provision the storage using your vendor documentation.

For local fixed disk storage:

 Ensure the storage doesn't contain a system volume, page file, or dump
files.

Provision the storage using your vendor documentation.

9. Start Windows PowerShell and use the Test-SRTopology cmdlet to determine if

you meet all the Storage Replica requirements. You can use the cmdlet in a
requirements-only mode for a quick test as well as a long running performance
evaluation mode.

For example, to validate the proposed nodes that each have a F: and G: volume
and run the test for 30 minutes:

PowerShell

MD c:\temp

Test-SRTopology -SourceComputerName SR-SRV05 -SourceVolumeName f: -

SourceLogVolumeName g: -DestinationComputerName SR-SRV06 -
DestinationVolumeName f: -DestinationLogVolumeName g: -
DurationInMinutes 30 -ResultPath c:\temp

） Important

When using a test server with no write IO load on the specified source volume
during the evaluation period, consider adding a workload or it will not
generate a useful report. You should test with production-like workloads in
order to see real numbers and recommended log sizes. Alternatively, simply
copy some files into the source volume during the test or download and run
DISKSPD to generate write IOs. For instance, a sample with a low write IO
workload for ten minutes to the D: volume:

Diskspd.exe -c1g -d600 -W5 -C5 -b8k -t2 -o2 -r -w5 -i100 -j100 d:\test

10. Examine the TestSrTopologyReport.html report shown in Figure 2 to ensure that

you meet the Storage Replica requirements.
 Figure 2: Storage replication topology report

Step 3: Set up server-to-server replication

Using Windows Admin Center

1. Add the source server.
a. Select the Add button.
b. Select Add server connection.
c. Type the name of the server and then select Submit.

2. On the All Connections page, select the source server.

3. Select Storage Replica from Tools panel.

4. Select New to create a new partnership. To create a new Azure VM to use as the
destination for the partnership:
a. Under Replicate with another server select Use a New Azure VM and then
select Next. If you don't see this option, make sure that you're using Windows
Admin Center version 1910 or a later version.
b. Specify your source server information and replication group name, and then
select Next.

This begins a process that automatically selects a Windows Server 2019 or

Windows Server 2016 Azure VM as a destination for the migration source.
 Storage Migration Service recommends VM sizes to match your source, but you
can override this by selecting See all sizes. Inventory data is used to
automatically configure your managed disks and their file systems, as well as
join your new Azure VM to your Active Directory domain.
c. After Windows Admin Center creates the Azure VM, provide a replication group
name and then select Create. Windows Admin Center then begins the normal
Storage Replica initial synchronization process to start protecting your data.

Here's a video showing how to use Storage Replica to migrate to Azure VMs.
https://www.youtube-nocookie.com/embed/_VqD7HjTewQ

5. Provide the details of the partnership, and then select Create (as shown in Figure
3).

Figure 3: Creating a new partnership

７ Note

Removing the partnership from Storage Replica in Windows Admin Center doesn't
remove the replication group name.

Using Windows PowerShell

Now you will configure server-to-server replication using Windows PowerShell. You
must perform all of the steps below on the nodes directly or from a remote
management computer that contains the Windows Server Remote Server Administration
Tools.
1. Ensure you are using an elevated Powershell console as an administrator.

2. Configure the server-to-server replication, specifying the source and destination

disks, the source and destination logs, the source and destination nodes, and the
log size.

PowerShell

New-SRPartnership -SourceComputerName sr-srv05 -SourceRGName rg01 -

SourceVolumeName f: -SourceLogVolumeName g: -DestinationComputerName
sr-srv06 -DestinationRGName rg02 -DestinationVolumeName f: -
DestinationLogVolumeName g:

Output:

PowerShell

DestinationComputerName : SR-SRV06
DestinationRGName : rg02
SourceComputerName : SR-SRV05
PSComputerName :

） Important

The default log size is 8GB. Depending on the results of the Test-SRTopology
cmdlet, you may decide to use -LogSizeInBytes with a higher or lower value.

3. To get replication source and destination state, use Get-SRGroup and Get-
SRPartnership as follows:

PowerShell

Get-SRGroup
Get-SRPartnership
(Get-SRGroup).replicas

Output:

PowerShell

CurrentLsn : 0
DataVolume : F:\
LastInSyncTime :
LastKnownPrimaryLsn : 1
LastOutOfSyncTime :
 NumOfBytesRecovered : 37731958784
NumOfBytesRemaining : 30851203072
PartitionId : c3999f10-dbc9-4a8e-8f9c-dd2ee6ef3e9f
PartitionSize : 68583161856
ReplicationMode : synchronous
ReplicationStatus : InitialBlockCopy
PSComputerName :

4. Determine the replication progress as follows:

a. On the source server, run the following command and examine events 5015,
5002, 5004, 1237, 5001, and 2200:

PowerShell

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica -max 20

b. On the destination server, run the following command to see the Storage
Replica events that show creation of the partnership. This event states the
number of copied bytes and the time taken. Example:

PowerShell

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica | Where-

Object {$_.ID -eq "1215"} | fl

Here's some example output:

TimeCreated : 4/8/2016 4:12:37 PM

ProviderName : Microsoft-Windows-StorageReplica
Id : 1215
Message : Block copy completed for replica.

ReplicationGroupName: rg02
ReplicationGroupId: {616F1E00-5A68-4447-830F-B0B0EFBD359C}
ReplicaName: f:\
ReplicaId: {00000000-0000-0000-0000-000000000000}
End LSN in bitmap:
LogGeneration: {00000000-0000-0000-0000-000000000000}
LogFileId: 0
CLSFLsn: 0xFFFFFFFF
Number of Bytes Recovered: 68583161856
Elapsed Time (ms): 117

７ Note
 Storage Replica dismounts the destination volumes and their drive letters
or mount points. This is by design.

c. Alternatively, the destination server group for the replica states the number of
byte remaining to copy at all times, and can be queried through PowerShell. For
example:

PowerShell

(Get-SRGroup).Replicas | Select-Object numofbytesremaining

As a progress sample (that will not terminate):

PowerShell

while($true) {

$v = (Get-SRGroup -Name "RG02").replicas | Select-Object

numofbytesremaining
[System.Console]::Write("Number of bytes remaining: {0}`r",
$v.numofbytesremaining)
Start-Sleep -s 5
}

d. On the destination server, run the following command and examine events
5009, 1237, 5001, 5015, 5005, and 2200 to understand the processing progress.
There should be no warnings of errors in this sequence. There will be many
1237 events; these indicate progress.

PowerShell

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica | FL

Step 4: Manage replication

Now you will manage and operate your server-to-server replicated infrastructure. You
can perform all of the steps below on the nodes directly or from a remote management
computer that contains the Windows Server Remote Server Administration Tools.

1. Use Get-SRPartnership and Get-SRGroup to determine the current source and

destination of replication and their status.
2. To measure replication performance, use the Get-Counter cmdlet on both the
source and destination nodes. The counter names are:

\Storage Replica Partition I/O Statistics(*)\Number of times flush paused

\Storage Replica Partition I/O Statistics(*)\Number of pending flush I/O

\Storage Replica Partition I/O Statistics(*)\Number of requests for last log

write

\Storage Replica Partition I/O Statistics(*)\Avg. Flush Queue Length

\Storage Replica Partition I/O Statistics(*)\Current Flush Queue Length

\Storage Replica Partition I/O Statistics(*)\Number of Application Write

Requests

\Storage Replica Partition I/O Statistics(*)\Avg. Number of requests per log

write

\Storage Replica Partition I/O Statistics(*)\Avg. App Write Latency

\Storage Replica Partition I/O Statistics(*)\Avg. App Read Latency

\Storage Replica Statistics(*)\Target RPO

\Storage Replica Statistics(*)\Current RPO

\Storage Replica Statistics(*)\Avg. Log Queue Length

\Storage Replica Statistics(*)\Current Log Queue Length

\Storage Replica Statistics(*)\Total Bytes Received

\Storage Replica Statistics(*)\Total Bytes Sent

\Storage Replica Statistics(*)\Avg. Network Send Latency

\Storage Replica Statistics(*)\Replication State

\Storage Replica Statistics(*)\Avg. Message Round Trip Latency

\Storage Replica Statistics(*)\Last Recovery Elapsed Time

\Storage Replica Statistics(*)\Number of Flushed Recovery Transactions

\Storage Replica Statistics(*)\Number of Recovery Transactions

\Storage Replica Statistics(*)\Number of Flushed Replication Transactions

 \Storage Replica Statistics(*)\Number of Replication Transactions

\Storage Replica Statistics(*)\Max Log Sequence Number

\Storage Replica Statistics(*)\Number of Messages Received

\Storage Replica Statistics(*)\Number of Messages Sent

For more information on performance counters in Windows PowerShell, see Get-

Counter.

3. To move the replication direction from one site, use the Set-SRPartnership cmdlet.

PowerShell

Set-SRPartnership -NewSourceComputerName sr-srv06 -SourceRGName rg02 -

DestinationComputerName sr-srv05 -DestinationRGName rg01

２ Warning

Windows Server prevents role switching when the initial sync is ongoing, as it
can lead to data loss if you attempt to switch before allowing initial replication
to complete. Don't force switch directions until the initial sync is complete.

Check the event logs to see the direction of replication change and recovery mode
occur, and then reconcile. Write IOs can then write to the storage owned by the
new source server. Changing the replication direction will block write IOs on the
previous source computer.

4. To remove replication, use Get-SRGroup , Get-SRPartnership , Remove-SRGroup , and

Remove-SRPartnership on each node. Ensure you run the Remove-SRPartnership

cmdlet on the current source of replication only, not on the destination server. Run
Remove-SRGroup on both servers. For example, to remove all replication from two
servers:

PowerShell

Get-SRPartnership
Get-SRPartnership | Remove-SRPartnership
Get-SRGroup | Remove-SRGroup

Replacing DFS Replication with Storage Replica

Many Microsoft customers deploy DFS Replication as a disaster recovery solution for
unstructured user data like home folders and departmental shares. DFS Replication has
shipped in Windows Server 2003 R2 and all later operating systems and operates on low
bandwidth networks, which makes it attractive for high latency and low change
environments with many nodes. However, DFS Replication has notable limitations as a
data replication solution:

It doesn't replicate in-use or open files.

It doesn't replicate synchronously.
Its asynchronous replication latency can be many minutes, hours, or even days.
It relies on a database that can require lengthy consistency checks after a power
interruption.
It's generally configured as multi-master, which allows changes to flow in both
directions, possibly overwriting newer data.

Storage Replica has none of these limitations. It does, however, have several that might
make it less interesting in some environments:

It only allows one-to-one replication between volumes. It's possible to replicate

different volumes between multiple servers.
While it supports asynchronous replication, it's not designed for low bandwidth,
high latency networks.
It doesn't allow user access to the protected data on the destination while
replication is ongoing

If these are not blocking factors, Storage Replica allows you to replace DFS Replication
servers with this newer technology. The process is, at a high level:

1. Install Windows Server on two servers and configure your storage. This could mean
upgrading an existing set of servers or cleanly installing.

2. Ensure that any data you want to replicate exists on one or more data volumes and
not on the C: drive. a. You can also seed the data on the other server to save time,
using a backup or file copies, as well as use thin provisioned storage. Making the
metadata-like security match perfectly is unnecessary, unlike DFS Replication.

3. Share the data on your source server and make it accessible through a DFS
namespace. This is important, to ensure that users can still access it if the server
name changes to one in a disaster site. a. You can create matching shares on the
destination server, which will be unavailable during normal operations, b. Don't
add the destination server to the DFS Namespaces namespace, or if you do, ensure
that all its folder targets are disabled.
 4. Enable Storage Replica replication and complete initial sync. Replication can be
either synchronous or asynchronous. a. However, synchronous is recommended in
order to guarantee IO data consistency on the destination server. b. We strongly
recommend enabling Volume Shadow Copies and periodically taking snapshots
with VSSADMIN or your other tools of choice. This will guarantee applications flush
their data files to disk consistently. In the event of a disaster, you can recover files
from snapshots on the destination server that might have been partially replicated
asynchronously. Snapshots replicate along with files.

5. Operate normally until there is a disaster.

6. Switch the destination server to be the new source, which surfaces its replicated
volumes to users.

7. If using synchronous replication, no data restore will be necessary unless the user
was using an application that was writing data without transaction protection (this
is irrespective of replication) during loss of the source server. If using asynchronous
replication, the need for a VSS snapshot mount is higher but consider using VSS in
all circumstances for application consistent snapshots.

8. Add the server and its shares as a DFS Namespaces folder target.

9. Users can then access their data.

７ Note

Disaster Recovery planning is a complex subject and requires great attention

to detail. Creation of runbooks and the performance of annual live failover
drills is highly recommended. When an actual disaster strikes, chaos will rule
and experienced personnel may be unavailable.

Adding an Azure VM connected to your

network via ExpressRoute
1. Create an ExpressRoute in the Azure portal.
After the ExpressRoute is approved, a resource group is added to the subscription
- navigate to Resource groups to view this new group. Take note of the virtual
 network name.

Figure 4: The resources associated with an ExpressRoute - take note of the

virtual network name

2. Create a new resource group.

3. Add a network security group. When creating it, select the subscription ID
associated with the ExpressRoute you created, and select the resource group you
just created as well.

Add any inbound and outbound security rules you need to the network security
group. For example, you might want to allow Remote Desktop access to the VM.

4. Create an Azure VM with the following settings (shown in Figure 5):

Public IP address: None

Virtual network: Select the virtual network you took note of from the
resource group added with the ExpressRoute.
Network security group (firewall): Select the network security group you
created previously.
 Figure 5: Creating a VM while selecting ExpressRoute network settings

5. After the VM is created, see Step 2: Provision operating system, features, roles,
storage, and network.

Related Topics
Storage Replica Overview
Stretch Cluster Replication Using Shared Storage
Cluster to Cluster Storage Replication
Storage Replica: Known Issues
Storage Replica: Frequently Asked Questions
Storage Spaces Direct in Windows Server 2016
Cluster to cluster Storage Replication
Article • 03/29/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

Storage Replica can replicate volumes between clusters, including the replication of
clusters using Storage Spaces Direct. The management and configuration is similar to
server-to-server replication.

You will configure these computers and storage in a cluster-to-cluster configuration,

where one cluster replicates its own set of storage with another cluster and its set of
storage. These nodes and their storage should be located in separate physical sites,
although it is not required.

） Important

In this test, the four servers are an example. You can use any number of servers
supported by Microsoft in each cluster, which is currently 8 for a Storage Spaces
Direct cluster and 64 for a shared storage cluster.

This guide does not cover configuring Storage Spaces Direct. For information about
configuring Storage Spaces Direct, see Storage Spaces Direct overview.

This walkthrough uses the following environment as an example:

Two member servers, named SR-SRV01 and SR-SRV02 that are later formed into a
cluster named SR-SRVCLUSA.

Two member servers named SR-SRV03 and SR-SRV04 that are later formed into a
cluster named SR-SRVCLUSB.

A pair of logical "sites" that represent two different data centers, with one called
Redmond and one called Bellevue.
FIGURE 1: Cluster to cluster Replication

Prerequisites
Active Directory Domain Services forest (does not need to run Windows Server
2016).
4-128 servers (two clusters of 2-64 servers) running Windows Server 2019 or
Windows Server 2016, Datacenter Edition. If you're running Windows Server 2019,
you can instead use Standard Edition if you're OK replicating only a single volume
up to 2 TB in size.
Two sets of storage, using SAS JBODs, fibre channel SAN, Shared VHDX, Storage
Spaces Direct, or iSCSI target. The storage should contain a mix of HDD and SSD
media. You will make each storage set available only to each of the clusters, with
no shared access between clusters.
Each set of storage must allow creation of at least two virtual disks, one for
replicated data and one for logs. The physical storage must have the same sector
sizes on all the data disks. The physical storage must have the same sector sizes on
all the log disks.
At least one ethernet/TCP connection on each server for synchronous replication,
but preferably RDMA.
Appropriate firewall and router rules to allow ICMP, SMB (port 445, plus 5445 for
SMB Direct) and WS-MAN (port 5985) bi-directional traffic between all nodes.
A network between servers with enough bandwidth to contain your IO write
workload and an average of =5ms round trip latency, for synchronous replication.
Asynchronous replication does not have a latency recommendation.
The replicated storage cannot be located on the drive containing the Windows
operating system folder.
 There are important considerations & limitations for Storage Spaces Direct
replication - please review the detailed information below.

Many of these requirements can be determined by using the Test-SRTopology cmdlet.

You get access to this tool if you install Storage Replica or the Storage Replica
Management Tools features on at least one server. There is no need to configure
Storage Replica to use this tool, only to install the cmdlet. More information is included
in the steps below.

Step 1: Provision operating system, features,

roles, storage, and network
1. Install Windows Server on all four server nodes with an installation type of
Windows Server (Desktop Experience).

2. Add network information and join them to the domain, then restart them.

） Important

From this point on, always logon as a domain user who is a member of the
built-in administrator group on all servers. Always remember to elevate your
Windows PowerShell and CMD prompts going forward when running on a
graphical server installation or on a Windows 10 computer.

3. Connect first set of JBOD storage enclosure, iSCSI target, FC SAN, or local fixed
disk (DAS) storage to the server in site Redmond.

4. Connect second set of storage to the server in site Bellevue.

5. As appropriate, install latest vendor storage and enclosure firmware and drivers,
latest vendor HBA drivers, latest vendor BIOS/UEFI firmware, latest vendor network
drivers, and latest motherboard chipset drivers on all four nodes. Restart nodes as
needed.

７ Note

Consult your hardware vendor documentation for configuring shared storage

and networking hardware.

6. Ensure that BIOS/UEFI settings for servers enable high performance, such as
disabling C-State, setting QPI speed, enabling NUMA, and setting highest memory
 frequency. Ensure power management in Windows Server is set to high
performance. Restart as required.

7. Configure roles as follows:

Graphical method

a. Run ServerManager.exe and create a server group, adding all server

nodes.

b. Install the File Server and Storage Replica roles and features on each of
the nodes and restart them.

Windows PowerShell method

On SR-SRV04 or a remote management computer, run the following

command in a Windows PowerShell console to install the required features
and roles for a stretch cluster on the four nodes and restart them:

PowerShell

$Servers = 'SR-SRV01','SR-SRV02','SR-SRV03','SR-SRV04'

$Servers | ForEach { Install-WindowsFeature -ComputerName $_ -Name

Storage-Replica,Failover-Clustering,FS-FileServer -
IncludeManagementTools -restart }

For more information on these steps, see Install or Uninstall Roles, Role
Services, or Features

8. Configure storage as follows:

） Important

You must create two volumes on each enclosure: one for data and one
for logs.
Log and data disks must be initialized as GPT, not MBR.
The two data volumes must be of identical size.
The two log volumes should be of identical size.
All replicated data disks must have the same sector sizes.
All log disks must have the same sector sizes.
The log volumes should use flash-based storage, such as SSD. Microsoft
recommends that the log storage be faster than the data storage. Log
 volumes must never be used for other workloads.
The data disks can use HDD, SSD, or a tiered combination and can use
either mirrored or parity spaces or RAID 1 or 10, or RAID 5 or RAID 50.
The log volume must be at least 8GB by default and may be larger or
smaller based on log requirements.
When using Storage Spaces Direct (Storage Spaces Direct) with an NVME
or SSD cache, you see a greater than expected increase in latency when
configuring Storage Replica replication between Storage Spaces Direct
clusters. The change in latency is proportionally much higher than you
see when using NVME and SSD in a performance + capacity
configuration and no HDD tier nor capacity tier.

This issue occurs due to architectural limitations within SR's log mechanism
combined with the extremely low latency of NVME when compared to slower
media. When using Storage Spaces Direct Storage Spaces Direct cache, all IO of SR
logs, along with all recent read/write IO of applications, will occur in the cache and
never on the performance or capacity tiers. This means that all SR activity happens
on the same speed media - this configuration is not supported not recommended
(see https://aka.ms/srfaq for log recommendations).

When using Storage Spaces Direct with HDDs, you cannot disable or avoid the
cache. As a workaround, if using just SSD and NVME, you can configure just
performance and capacity tiers. If using that configuration, and by placing the SR
logs on the performance tier only with the data volumes they service being on the
capacity tier only, you will avoid the high latency issue described above. The same
could be done with a mix of faster and slower SSDs and no NVME.

This workaround is of course not ideal and some customers may not be able to
make use of it. The SR team is working on optimizations and updated log
mechanism for the future to reduce these artificial bottlenecks that occur. There is
no ETA for this, but when available to TAP customers for testing, this FAQ will be
updated.

For JBOD enclosures:

1. Ensure that each cluster can see that site's storage enclosures only and that the
SAS connections are correctly configured.

2. Provision the storage using Storage Spaces by following Steps 1-3 provided in the
Deploy Storage Spaces on a Stand-Alone Server using Windows PowerShell or
Server Manager.
 For iSCSI Target storage:

1. Ensure that each cluster can see that site's storage enclosures only. You should use
more than one single network adapter if using iSCSI.

2. Provision the storage using your vendor documentation. If using Windows-based

iSCSI Targeting, consult iSCSI Target Block Storage, How To.

For FC SAN storage:

1. Ensure that each cluster can see that site's storage enclosures only and that you
have properly zoned the hosts.

2. Provision the storage using your vendor documentation.

For Storage Spaces Direct:

1. Ensure that each cluster can see that site's storage enclosures only by deploying
Storage Spaces Direct.

2. Ensure that the SR log volumes will always be on the fastest flash storage and the
data volumes on slower high capacity storage.

3. Start Windows PowerShell and use the Test-SRTopology cmdlet to determine if you
meet all the Storage Replica requirements. You can use the cmdlet in a
requirements-only mode for a quick test as well as a long running performance
evaluation mode. For example,

PowerShell

MD c:\temp

Test-SRTopology -SourceComputerName SR-SRV01 -SourceVolumeName f: -

SourceLogVolumeName g: -DestinationComputerName SR-SRV03 -
DestinationVolumeName f: -DestinationLogVolumeName g: -
DurationInMinutes 30 -ResultPath c:\temp

） Important

When using a test server with no write IO load on the specified source volume
during the evaluation period, consider adding a workload or it will not
generate a useful report. You should test with production-like workloads in
order to see real numbers and recommended log sizes. Alternatively, simply
copy some files into the source volume during the test or download and run
DISKSPD to generate write IOs. For instance, a sample with a low write IO
 workload for five minutes to the D: volume: Diskspd.exe -c1g -d300 -W5 -C5 -
b8k -t2 -o2 -r -w5 -h d:\test.dat

4. Examine the TestSrTopologyReport.html report to ensure that you meet the

Storage Replica requirements.

Step 2: Configure two Scale-Out File Server

Failover Clusters
You will now create two normal failover clusters. After configuration, validation, and
testing, you will replicate them using Storage Replica. You can perform all of the steps
below on the cluster nodes directly or from a remote management computer that
contains the Windows Server Remote Server Administration Tools.

Graphical method
1. Run cluadmin.msc against a node in each site.

2. Validate the proposed cluster and analyze the results to ensure you can continue.
The example used below are SR-SRVCLUSA and SR-SRVCLUSB.

3. Create the two clusters. Ensure that the cluster names are 15 characters or fewer.

4. Configure a File Share Witness or Cloud Witness.

 ７ Note

WIndows Server now includes an option for Cloud (Azure)-based Witness. You
can choose this quorum option instead of the file share witness.

２ Warning

For more information about quorum configuration, see the Witness

Configuration section in Configure and Manage Quorum. For more
information on the Set-ClusterQuorum cmdlet, see Set-ClusterQuorum.

5. Add one disk in the Redmond site to the cluster CSV. To do so, right click a source
disk in the Disks node of the Storage section, and then click Add to Cluster Shared
Volumes.

6. Create the clustered Scale-Out File Servers on both clusters using the instructions
in Configure Scale-Out File Server

Windows PowerShell method

1. Test the proposed cluster and analyze the results to ensure you can continue:

PowerShell

Test-Cluster SR-SRV01,SR-SRV02
Test-Cluster SR-SRV03,SR-SRV04

2. Create the clusters (you must specify your own static IP addresses for the clusters).
Ensure that each cluster name is 15 characters or fewer:

PowerShell

New-Cluster -Name SR-SRVCLUSA -Node SR-SRV01,SR-SRV02 -StaticAddress

<your IP here>
New-Cluster -Name SR-SRVCLUSB -Node SR-SRV03,SR-SRV04 -StaticAddress
<your IP here>

3. Configure a File Share Witness or Cloud (Azure) witness in each cluster that points
to a share hosted on the domain controller or some other independent server. For
example:

PowerShell
 Set-ClusterQuorum -FileShareWitness \\someserver\someshare

７ Note

WIndows Server now includes an option for Cloud (Azure)-based Witness. You
can choose this quorum option instead of the file share witness.

２ Warning

For more information about quorum configuration, see the Witness

Configuration section in Configure and Manage Quorum. For more
information on the Set-ClusterQuorum cmdlet, see Set-ClusterQuorum.

4. Create the clustered Scale-Out File Servers on both clusters using the instructions
in Configure Scale-Out File Server

Step 3: Set up Cluster to Cluster Replication

using Windows PowerShell
Now you will set up cluster-to-cluster replication using Windows PowerShell. You can
perform all of the steps below on the nodes directly or from a remote management
computer that contains the Windows Server Remote Server Administration Tools

1. Grant the first cluster full access to the other cluster by running the Grant-
SRAccess cmdlet on any node in the first cluster, or remotely. Windows Server
Remote Server Administration Tools

PowerShell

Grant-SRAccess -ComputerName SR-SRV01 -Cluster SR-SRVCLUSB

2. Grant the second cluster full access to the other cluster by running the Grant-
SRAccess cmdlet on any node in the second cluster, or remotely.

PowerShell

Grant-SRAccess -ComputerName SR-SRV03 -Cluster SR-SRVCLUSA

3. Configure the cluster-to-cluster replication, specifying the source and destination
disks, the source and destination logs, the source and destination cluster names,
and the log size. You can perform this command locally on the server or using a
remote management computer.

PowerShell

New-SRPartnership -SourceComputerName SR-SRVCLUSA -SourceRGName rg01 -

SourceVolumeName c:\ClusterStorage\Volume2 -SourceLogVolumeName f: -
DestinationComputerName SR-SRVCLUSB -DestinationRGName rg02 -
DestinationVolumeName c:\ClusterStorage\Volume2 -
DestinationLogVolumeName f:

２ Warning

The default log size is 8GB. Depending on the results of the Test-SRTopology
cmdlet, you may decide to use -LogSizeInBytes with a higher or lower value.

4. To get replication source and destination state, use Get-SRGroup and Get-
SRPartnership as follows:

PowerShell

Get-SRGroup
Get-SRPartnership
(Get-SRGroup).replicas

5. Determine the replication progress as follows:

a. On the source server, run the following command and examine events 5015,
5002, 5004, 1237, 5001, and 2200:

PowerShell

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica -max 20

b. On the destination server, run the following command to see the Storage
Replica events that show creation of the partnership. This event states the
number of copied bytes and the time taken. Example:

PowerShell

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica | Where-

Object {$_.ID -eq "1215"} | Format-List
 Here's an example of the output:

TimeCreated : 4/8/2016 4:12:37 PM

ProviderName : Microsoft-Windows-StorageReplica
Id : 1215
Message : Block copy completed for replica.
ReplicationGroupName: rg02
ReplicationGroupId:
{616F1E00-5A68-4447-830F-B0B0EFBD359C}
ReplicaName: f:\
ReplicaId: {00000000-0000-0000-0000-000000000000}
End LSN in bitmap:
LogGeneration: {00000000-0000-0000-0000-000000000000}
LogFileId: 0
CLSFLsn: 0xFFFFFFFF
Number of Bytes Recovered: 68583161856
Elapsed Time (seconds): 117

c. Alternately, the destination server group for the replica states the number of
byte remaining to copy at all times, and can be queried through PowerShell. For
example:

PowerShell

(Get-SRGroup).Replicas | Select-Object numofbytesremaining

As a progress sample (that will not terminate):

PowerShell

while($true) {
$v = (Get-SRGroup -Name "Replication 2").replicas | Select-Object
numofbytesremaining
[System.Console]::Write("Number of bytes remaining: {0}`n",
$v.numofbytesremaining)
Start-Sleep -s 5
}

6. On the destination server in the destination cluster, run the following command
and examine events 5009, 1237, 5001, 5015, 5005, and 2200 to understand the
processing progress. There should be no warnings of errors in this sequence. There
will be many 1237 events; these indicate progress.

PowerShell
 Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica | FL

７ Note

The destination cluster disk will always show as Online (No Access) when
replicated.

Step 4: Manage replication

Now you will manage and operate your cluster-to-cluster replication. You can perform
all of the steps below on the cluster nodes directly or from a remote management
computer that contains the Windows Server Remote Server Administration Tools.

1. Use Get-ClusterGroup or Failover Cluster Manager to determine the current

source and destination of replication and their status. Windows Server Remote
Server Administration Tools

2. To measure replication performance, use the Get-Counter cmdlet on both the

source and destination nodes. The counter names are:

\Storage Replica Partition I/O Statistics(*)\Number of times flush paused

\Storage Replica Partition I/O Statistics(*)\Number of pending flush I/O

\Storage Replica Partition I/O Statistics(*)\Number of requests for last log

write

\Storage Replica Partition I/O Statistics(*)\Avg. Flush Queue Length

\Storage Replica Partition I/O Statistics(*)\Current Flush Queue Length

\Storage Replica Partition I/O Statistics(*)\Number of Application Write

Requests

\Storage Replica Partition I/O Statistics(*)\Avg. Number of requests per log

write

\Storage Replica Partition I/O Statistics(*)\Avg. App Write Latency

\Storage Replica Partition I/O Statistics(*)\Avg. App Read Latency

\Storage Replica Statistics(*)\Target RPO

\Storage Replica Statistics(*)\Current RPO

 \Storage Replica Statistics(*)\Avg. Log Queue Length

\Storage Replica Statistics(*)\Current Log Queue Length

\Storage Replica Statistics(*)\Total Bytes Received

\Storage Replica Statistics(*)\Total Bytes Sent

\Storage Replica Statistics(*)\Avg. Network Send Latency

\Storage Replica Statistics(*)\Replication State

\Storage Replica Statistics(*)\Avg. Message Round Trip Latency

\Storage Replica Statistics(*)\Last Recovery Elapsed Time

\Storage Replica Statistics(*)\Number of Flushed Recovery Transactions

\Storage Replica Statistics(*)\Number of Recovery Transactions

\Storage Replica Statistics(*)\Number of Flushed Replication Transactions

\Storage Replica Statistics(*)\Number of Replication Transactions

\Storage Replica Statistics(*)\Max Log Sequence Number

\Storage Replica Statistics(*)\Number of Messages Received

\Storage Replica Statistics(*)\Number of Messages Sent

For more information on performance counters in Windows PowerShell, see Get-

Counter.

3. To move the replication direction from one site, use the Set-SRPartnership cmdlet.

PowerShell

Set-SRPartnership -NewSourceComputerName SR-SRVCLUSB -SourceRGName rg02

-DestinationComputerName SR-SRVCLUSA -DestinationRGName rg01

７ Note

Windows Server prevents role switching when initial sync is ongoing, as it can
lead to data loss if you attempt to switch before allowing initial replication to
complete. Do not force switch directions until initial sync is complete.
 Check the event logs to see the direction of replication change and recovery mode
occur, and then reconcile. Write IOs can then write to the storage owned by the
new source server. Changing the replication direction will block write IOs on the
previous source computer.

７ Note

The destination cluster disk will always show as Online (No Access) when
replicated.

4. To change the log size from the default 8GB, use Set-SRGroup on both the source
and destination Storage Replica groups.

） Important

The default log size is 8GB. Depending on the results of the Test-SRTopology
cmdlet, you may decide to use -LogSizeInBytes with a higher or lower value.

5. To remove replication, use Get-SRGroup, Get-SRPartnership, Remove-SRGroup,

and Remove-SRPartnership on each cluster.

PowerShell

Get-SRPartnership | Remove-SRPartnership
Get-SRGroup | Remove-SRGroup

７ Note

Storage Replica dismounts the destination volumes. This is by design.

Additional References
Storage Replica Overview
Stretch Cluster Replication Using Shared Storage
Server to Server Storage Replication
Storage Replica: Known Issues
Storage Replica: Frequently Asked Questions
Storage Spaces Direct in Windows Server 2016
Cluster to Cluster Storage Replica cross
region in Azure
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

You can configure Cluster to Cluster Storage Replicas for cross-region applications in
Azure. In the examples below, we use a two-node cluster, but Cluster to Cluster storage
replica isn't restricted to a two-node cluster. The illustration below is a two-node
Storage Space Direct cluster that can communicate with each other, are in the same
domain, and are cross-region.

Watch the video below for a complete walk-through of the process.

https://www.microsoft.com/en-us/videoplayer/embed/RE26xeW?postJsllMsg=true

） Important

All referenced examples are specific to the illustration above.

1. In the Azure portal, create resource groups in two different regions.

For example, SR-AZ2AZ in West US 2 and SR-AZCROSS in West Central US, as

shown above.

2. Create two availability sets , one in each resource group for each cluster.
 Availability set (az2azAS1) in (SR-AZ2AZ)
Availability set (azcross-AS) in (SR-AZCROSS)

3. Create two virtual networks

Create the virtual network (az2az-Vnet) in the first resource group (SR-
AZ2AZ), having one subnet and one Gateway subnet.
Create the virtual network (azcross-VNET) in the second resource group
(SR-AZCROSS), having one subnet and one Gateway subnet.

4. Create two network security groups

Create the network security group (az2az-NSG) in the first resource group
(SR-AZ2AZ).
Create the network security group (azcross-NSG) in the second resource
group (SR-AZCROSS).

Add one Inbound security rule for RDP:3389 to both Network Security Groups. You
can choose to remove this rule once you finish your setup.

5. Create Windows Server virtual machines in the previously created resource

groups.

Domain Controller (az2azDC). You can choose to create a 3rd availability set for
your domain controller or add the domain controller in one of the two availability
set. If you are adding this to the availability set created for the two clusters, assign
it a Standard public IP address during VM creation.

Install Active Directory Domain Service.

Create a domain (contoso.com)
Create a user with administrator privileges (contosoadmin)

Create two virtual machines (az2az1, az2az2) in the resource group (SR-AZ2AZ)
using virtual network (az2az-Vnet) and network security group (az2az-NSG) in
availability set (az2azAS1). Assign a standard public IP address to each virtual
machine during the creation itself.

Add at-least two managed disks to each machine

Install Failover Clustering and the Storage Replica feature

Create two virtual machines (azcross1, azcross2) in the resource group (SR-
AZCROSS) using virtual network (azcross-VNET) and network security group
(azcross-NSG) in availability set (azcross-AS). Assign standard Public IP address to
each virtual machine during the creation itself
 Add at least two managed disks to each machine
Install Failover Clustering and the Storage Replica feature

Connect all the nodes to the domain and provide administrator privileges to the
previously created user.

Change the DNS Server of the virtual network to domain controller private IP
address.

In the example, the domain controller az2azDC has private IP address

(10.3.0.8). In the Virtual Network (az2az-Vnet and azcross-VNET) change DNS
Server 10.3.0.8.

In the example, connect all the nodes to "contoso.com" and provide

administrator privileges to "contosoadmin".

Login as contosoadmin from all the nodes.

6. Create the clusters (SRAZC1, SRAZCross).

Below is the PowerShell commands for the example

PowerShell

New-Cluster -Name SRAZC1 -Node az2az1,az2az2 –StaticAddress

10.3.0.100

PowerShell

New-Cluster -Name SRAZCross -Node azcross1,azcross2 –StaticAddress

10.0.0.10

7. Enable storage spaces direct.

PowerShell

Enable-clusterS2D

７ Note

For each cluster create virtual disk and volume. One for the data and another
for the log.
 8. Create an internal Standard SKU Load Balancer for each cluster (azlbr1,
azlbazcross).

Provide the Cluster IP address as static private IP address for the load balancer.

azlbr1 => Frontend IP: 10.3.0.100 (Pick up an unused IP address from the
Virtual network (az2az-Vnet) subnet)
Create Backend Pool for each load balancer. Add the associated cluster
nodes.
Create Health Probe: port 59999
Create Load Balance Rule: Allow HA ports, with enabled Floating IP.

Provide the Cluster IP address as static private IP address for the load balancer.

azlbazcross => Frontend IP: 10.0.0.10 (Pick up an unused IP address from the
Virtual network (azcross-VNET) subnet)
Create Backend Pool for each load balancer. Add the associated cluster
nodes.
Create Health Probe: port 59999
Create Load Balance Rule: Allow HA ports, with enabled Floating IP.

9. Create Virtual network gateway for Vnet-to-Vnet connectivity.

Create the first virtual network gateway (az2az-VNetGateway) in the first

resource group (SR-AZ2AZ)

Gateway Type = VPN, and VPN type = Route-based

Create the second Virtual network gateway (azcross-VNetGateway) in the

second resource group (SR-AZCROSS)

Gateway Type = VPN, and VPN type = Route-based

Create a Vnet-to-Vnet connection from first Virtual network gateway to

second Virtual network gateway. Provide a shared key

Create a Vnet-to-Vnet connection from second Virtual network gateway to

first Virtual network gateway. Provide the same shared key as provided in the
step above.

10. On each cluster node, open port 59999 (Health Probe).

Run the following command on each node:

PowerShell
 netsh advfirewall firewall add rule name=PROBEPORT dir=in
protocol=tcp action=allow localport=59999 remoteip=any profile=any

11. Instruct the cluster to listen for Health Probe messages on Port 59999 and respond
from the node that currently owns this resource.

Run it once from any one node of the cluster, for each cluster.

In our example, make sure to change the "ILBIP" according to your configuration
values. Run the following command from any one node az2az1/az2az2

PowerShell

$ClusterNetworkName = "Cluster Network 1" # Cluster network name (Use

Get-ClusterNetwork on Windows Server 2012 or higher to find the name.
And use Get-ClusterResource to find the IPResourceName).
$IPResourceName = "Cluster IP Address" # IP Address cluster resource
name.
$ILBIP = "10.3.0.100" # IP Address in Internal Load Balancer (ILB) -
The static IP address for the load balancer configured in the Azure
portal.
[int]$ProbePort = 59999
Get-ClusterResource $IPResourceName | Set-ClusterParameter -Multiple
@{"Address"="$ILBIP";"ProbePort"=$ProbePort;"SubnetMask"="255.255.255.2
55";"Network"="$ClusterNetworkName";"ProbeFailureThreshold"=5;"EnableDh
cp"=0}

12. Run the following command from any one node azcross1/azcross2

PowerShell

$ClusterNetworkName = "Cluster Network 1" # Cluster network name (Use

Get-ClusterNetwork on Windows Server 2012 or higher to find the name.
And use Get-ClusterResource to find the IPResourceName).
$IPResourceName = "Cluster IP Address" # IP Address cluster resource
name.
$ILBIP = "10.0.0.10" # IP Address in Internal Load Balancer (ILB) -
The static IP address for the load balancer configured in the Azure
portal.
[int]$ProbePort = 59999
Get-ClusterResource $IPResourceName | Set-ClusterParameter -Multiple
@{"Address"="$ILBIP";"ProbePort"=$ProbePort;"SubnetMask"="255.255.255.2
55";"Network"="$ClusterNetworkName";"ProbeFailureThreshold"=5;"EnableDh
cp"=0}

Make sure both clusters can connect / communicate with each other.
 Either use "Connect to Cluster" feature in Failover cluster manager to connect to
the other cluster or check other cluster responds from one of the nodes of the
current cluster.

From the example we've been using:

PowerShell

Get-Cluster -Name SRAZC1 (ran from azcross1)

PowerShell

Get-Cluster -Name SRAZCross (ran from az2az1)

13. Create cloud witness for both clusters. Create two storage accounts
(az2azcw,azcrosssa) in Azure, one for each cluster in each resource group (SR-
AZ2AZ, SR-AZCROSS).

Copy the storage account name and key from "access keys"
Create the cloud witness from "failover cluster manager" and use the above
account name and key to create it.

14. Run cluster validation tests before moving on to the next step

15. Start Windows PowerShell and use the Test-SRTopology cmdlet to determine if you
meet all the Storage Replica requirements. You can use the cmdlet in a
requirements-only mode for a quick test as well as a long running performance
evaluation mode.

16. Configure cluster-to-cluster storage replica. Grant access from one cluster to
another cluster in both directions:

From our example:

PowerShell

Grant-SRAccess -ComputerName az2az1 -Cluster SRAZCross

If you're using Windows Server 2016, then also run this command:

PowerShell

Grant-SRAccess -ComputerName azcross1 -Cluster SRAZC1

 17. Create SR-Partnership for the two clusters:

For cluster SRAZC1

Volume location:- c:\ClusterStorage\DataDisk1
Log location:- g:
For cluster SRAZCross
Volume location:- c:\ClusterStorage\DataDiskCross
Log location:- g:

Run the command:

PowerShell

PowerShell

New-SRPartnership -SourceComputerName SRAZC1 -SourceRGName rg01 -

SourceVolumeName c:\ClusterStorage\DataDisk1 -SourceLogVolumeName g: -
DestinationComputerName SRAZCross -DestinationRGName rg02 -
DestinationVolumeName c:\ClusterStorage\DataDiskCross -
DestinationLogVolumeName g:
Cluster to cluster Storage Replica within
the same region in Azure
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

You can configure cluster to cluster storage replication within the same region in Azure.
In the examples below, we use a two-node cluster, but cluster to cluster storage replica
isn't restricted to a two-node cluster. The illustration below is a two-node Storage Space
Direct cluster that can communicate with each other, are in the same domain, and within
the same region.

Watch the videos below for a complete walk-through of the process.

Part one
https://www.microsoft.com/en-us/videoplayer/embed/RE26f2Y?postJsllMsg=true

Part two
https://www.microsoft.com/en-us/videoplayer/embed/RE269Pq?postJsllMsg=true

） Important

All referenced examples are specific to the illustration above.

1. Create a resource group in the Azure portal in a region (SR-AZ2AZ in West US
2).

2. Create two availability sets in the resource group (SR-AZ2AZ) created above,
one for each cluster. a. Availability set (az2azAS1) b. Availability set (az2azAS2)

3. Create a virtual network (az2az-Vnet) in the previously created resource group

(SR-AZ2AZ), having at-least one subnet.

4. Create a network security group (az2az-NSG), and add one Inbound security rule
for RDP:3389. You can choose to remove this rule once you finish your setup.

5. Create Windows Server virtual machines in the previously created Resource

group (SR-AZ2AZ). Use the previously created virtual network (az2az-Vnet) and
network security group (az2az-NSG).

Domain Controller (az2azDC). You can choose to create a third availability set for
your domain controller or add the domain controller in one of the two availability
sets. If you are adding this to the availability set created for the two clusters, assign
it a Standard public IP address during VM creation.

Install Active Directory Domain Service.

Create a domain (Contoso.com)
Create a user with administrator privileges (contosoadmin)
Create two virtual machines (az2az1, az2az2) in the first availability set
(az2azAS1). Assign a standard Public IP address to each virtual machine
during the creation itself.
Add at-least 2 managed disks to each machine
Install Failover Clustering and Storage Replica feature
Create two virtual machines (az2az3, az2az4) in the second availability set
(az2azAS2). Assign standard Public IP address to each virtual machine during
the creation itself.
Add at-least 2 managed disks to each machine.
Install Failover Clustering and Storage Replica feature.

6. Connect all the nodes to the domain and provide Administrator privileges to the
previously created user.

7. Change the DNS Server of the virtual network to domain controller private IP
address.

8. In our example, the domain controller az2azDC has private IP address (10.3.0.8). In
the Virtual Network (az2az-Vnet) change DNS Server 10.3.0.8. Connect all the
nodes to "Contoso.com" and provide administrator privileges to "contosoadmin".
 Login as contosoadmin from all the nodes.

9. Create the clusters (SRAZC1, SRAZC2). Below is the PowerShell commands for our
example

PowerShell

New-Cluster -Name SRAZC1 -Node az2az1,az2az2 –StaticAddress 10.3.0.100

PowerShell

New-Cluster -Name SRAZC2 -Node az2az3,az2az4 –StaticAddress 10.3.0.101

10. Enable storage spaces direct

PowerShell

Enable-clusterS2D

For each cluster create virtual disk and volume. One for the data and another for
the log.

11. Create an internal Standard SKU Load Balancer for each cluster (azlbr1,azlbr2).

Provide the Cluster IP address as static private IP address for the load balancer.

azlbr1 => Frontend IP: 10.3.0.100 (Pick up an unused IP address from the
Virtual network (az2az-Vnet) subnet)
Create Backend Pool for each load balancer. Add the associated cluster
nodes.
Create Health Probe: port 59999
Create Load Balance Rule: Allow HA ports, with enabled Floating IP.

Provide the Cluster IP address as static private IP address for the load balancer.

azlbr2 => Frontend IP: 10.3.0.101 (Pick up an unused IP address from the
Virtual network (az2az-Vnet) subnet)
Create Backend Pool for each load balancer. Add the associated cluster
nodes.
Create Health Probe: port 59999
Create Load Balance Rule: Allow HA ports, with enabled Floating IP.

12. On each cluster node, open port 59999 (Health Probe).

 Run the following command on each node:

PowerShell

netsh advfirewall firewall add rule name=PROBEPORT dir=in protocol=tcp

action=allow localport=59999 remoteip=any profile=any

13. Instruct the cluster to listen for Health Probe messages on Port 59999 and respond
from the node that currently owns this resource. Run it once from any one node of
the cluster, for each cluster.

In our example, make sure to change the "ILBIP" according to your configuration
values. Run the following command from any one node az2az1/az2az2:

PowerShell

$ClusterNetworkName = "Cluster Network 1" # Cluster network name (Use

Get-ClusterNetwork on Windows Server 2012 or higher to find the name.
And use Get-ClusterResource to find the IPResourceName).
$IPResourceName = "Cluster IP Address" # IP Address cluster resource
name.
$ILBIP = "10.3.0.100" # IP Address in Internal Load Balancer (ILB) -
The static IP address for the load balancer configured in the Azure
portal.
[int]$ProbePort = 59999
Get-ClusterResource $IPResourceName | Set-ClusterParameter -Multiple
@{"Address"="$ILBIP";"ProbePort"=$ProbePort;"SubnetMask"="255.255.255.2
55";"Network"="$ClusterNetworkName";"ProbeFailureThreshold"=5;"EnableDh
cp"=0}

14. Run the following command from any one node az2az3/az2az4.

PowerShell

$ClusterNetworkName = "Cluster Network 1" # Cluster network name (Use

Get-ClusterNetwork on Windows Server 2012 or higher to find the name.
And use Get-ClusterResource to find the IPResourceName).
$IPResourceName = "Cluster IP Address" # IP Address cluster resource
name.
$ILBIP = "10.3.0.101" # IP Address in Internal Load Balancer (ILB) -
The static IP address for the load balancer configured in the Azure
portal.
[int]$ProbePort = 59999
Get-ClusterResource $IPResourceName | Set-ClusterParameter -Multiple
@{"Address"="$ILBIP";"ProbePort"=$ProbePort;"SubnetMask"="255.255.255.2
55";"Network"="$ClusterNetworkName";"ProbeFailureThreshold"=5;"EnableDh
cp"=0}
 Make sure both clusters can connect / communicate with each other.

Either use "Connect to Cluster" feature in Failover cluster manager to connect to

the other cluster or check other cluster responds from one of the nodes of the
current cluster.

PowerShell

Get-Cluster -Name SRAZC1 (ran from az2az3)

PowerShell

Get-Cluster -Name SRAZC2 (ran from az2az1)

15. Create cloud witnesses for both clusters. Create two storage accounts (az2azcw,
az2azcw2) in azure one for each cluster in the same resource group (SR-AZ2AZ).

Copy the storage account name and key from "access keys"
Create the cloud witness from "failover cluster manager" and use the above
account name and key to create it.

16. Run cluster validation tests before moving on to the next step.

17. Start Windows PowerShell and use the Test-SRTopology cmdlet to determine if you
meet all the Storage Replica requirements. You can use the cmdlet in a
requirements-only mode for a quick test as well as a long-running performance
evaluation mode.

18. Configure cluster-to-cluster Storage Replica.

Grant access from one cluster to another cluster in both directions:

In our example:

PowerShell

Grant-SRAccess -ComputerName az2az1 -Cluster SRAZC2

If you're using Windows Server 2016 then also run this command:

PowerShell

Grant-SRAccess -ComputerName az2az3 -Cluster SRAZC1

 19. Create SRPartnership for the clusters:

For cluster SRAZC1.

Volume location:- c:\ClusterStorage\DataDisk1
Log location:- g:
For cluster SRAZC2
Volume location:- c:\ClusterStorage\DataDisk2
Log location:- g:

Run the following command:

PowerShell

New-SRPartnership -SourceComputerName SRAZC1 -SourceRGName rg01 -

SourceVolumeName c:\ClusterStorage\DataDisk1 -SourceLogVolumeName g: -
DestinationComputerName **SRAZC2** -DestinationRGName rg02 -
DestinationVolumeName c:\ClusterStorage\DataDisk2 -DestinationLogVolumeName
g:
Known issues with Storage Replica
Article • 05/31/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

This article describes some of the known issues with Storage Replica in Windows Server.

Disks offline after removing replication and you

can't configure replication
You may be unable to provision replication on a volume that was previously replicated
or may find unmountable volumes. Disks can remain offline when replication isn't
removed or when you reinstall the operating system on a computer that was previously
replicating data.

To fix, you must clear the hidden Storage Replica partition off the disks and return them
to a writeable state using the Clear-SRMetadata cmdlet.

To remove all orphaned Storage Replica partition databases slots and remount all
partitions, use the -AllPartitions parameter as follows:

PowerShell

Clear-SRMetadata -AllPartitions

To remove all orphaned Storage Replica log data, use the -AllLogs parameter as
follows:

PowerShell

Clear-SRMetadata -AllLogs

To remove all orphaned failover cluster configuration data, use the -

AllConfiguration parameter as follows:

PowerShell

Clear-SRMetadata -AllConfiguration
 To remove individual replication group metadata, use the -Name parameter and
specify a replication group as follows:

PowerShell

Clear-SRMetadata -Name RG01 -Logs -Partition

The server may need to restart after cleaning the partition database. You can prevent the
server from rebooting temporarily with -NoRestart but you shouldn't skip restarting if
requested by the cmdlet. This cmdlet doesn't remove data volumes nor data contained
within those volumes.

During initial sync, event ID 4004 warnings are

seen in the event log
After configuring replication, during initial sync both the source and destination servers
may show multiple warnings events with event ID 4004 in the StorageReplica\Admin
event log. The event description shows the status "insufficient system resources exist to
complete the API". You're likely to see 5014 errors as well. These events indicate that the
servers don't have enough available memory (RAM) to perform both initial
synchronization and run workloads. Either add RAM or reduce the used RAM from
features and applications other than Storage Replica.

Virtual machines stop responding after

configuring in-guest replication
Virtual machines stop responding after you configure replication when using in-guest
clustering and Storage Replica on a shared VHDX (not a cluster shared volume). If you
restart the Hyper-V host, the virtual machines start responding, however the replication
configuration won't be complete and no replication will occur.

This behavior occurs when you're using fltmc.exe attach svhdxflt to bypass the
requirement for the Hyper-V host running a CSV. Use of this command isn't supported
and is intended only for test and demonstration purposes.

The cause of the slowdown is an interoperability issue between Storage QoS in Windows
Server and the manually attached Shared VHDX filter. To resolve this issue, disable the
Storage QoS filter driver and restart the Hyper-V host:

Console
 SC config storqosflt start= disabled

Can't configure replication when using New-

Volume and differing storage
When using the New-Volume cmdlet along with differing sets of storage on the source
and destination server, such as two different SANs or two JBODs with differing disks, you
may not be able to configure replication using New-SRPartnership . The error shown may
include:

Output

Data partition sizes are different in those two groups

Use the New-Partition** cmdlet to create volumes and format them instead of New-
Volume , as the latter cmdlet may round the volume size on differing storage arrays. If

you've already created an NTFS volume, you can use Resize-Partition to grow or
shrink one of the volumes to match the other. You can't use this method with ReFS
volumes. If using Diskmgmt or Server Manager, no rounding will occur.

Running Test-SRTopology fails with name-

related errors
When attempting to use Test-SRTopology , you receive one of the following errors:

ERROR EXAMPLE 1:

Output

WARNING: Invalid value entered for target computer name: sr-srv03. Test-
SrTopology cmdlet does not accept IP address as input for target computer
name parameter. NetBIOS names and fully qualified domain names are
acceptable inputs
WARNING: System.Exception
WARNING: at
Microsoft.FileServices.SR.Powershell.TestSRTopologyCommand.BeginProcessing()
Test-SRTopology : Invalid value entered for target computer name: sr-srv03.
Test-SrTopology cmdlet does not accept IP address as input for target
computer name parameter. NetBIOS names and fully qualified domain names are
acceptable inputs
At line:1 char:1
+ Test-SRTopology -SourceComputerName sr-srv01 -SourceVolumeName d: -So ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 + CategoryInfo : InvalidArgument: (:) [Test-SRTopology], Exception
+ FullyQualifiedErrorId :
TestSRTopologyFailure,Microsoft.FileServices.SR.Powershell.TestSRTopologyCom
mand

ERROR EXAMPLE 2:

Output

WARNING: Invalid value entered for source computer name

ERROR EXAMPLE 3:

Output

The specified volume cannot be found G: cannot be found on computer

SRCLUSTERNODE1

This cmdlet has limited error reporting in Windows Server and will return the same
output for many common issues. The error can appear for the following reasons:

You're logged on to the source computer as a local user, not a domain user.

The destination computer isn't running or isn't accessible over the network.

You specified an incorrect name for the destination computer.

You specified an IP address for the destination server.

The destination computer firewall is blocking access to PowerShell and/or CIM

calls.

The destination computer isn't running the WMI service.

You didn't use CREDSSP when running the Test-SRTopology cmdlet remotely from
a management computer.

The source or destination volume specified is a local disk on a cluster node, not a
clustered disk.

Configuring new Storage Replica partnership

returns an error - "Failed to provision partition"
When attempting to create a new replication partnership with New-SRPartnership , you
receive the following error:

Output

New-SRPartnership : Unable to create replication group test01, detailed

reason: Failed to provision partition ed0dc93f-107c-4ab4-a785-afd687d3e734.
At line: 1 char: 1
+ New-SRPartnership -SourceComputerName srv1 -SourceRGName test01
+ Categorylnfo : ObjectNotFound: (MSFT_WvrAdminTasks : root/ Microsoft/. .
s) CNew-SRPartnership], CimException
+ FullyQua1ifiedErrorId : Windows System Error 1168 ,New-SRPartnership

You'll experience this error when selecting a data volume that is on the same partition as
the system drive (that is, the C: drive with its Windows folder). For instance, on a drive
that contains both the C: and D: volumes created from the same partition. Using a
system drive isn't supported in Storage Replica; you must pick a different volume to
replicate.

Attempting to grow a replicated volume fails

due to missing update
When attempting to grow or extend a replicated volume, you receive the following
error:

Output

Resize-Partition -DriveLetter d -Size 44GB

Resize-Partition : The operation failed with return code 8
At line:1 char:1
+ Resize-Partition -DriveLetter d -Size 44GB
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified:
(StorageWMI:ROOT/Microsoft/.../MSFT_Partition
[Resize-Partition], CimException
+ FullyQualifiedErrorId : StorageWMI 8,Resize-Partition

If using the Disk Management MMC snap-in, you receive this error:

Output

Element not found

You'll receive The operation failed with return code 8 even if you correctly enable
volume resizing on the source server using the command Set-SRGroup -Name rg01 -
AllowVolumeResize $TRUE .

The issue was fixed in Cumulative Update for Windows 10, version 1607 (Anniversary
Update) and Windows Server 2016: December 9, 2016 (KB3201845 ).

Attempting to grow a replicated volume fails

due to missing step
If you attempt to resize a replicated volume on the source server without setting -
AllowResizeVolume $TRUE first, you'll receive the following errors:

Output

Resize-Partition -DriveLetter I -Size 8GB

Resize-Partition : Failed

Activity ID: {87aebbd6-4f47-4621-8aa4-5328dfa6c3be}

At line:1 char:1
+ Resize-Partition -DriveLetter I -Size 8GB
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified:
(StorageWMI:ROOT/Microsoft/.../MSFT_Partition) [Resize-Partition],
CimException
+ FullyQualifiedErrorId : StorageWMI 4,Resize-Partition

Storage Replica Event log error 10307:

Attempted to resize a partition that is protected by Storage Replica.

DeviceName: \Device\Harddisk1\DR1
PartitionNumber: 7
PartitionId: {b71a79ca-0efe-4f9a-a2b9-3ed4084a1822}

Guidance: To grow a source data partition, set the policy on the replication
group containing the data partition.

PowerShell

Set-SRGroup -ComputerName [ComputerName] -Name [ReplicationGroupName] -

AllowVolumeResize $true

Before you grow the source data partition, ensure that the destination data partition has
enough space to grow to an equal size. Shrinking of data partition protected by Storage
Replica is blocked.
Disk Management Snap-in Error:

Output

An unexpected error has occurred

After resizing the volume, remember to disable resizing with Set-SRGroup -Name rg01 -
AllowVolumeResize $FALSE . This parameter prevents admins from attempting to resize

volumes prior to ensuring that there's sufficient space on the destination volume,
typically because they were unaware of Storage Replica's presence.

Moving a physical disk resource between sites

on an asynchronous stretch cluster fails
When attempting to move a physical disk resource (PDR) attached role in order to move
the associated storage in an asynchronous stretch cluster, you receive an error. For
example, trying moving a file server role to the asynchronous site.

If using the Failover Cluster Manager snap-in:

Output

Error
The operation has failed.
The action 'Move' did not complete.
Error Code: 0x80071398
The operation failed because either the specified cluster node is not the
owner of the group, or the node is not a possible owner of the group

If using the Cluster PowerShell cmdlet:

PowerShell

Move-ClusterGroup -Name sr-fs-006 -Node sr-srv07

Move-ClusterGroup : An error occurred while moving the clustered role 'sr-
fs-006'.
The operation failed because either the specified cluster node is not the
owner of the group, or the node is not a possible owner of the group
At line:1 char:1
+ Move-ClusterGroup -Name sr-fs-006 -Node sr-srv07
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Move-ClusterGroup],
ClusterCmdletException
+ FullyQualifiedErrorId : Move-
ClusterGroup,Microsoft.FailoverClusters.PowerShell.MoveClusterGroupCommand
Use Set-SRPartnership to move these PDR disks in an asynchronous stretched cluster.
The moved behavior has been changed beginning with Windows Server 2019 to allow
manual and automated failovers with asynchronous replication, based on customer
feedback.

Attempting to add disks to a two-node

asymmetric cluster returns "No disks suitable
for cluster disks found"
When attempting to provision a cluster with only two nodes, prior to adding Storage
Replica stretch replication, you attempt to add the disks in the second site to the
Available Disks. You receive the following error:

Output

No disks suitable for cluster disks found. For diagnostic information about
disks available to the cluster, use the Validate a Configuration Wizard to
run Storage tests.

You won't experience the error if you have at least three nodes in the cluster. To add the
storage, you can run the following command on the node in the second site:

PowerShell

Get-ClusterAvailableDisk -All | Add-ClusterDisk

The command won't work with node local storage. You can use Storage Replica to
replicate a stretch cluster between two total nodes, each one using its own set of
shared storage.

Event ID 1241 warning repeated during initial

sync
When specifying a replication partnership is asynchronous, the source computer
repeatedly logs event ID 1241 warning events in the Storage Replica Admin channel. For
example:

Output

Log Name: Microsoft-Windows-StorageReplica/Admin

Source: Microsoft-Windows-StorageReplica
 Date: 3/21/2017 3:10:41 PM
Event ID: 1241
Task Category: (1)
Level: Warning
Keywords: (1)
User: SYSTEM
Computer: sr-srv05.corp.contoso.com
Description:
The Recovery Point Objective (RPO) of the asynchronous destination is
unavailable.

LocalReplicationGroupName: rg01
LocalReplicationGroupId: {e20b6c68-1758-4ce4-bd3b-84a5b5ef2a87}
LocalReplicaName: f:\
LocalPartitionId: {27484a49-0f62-4515-8588-3755a292657f}
ReplicaSetId: {1f6446b5-d5fd-4776-b29b-f235d97d8c63}
RemoteReplicationGroupName: rg02
RemoteReplicationGroupId: {7f18e5ea-53ca-4b50-989c-9ac6afb3cc81}
TargetRPO: 30

Event ID 1241, "The Recovery Point Objective (RPO) of the asynchronous destination is
unavailable" is typically due to one of the following reasons:

The asynchronous destination is currently disconnected. The RPO may become

available after the connection is restored.

The asynchronous destination is unable to keep pace with the source such that the
most recent destination log record is no longer present in the source log. The
destination will start block copying. The RPO should become available after block
copying completes.

During initial sync, the event is expected behavior and can safely be ignored. The event
behavior may change in a later release. If you see this behavior during ongoing
asynchronous replication, investigate the partnership to determine why replication is
delayed beyond your configured RPO (30 seconds, by default).

Event ID 4004 warning repeated after

rebooting a replicated node
Under rare circumstances, rebooting a server that is in a partnership leads to replication
failing and the rebooted node logging event ID 4004 warning events with an access
denied error.

Output
 Log Name: Microsoft-Windows-StorageReplica/Admin
Source: Microsoft-Windows-StorageReplica
Date: 3/21/2017 11:43:25 AM
Event ID: 4004
Task Category: (7)
Level: Warning
Keywords: (256)
User: SYSTEM
Computer: server.contoso.com
Description:
Failed to establish a connection to a remote computer.

RemoteComputerName: server
LocalReplicationGroupName: rg01
LocalReplicationGroupId: {a386f747-bcae-40ac-9f4b-1942eb4498a0}
RemoteReplicationGroupName: rg02
RemoteReplicationGroupId: {a386f747-bcae-40ac-9f4b-1942eb4498a0}
ReplicaSetId: {00000000-0000-0000-0000-000000000000}
RemoteShareName:{a386f747-bcae-40ac-9f4b-1942eb4498a0}.{00000000-0000-0000-
0000-000000000000}
Status: {Access Denied}
A process has requested access to an object, but has not been granted those
access rights.

Guidance: Possible causes include network failures, share creation failures

for the remote replication group, or firewall settings. Make sure SMB
traffic is allowed and there are no connectivity issues between the local
computer and the remote computer. You should expect this event when
suspending replication or removing a replication partnership.

Note the Status: "{Access Denied}" and the message A process has requested access
to an object, but has not been granted those access rights. This is a known issue

within Storage Replica and was fixed in Quality Update September 12, 2017
KB4038782 (OS Build 14393.1715).

Error "Failed to bring the resource 'Cluster Disk

x' online." with a stretch cluster
When attempting to bring a cluster disk online after a successful failover, where you're
attempting to make the original source site primary again, you receive an error in
Failover Cluster Manager. For example:

Output

Error
The operation has failed.
Failed to bring the resource 'Cluster Disk 2' online.
 Error Code: 0x80071397
The operation failed because either the specified cluster node is not the
owner of the resource, or the node is not a possible owner of the resource.

If you attempt to move the disk or CSV manually, you receive another error. For
example:

Output

Error
The operation has failed.
The action 'Move' did not complete.

Error Code: 0x8007138d

A cluster node is not available for this operation

This issue is caused by one or more uninitialized disks being attached to one or more
cluster nodes. To resolve the issue, initialize all attached storage using DiskMgmt.msc,
DISKPART.EXE, or the Initialize-Disk PowerShell cmdlet.

We're working on providing an update that permanently resolves this issue. Contact
Microsoft Support for more information.

GPT error when attempting to create a new SR

partnership
Running New-SRPartnership fails with the error:

Output

Disk layout type for volume \\?\Volume{GUID}\ is not a valid GPT style
layout.
New-SRPartnership : Unable to create replication group SRG01, detailed
reason: Disk layout type for volume
\\?\Volume{GUID}\ is not a valid GPT style layout.
At line:1 char:1
+ New-SRPartnership -SourceComputerName nodesrc01 -SourceRGName SRG01 ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified:
(MSFT_WvrAdminTasks:root/Microsoft/...T_WvrAdminTasks) [New-SRPartnership],
CimException
+ FullyQualifiedErrorId : Windows System Error 5078,New-SRPartnership

In the Failover Cluster Manager GUI, there's no capability to set up replication for the
disk.
Running Test-SRTopology fails with the following output:

Output

WARNING: Object reference not set to an instance of an object.

WARNING: System.NullReferenceException
WARNING: at
Microsoft.FileServices.SR.Powershell.MSFTPartition.GetPartitionInStorageNode
ByAccessPath(String AccessPath, String ComputerName, MIObject StorageNode)
at Microsoft.FileServices.SR.Powershell.Volume.GetVolume(String Path,
String ComputerName)
at
Microsoft.FileServices.SR.Powershell.TestSRTopologyCommand.BeginProcessing()
Test-SRTopology : Object reference not set to an instance of an object.
At line:1 char:1
+ Test-SRTopology -SourceComputerName nodesrc01 -SourceVolumeName U: - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Test-SRTopology],
NullReferenceException
+ FullyQualifiedErrorId :
TestSRTopologyFailure,Microsoft.FileServices.SR.Powershell.TestSRTopologyCom
mand

The error is caused by the cluster functional level still being set to Windows Server 2012
R2 (that is, FL 8). Storage Replica is supposed to return a specific error here but instead
returns an incorrect error mapping.

From an elevated PowerShell session, run the following command on each node.

PowerShell

Get-Cluster | fl *

If the ClusterFunctionalLevel attribute is 9 or higher, that is the version needed to

implement Storage Replica. If ClusterFunctionalLevel isn't 9 , the
ClusterFunctionalLevel will need to be updated in order to implement Storage Replica
on this node.

To resolve the issue, raise the cluster functional level by running the PowerShell cmdlet:
Update-ClusterFunctionalLevel.

Small unknown volume listed in DISKMGMT for

each replicated volume
When running the Disk Management snap-in (DISKMGMT.MSC), you notice one or more
volumes listed with no label or drive letter and 1 MB in size. You may be able to delete
the unknown volume or you may receive:

Output

An Unexpected Error has Occurred

The message above is expected behavior and is by design. The listed items aren't
volumes, but are partitions. Storage Replica creates a 512-KB partition as a database slot
for replication operations (the legacy DiskMgmt.msc tool rounds to the nearest MB).
Having a partition like this for each replicated volume is normal and desirable. Once the
disk is no longer used by Storage Replica, you're free to delete this 512-KB partition; in-
use partitions can't be deleted. The partition will never grow or shrink. If you're
recreating replication, we recommend leaving the partition as Storage Replica will claim
unused ones.

To view details, use the DISKPART tool or Get-Partition cmdlet. These partitions will
have a GPT Type of 558d43c5-a1ac-43c0-aac8-d1472b2923d1 .

A Storage Replica node hangs when creating

snapshots
Creating a VSS snapshot (through backup, VSSADMIN, etc.) causes a Storage Replica
node hangs, you must force a restart of the node to recover. There's no error, just a hard
hang of the server.

This issue occurs when you create a VSS snapshot of the log volume. The underlying
cause is a legacy design aspect of VSS, not Storage Replica. The resulting behavior when
you snapshot the Storage Replica log volume is a VSS I/O queuing mechanism
deadlocks the server.

To prevent this behavior, don't snapshot Storage Replica log volumes. There's no need
to snapshot Storage Replica log volumes, as these logs can't be restored. Furthermore,
the log volume should never contain any other workloads, so no snapshot is needed in
general.

High IO latency when using Storage Spaces

Direct with Storage Replica
When using Storage Spaces Direct with an NVMe (nonvolatile memory express) device
or SSD (solid-state drive) cache, you see a greater than expected increase in latency
when configuring Storage Replica replication between Storage Spaces Direct clusters.
The change in latency is proportionally much higher than you see when using NVMe
and SSD in a performance + capacity configuration and no HDD tier nor capacity tier.

This issue occurs due to architectural limitations within Storage Replica's log mechanism
combined with the low latency of NVMe when compared to slower media. With Storage
Spaces Direct cache, all I/O of Storage Replica logs, along with all recent read/write IO of
applications, will occur in the cache and never on the performance or capacity tiers.
Meaning that all Storage Replica activity happens on the same speed media - the
configuration is supported but not recommended (see Frequently asked questions
about Storage Replica for log recommendations).

When using Storage Spaces Direct with HDDs, you can't disable or avoid the cache. As a
workaround, if using just SSD and NVMe, you can configure just performance and
capacity tiers. If using that configuration, and by placing the SR logs on the performance
tier only with the data volumes they service being on the capacity tier only, you'll avoid
the high latency issue described above. The same could be done with a mix of faster and
slower SSDs and no NVMe.

This workaround isn't ideal and some customers may not be able to make use of it. The
Storage Replica team is working on optimizations and an updated log mechanism for
the future to reduce these artificial bottlenecks. This v1.1 log first became available in
Windows Server 2019 and its improved performance is described in on the Server
Storage Blog .

Error "Could not find file" when running Test-

SRTopology between two clusters
Running Test-SRTopology between two clusters and their CSV paths fails with the error:

Output

Validating data and log volumes...

Measuring Storage Replica recovery and initial synchronization
performance...
WARNING: Could not find file
'\\SERVER01\C$\CLUSTERSTORAGE\VOLUME1TestSRTopologyRecoveryTest\SRRecoveryTe
stFile01.txt'.
WARNING: System.IO.FileNotFoundException
WARNING: at System.IO.__Error.WinIOError(Int32 errorCode, String
maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access,
 Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize,
FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean
bFromProxy, Boolean useLongPath, Boolean checkHost) at
System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access,
FileShare share, Int32 bufferSize, FileOptions options) at
Microsoft.FileServices.SR.Powershell.TestSRTopologyCommand.GenerateWriteIOWo
rkload(String Path, UInt32 IoSizeInBytes, UInt32 Parallel IoCount, UInt32
Duration)at Microsoft.FileServices.SR.Powershell.TestSRTopologyCommand.
<>c__DisplayClass75_0.<PerformRecoveryTest>b__0()at
System.Threading.Tasks.Task.Execute()
Test-SRTopology : Could not find file
'\\SERVER01\C$\CLUSTERSTORAGE\VOLUME1TestSRTopologyRecoveryTest\SRRecoveryTe
stFile01.txt'.
At line:1 char:1
+ Test-SRTopology -SourceComputerName ClusterA -SourceVolumeName ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (:) [Test-SRTopology],
FileNotFoundException
+ FullyQualifiedErrorId :
TestSRTopologyFailure,Microsoft.FileServices.SR.Powershell.TestSRTopologyCom
mand

The error shown in the example is caused by a known code defect in Windows Server
2016. This issue was fixed in Windows Server 2019 and the associated RSAT tools. For a
downlevel resolution, contact Microsoft Support. There's no workaround.

Error "specified volume couldn't be found"

when running Test-SRTopology between two
clusters
Running Test-SRTopology between two clusters and their CSV paths fail with error:

Output

Test-SRTopology : The specified volume C:\ClusterStorage\Volume1 cannot be

found on computer RRN44-14-09. If this is a cluster node, the volume must be
part of a role or CSV; volumes in Available Storage are not accessible
At line:1 char:1
+ Test-SRTopology -SourceComputerName RRN44-14-09 -SourceVolumeName C:\ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (:) [Test-SRTopology],
Exception
+ FullyQualifiedErrorId :
TestSRTopologyFailure,Microsoft.FileServices.SR.Powershell.TestSRTopologyCom
mand
When specifying the source node CSV as the source volume, you must select the node
that owns the CSV. You can either move the CSV to the specified node or change the
node name you specified in -SourceComputerName . An improved message was introduced
beginning with Windows Server 2019.

Unable to access the data drive in Storage

Replica after unexpected reboot when
BitLocker is enabled
If BitLocker is enabled on both drives (the Log Drive and Data Drive), the Primary Server
reboots then you're unable to access the Primary Drive even after unlocking the Log
Drive from BitLocker.

To recover the data or access the drive, you need to unlock the log drive first, and then
open Diskmgmt.msc to locate the data drive. Mark the data drive offline and online
again. Locate the BitLocker icon on the drive and unlock the drive.

Issue unlocking the Data drive on secondary

server after breaking the Storage Replica
partnership
After disabling the SR Partnership and removing the Storage Replica partnership, it's
expected if you're unable to unlock the Secondary Server's Data drive with its respective
password or key.

You need to use key or password of primary server's data drive to unlock the secondary
server's data drive.

Test Failover doesn't mount when using

asynchronous replication
Running Mount-SRDestination to bring a destination volume online as part of Test
Failover fails with error:

Output

Mount-SRDestination: Unable to mount SR group <TEST>, detailed reason: The

group or resource is not in the correct state to perform the supported
operation.
 At line:1 char:1
+ Mount-SRDestination -ComputerName SRV1 -Name TEST -TemporaryP . . .
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (MSFT WvrAdminTasks :
root/Microsoft/...(MSFT WvrAdminTasks : root/Microsoft/. T_WvrAdminTasks)
(Mount-SRDestination], CimException
+ FullyQua1ifiedErrorId : Windows System Error 5823, Mount-
SRDestination.

If using a synchronous partnership type, test failover works normally.

There's a known code defect in Windows Server, version 1709, which caused this error
shown. To resolve this issue, install the October 18, 2018 update . This issue isn't
present in Windows Server 2019 and newer.

Unable to setup Storage Replica with physical

sector sizes greater than 4K
Storage Replica does not support disks with physical sector sizes great than 4K today.
We are exploring implementing this feature in future releases.

Next steps
Now you understand some of the known issues with Storage Replica in Windows
Servers, here are some articles that might help you as you use it.

Storage Replica
Stretch Cluster Replication Using Shared Storage
Server to Server Storage Replication
Cluster to Cluster Storage Replication
Storage Replica: Frequently Asked Questions
Storage Spaces Direct
Frequently asked questions about
Storage Replica
FAQ

Applies to: Windows Server 2019, Windows Server 2016

This topic contains answers to frequently asked questions (FAQs) about Storage Replica.

Is Storage Replica supported on Azure?

Yes. You can use the following scenarios with Azure:

Server-to-server replication inside Azure (synchronously or asynchronously

between IaaS VMs in one or two datacenter fault domains, or asynchronously
between two separate regions)
Server-to-server asynchronous replication between Azure and on-premises (using
VPN or Azure ExpressRoute)
Cluster-to-cluster replication inside Azure (synchronously or asynchronously
between IaaS VMs in one or two datacenter fault domains, or asynchronously
between two separate regions)
Cluster-to-cluster asynchronous replication between Azure and on-premises (using
VPN or Azure ExpressRoute)
Stretch clustering using Azure Shared Disks (synchronously or asynchronously
between IaaS VMs in one or two datacenter fault domains, or asynchronously
between two separate regions)

Further notes on guest clustering in Azure can be found at: Deploying IaaS VM Guest
Clusters in Microsoft Azure .

Important notes:

There are Azure Resource Manager templates for Storage Spaces Direct-based
Storage Replica clustering at Create a Storage Spaces Direct SOFS Clusters with
Storage Replica for Disaster Recovery across Azure Regions .
Cluster to cluster RPC communication in Azure (required by the cluster APIs for
granting access between cluster) requires configuring network access for the CNO.
You must allow TCP port 135 and the dynamic range above TCP port 49152.
Reference Building Windows Server Failover Cluster on Azure IAAS VM – Part 2
Network and Creation.
 It's possible to use two-node guest clusters, where each node is using loopback
iSCSI for an asymmetric cluster replicated by Storage Replica. But this will likely
have very poor performance and should be used only for very limited workloads or
testing.

How do I see the progress of replication

during initial sync?
The Event 1237 messages shown in the Storage Replica Admin even log on the
destination server show number of bytes copied and bytes remaining every 10 seconds.
You can also use the Storage Replica performance counter on the destination showing
\Storage Replica Statistics\Total Bytes Received for one or more replicated volumes.
You can also query the replication group using Windows PowerShell. For instance, this
sample command gets the name of the groups on the destination then queries one
group named Replication 2 every 10 seconds to show progress:

Get-SRGroup

do{
$r=(Get-SRGroup -Name "Replication 2").replicas
[System.Console]::Write("Number of remaining bytes {0}`n",
$r.NumOfBytesRemaining)
Start-Sleep 10
}until($r.ReplicationStatus -eq 'ContinuouslyReplicating')
Write-Output "Replica Status: "$r.replicationstatus

Can I specify specific network interfaces

to be used for replication?
Yes, using Set-SRNetworkConstraint . This cmdlet operates at the interface layer and be
used on both cluster and non-cluster scenarios. For example, with a standalone server
(on each node):

Get-SRPartnership

Get-NetIPConfiguration
Note the gateway and interface information (on both servers) and the partnership
directions. Then run:

Set-SRNetworkConstraint -SourceComputerName sr-srv06 -SourceRGName rg02 -

SourceNWInterface 2 -DestinationComputerName sr-srv05 -
DestinationNWInterface 3 -DestinationRGName rg01

Get-SRNetworkConstraint

Update-SmbMultichannelConnection

For configuring network constraints on a stretch cluster:

Set-SRNetworkConstraint -SourceComputerName sr-cluster01 -SourceRGName

group1 -SourceNWInterface "Cluster Network 1","Cluster Network 2" -
DestinationComputerName sr-cluster02 -DestinationRGName group2 -
DestinationNWInterface "Cluster Network 1","Cluster Network 2"

Can I configure one-to-many replication

or transitive (A to B to C) replication?
No, Storage Replica supports only one to one replication of a server, cluster, or stretch
cluster node. This may change in a later release. You can of course configure replication
between various servers of a specific volume pair, in either direction. For instance, Server
1 can replicate its D volume to server 2, and its E volume from Server 3.

Can I grow or shrink replicated volumes

replicated by Storage Replica?
You can grow (extend) volumes, but not shrink them. By default, Storage Replica
prevents administrators from extending replicated volumes; use the Set-SRGroup -
AllowVolumeResize $TRUE option on the source group, prior to resizing. For example:

1. Use against the source computer: Set-SRGroup -Name YourRG -AllowVolumeResize

$TRUE

2. Grow the volume using whatever technique you prefer

 3. Use against the source computer: Set-SRGroup -Name YourRG -AllowVolumeResize
$FALSE

Can I bring a destination volume online

for read-only access?
Not in Windows Server 2016. Storage Replica dismounts the destination volume when
replication begins.

However, in Windows Server 2019 and Windows Server Semi-Annual Channel starting
with version, 1709, the option to mount the destination storage is now possible - this
feature is called "Test Failover". To do this, you must have an unused, NTFS or ReFS
formatted volume that is not currently replicating on the destination. Then you can
mount a snapshot of the replicated storage temporarily for testing or backup purposes.

For example, to create a test failover where you are replicating a volume "D:" in the
Replication Group "RG2" on the destination server "SRV2" and have a "T:" drive on SRV2
that is not being replicated:

Mount-SRDestination -Name RG2 -Computername SRV2 -TemporaryPath T:\

The replicated volume D: is now accessible on SRV2. You can read and write to it
normally, copy files off it, or run an online backup that you save elsewhere for
safekeeping, under the D: path. The T: volume will only contain log data.

To remove the test failover snapshot and discard its changes:

Dismount-SRDestination -Name RG2 -Computername SRV2

You should only use the test failover feature for short-term temporary operations. It is
not intended for long term usage. When in use, replication continues to the real
destination volume.

Can I configure Scale-out File Server

(SOFS) in a stretch cluster?
While technically possible, this is not a recommended configuration due to the lack of
site awareness in the compute nodes contacting the SOFS. If using campus-distance
networking, where latencies are typically sub-millisecond, this configuration typically
works without issues.
If configuring cluster-to-cluster replication, Storage Replica fully supports Scale-out File
Servers, including the use of Storage Spaces Direct, when replicating between two
clusters.

Is CSV required to replicate in a stretch

cluster or between clusters?
No. You can replicate with CSV or persistent disk reservation (PDR) owned by a cluster
resource, such as a File Server role.

If configuring cluster-to-cluster replication, Storage Replica fully supports Scale-out File

Servers, including the use of Storage Spaces Direct, when replicating between two
clusters.

Can I configure Storage Spaces Direct in

a stretch cluster with Storage Replica?
This is not a supported configuration in Windows Server. This may change in a later
release. If configuring cluster-to-cluster replication, Storage Replica fully supports Scale
Out File Servers and Hyper-V Servers, including the use of Storage Spaces Direct.

How do I configure asynchronous

replication?
Specify New-SRPartnership -ReplicationMode and provide argument Asynchronous. By
default, all replication in Storage Replica is synchronous. You can also change the mode
with Set-SRPartnership -ReplicationMode .

How do I prevent automatic failover of

a stretch cluster?
To prevent automatic failover, you can use PowerShell to configure Get-ClusterNode -
Name "NodeName").NodeWeight=0 . This removes the vote on each node in the disaster

recovery site. Then you can use Start-ClusterNode -PreventQuorum on nodes in the
primary site and Start-ClusterNode -ForceQuorum on nodes in the disaster site to force
failover. There is no graphical option for preventing automatic failover, and preventing
automatic failover is not recommended.

How do I disable virtual machine

resiliency?
To prevent the new Hyper-V virtual machine resiliency feature from running and
therefore pausing virtual machines instead of failing them over to the disaster recovery
site, run (Get-Cluster).ResiliencyDefaultPeriod=0

How can I reduce time for initial

synchronization?
You can use thin-provisioned storage as one way to speed up initial sync times. Storage
Replica queries for and automatically uses thin-provisioned storage, including non-
clustered Storage Spaces, Hyper-V dynamic disks, and SAN LUNs. Once initial replication
is initiated, the volume will not be able to shrink or trim.

You can also use seeded data volumes to reduce bandwidth usage and sometimes time,
by ensuring that the destination volume has some subset of data from the primary then
using the Seeded option in Failover Cluster Manager or New-SRPartnership . If the
volume is mostly empty, using seeded sync may reduce time and bandwidth usage.
There are multiple ways to seed data, with varying degrees of efficacy:

Previous replication - by replicating with normal initial sync locally between nodes
containing the disks and volumes, removing replication, shipping the destination
disks elsewhere, then adding replication with the seeded option. This is the most
effective method as Storage Replica guaranteed a block-copy mirror and the only
thing to replicate is delta blocks.
Restored snapshot or restored snapshot-based backup - by restoring a volume-
based snapshot onto the destination volume, there should be minimal differences
in the block layout. This is the next most effective method as blocks are likely to
match thanks to volume snapshots being mirror images.
Copied files - by creating a new volume on the destination that has never been
used before and performing a full robocopy /MIR tree copy of the data, there are
likely to be block matches. Using Windows File Explorer or copying some portion
of the tree will not create many block matches. Copying files manually is the least
effective method of seeding.
Can I delegate users to administer
replication?
You can use the Grant-SRDelegation cmdlet. This allows you to set specific users in
server to server, cluster to cluster, and stretch cluster replication scenarios as having the
permissions to create, modify, or remove replication, without being a member of the
local administrators group. For example:

Grant-SRDelegation -UserName contso\tonywang

The cmdlet will remind you that the user needs to log off and on of the server they are
planning to administer in order for the change to take effect. You can use Get-
SRDelegation and Revoke-SRDelegation to further control this.

What are my backup and restore

options for replicated volumes?
Storage Replica supports backing up and restoring the source volume. It also supports
creating and restoring snapshots of the source volume. You cannot back up or restore
the destination volume while protected by Storage Replica, as it is not mounted nor
accessible. If you experience a disaster where the source volume is lost, using Set-
SRPartnership to promote the previous destination volume to now be a read/writable

source will allow you to back up or restore that volume. You can also remove replication
with Remove-SRPartnership and Remove-SRGroup to remount that volume as
read/writable.

To create periodic application consistent snapshots, you can use VSSADMIN.EXE on the
source server to snapshot replicated data volumes. For example, where you are
replicating the F: volume with Storage Replica:

vssadmin create shadow /for=F:

Then, after you switch replication direction, remove replication, or are simply still on the
same source volume, you can restore any snapshot to its point in time. For example, still
using F:
 vssadmin list shadows
vssadmin revert shadow /shadow={shadown copy ID GUID listed previously}

You can also schedule this tool to run periodically using a scheduled task. For more
information on using VSS, review Vssadmin. There is no need or value in backing up the
log volumes. Attempting to do so will be ignored by VSS.

Use of Windows Server Backup, Microsoft Azure Backup, Microsoft DPM, or other
snapshot, VSS, virtual machine, or file-based technologies are supported by Storage
Replica as long as they operate within the volume layer. Storage Replica does not
support block-based backup and restore.

What network ports does Storage

Replica require?
Storage Replica relies on SMB and WSMAN for its replication and management. This
means the following ports are required:

445 (SMB - replication transport protocol, cluster RPC management protocol)

5445 (iWARP SMB - only needed when using iWARP RDMA networking)
5985 (WSManHTTP - Management protocol for WMI/CIM/PowerShell)

７ Note

The Test-SRTopology cmdlet requires ICMPv4/ICMPv6, but not for replication or

management.

What are the log volume best practices?

The optimal size of the log varies widely per environment and workload, and is
determined by how much write IO your workload performs.

A larger or smaller log doesn't make you any faster or slower

A larger or smaller log doesn't have any bearing on a 10GB data volume versus a
10TB data volume, for instance

A larger log simply collects and retains more write IOs before they are wrapped out. This
allows an interruption in service between the source and destination computer – such as
a network outage or the destination being offline - to go longer. If the log can hold 10
hours of writes, and the network goes down for 2 hours, when the network returns the
source can simply play the delta of unsynced changes back to the destination very fast
and you are protected again very quickly. If the log holds 10 hours and the outage is 2
days, the source now has to play back from a different log called the bitmap – and will
likely be slower to get back into sync. Once in sync it returns to using the log.

Storage Replica relies on the log for all write performance. Log performance critical to
replication performance. You must ensure that the log volume performs better than the
data volume, as the log will serialize and sequentialize all write IO. You should always
use flash media like SSD on log volumes. You must never allow any other workloads to
run on the log volume, the same way you would never allow other workloads to run on
SQL database log volumes.

Again: Microsoft strongly recommends that the log storage be faster than the data
storage and that log volumes must never be used for other workloads.

You can get log sizing recommendations by running the Test-SRTopology tool.
Alternatively, you can use performance counters on existing servers to make a log size
judgment. The formula is simple: monitor the data disk throughput (Avg Write
Bytes/Sec) under the workload and use it to calculate the amount of time it will take to
fill up the log of different sizes. For example, data disk throughput of 50 MB/s will cause
the log of 120GB to wrap in 120GB/50MB seconds or 2400 seconds or 40 minutes. So
the amount of time that the destination server could be unreachable before the log
wrapped is 40 minutes. If the log wraps but the destination becomes reachable again,
the source would replay blocks via the bit map log instead of the main log. The size of
the log does not have an effect on performance.

ONLY the Data disk from the Source cluster should be backed-up. The Storage Replica
Log disks should NOT be backed-up since a backup can conflict with Storage Replica
operations.

Why would you choose a stretch cluster

versus cluster-to-cluster versus server-
to-server topology?
Storage Replica comes in three main configurations: stretch cluster, cluster-to-cluster,
and server-to-server. There are different advantages to each.

The stretch cluster topology is ideal for workloads requiring automatic failover with
orchestration, such as Hyper-V private cloud clusters and SQL Server FCI. It also has a
built-in graphical interface using Failover Cluster Manager. It utilizes the classic
asymmetric cluster shared storage architecture of Storage Spaces, SAN, iSCSI, and RAID
via persistent reservation. You can run this with as few as 2 nodes.

The cluster-to-cluster topology uses two separate clusters and is ideal for administrators
who want manual failover, especially when the second site is provisioned for disaster
recovery and not everyday usage. Orchestration is manual. Unlike stretch cluster,
Storage Spaces Direct can be used in this configuration (with caveats - see the Storage
Replica FAQ and cluster-to-cluster documentation). You can run this with as few as four
nodes.

The server-to-server topology is ideal for customers running hardware that cannot be
clustered. It requires manual failover and orchestration. It is ideal for inexpensive
deployments between branch offices and central data centers, especially when using
asynchronous replication. This configuration can often replace instances of DFSR-
protected File Servers used for single-master disaster recovery scenarios.

In all cases, the topologies support both running on physical hardware as well as virtual
machines. When in virtual machines, the underlying hypervisor doesn't require Hyper-V;
it can be VMware, KVM, Xen, etc.

Storage Replica also has a server-to-self mode, where you point replication to two
different volumes on the same computer.

Is Data Deduplication supported with

Storage Replica?
Yes, Data Deduplcation is supported with Storage Replica. Enable Data Deduplication on
a volume on the source server, and during replication the destination server receives a
deduplicated copy of the volume.

While you should install Data Deduplication on both the source and destination servers
(see Installing and enabling Data Deduplication), it's important not to enable Data
Deduplication on the destination server. Storage Replica allows writes only on the
source server. Because Data Deduplication makes writes to the volume, it should run
only on the source server.

Can I replicate between Windows Server

2019 and Windows Server 2016?
Unfortunately, we don't support creating a new partnership between Windows Server
2019 and Windows Server 2016. You can safely upgrade a server or cluster running
Windows Server 2016 to Windows Server 2019 and any existing partnerships will
continue to work.

However, to get the improved replication performance of Windows Server 2019, all
members of the partnership must run Windows Server 2019 and you must delete
existing partnerships and associated replication groups and then recreate them with
seeded data (either when creating the partnership in Windows Admin Center or with the
New-SRPartnership cmdlet).

How do I report an issue with Storage

Replica or this guide?
For technical assistance with Storage Replica, you can post at the Microsoft forums. You
can also email [email protected] for questions on Storage Replica. For issues with
this documentation, see Feedback section at the bottom of this page, and select This
page.

Can Storage Replica be configured to

replicate in both directions?
Storage Replica is a one-way replication technology. It will only replicate from the
source to the destination on a per volume basis. This direction can be reversed at any
time, but is still only in one direction. However, that does not mean you cannot have a
set of volumes (source and destination) replicate in one direction and a different set of
drives (source and destination) replicate in the opposite direction. For example, you
have want to have server to server replication configured. Server1 and Server2 each
have drive letters L:, M:, N:, and O: and you wish to replicate drive M: from Server1 to
Server2 but drive O: replicate from Server2 to Server1. This can be done as long as there
are separate log drives for each of the groups. I.E.

Server1 source drive M: with source log drive L: replicating to Server2 destination
drive M: with destination log drive L:
Server2 source drive O: with source log drive N: replicating to Server1 destination
drive O: with destination log drive N:
Can you place cluster disks in
maintenance mode?
Storage Replica will block any cluster disks from entering maintenance mode. For tasks
such as enabling or disabling Bitlocker, the disks must be in maintenance mode. To
perform tasks that require the disks to be in maintenance mode, the partnership would
need to be broken first and created again once it is complete.

Related topics
Storage Replica Overview
Stretch Cluster Replication Using Shared Storage
Server to Server Storage Replication
Cluster to Cluster Storage Replication
Storage Replica: Known Issues

See also
Storage Overview
Storage Spaces Direct
Storage Spaces overview
Article • 03/21/2023

Storage Spaces can help protect your data from drive failures. It's a technology in
Windows and Windows Server and is conceptually similar to redundant array of
independent disks (RAID), implemented in software. You can use Storage Spaces to
group three or more drives into a storage pool and then use capacity from that pool to
create Storage Spaces. These drives typically store extra copies of your data, so if one of
your drives fails, you still have an intact copy of your data. If you run low on capacity,
just add more drives to the storage pool.

There are four main ways to use Storage Spaces:

On a Windows PC. For more information, see Storage Spaces in Windows 10 .

On a stand-alone server with all storage in a single server. For more information,
see Deploy Storage Spaces on a stand-alone server.
On a clustered server using Storage Spaces Direct with local, direct-attached
storage in each cluster node. For more information, see Storage Spaces Direct
overview.
On a clustered server with one or more shared SAS storage enclosures holding
all drives. For more information, see Storage Spaces on a cluster with shared SAS
overview.
Deploy Storage Spaces on a stand-alone
server
Article • 03/15/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012

This article describes how to deploy Storage Spaces on a stand-alone server. For
information about how to create a clustered storage space, see Deploy a Storage Spaces
cluster on Windows Server 2012 R2.

To create a storage space, you must first create one or more storage pools. A storage
pool is a collection of physical disks. A storage pool enables storage aggregation, elastic
capacity expansion, and delegated administration.

From a storage pool, you can create one or more virtual disks. These virtual disks are
also referred to as storage spaces. A storage space appears to the Windows operating
system as a regular disk from which you can create formatted volumes. When you create
a virtual disk through the File and Storage Services user interface, you can configure the
resiliency type (simple, mirror, or parity), the provisioning type (thin or fixed), and the
size. Through Windows PowerShell, you can set other parameters such as the number of
columns, the interleave value, and which physical disks in the pool to use. For
information about these other parameters, see New-VirtualDisk and the Windows Server
Storage forum.

７ Note

You can't use a storage space to host the Windows operating system.

From a virtual disk, you can create one or more volumes. When you create a volume,
you can configure the size, drive letter or folder, file system (NTFS file system or Resilient
File System (ReFS)), allocation unit size, and an optional volume label.

The following figure illustrates the Storage Spaces workflow.

 ７ Note

This topic includes sample Windows PowerShell cmdlets that you can use to
automate some of the procedures described. For more information, see What is
PowerShell.

Prerequisites
To use Storage Spaces on a stand-alone Windows Server−based server, make sure that
the physical disks that you want to use meet the following prerequisites.

） Important

If you want to learn how to deploy Storage Spaces on a failover cluster, see Deploy
a Storage Spaces cluster on Windows Server 2012 R2. A failover cluster
deployment has different prerequisites, such as supported disk bus types,
supported resiliency types, and the required minimum number of disks.

Area Requirement Notes

 Area Requirement Notes

Disk bus types - Serial Attached SCSI (SAS) You can also use USB drives.
- Serial Advanced Technology Attachment However, it's not optimal to use
(SATA) USB drives in a server environment.
- iSCSI and Fibre Channel Controllers. Storage Spaces is supported on
iSCSI and Fibre Channel (FC)
controllers as long as the virtual
disks created on top of them are
nonresilient (Simple with any
number of columns).

Disk - Physical disks must be at least 4 GB

configuration - Disks must be blank and not formatted.
Don't create volumes.

HBA - Simple host bus adapters (HBAs) that Storage Spaces is compatible only
considerations don't support RAID functionality are with HBAs where you can
recommended completely disable all RAID
- If RAID-capable, HBAs must be in non- functionality.
RAID mode with all RAID functionality
disabled
- Adapters must not abstract the physical
disks, cache data, or obscure any
attached devices. This guideline includes
enclosure services that are provided by
attached just-a-bunch-of-disks (JBOD)
devices.

JBOD - JBOD enclosures are optional If the EnclosureNumber and

enclosures - We recommend using Storage Spaces SlotNumber fields contain values,
certified enclosures listed on the then the enclosure supports these
Windows Server Catalog features.
- If you're using a JBOD enclosure, verify
with your storage vendor that the
enclosure supports Storage Spaces to
ensure full functionality
- To determine whether the JBOD
enclosure supports enclosure and slot
identification, run the following Windows
PowerShell cmdlet:

Get-PhysicalDisk | ? {$_.BusType –eq

"SAS"} | fc

To plan for the number of physical disks and the desired resiliency type for a stand-
alone server deployment, use the following guidelines.
Resiliency type Disk When to use
requirements

Simple Requires at Don't use to host

least one irreplaceable data. Simple
- Stripes data across physical disks physical disk. spaces don't protect
- Maximizes disk capacity and increases against disk failure.
throughput
- No resiliency (doesn't protect from disk failure) Use to host temporary or
easily recreated data at a
reduced cost.

Suited for high-

performance workloads
where resiliency isn't
required or is already
provided by the
application.

Mirror Requires at Use for most deployments.

least two For example, mirror spaces
- Stores two or three copies of the data across physical disks are suited for a general-
the set of physical disks to protect from purpose file share or a
- Increases reliability, but reduces capacity. single disk virtual hard disk (VHD)
Duplication occurs with every write. A mirror failure. library.
space also stripes the data across multiple
physical drives. Requires at
- Greater data throughput and lower access least five
latency than parity physical disks
- Uses dirty region tracking (DRT) to track to protect from
modifications to the disks in the pool. When the two
system resumes from an unplanned shutdown simultaneous
and the spaces are brought back online, DRT disk failures.
makes disks in the pool consistent with each
other.

Parity Requires at Use for workloads that are

least three highly sequential, such as
- Stripes data and parity information across physical disks archive or backup.
physical disks to protect from
- Increases reliability when it's compared to a single disk
simple space, but somewhat reduces capacity failure.
- Increases resiliency through journaling. This
function helps prevent data corruption if an
unplanned shutdown occurs.

Step 1: Create a storage pool

You must first group available physical disks into one or more storage pools.

1. In the Server Manager navigation pane, select File and Storage Services.

2. Under Volumes, select Storage Pools.

By default, available disks are included in a pool that is named the primordial pool.
If no primordial pool is listed under STORAGE POOLS, this situation indicates that
the storage doesn't meet the requirements for Storage Spaces. Make sure that the
disks meet the requirements that are outlined in the Prerequisites section.

 Tip

If you select the Primordial storage pool, the available physical disks are listed
under PHYSICAL DISKS.

3. Under STORAGE POOLS, select the TASKS list, and then select New Storage Pool.
The New Storage Pool Wizard opens.

4. On the Before you begin page, select Next.

5. On the Specify a storage pool name and subsystem page, enter a name and
optional description for the storage pool, select the group of available physical
disks that you want to use, and then select Next.

6. On the Select physical disks for the storage pool page, do the following, and then
select Next:

a. Select the check box next to each physical disk that you want to include in the
storage pool.

b. If you want to designate one or more disks as hot spares, under Allocation,
select the drop-down arrow, then select Hot Spare.

7. On the Confirm selections page, verify that the settings are correct, and then
select Create.

8. On the View results page, verify that all tasks completed, and then select Close.

７ Note

Optionally, to continue directly to the next step, you can select the Create a
virtual disk when this wizard closes check box.
 9. Under STORAGE POOLS, verify that the new storage pool is listed.

Windows PowerShell equivalent commands for creating

storage pools
The following Windows PowerShell cmdlet or cmdlets perform the same function as the
preceding procedure. Enter each cmdlet on a single line, even though they may appear
word-wrapped across several lines here because of formatting constraints.

The following example shows which physical disks are available in the primordial pool.

PowerShell

Get-StoragePool -IsPrimordial $true | Get-PhysicalDisk -CanPool $True

The following example creates a new storage pool named StoragePool1 that uses all
available disks.

PowerShell

New-StoragePool –FriendlyName StoragePool1 –StorageSubsystemFriendlyName

"Windows Storage*" –PhysicalDisks (Get-PhysicalDisk –CanPool $True)

The following example creates a new storage pool, StoragePool1, that uses four of the
available disks.

PowerShell

New-StoragePool –FriendlyName StoragePool1 –StorageSubsystemFriendlyName

"Windows Storage*" –PhysicalDisks (Get-PhysicalDisk PhysicalDisk1,
PhysicalDisk2, PhysicalDisk3, PhysicalDisk4)

The following example sequence of cmdlets shows how to add an available physical disk
PhysicalDisk5 as a hot spare to the storage pool StoragePool1.

PowerShell

$PDToAdd = Get-PhysicalDisk –FriendlyName PhysicalDisk5

Add-PhysicalDisk –StoragePoolFriendlyName StoragePool1 –PhysicalDisks
$PDToAdd –Usage HotSpare

Step 2: Create a virtual disk

Next, you must create one or more virtual disks from the storage pool. When you create
a virtual disk, you can select how the data is laid out across the physical disks. This
selection affects both reliability and performance. You can also select whether to create
thin- or fixed-provisioned disks.

1. If the New Virtual Disk Wizard isn't open yet, on the Storage Pools page in Server
Manager, under STORAGE POOLS, make sure that the desired storage pool is
selected.

2. Under VIRTUAL DISKS, select the TASKS list, and then select New Virtual Disk. The
New Virtual Disk Wizard opens.

3. On the Before you begin page, select Next.

4. On the Select the storage pool page, select the desired storage pool, and then
select Next.

5. On the Specify the virtual disk name page, enter a name and optional description,
then select Next.

6. On the Select the storage layout page, select the desired layout, then select Next.

７ Note

If you select a layout where you do not have enough physical disks, you
receive an error message when you select Next. For information about which
layout to use and the disk requirements, see Prerequisites.

7. If you selected Mirror as the storage layout, and you've five or more disks in the
pool, the Configure the resiliency settings page appears. Select one of the
following options:

Two-way mirror
Three-way mirror

8. On the Specify the provisioning type page, select one of the following options,
then select Next.

Thin

With thin provisioning, space is allocated on an as-needed basis. This

selection optimizes the use of available storage. However, because this
setting enables you to over-allocate storage, you must carefully monitor how
much disk space is available.
 Fixed

With fixed provisioning, the storage capacity is allocated immediately, at the

time, a virtual disk is created. Therefore, fixed provisioning uses space from
the storage pool that is equal to the virtual disk size.

 Tip

With Storage Spaces, you can create both thin- and fixed-provisioned
virtual disks in the same storage pool. For example, you can use a thin-
provisioned virtual disk to host a database and a fixed-provisioned
virtual disk to host the associated log files.

9. On the Specify the size of the virtual disk page, perform one of the following
actions:

If you selected thin provisioning in the previous step, follow these steps:
a. In the Virtual disk size box, enter a virtual disk size.
b. Select the units (MB, GB, or TB), and then select Next.

If you selected fixed provisioning in the previous step, select one of the
following options:

Specify size

To specify a size, enter a value in the Virtual disk size box, then select the
units (MB, GB, or TB).

７ Note

If you use a storage layout other than simple, the virtual disk uses
more free space than the size that you specify. To avoid a potential
error in which the size of the volume exceeds the storage pool free
space, select the Create the largest virtual disk possible, up to the
specified size check box.

Maximum size

Select this option to create a virtual disk that uses the maximum capacity
of the storage pool.

10. On the Confirm selections page, verify that the settings are correct, and then
select Create.
 11. On the View results page, verify that all tasks are completed, and then select
Close.

 Tip

By default, the Create a volume when this wizard closes check box is
selected. This takes you directly to the next step.

Windows PowerShell equivalent commands for creating

virtual disks
The following Windows PowerShell cmdlets perform the same function as the preceding
procedure. Enter each cmdlet on a single line, even though they may appear word-
wrapped across several lines here because of formatting constraints.

The following example creates a 50-GB virtual disk named VirtualDisk1 on a storage
pool named StoragePool1.

PowerShell

New-VirtualDisk –StoragePoolFriendlyName StoragePool1 –FriendlyName

VirtualDisk1 –Size (50GB)

The following example creates a mirrored virtual disk named VirtualDisk1 on a storage
pool named StoragePool1. The disk uses the storage pool's maximum storage capacity.

PowerShell

New-VirtualDisk –StoragePoolFriendlyName StoragePool1 –FriendlyName

VirtualDisk1 –ResiliencySettingName Mirror –UseMaximumSize

The following example creates a 50-GB virtual disk named VirtualDisk1 on a storage
pool that is named StoragePool1. The disk uses the thin provisioning type.

PowerShell

New-VirtualDisk –StoragePoolFriendlyName StoragePool1 –FriendlyName

VirtualDisk1 –Size (50GB) –ProvisioningType Thin

The following example creates a virtual disk named VirtualDisk1 on a storage pool
named StoragePool1. The virtual disk uses three-way mirroring and has a fixed size of 20
GB.
 ７ Note

You must have at least five physical disks in the storage pool for this cmdlet to
work. (This does not include any disks that are allocated as hot spares.)

PowerShell

New-VirtualDisk -StoragePoolFriendlyName StoragePool1 -FriendlyName

VirtualDisk1 -ResiliencySettingName Mirror -NumberOfDataCopies 3 -Size 20GB
-ProvisioningType Fixed

Step 3: Create a volume

Next, you must create a volume from the virtual disk. You can assign an optional drive
letter or folder, then format the volume with a file system.

1. If the New Volume Wizard isn't open yet, on the Storage Pools page in Server
Manager, under VIRTUAL DISKS, right-click the desired virtual disk, and then select
New Volume.

The New Volume Wizard opens.

2. On the Before you begin page, select Next.

3. On the Select the server and disk page, do the following, and then select Next.

a. In the Server area, select the server on which you want to provision the volume.

b. In the Disk area, select the virtual disk on which you want to create the volume.

4. On the Specify the size of the volume page, enter a volume size, specify the units
(MB, GB, or TB), and then select Next.

5. On the Assign to a drive letter or folder page, configure the desired option, and
then select Next.

6. On the Select file system settings page, do the following, and then select Next.

a. In the File system list, select either NTFS or ReFS.

b. In the Allocation unit size list, either leave the setting at Default or set the
allocation unit size.

７ Note
 For more information about allocation unit size, see Default cluster size for
NTFS, FAT, and exFAT .

c. Optionally, in the Volume label box, enter a volume label name, for example HR
Data.

7. On the Confirm selections page, verify that the settings are correct, and then
select Create.

8. On the View results page, verify that all tasks are completed, and then select
Close.

9. To verify that the volume was created, in Server Manager, select the Volumes page.
The volume is listed under the server where it was created. You can also verify that
the volume was created in Windows Explorer.

Windows PowerShell equivalent commands for creating

volumes
The following Windows PowerShell cmdlet performs the same function as the previous
procedure. Enter the command on a single line.

The following example initializes the disks for virtual disk VirtualDisk1, creates a partition
with an assigned drive letter, and then formats the volume with the default NTFS file
system.

PowerShell

Get-VirtualDisk –FriendlyName VirtualDisk1 | Get-Disk | Initialize-Disk –

Passthru | New-Partition –AssignDriveLetter –UseMaximumSize | Format-Volume

Additional information
Storage Spaces overview
Windows PowerShell cmdlets in Storage
Deploy Clustered Storage Spaces
The forums at Windows Server Storage
Tutorial: Enable storage bus cache with
Storage Spaces on standalone servers
Article • 02/10/2022

Applies to: Windows Server 2022

The storage bus cache for standalone servers can significantly improve read and write
performance, while maintaining storage efficiency and keeping the operational costs
low. Similar to its implementation for Storage Spaces Direct, this feature binds together
faster media (for example, SSD) with slower media (for example, HDD) to create tiers. By
default, only a portion of the faster media tier is reserved for the cache.

Resiliency Cache type

None (Simple space) Read and write

Mirror accelerated parity Read

If your system does not require resiliency or has external backups, the storage bus cache
will support both read and write caching. For resilient systems, the storage bus cache
will only serve as a read cache and it is recommended to pick ReFS Mirror-accelerated
parity as the volume resiliency. This combination improves random read performance as
data is read from the parity tier and cached on the faster mirror tier. The mirror tier also
provides write caching capabilities if the Provision Mode is set to Shared (default).
In this tutorial, you learn about:

＂ What the storage bus cache is

＂ How to enable the storage bus cache
＂ Managing the cache after deployment

Prerequisites

Consider storage bus cache if:

Your server runs Windows Server 2022; and
Your server has 2 media/ drive types, one of which must be HDD (for example:
SSD+HDD or NVMe+HDD); and
Your server has the Failover Clustering feature installed

You can't use storage bus cache if:

Your server runs Windows Server 2016 or 2019; or
Your server has an all flash configuration; or
You server is a member of a Failover Cluster

７ Note

This feature requires your server to have the Failover Clustering feature installed
but your server cannot be a part of a Failover Cluster.

Feature overview
This section explains what each configurable field of the storage bus cache is and
applicable values.

PowerShell

Get-StorageBusCache

The output should resemble below when not enabled:

PowerShell
 ProvisionMode : Shared
SharedCachePercent : 15
CacheMetadataReserveBytes : 34359738368
CacheModeHDD : ReadWrite
CacheModeSSD : WriteOnly
CachePageSizeKBytes : 16
Enabled : False

７ Note

For general use, default settings are recommended. Any changes must be made
prior to enabling the storage bus cache.

Provision Mode
This field determines if the entire faster media tier or only a portion of it will be used for
caching. This field cannot be modified after enabling the storage bus cache.

Shared (default): The cache will only take up a portion of the faster media tier. The
exact percentage is configurable by the Shared Cache Percentage field below.
Cache: Dedicate majority of the faster media tier to caching as opposed to only a
portion. The implementation is similar to the storage bus cache in Storage Spaces
Direct.

Shared cache percentage

This field is only applicable when the Provision Mode is set to Shared. The default value
is 15% and the field can be set from 5% to 90%. A value over 50% is not recommended
when using mirror accelerated parity volumes as there needs to be a balance between
the cache and the mirror tier.

Enabled
This field refers to the state of the storage bus cache and can either be True or False.

Advanced fields

） Important
 Changes to these fields are not recommended. Adjustments after enabling the
storage bus cache cannot be made.

Cache metadata reserve bytes: The amount of disk space (in bytes) reserved for
Storage Spaces. This field is only applied if the Provision Mode is Cache.

Cache mode HDD: The default is to allow the HDD capacity devices to cache reads
and writes. For Simple spaces, this setting can be set to ReadWrite or WriteOnly.

Cache mode SSD: For future use when all flash systems are supported. The default
is to allow the SSD capacity devices to cache writes only.

Cache page size KBytes: This field can be set to 8, 16 (default), 32 and 64.

Enable storage bus cache in PowerShell

This section is a step-by-step guide on how to enable the storage bus cache for your
stand-alone server in PowerShell.

1. Import the module

PowerShell

Import-Module StorageBusCache

2. Configure storage bus cache settings

Default settings are recommended, skip this step to continue with the defaults.

） Important

If configuration is needed, do so before enabling the storage bus cache. Refer

to Feature overview section for details of the fields.

3. Check the drive status

PowerShell

Get-PhysicalDisk

The output should resemble the image below, where the Number column shows
values under 500 and the CanPool column shows True for all non-boot drives.
4. Enable storage bus cache

PowerShell

Enable-StorageBusCache

This step will:

＂ Create a storage pool with all the available drives

＂ Bind the fast and slow media and form the cache
＂ Add the storage bus cache with default or custom settings

You can run Get-StoragePool to see the name of the storage pool and Get-
PhysicalDisk again to see the effects of enabling storage bus cache. The output

should resemble the image below, where the Number column shows values over
500 (indicating the drive is claimed by the storage bus) and the CanPool column
now shows False for all non-boot drives. If the ProvisionMode was set to Cache
prior to enabling, the Usage column will show as Journal for the faster drives.
 5. Check the storage bus cache state

Check that the fields are correct and the Enabled field is now set to true.

PowerShell

Get-StorageBusCache

The output should resemble below:

PowerShell

ProvisionMode : Shared
SharedCachePercent : 15
CacheMetadataReserveBytes : 34359738368
CacheModeHDD : ReadWrite
CacheModeSSD : WriteOnly
CachePageSizeKBytes : 16
Enabled : True

Now the storage bus cache has been successfully enabled, the next step is to create a
volume.

Create a volume

Volumes with resiliency:

The PowerShell cmdlet below creates a 1TiB mirror-accelerated parity volume with a
Mirror:Parity ratio of 20:80, which is the recommended configuration for most
workloads. For more information, see Mirror-accelerated parity.

PowerShell

New-Volume –FriendlyName "TestVolume" -FileSystem ReFS -

StoragePoolFriendlyName Storage* -StorageTierFriendlyNames MirrorOnSSD,
ParityOnHDD -StorageTierSizes 200GB, 800GB

Volumes without resiliency:

The PowerShell cmdlet below creates a 1TB Simple volume that cannot tolerate any disk
failure. Both read and write caching is supported.

PowerShell
 New-Volume -FriendlyName "TestVolume" -FileSystem ReFS -
StoragePoolFriendlyName Storage* -ResiliencySettingName Simple -Size 1TB

Making changes after enabling storage bus

cache
After running Enable-StorageBusCache , the Provision mode, Shared cache percent, Cache
metadata reserve bytes, Cache mode HDD, Cache mode SSD, and Cache page size
cannot be modified. Limited changes can be made to the physical setup, below are
some common scenarios.

Adding or replacing capacity drives (HDDs)

Once the drive has been manually added, run the cmdlet below to finish the intake
process.

PowerShell

Update-StorageBusCache

Adding or replacing cache drives (NVMes or SSDs)

There is no cmdlet to unbind/rebind existing bindings and balance out the relationship.
The steps below will cause the existing read cache to be lost.

PowerShell

Remove-StorageBusBinding
New-StorageBusBinding

Check and balance cache and capacity bindings

Use the following cmdlet to check the existing cache and capacity bindings.

PowerShell

Get-StorageBusBinding

In the example below, the first column lists out capacity drives and the third column lists
out the cache drives that they are bound to. Follow the instructions in Adding or
replacing cache drives to balance, the existing cache will not be preserved.

Storage bus cache FAQ

This section answers frequently asked questions about the storage bus cache on
Windows Server 2022

Why does the Failover Clustering feature need to be

installed when the server is not a part of a Failover
Cluster?
This feature is designed for standalone servers but built on top of the storage bus layer
(SBL) cache for Storage Spaces Direct. The Failover Clustering feature needs to be
installed as clustering components are needed.

Will the storage bus cache work with an all flash

configuration?
No, this feature will only work when there are two media types, one of which must be
HDD. This will not work with RAID, SAN, or all flash systems.

How can the storage bus cache settings be change?

See example below for changing the Provision mode from Shared (default) to Cache.
Note that default settings are recommended and any changes should be made before
the storage bus cache is enabled.

PowerShell

Set-StorageBusCache -ProvisionMode Cache

Troubleshoot Storage Spaces and
Storage Spaces Direct health and
operational states
Article • 04/28/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019, Windows Server 2016, Windows Server 2012 R2, Windows
Server 2012, Windows 10, Windows 8.1

This topic describes the health and operational states of storage pools, virtual disks
(which sit underneath volumes in Storage Spaces), and drives in Storage Spaces Direct
and Storage Spaces. These states can be invaluable when trying to troubleshoot various
issues such as why you can't delete a virtual disk because of a read-only configuration. It
also discusses why a drive can't be added to a pool (the CannotPoolReason).

Storage Spaces has three primary objects - physical disks (hard drives, SSDs, etc.) that
are added to a storage pool, virtualizing the storage so that you can create virtual disks
from free space in the pool, as shown here. Pool metadata is written to each drive in the
pool. Volumes are created on top of the virtual disks and store your files, but we're not
going to talk about volumes here.
You can view health and operational states in Server Manager, or with PowerShell. Here's
an example of a variety of (mostly bad) health and operational states on a Storage
Spaces Direct cluster that's missing most of its cluster nodes (right-click the column
headers to add Operational Status). This isn't a happy cluster.

Storage pool states

Every storage pool has a health status - Healthy, Warning, or Unknown/Unhealthy, as
well as one or more operational states.

To find out what state a pool is in, use the following PowerShell commands:

PowerShell

Get-StoragePool -IsPrimordial $False | Select-Object HealthStatus,

OperationalStatus, ReadOnlyReason

Here's an example output showing a storage pool in the Unknown health state with the
Read-only operational status:

FriendlyName OperationalStatus HealthStatus IsPrimordial

IsReadOnly
------------ ----------------- ------------ ------------ ----
------
S2D on StorageSpacesDirect1 Read-only Unknown False True
The following sections list the health and operational states.

Pool health state: Healthy

Operational state Description

OK The storage pool is healthy.

Pool health state: Warning

When the storage pool is in the Warning health state, it means that the pool is
accessible, but one or more drives failed or are missing. As a result, your storage pool
might have reduced resilience.

Operational Description
state

Degraded There are failed or missing drives in the storage pool. This condition occurs only
with drives hosting pool metadata.

Action: Check the state of your drives and replace any failed drives before there
are additional failures.

Pool health state: Unknown or Unhealthy

When a storage pool is in the Unknown or Unhealthy health state, it means that the
storage pool is read-only and can't be modified until the pool is returned to the
Warning or OK health states.

Operational Read-only Description

state reason
 Operational Read-only Description
state reason

Read-only Incomplete This can occur if the storage pool loses its quorum, which means
that most drives in the pool have failed or are offline for some
reason. When a pool loses its quorum, Storage Spaces automatically
sets the pool configuration to read-only until enough drives become
available again.

Action:
1. Reconnect any missing drives, and if you're using Storage Spaces
Direct, bring all servers online.
2. Set the pool back to read-write by opening a PowerShell session
with administrative permissions and then typing:

Get-StoragePool <PoolName> -IsPrimordial $False | Set-

StoragePool -IsReadOnly $false

Policy An administrator set the storage pool to read-only.

Action: To set a clustered storage pool to read-write access in

Failover Cluster Manager, go to Pools, right-click the pool and then
select Bring Online.

For other servers and PCs, open a PowerShell session with

administrative permissions and then type:

Get-StoragePool <PoolName> | Set-StoragePool -IsReadOnly $false

Starting Storage Spaces is starting or waiting for drives to be connected in

the pool. This should be a temporary state. Once completely started,
the pool should transition to a different operational state.

Action: If the pool stays in the Starting state, make sure that all
drives in the pool are connected properly.

See also, the Windows Server storage forum.

Virtual disk states

In Storage Spaces, volumes are placed on virtual disks (storage spaces) that are carved
out of free space in a pool. Every virtual disk has a health status - Healthy, Warning,
Unhealthy, or Unknown as well as one or more operational states.

To find out what state virtual disks are in, use the following PowerShell commands:
 PowerShell

Get-VirtualDisk | Select-Object FriendlyName,HealthStatus,

OperationalStatus, DetachedReason

Here's an example of output showing a detached virtual disk and a

degraded/incomplete virtual disk:

FriendlyName HealthStatus OperationalStatus DetachedReason

------------ ------------ ----------------- --------------
Volume1 Unknown Detached By Policy
Volume2 Warning {Degraded, Incomplete} None

The following sections list the health and operational states.

Virtual disk health state: Healthy

Operational Description
state

OK The virtual disk is healthy.

Suboptimal Data isn't written evenly across drives.

Action: Optimize drive usage in the storage pool by running the Optimize-
StoragePool cmdlet.

Virtual disk health state: Warning

When the virtual disk is in a Warning health state, it means that one or more copies of
your data are unavailable, but Storage Spaces can still read at least one copy of your
data.

Operational Description
state

In service Windows is repairing the virtual disk, such as after adding or removing a drive.
When the repair is complete, the virtual disk should return to the OK health state.
 Operational Description
state

Incomplete The resilience of the virtual disk is reduced because one or more drives failed or
are missing. However, the missing drives contain up-to-date copies of your data.

Action:
1. Reconnect any missing drives, replace any failed drives, and if you're using
Storage Spaces Direct, bring online any servers that are offline.
2. If you're not using Storage Spaces Direct, next repair the virtual disk using the
Repair-VirtualDisk cmdlet.
Storage Spaces Direct automatically starts a repair if needed after reconnecting or
replacing a drive.

Degraded The resilience of the virtual disk is reduced because one or more drives failed or
are missing, and there are outdated copies of your data on these drives.

Action:
1. Reconnect any missing drives, replace any failed drives, and if you're using
Storage Spaces Direct, bring online any servers that are offline.
2. If you're not using Storage Spaces Direct, next repair the virtual disk using the
Repair-VirtualDisk cmdlet.
Storage Spaces Direct automatically starts a repair if needed after reconnecting or
replacing a drive.

Virtual disk health state: Unhealthy

When a virtual disk is in an Unhealthy health state, some or all of the data on the virtual
disk is currently inaccessible.

Operational state Description

No redundancy The virtual disk has lost data because too many drives failed.

Action: Replace failed drives and then restore your data from backup.

Virtual disk health state: Information/Unknown

The virtual disk can also be in the Information health state (as shown in the Storage
Spaces Control Panel item) or Unknown health state (as shown in PowerShell) if an
administrator took the virtual disk offline or the virtual disk has become detached.

Operational Detached Description

state reason
Operational Detached Description
state reason

Detached By Policy An administrator took the virtual disk offline, or set the virtual disk to
require manual attachment, in which case you'll have to manually
attach the virtual disk every time Windows restarts.,

Action: Bring the virtual disk back online. To do so when the virtual
disk is in a clustered storage pool, in Failover Cluster Manager select
Storage > Pools > Virtual Disks, select the virtual disk that shows
the Offline status and then select Bring Online.

To bring a virtual disk back online when not in a cluster, open a

PowerShell session as an Administrator and then try using the
following command:

Get-VirtualDisk | Where-Object -Filter { $_.OperationalStatus -

eq "Detached" } | Connect-VirtualDisk

To automatically attach all non-clustered virtual disks after Windows

restarts, open a PowerShell session as an Administrator and then use
the following command:

Get-VirtualDisk | Set-VirtualDisk -ismanualattach $false

Majority Too many drives used by this virtual disk failed, are missing, or have
Disks stale data.
Unhealthy
Action:
1. Reconnect any missing drives, and if you're using Storage Spaces
Direct, bring online any servers that are offline.
2. After all drives and servers are online, replace any failed drives.
See Health Service for details.
Storage Spaces Direct automatically starts a repair if needed after
reconnecting or replacing a drive.
3. If you're not using Storage Spaces Direct, next repair the virtual
disk using the Repair-VirtualDisk cmdlet.

If more disks failed than you have copies of your data and the virtual
disk wasn't repaired in-between failures, all data on the virtual disk is
permanently lost. In this unfortunate case, delete the virtual disk,
create a new virtual disk, and then restore from a backup.
 Operational Detached Description
state reason

Incomplete Not enough drives are present to read the virtual disk.

Action:
1. Reconnect any missing drives, and if you're using Storage Spaces
Direct, bring online any servers that are offline.
2. After all drives and servers are online, replace any failed drives.
See Health Service for details.
Storage Spaces Direct automatically starts a repair if needed after
reconnecting or replacing a drive.
3. If you're not using Storage Spaces Direct, next repair the virtual
disk using the Repair-VirtualDisk cmdlet.

If more disks failed than you have copies of your data and the virtual
disk wasn't repaired in-between failures, all data on the virtual disk is
permanently lost. In this unfortunate case, delete the virtual disk,
create a new virtual disk, and then restore from a backup.

Timeout Attaching the virtual disk took too long

Action: This shouldn't happen often, so you might try see if the
condition passes in time. Or you can try disconnecting the virtual
disk with the Disconnect-VirtualDisk cmdlet, then using the Connect-
VirtualDisk cmdlet to reconnect it.

Drive (physical disk) states

The following sections describe the health states a drive can be in. Drives in a pool are
represented in PowerShell as physical disk objects.

Drive health state: Healthy

Operational Description
state

OK The drive is healthy.

In service The drive is performing some internal housekeeping operations. When the action
is complete, the drive should return to the OK health state.

Drive health state: Warning

A drive in the Warning state can read and write data successfully but has an issue.
Operational Description
state

Lost The drive is missing. If you're using Storage Spaces Direct, this could be because
communication a server is down.

Action: If you're using Storage Spaces Direct, bring all servers back online. If
that doesn't fix it, reconnect the drive, replace it, or try getting detailed
diagnostic info about this drive by following the steps in Troubleshooting using
Windows Error Reporting > Physical disk timed out.

Removing from Storage Spaces is in the process of removing the drive from its storage pool.
pool
This is a temporary state. After the removal is complete, if the drive is still
attached to the system, the drive transitions to another operational state
(usually OK) in a primordial pool.

Starting Storage Spaces is in the process of putting the drive in maintenance mode after
maintenance an administrator put the drive in maintenance mode. This is a temporary state -
mode the drive should soon be in the In maintenance mode state.

In maintenance An administrator placed the drive in maintenance mode, halting reads and
mode writes from the drive. This is usually done before updating drive firmware, or
when testing failures.

Action: To take the drive out of maintenance mode, use the Disable-
StorageMaintenanceMode cmdlet.

Stopping An administrator took the drive out of maintenance mode, and Storage Spaces
maintenance is in the process of bringing the drive back online. This is a temporary state -
mode the drive should soon be in another state - ideally Healthy.

Predictive The drive reported that it's close to failing.

failure
Action: Replace the drive.

IO error There was a temporary error accessing the drive.

Action:
1. If the drive doesn't transition back to the OK state, you can try using the
Reset-PhysicalDisk cmdlet to wipe the drive.
2. Use Repair-VirtualDisk to restore the resiliency of affected virtual disks.
3. If this keeps happening, replace the drive.
 Operational Description
state

Transient error There was a temporary error with the drive. This usually means the drive was
unresponsive, but it could also mean that the Storage Spaces protective
partition was inappropriately removed from the drive.

Action:
1. If the drive doesn't transition back to the OK state, you can try using the
Reset-PhysicalDisk cmdlet to wipe the drive.
2. Use Repair-VirtualDisk to restore the resiliency of affected virtual disks.
3. If this keeps happening, replace the drive, or try getting detailed diagnostic
info about this drive by following the steps in Troubleshooting using Windows
Error Reporting > Physical disk failed to come online.

Abnormal The drive is performing slowly, as measured by the Health Service in Storage
latency Spaces Direct.

Action: If this keeps happening, replace the drive so it doesn't reduce the
performance of Storage Spaces as a whole.

Drive health state: Unhealthy

A drive in the Unhealthy state can't currently be written to or accessed.

Operational Description
state

Not usable This drive can't be used by Storage Spaces. For more info see Storage Spaces
Direct hardware requirements; if you're not using Storage Spaces Direct, see
Storage Spaces overview.

Split The drive has become separated from the pool.

Action: Reset the drive, erasing all data from the drive and adding it back to the
pool as an empty drive. To do so, open a PowerShell session as an administrator,
run the Reset-PhysicalDisk cmdlet, and then run Repair-VirtualDisk.

To get detailed diagnostic info about this drive, follow the steps in
Troubleshooting using Windows Error Reporting > Physical disk failed to come
online.
 Operational Description
state

Stale Storage Spaces found old metadata on the drive.

metadata
Action: This should be a temporary state. If the drive doesn't transition back to
OK, you can run Repair-VirtualDisk to start a repair operation on affected virtual
disks. If that doesn't resolve the issue, you can reset the drive with the Reset-
PhysicalDisk cmdlet, wiping all data from the drive, and then run Repair-
VirtualDisk.

Unrecognized Storage Spaces found unrecognized metadata on the drive, which usually means
metadata that the drive has metadata from a different pool on it.

Action: To wipe the drive and add it to the current pool, reset the drive. To reset
the drive, open a PowerShell session as an administrator, run the Reset-
PhysicalDisk cmdlet, and then run Repair-VirtualDisk.

Failed media The drive failed and won't be used by Storage Spaces anymore.

Action: Replace the drive.

To get detailed diagnostic info about this drive, follow the steps in
Troubleshooting using Windows Error Reporting > Physical disk failed to come
online.

Device There was a hardware failure on this drive.

hardware
failure Action: Replace the drive.

Updating Windows is updating the firmware on the drive. This is a temporary state that
firmware usually lasts less than a minute and during which time other drives in the pool
handle all reads and writes. For more info, see Update drive firmware.

Starting The drive is getting ready for operation. This should be a temporary state - once
complete, the drive should transition to a different operational state.

Reasons a drive can't be pooled

Some drives just aren't ready to be in a storage pool. You can find out why a drive isn't
eligible for pooling by looking at the CannotPoolReason property of a physical disk.
Here's an example PowerShell script to display the CannotPoolReason property:

PowerShell

Get-PhysicalDisk | Format-Table
FriendlyName,MediaType,Size,CanPool,CannotPoolReason
Here's an example output:

FriendlyName MediaType Size CanPool CannotPoolReason

------------ --------- ---- ------- ----------------
ATA MZ7LM120HCFD00D3 SSD 120034123776 False Insufficient Capacity
Msft Virtual Disk SSD 10737418240 True
Generic Physical Disk SSD 119990648832 False In a Pool

The following table gives a little more detail on each of the reasons.

Reason Description

In a pool The drive already belongs to a storage pool.

Drives can belong to only a single storage pool at a time. To use this drive in
another storage pool, first remove the drive from its existing pool, which tells
Storage Spaces to move the data on the drive to other drives on the pool. Or reset
the drive if the drive has been disconnected from its pool without notifying Storage
Spaces.

To safely remove a drive from a storage pool, use Remove-PhysicalDisk, or go to

Server Manager > File and Storage Services > Storage Pools, > Physical Disks,
right-click the drive and then select Remove Disk.

To reset a drive, use Reset-PhysicalDisk.

Not The drive isn't in a healthy state and might need to be replaced.
healthy

Removable The drive is classified as a removable drive.

media
Storage Spaces doesn't support media that are recognized by Windows as
removable, such as Blu-Ray drives. Although many fixed drives are in removable
slots, in general, media that are classified by Windows as removable aren't suitable
for use with Storage Spaces.

In use by The drive is currently used by a Failover Cluster.

cluster
Reason Description

Offline The drive is offline.

To bring all offline drives online and set to read/write, open a PowerShell session as
an administrator and use the following scripts:

Get-Disk | Where-Object -Property OperationalStatus -EQ "Offline" | Set-Disk -

IsOffline $false

Get-Disk | Where-Object -Property IsReadOnly -EQ $true | Set-Disk -IsReadOnly

$false

Insufficient This typically occurs when there are partitions taking up the free space on the drive.
capacity
Action: Delete any volumes on the drive, erasing all data on the drive. One way to
do that is to use the Clear-Disk PowerShell cmdlet.

Verification The Health Service is checking to see if the drive or firmware on the drive is
in progress approved for use by the server administrator.

Verification The Health Service couldn't check to see if the drive or firmware on the drive is
failed approved for use by the server administrator.

Firmware The firmware on the physical drive isn't in the list of approved firmware revisions
not specified by the server administrator by using the Health Service.
compliant

Hardware The drive isn't in the list of approved storage models specified by the server
not administrator by using the Health Service.
compliant

Additional References
Storage Spaces Direct
Storage Spaces Direct hardware requirements
Understanding cluster and pool quorum
Storage Spaces Direct overview
Article • 04/18/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019, Windows Server 2016

Storage Spaces Direct is a feature of Azure Stack HCI and Windows Server that enables
you to cluster servers with internal storage into a software-defined storage solution.

This article provides an overview of Storage Spaces Direct, how it works, when to use it,
and its key benefits. You can also explore videos and real-world customer stories in this
article to learn more about Storage Spaces Direct.

To get started, try Storage Spaces Direct in Microsoft Azure, or download a 180-day-
licensed evaluation copy of Windows Server from Windows Server Evaluations . To
know the minimum hardware requirements for Storage Spaces Direct on Windows
Server and Azure Stack HCI, see System requirements for Windows Server and System
requirements for Azure Stack HCI, respectively. To deploy Storage Spaces Direct as part
of Azure Stack HCI, see Deploy the Azure Stack HCI operating system.

What is Storage Spaces Direct?

Storage Spaces Direct is a software-defined storage solution that allows you to share
storage resources in your converged and hyperconverged IT infrastructure. It enables
you to combine internal storage drives on a cluster of physical servers (2 and up to 16)
into a software-defined pool of storage. This storage pool has cache, tiers, resiliency,
and erasure coding across columns—all configured and managed automatically.

You can scale out the storage capacity of your cluster by adding more drives or adding
more servers in the cluster. Storage Spaces Direct automatically onboards the new
drives and rebalances the storage pool. It also automatically uses the fastest storage
media present to provide built-in and always-on cache.

Storage Spaces Direct is a core technology of Azure Stack HCI, versions 21H2 and 20H2.
It’s also included in the Datacenter editions of Windows Server 2022, Windows Server
2019, Windows Server 2016, Windows Server Insider Preview Builds and Azure Edition
of Windows Server 2022.

You can deploy Storage Spaces Direct on a cluster of physical servers or on virtual
machine (VM) guest clusters. If deploying it on a hyperconverged cluster of physical
servers, we recommend using Azure Stack HCI servers. To deploy Storage Spaces Direct
as part of Azure Stack HCI, see Deploy the Azure Stack HCI operating system.

Deploying Storage Spaces Direct on VM guest clusters delivers virtual shared storage
across a set of VMs on top of a private or public cloud. In production environments, this
deployment is supported only in Windows Server. For information about how to deploy
Storage Spaces Direct on VM guest clusters in Windows Server, see Using Storage
Spaces Direct in guest virtual machine clusters.

For testing and evaluation purposes only, you can deploy Storage Spaces Direct on VM
guest clusters in Azure Stack HCI test environments. For information about deploying it
in Azure Stack HCI test environments, see Tutorial: Create a VM-based lab for Azure
Stack HCI.

How it works
Storage Spaces Direct applies many of the features in Windows Server, such as Failover
Clustering, the Cluster Shared Volume (CSV) file system, Server Message Block (SMB) 3,
and Storage Spaces. It also introduces a new technology called Software Storage Bus.

Storage Spaces Direct creates a software-defined storage solution by combining the

internal storage drives on a cluster of industry-standard servers. You start by connecting
your servers with internal storage drives over Ethernet to form a cluster—no special
cable or storage fabric is required. When you enable Storage Spaces Direct on this
cluster, it combines the storage drives from each of those servers into one software-
defined pool of virtually shared storage.

You then create volumes from that pool of storage, into which you can store your data.
These volumes run CSV file system. This means that to each server these volumes look
and act as if they're mounted locally. With built-in fault tolerance in these volumes, your
data stays online and accessible even if a drive fails or the entire node goes offline.

In these volumes, you can place your files, such as .vhd and .vhdx for VMs. You can use
the cluster running Storage Spaces Direct as:

Scale-out File Server (SoFS) by exposing the volumes over the network as SMB3
File Shares.
Hyperconverged system by enabling Hyper-V on the cluster and placing your VMs
directly on top of the volumes.

The following section describes the features and components of a Storage Spaces Direct
stack.
Networking Hardware. Storage Spaces Direct uses SMB3, including SMB Direct and
SMB Multichannel, over Ethernet to communicate between servers. We strongly
recommend using 10+ GbE with remote-direct memory access (RDMA), either iWARP or
RoCE.

Storage Hardware. Storage Spaces Direct requires 2 and up to 16 Microsoft-approved

servers with direct-attached SATA, SAS, NVMe, or persistent memory drives that are
physically attached to just one server each. Each server must have at least two solid-
state drives, and at least four more drives. The SATA and SAS devices should be behind a
host-bus adapter (HBA) and SAS expander.

Failover Clustering. Storage Spaces Direct uses the built-in clustering feature of Azure
Stack HCI and Windows Server to connect the servers.
Software Storage Bus. The Software Storage Bus spans the cluster and establishes a
software-defined storage fabric whereby all the servers can see all of each other's local
drives. You can think of it as replacing costly and restrictive Fibre Channel or Shared SAS
cabling.

Storage Bus Layer Cache. The Software Storage Bus dynamically binds the fastest drives
present (for example, SSD) to slower drives (for example, HDDs) to provide server-side
read/write caching that accelerates IO and boosts throughput.

Storage Pool. The collection of drives that form the basis of Storage Spaces is called the
storage pool. It's automatically created, and all eligible drives are automatically
discovered and added to it. We strongly recommend you use one pool per cluster, with
the default settings. To learn more about the Storage Pool, see the Deep Dive into the
Storage Pool blog.

Storage Spaces. Storage Spaces provides fault tolerance to "virtual disks" using
mirroring, erasure coding, or both. You can think of it as distributed, software-defined
RAID using the drives in the pool. In Storage Spaces Direct, these virtual disks typically
have resiliency to two simultaneous drive or server failures (for example, 3-way
mirroring, with each data copy in a different server)—though chassis and rack fault
tolerance is also available.

Resilient File System (ReFS). ReFS is the premier filesystem purpose-built for
virtualization. It includes dramatic accelerations for .vhdx file operations, such as
creation, expansion, and checkpoint merging, and built-in checksums to detect and
correct bit errors. It also introduces real-time tiers that rotate data between "hot" and
"cold" storage tiers in real-time, based on usage.

Cluster Shared Volumes. The CSV file system unifies all the ReFS volumes into a single
namespace accessible through any server. To each server, every volume looks and acts
like it's mounted locally.

Scale-Out File Server. This final layer is necessary only in converged deployments. It
provides remote file access by using the SMB3 access protocol to clients, such as
another cluster running Hyper-V, over the network, effectively turning Storage Spaces
Direct into network-attached storage (NAS).

Key benefits
Storage Spaces Direct offers the following key benefits:

Image Description
 Image Description

Simplicity. Go from industry-standard servers running Windows Server or Azure Stack

HCI, to your first Storage Spaces Direct cluster in under 15 minutes. For System Center
users, deployment is just one checkbox.

High performance. Whether all-flash or hybrid, Storage Spaces Direct can exceed 13.7
million IOPS per server . The hypervisor-embedded architecture of Storage Spaces
Direct provides consistent, low latency, built-in read/write cache, and support for
cutting-edge NVMe drives mounted directly on the PCIe bus.

Fault tolerance. Built-in resiliency handles drive, server, or component failures with
continuous availability. Larger deployments can also be configured for chassis and rack
fault tolerance. When hardware fails, just swap it out; the software heals itself, with no
complicated management steps.

Resource efficiency. Erasure coding delivers up to 2.4x greater storage efficiency, with
unique innovations, such as Local Reconstruction Codes and ReFS real-time tiers to
extend these gains to hard disk drives and mixed hot or cold workloads, all while
minimizing CPU consumption to give resources back to where they're needed the most
—to the VMs.

Manageability. Use Storage QoS Controls to keep busy VMs in check with minimum and
maximum per-VM IOPS limits. The Health Service provides continuous built-in
monitoring and alerting. New APIs make it easy to collect rich, cluster-wide performance
and capacity metrics.

Scalability. Go up to 16 servers and over 400 drives, for up to 4 petabyte (4,000

terabytes) of storage per cluster. To scale out, add more drives or add more servers;
Storage Spaces Direct automatically onboards new drives and begin using them. Storage
efficiency and performance improve predictably at scale.

When to use it
Storage Spaces Direct is a core technology of Azure Stack HCI and Windows Server. It
provides an ideal network storage solution when you want to:

Scale up or scale out your network storage capacity. You can add more drives or
add more servers to expand your network storage capacity, still keeping your data
secured and accessible. If a drive within the storage pool fails or the entire node
goes offline, all data stays online and accessible.
Share the same set of data from different locations at the same time. The storage
pool that Storage Spaces Direct creates looks and acts like a network share. Your
network users can access the stored data anytime from any location, without
worrying about the physical location of their stored data.
 Use a mix of storage media. With Storage Spaces Direct, you can combine different
types of storage media in your server cluster to form the software-defined storage
pool. The software automatically decides which media to use based on data—
active data on faster media and other infrequently used data on slower media.

Deployment options
Storage Spaces Direct supports the following two deployment options:

Hyperconverged
Converged

７ Note

Azure Stack HCI supports only hyperconverged deployment.

Hyperconverged deployment
In a hyperconverged deployment, you use single cluster for both compute and storage.
The hyperconverged deployment option runs Hyper-V virtual machines or SQL Server
databases directly on the servers providing the storage—storing their files on the local
volumes. This eliminates the need to configure file server access and permissions, which
in turn reduces hardware costs for small-to-medium business and remote or branch
office deployments. To deploy Storage Spaces Direct on Windows Server, see Deploy
Storage Spaces Direct on Windows Server. To deploy Storage Spaces Direct as part of
Azure Stack HCI, see What is the deployment process for Azure Stack HCI?

Converged deployment
In a converged deployment, you use separate clusters for storage and compute. The
converged deployment option, also known as 'disaggregated,' layers a Scale-out File
Server (SoFS) atop Storage Spaces Direct to provide network-attached storage over
SMB3 file shares. This allows for scaling compute and workload independently from the
storage cluster, essential for larger-scale deployments such as Hyper-V IaaS
(Infrastructure as a Service) for service providers and enterprises.

Manage and monitor

You can use the following tools to manage and monitor Storage Spaces Direct:

Name Graphical or command- Paid or

line? included?

Windows Admin Center Graphical Included

Server Manager & Failover Cluster Manager Graphical Included

Windows PowerShell Command-line Included

System Center Virtual Machine Manager Graphical Paid

(SCVMM)
& Operations Manager
Videos
Storage Spaces Direct Overview (5 minutes)
https://www.youtube-nocookie.com/embed/raeUiNtMk0E

Storage Spaces Direct at Microsoft Ignite 2018 (1 hour)

https://www.youtube-nocookie.com/embed/5kaUiW3qo30

Storage Spaces Direct at Microsoft Ignite 2017 (1 hour)

https://www.youtube-nocookie.com/embed/YDr2sqNB-3c

Storage Spaces Direct launch event at Microsoft Ignite 2016 (1 hour)

https://www.youtube-nocookie.com/embed/LK2ViRGbWs

Customer stories
There are over 10,000 clusters worldwide running Storage Spaces Direct.
Organizations of all sizes, from small businesses deploying just two nodes, to large
enterprises and governments deploying hundreds of nodes, depend on Storage Spaces
Direct for their critical applications and infrastructure.

Visit Microsoft.com/HCI to read their stories.

Additional references
Fault tolerance and storage efficiency
Storage Replica
Storage at Microsoft blog
Storage Spaces Direct throughput with iWARP (TechNet blog)
What's New in Failover Clustering in Windows Server
Storage Quality of Service
Windows IT Pro Support
Understanding the storage pool cache
Article • 04/18/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019

Storage Spaces Direct, the foundational storage virtualization technology behind Azure
Stack HCI and Windows Server, features a built-in server-side cache to maximize storage
performance while reducing costs. It's a large, persistent, real-time read and write cache
that is configured automatically upon deployment. In most cases, no manual
management whatsoever is required. How the cache works depends on the types of
drives present.

Drive types and deployment options

Storage Spaces Direct currently works with four types of drives:

Type Description
of
drive

PMem refers to persistent memory, a new type of low latency, high performance storage.

NVMe (Non-Volatile Memory Express) refers to solid-state drives that sit directly on the
PCIe bus. Common form factors are 2.5" U.2, PCIe Add-In-Card (AIC), and M.2. NVMe
offers higher IOPS and I/O throughput with lower latency than any other type of drive we
support today except PMem.

SSD refers to solid-state drives, which connect via conventional SATA or SAS.

HDD refers to rotational, magnetic hard disk drives, which offer vast storage capacity at a
low cost.

These can be combined in various ways, which we group into two categories: "all-flash"
and "hybrid". Deployments with all HDD are not supported.

７ Note

This article covers cache configurations with NVMe, SSD, and HDD. For information
on using persistent memory as a cache, see Understand and deploy persistent
 memory.

All-flash deployment possibilities

All-flash deployments aim to maximize storage performance and do not include HDD.

Hybrid deployment possibilities

Hybrid deployments aim to balance performance and capacity or to maximize capacity,
and do include HDD.

７ Note

Hybrid deployment is not supported in single server configuration. All flat single
storage type configurations (for example all-NVMe or all-SSD) is the only
supported storage type for single server.

Cache drives are selected automatically

In deployments with multiple types of drives, Storage Spaces Direct automatically uses
all drives of the fastest type for caching. The remaining drives are used for capacity.

Which type is "fastest" is determined according to the following hierarchy.

For example, if you have NVMe and SSDs, the NVMe will cache for the SSDs.

If you have SSDs and HDDs, the SSDs will cache for the HDDs.

７ Note

Cache drives do not contribute usable storage capacity to the cluster. All data
stored in the cache is also stored elsewhere, or will be once it de-stages. This
means the total raw storage capacity of your cluster is the sum of your capacity
drives only.

When all drives are of the same type, no cache is configured automatically. You have the
option to manually configure higher-endurance drives to cache for lower-endurance
drives of the same type – see the Manual configuration section to learn how.

 Tip

In some cases, using the storage pool cache does not make sense. For example, in
all-NVMe or all-SSD deployments, especially at very small scale, having no drives
"spent" on cache can improve storage efficiency and maximize performance.
Likewise, small remote or branch office deployments may have limited space for
cache drives.

Cache behavior is set automatically

The behavior of the cache is determined automatically based on the type(s) of drives
that are being cached for. When caching for flash drives (such as NVMe caching for
SSDs), only writes are cached. When caching for rotating disk drives (such as SSDs
caching for HDDs), both reads and writes are cached.
Write-only caching for all-flash deployments
Caching can be used in an all-flash scenario, for example using NVMe as cache to
accelerate the performance of SSDs. When caching for all-flash deployments, only writes
are cached. This reduces wear on the capacity drives because many writes and re-writes
can coalesce in the cache and then de-stage only as needed, reducing the cumulative
traffic to the capacity drives and extending their lifespan. For this reason, we
recommend selecting higher-endurance, write-optimized drives for the cache. The
capacity drives may reasonably have lower write endurance.

Because reads do not significantly affect the lifespan of flash, and because SSDs
universally offer low read latency, reads are not cached: they are served directly from the
capacity drives (except when the data was written so recently that it has not yet been
de-staged). This allows the cache to be dedicated entirely to writes, maximizing its
effectiveness.

This results in write characteristics, such as write latency, being dictated by the cache
drives, while read characteristics are dictated by the capacity drives. Both are consistent,
predictable, and uniform.

Read/write caching for hybrid deployments

When caching for HDD, both reads and writes are cached, to provide flash-like latency
(often ~10x better) for both. The read cache stores recently and frequently read data for
fast access and to minimize random traffic to the HDDs. (Because of seek and rotational
delays, the latency and lost time incurred by random access to an HDD is significant.)
Writes are cached to absorb bursts and, as before, to coalesce writes and re-writes and
minimize the cumulative traffic to the capacity drives.

Storage Spaces Direct implements an algorithm that de-randomizes writes before de-
staging them, to emulate an IO pattern to disk that seems sequential even when the
actual I/O coming from the workload (such as virtual machines) is random. This
maximizes the IOPS and throughput to the HDDs.

Caching in deployments with NVMe, SSD, and HDD

When drives of all three types are present, the NVMe drives provide caching for both
the SSDs and the HDDs. The behavior is as described above: only writes are cached for
the SSDs, and both reads and writes are cached for the HDDs. The burden of caching for
the HDDs is distributed evenly among the cache drives.

Summary
This table summarizes which drives are used for caching, which are used for capacity,
and what the caching behavior is for each deployment possibility.

Deployment Cache drives Capacity Cache behavior (default)

drives

All NVMe None (Optional: configure NVMe Write-only (if configured)

manually)

All SSD None (Optional: configure SSD Write-only (if configured)

manually)

NVMe + SSD NVMe SSD Write-only

NVMe + HDD NVMe HDD Read + Write

SSD + HDD SSD HDD Read + Write

NVMe + SSD + NVMe SSD + HDD Read + Write for HDD, Write-
HDD only for SSD

Server-side architecture
The cache is implemented at the drive level: individual cache drives within one server are
bound to one or many capacity drives within the same server.
Because the cache is below the rest of the Windows software-defined storage stack, it
does not have nor need any awareness of concepts such as Storage Spaces or fault
tolerance. You can think of it as creating "hybrid" (part flash, part disk) drives which are
then presented to the operating system. As with an actual hybrid drive, the real-time
movement of hot and cold data between the faster and slower portions of the physical
media is nearly invisible to the outside.

Given that resiliency in Storage Spaces Direct is at least server-level (meaning data
copies are always written to different servers; at most one copy per server), data in the
cache benefits from the same resiliency as data not in the cache.

For example, when using three-way mirroring, three copies of any data are written to
different servers, where they land in cache. Regardless of whether they are later de-
staged or not, three copies will always exist.

Drive bindings are dynamic

The binding between cache and capacity drives can have any ratio, from 1:1 up to 1:12
and beyond. It adjusts dynamically whenever drives are added or removed, such as
when scaling up or after failures. This means you can add cache drives or capacity drives
independently, whenever you want.
We recommend making the number of capacity drives a multiple of the number of
cache drives, for symmetry. For example, if you have 4 cache drives, you will experience
more even performance with 8 capacity drives (1:2 ratio) than with 7 or 9.

Handling cache drive failures

When a cache drive fails, any writes which have not yet been de-staged are lost to the
local server, meaning they exist only on the other copies (in other servers). Just like after
any other drive failure, Storage Spaces can and does automatically recover by consulting
the surviving copies.

For a brief period, the capacity drives which were bound to the lost cache drive will
appear unhealthy. Once the cache rebinding has occurred (automatic) and the data
repair has completed (automatic), they will resume showing as healthy.

This scenario is why at minimum two cache drives are required per server to preserve
performance.
You can then replace the cache drive just like any other drive replacement.

７ Note

You may need to power down to safely replace NVMe that is Add-In Card (AIC) or
M.2 form factor.

Relationship to other caches

There are several other unrelated caches in the Windows software-defined storage
stack. Examples include the Storage Spaces write-back cache and the Cluster Shared
Volume (CSV) in-memory read cache.

With Azure Stack HCI, the Storage Spaces write-back cache should not be modified from
its default behavior. For example, parameters such as -WriteCacheSize on the New-
Volume cmdlet should not be used.

You may choose to use the CSV cache, or not – it's up to you. It is on by default in Azure
Stack HCI, but it does not conflict with the cache described in this topic in any way. In
certain scenarios it can provide valuable performance gains. For more information, see
Use the CSV in-memory read cache with Azure Stack HCI.

Manual configuration
For most deployments, manual configuration is not required. In case you do need it, see
the following sections.

If you need to make changes to the cache device model after setup, edit the Health
Service's Support Components Document, as described in Health Service overview.

Specify cache drive model

In deployments where all drives are of the same type, such as all-NVMe or all-SSD
deployments, no cache is configured because Windows cannot distinguish
characteristics like write endurance automatically among drives of the same type.

To use higher-endurance drives to cache for lower-endurance drives of the same type,
you can specify which drive model to use with the -CacheDeviceModel parameter of the
Enable-ClusterS2D cmdlet. All drives of that model will be used for caching.
  Tip

Be sure to match the model string exactly as it appears in the output of Get-
PhysicalDisk.

Example
First, get a list of physical disks:

PowerShell

Get-PhysicalDisk | Group Model -NoElement

Here's some example output:

Count Name
----- ----
8 FABRIKAM NVME-1710
16 CONTOSO NVME-1520

Then enter the following command, specifying the cache device model:

PowerShell

Enable-ClusterS2D -CacheDeviceModel "FABRIKAM NVME-1710"

You can verify that the drives you intended are being used for caching by running Get-
PhysicalDisk in PowerShell and verifying that their Usage property says "Journal".

Manual deployment possibilities

Manual configuration enables the following deployment possibilities:
Set cache behavior
It is possible to override the default behavior of the cache. For example, you can set it to
cache reads even in an all-flash deployment. We discourage modifying the behavior
unless you are certain the default does not suit your workload.

To override the behavior, use Set-ClusterStorageSpacesDirect cmdlet and its -

CacheModeSSD and -CacheModeHDD parameters. The CacheModeSSD parameter sets
the cache behavior when caching for SSD. The CacheModeHDD parameter sets cache
behavior when caching for HDD.

You can use Get-ClusterStorageSpacesDirect to verify the behavior is set.

Example
First, get the Storage Spaces Direct settings:

PowerShell

Get-ClusterStorageSpacesDirect

Here's some example output:

CacheModeHDD : ReadWrite
CacheModeSSD : WriteOnly

Then, do the following:

PowerShell

Set-ClusterStorageSpacesDirect -CacheModeSSD ReadWrite

Get-ClusterS2D

Here's some example output:

CacheModeHDD : ReadWrite
CacheModeSSD : ReadWrite
Sizing the cache
The cache should be sized to accommodate the working set (the data being actively
read or written at any given time) of your applications and workloads.

This is especially important in hybrid deployments with hard disk drives. If the active
working set exceeds the size of the cache, or if the active working set drifts too quickly,
read cache misses will increase and writes will need to be de-staged more aggressively,
hurting overall performance.

You can use the built-in Performance Monitor (PerfMon.exe) utility in Windows to
inspect the rate of cache misses. Specifically, you can compare the Cache Miss
Reads/sec from the Cluster Storage Hybrid Disk counter set to the overall read IOPS of
your deployment. Each "Hybrid Disk" corresponds to one capacity drive.

For example, 2 cache drives bound to 4 capacity drives results in 4 "Hybrid Disk" object
instances per server.

There is no universal rule, but if too many reads are missing the cache, it may be
undersized and you should consider adding cache drives to expand your cache. You can
add cache drives or capacity drives independently whenever you want.

Next steps
For additional storage knowledge, see also:
Fault tolerance and storage efficiency
Cluster and pool quorum
Fault tolerance and storage efficiency
on Azure Stack HCI and Windows Server
clusters
Article • 04/18/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019

This article explains the resiliency options available and outlines the scale requirements,
storage efficiency, and general advantages and tradeoffs of each.

Overview
Storage Spaces Direct provides fault tolerance, often called "resiliency," for your data. Its
implementation is similar to RAID, except distributed across servers and implemented in
software.

As with RAID, there are a few different ways Storage Spaces can do this, which make
different tradeoffs between fault tolerance, storage efficiency, and compute complexity.
These broadly fall into two categories: "mirroring" and "parity," the latter sometimes
called "erasure coding."

Mirroring
Mirroring provides fault tolerance by keeping multiple copies of all data. This most
closely resembles RAID-1. How that data is striped and placed is non-trivial (see this
blog to learn more), but it is absolutely true to say that any data stored using
mirroring is written, in its entirety, multiple times. Each copy is written to different
physical hardware (different drives in different servers) that are assumed to fail
independently.

You can choose between two flavors of mirroring – "two-way" and "three-way."

Two-way mirror
Two-way mirroring writes two copies of everything. Its storage efficiency is 50 percent –
to write 1 TB of data, you need at least 2 TB of physical storage capacity. Likewise, you
need at least two hardware 'fault domains' – with Storage Spaces Direct, that means two
servers.

２ Warning

If you have more than two servers, we recommend using three-way mirroring
instead.

Three-way mirror
Three-way mirroring writes three copies of everything. Its storage efficiency is 33.3
percent – to write 1 TB of data, you need at least 3 TB of physical storage capacity.
Likewise, you need at least three hardware fault domains – with Storage Spaces Direct,
that means three servers.

Three-way mirroring can safely tolerate at least two hardware problems (drive or server)
at a time. For example, if you're rebooting one server when suddenly another drive or
server fails, all data remains safe and continuously accessible.

Parity
Parity encoding, often called "erasure coding," provides fault tolerance using bitwise
arithmetic, which can get remarkably complicated . The way this works is less obvious
than mirroring, and there are many great online resources (for example, this third-party
Dummies Guide to Erasure Coding ) that can help you get the idea. Sufficed to say it
provides better storage efficiency without compromising fault tolerance.

Storage Spaces offers two flavors of parity – "single" parity and "dual" parity, the latter
employing an advanced technique called "local reconstruction codes" at larger scales.

） Important

We recommend using mirroring for most performance-sensitive workloads. To

learn more about how to balance performance and capacity depending on your
workload, see Plan volumes.

Single parity
Single parity keeps only one bitwise parity symbol, which provides fault tolerance
against only one failure at a time. It most closely resembles RAID-5. To use single parity,
you need at least three hardware fault domains – with Storage Spaces Direct, that means
three servers. Because three-way mirroring provides more fault tolerance at the same
scale, we discourage using single parity. But, it's there if you insist on using it, and it is
fully supported.

２ Warning

We discourage using single parity because it can only safely tolerate one hardware
failure at a time: if you're rebooting one server when suddenly another drive or
server fails, you will experience downtime. If you only have three servers, we
recommend using three-way mirroring. If you have four or more, see the next
section.

Dual parity
Dual parity implements Reed-Solomon error-correcting codes to keep two bitwise parity
symbols, thereby providing the same fault tolerance as three-way mirroring (i.e. up to
two failures at once), but with better storage efficiency. It most closely resembles RAID-
6. To use dual parity, you need at least four hardware fault domains – with Storage
Spaces Direct, that means four servers. At that scale, the storage efficiency is 50% – to
store 2 TB of data, you need 4 TB of physical storage capacity.
The storage efficiency of dual parity increases the more hardware fault domains you
have, from 50 percent up to 80 percent. For example, at seven (with Storage Spaces
Direct, that means seven servers) the efficiency jumps to 66.7 percent – to store 4 TB of
data, you need just 6 TB of physical storage capacity.

See the Summary section for the efficiency of dual party and local reconstruction codes
at every scale.

Local reconstruction codes

Storage Spaces introduces an advanced technique developed by Microsoft Research
called "local reconstruction codes," or LRC. At large scale, dual parity uses LRC to split its
encoding/decoding into a few smaller groups, to reduce the overhead required to make
writes or recover from failures.

With hard disk drives (HDD) the group size is four symbols; with solid-state drives (SSD),
the group size is six symbols. For example, here's what the layout looks like with hard
disk drives and 12 hardware fault domains (meaning 12 servers) – there are two groups
of four data symbols. It achieves 72.7 percent storage efficiency.
We recommend this in-depth yet eminently readable walk-through of how local
reconstruction codes handle various failure scenarios, and why they're appealing , by
Claus Joergensen .

Mirror-accelerated parity
A Storage Spaces Direct volume can be part mirror and part parity. Writes land first in
the mirrored portion and are gradually moved into the parity portion later. Effectively,
this is using mirroring to accelerate erasure coding .

To mix three-way mirror and dual parity, you need at least four fault domains, meaning
four servers.

The storage efficiency of mirror-accelerated parity is in between what you'd get from
using all mirror or all parity, and depends on the proportions you choose. For example,
the demo at the 37-minute mark of this presentation shows various mixes achieving 46
percent, 54 percent, and 65 percent efficiency with 12 servers.

） Important

We recommend using mirroring for most performance-sensitive workloads. To

learn more about how to balance performance and capacity depending on your
workload, see Plan volumes.

Summary
This section summarizes the resiliency types available in Storage Spaces Direct, the
minimum scale requirements to use each type, how many failures each type can
tolerate, and the corresponding storage efficiency.

Resiliency types

Resiliency Failure tolerance Storage efficiency

Two-way mirror 1 50.0%

Three-way mirror 2 33.3%

Dual parity 2 50.0% - 80.0%

Mixed 2 33.3% - 80.0%

Minimum scale requirements

Resiliency Minimum required fault domains

Two-way mirror 2

Three-way mirror 3

Dual parity 4

Mixed 4

 Tip

Unless you are using chassis or rack fault tolerance, the number of fault domains
refers to the number of servers. The number of drives in each server does not affect
which resiliency types you can use, as long as you meet the minimum requirements
for Storage Spaces Direct.

Dual parity efficiency for hybrid deployments

This table shows the storage efficiency of dual parity and local reconstruction codes at
each scale for hybrid deployments which contain both hard disk drives (HDD) and solid-
state drives (SSD).

Fault domains Layout Efficiency

2 – –

3 – –

4 RS 2+2 50.0%

5 RS 2+2 50.0%

6 RS 2+2 50.0%

7 RS 4+2 66.7%

8 RS 4+2 66.7%

9 RS 4+2 66.7%

10 RS 4+2 66.7%

11 RS 4+2 66.7%
 Fault domains Layout Efficiency

12 LRC (8, 2, 1) 72.7%

13 LRC (8, 2, 1) 72.7%

14 LRC (8, 2, 1) 72.7%

15 LRC (8, 2, 1) 72.7%

16 LRC (8, 2, 1) 72.7%

Dual parity efficiency for all-flash deployments

This table shows the storage efficiency of dual parity and local reconstruction codes at
each scale for all-flash deployments which contain only solid-state drives (SSD). The
parity layout can use larger group sizes and achieve better storage efficiency in an all-
flash configuration.

Fault domains Layout Efficiency

2 – –

3 – –

4 RS 2+2 50.0%

5 RS 2+2 50.0%

6 RS 2+2 50.0%

7 RS 4+2 66.7%

8 RS 4+2 66.7%

9 RS 6+2 75.0%

10 RS 6+2 75.0%

11 RS 6+2 75.0%

12 RS 6+2 75.0%

13 RS 6+2 75.0%

14 RS 6+2 75.0%

15 RS 6+2 75.0%

16 LRC (12, 2, 1) 80.0%

Examples
Unless you have only two servers, we recommend using three-way mirroring and/or
dual parity, because they offer better fault tolerance. Specifically, they ensure that all
data remains safe and continuously accessible even when two fault domains – with
Storage Spaces Direct, that means two servers - are affected by simultaneous failures.

Examples where everything stays online

These six examples show what three-way mirroring and/or dual parity can tolerate.

1. One drive lost (includes cache drives)

2. One server lost

3. One server and one drive lost

4. Two drives lost in different servers

5. More than two drives lost, so long as at most two servers are affected
6. Two servers lost
...in every case, all volumes will stay online. (Make sure your cluster maintains quorum.)

Examples where everything goes offline

Over its lifetime, Storage Spaces can tolerate any number of failures, because it restores
to full resiliency after each one, given sufficient time. However, at most two fault
domains can safely be affected by failures at any given moment. The following are
therefore examples of what three-way mirroring and/or dual parity cannot tolerate.

7. Drives lost in three or more servers at once

8. Three or more servers lost at once

Usage
Check out Create volumes.

Next steps
For further reading on subjects mentioned in this article, see the following:

Erasure Coding in Azure by Microsoft Research

Local Reconstruction Codes and Accelerating Parity Volumes
Volumes in the Storage Management API
Capacity Calculator PREVIEW for Storage Spaces Direct
Drive symmetry considerations for
Azure Stack HCI and Windows Server
clusters
Article • 04/18/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019

Azure Stack HCI and Windows Server clusters work best when every server has exactly
the same drives.

In reality, we recognize this is not always practical. Today, you may buy spacious 3 TB
hard drives; next year, it may become impossible to find drives that small. Therefore,
some amount of mixing-and-matching is expected and supported. Keep in mind,
however, that more symmetry is always better.

This article explains the constraints and provides examples of supported and
unsupported configurations in Storage Spaces Direct, the underlying storage
virtualization technology behind Azure Stack HCI and Windows Server.

Constraints
This section explains constraints in terms of drive type, model, size, and number of
drives.

Type
All servers should have the same types of drives.

For example, if one server has NVMe, they should all have NVMe.

Number
All servers should have the same number of drives of each type.

For example, if one server has six SSD, they should all have six SSD.

７ Note
 It is okay for the number of drives to differ temporarily during failures or while
adding or removing drives.

Model
We recommend using drives of the same model and firmware version whenever
possible. If you can't, carefully select drives that are as similar as possible. We discourage
mixing-and-matching drives of the same type with sharply different performance or
endurance characteristics (unless one is cache and the other is capacity) because
Storage Spaces Direct distributes IO evenly and doesn't discriminate based on model.

７ Note

It is okay to mix-and-match similar SATA and SAS drives.

Size
We recommend using drives of the same sizes whenever possible. Using capacity drives
of different sizes may result in some unusable capacity, and using cache drives of
different sizes may not improve cache performance. See the next section for details.

２ Warning

Differing capacity drives sizes across servers may result in stranded capacity.

Understand: capacity imbalance

Storage Spaces Direct is robust enough to handle capacity imbalance across drives and
across servers. Even if the imbalance is severe, everything will continue to work.
However, depending on several factors, capacity that isn't available in every server may
not be usable.

To see why this happens, consider the simplified illustration below. Each colored box
represents one copy of mirrored data. For example, the boxes marked A, A', and A'' are
three copies of the same data. To honor server fault tolerance, these copies must be
stored in different servers.

Stranded capacity
As drawn, Server 1 (10 TB) and Server 2 (10 TB) are full. Server 3 has larger drives,
therefore its total capacity is larger (15 TB). However, to store more three-way mirror
data on Server 3 would require copies on Server 1 and Server 2 too, which are already
full. The remaining 5 TB capacity on Server 3 can't be used – it's "stranded" capacity.

Optimal placement
Conversely, with four servers of 10 TB, 10 TB, 10 TB, and 15 TB capacity and three-way
mirror resiliency, it is possible to validly place copies in a way that uses all available
capacity, as drawn. Whenever this is possible, the Storage Spaces Direct allocator will
find and use the optimal placement, leaving no stranded capacity.
The number of servers, the resiliency, the severity of the capacity imbalance, and other
factors affect whether there is stranded capacity. The most prudent general rule is to
assume that only capacity available in every server is guaranteed to be usable.

Understand: cache imbalance

Storage Spaces Direct can also withstand a cache imbalance across drives and across
servers. Even if the imbalance is severe, everything will continue to work. Moreover, it
always uses all available cache to the fullest.

Using cache drives of different sizes may not improve cache performance uniformly or
predictably: only IO to drive bindings with larger cache drives may see improved
performance. Storage Spaces Direct distributes IO evenly across bindings and doesn't
discriminate based on cache-to-capacity ratio.
  Tip

See Understanding the storage pool cache to learn more about cache bindings.

Example configurations
Here are some supported and unsupported configurations:

Supported: different models between servers

The first two servers use NVMe model "X" but the third server uses NVMe model "Z",
which is very similar.

Server 1 Server 2 Server 3

2 x NVMe Model X (cache) 2 x NVMe Model X (cache) 2 x NVMe Model Z (cache)

10 x SSD Model Y (capacity) 10 x SSD Model Y (capacity) 10 x SSD Model Y (capacity)

This is supported.

Supported: different models within server

Every server uses some different mix of HDD models "Y" and "Z", which are very similar.
Every server has 10 total HDD.

Server 1 Server 2 Server 3

2 x SSD Model X (cache) 2 x SSD Model X (cache) 2 x SSD Model X (cache)

7 x HDD Model Y (capacity) 5 x HDD Model Y (capacity) 1 x HDD Model Y (capacity)

3 x HDD Model Z (capacity) 5 x HDD Model Z (capacity) 9 x HDD Model Z (capacity)

This is supported.

Supported: different sizes across servers

The first two servers use 4 TB HDD but the third server uses very similar 6 TB HDD.

Server 1 Server 2 Server 3

2 x 800 GB NVMe (cache) 2 x 800 GB NVMe (cache) 2 x 800 GB NVMe (cache)

4 x 4 TB HDD (capacity) 4 x 4 TB HDD (capacity) 4 x 6 TB HDD (capacity)

This is supported, although it will result in stranded capacity.

Supported: different sizes within server

Every server uses some different mix of 1.2 TB and very similar 1.6 TB SSD. Every server
has 4 total SSD.

Server 1 Server 2 Server 3

3 x 1.2 TB SSD (cache) 2 x 1.2 TB SSD (cache) 4 x 1.2 TB SSD (cache)

1 x 1.6 TB SSD (cache) 2 x 1.6 TB SSD (cache) -

20 x 4 TB HDD (capacity) 20 x 4 TB HDD (capacity) 20 x 4 TB HDD (capacity)

This is supported.

Not supported: different types of drives across servers

Server 1 has NVMe but the others don't.

Server 1 Server 2 Server 3

 Server 1 Server 2 Server 3

6 x NVMe (cache) - -

- 6 x SSD (cache) 6 x SSD (cache)

18 x HDD (capacity) 18 x HDD (capacity) 18 x HDD (capacity)

This isn't supported. The types of drives should be the same in every server.

Not supported: different number of each type across

servers
Server 3 has more drives than the others.

Server 1 Server 2 Server 3

2 x NVMe (cache) 2 x NVMe (cache) 4 x NVMe (cache)

10 x HDD (capacity) 10 x HDD (capacity) 20 x HDD (capacity)

This isn't supported. The number of drives of each type should be the same in every
server.

Not supported: only HDD drives

All servers have only HDD drives connected.

Server 1 Server 2 Server 3

18 x HDD (capacity) 18 x HDD (capacity) 18 x HDD (capacity)

This isn't supported. You need to add a minimum of two cache drives (NvME or SSD)
attached to each of the servers.

Summary
To recap, every server in the cluster should have the same types of drives and the same
number of each type. It's supported to mix-and-match drive models and drive sizes as
needed, with the considerations above.

Constraint State

Same types of drives in every server Required

 Constraint State

Same number of each type in every server Required

Same drive models in every server Recommended

Same drive sizes in every server Recommended

Next steps
For related information, see also:

System requirements
Choose drives
Understand and monitor storage resync
Article • 04/18/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019

Storage resync alert is a capability of Storage Spaces Direct in Azure Stack HCI and
Windows Server. It allows the Health Service to throw a fault, notifying you about the
resync. This helps prevent you from accidentally taking down more servers, which could
affect multiple fault domains resulting in your cluster going down.

This article provides an overview of storage resync and how you can monitor it in a
failover cluster with Storage Spaces Direct.

About storage resync

Let's start with a simple example to understand how storage could get out of sync. Keep
in mind that any shared-nothing (local drives only) distributed storage solution exhibits
this behavior. The following section demonstrates how storage gets out of sync when
one server node goes down. Its drives don't get updated until it comes back online—
this behavior is applicable to any hyperconverged architecture.

Suppose you want to store the string "HELLO".

Assuming that you have three-way mirror resiliency, you have three copies of this string.
If you take down server #1 temporarily (for maintenance), then you can't access copy #1.
Suppose you update the string from "HELLO" to "HELP!" at this time.

After you update the string, copy #2 and #3 are updated successfully. However, copy #1
can't be accessed because server #1 is down temporarily (for maintenance).
You now have copy #1 with out-of-sync data. The operating system uses granular dirty
region tracking to keep track of the bits that are out of sync. This way when server #1
comes back online, you can sync the changes by reading the data from copy #2 or #3
and overwriting the data in copy #1. With this approach, you need to copy over only
that data which is stale, instead of resyncing all of the data from server #2 or server #3.

The preceding section described how data could get out of sync. But what does this
look like at a high level? Suppose you have a three-server, hyper-converged cluster.
When server #1 is in maintenance, you see it as being down. When you bring server #1
back up, it starts resyncing all of its storage using the granular dirty region tracking
(explained in the preceding section). Once the data is all back in sync, all servers are
shown as up.

The following GIF shows how storage resync works in a hyper-converged cluster:

How to monitor storage resync

Starting with Windows Server 2019, we added a new fault to the Health Service that
shows up when your storage is resyncing.

To view this fault in PowerShell, run the following cmdlet:

PowerShell

Get-HealthFault

This new fault appears in PowerShell, in the cluster validation report, and anywhere else
that builds on Health faults.

To get a deeper view, you can query the time series database in PowerShell, as follows:

PowerShell

Get-ClusterNode | Get-ClusterPerf -ClusterNodeSeriesName

ClusterNode.Storage.Degraded

Here's an example of the output:

PowerShell

Object Description: ClusterNode Server1

Series Time Value Unit

------ ---- ----- ----
ClusterNode.Storage.Degraded 01/11/2019 16:26:48 214 GB

Windows Admin Center uses Health faults to set the status and color of the cluster
nodes. On the HCI Dashboard, this new fault enables the cluster nodes to transition
from red (down) to yellow (resyncing) to green (up), instead of going straight from red
to green.

The following image compares how storage resync progresses in Windows Server 2016
vs. Windows Server 2019.
By showing the overall storage resync progress, you can accurately know how much
data is out of sync and whether your system is making forward progress. In Windows
Admin Center, go to the Dashboard to see the new alert, as shown in the following
screenshot:

The alert is useful in notifying you when resync is happening, so that you don't
accidentally take down more servers (which could cause multiple fault domains to be
affected, resulting in your cluster going down).

To get a detailed view of how storage resync appears on a per-server basis in Windows
Admin Center, navigate to the Servers page, click Inventory, and then choose a specific
server. Navigate to your server and look at the Storage chart to see the amount of data
that needs to be repaired in a purple line with an exact number right above it. This
amount increases when the server is down (more data needs to be resynced), and
decreases gradually when the server comes back online (data is being synced). When
the amount of data that needs to be repaired is 0, your storage is done resyncing—
you're now free to take down a server if you need to.

The following screenshot displays the server view in Windows Admin Center:

How to monitor storage resync in Windows

Server 2016
The alert available in Windows Server 2019 and later is helpful in getting a holistic view
of what is happening at the storage layer. It summarizes the information that you can
get from the Get-StorageJob cmdlet. This cmdlet returns information about long-
running storage module jobs, such as a repair operation on a storage space, as shown in
the following example output.

PowerShell

Get-StorageJob

Here's an example output:

PowerShell

Name ElapsedTime JobState

PercentComplete IsBackgroundTask
---- ----------- -------- ----------
----- ----------------
Regeneration 00:01:19 Running 50
True
This view is more granular because the storage jobs are listed per volume. You can see
the list of jobs that are running and you can track their individual progress. This cmdlet
works on both Windows Server 2016 and 2019.

Additional References
Taking a server offline for maintenance
Storage Spaces Direct overview
Understanding cluster and pool quorum
Article • 04/18/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server

Windows Server Failover Clustering provides high availability for workloads running on
Azure Stack HCI and Windows Server clusters. These resources are considered highly
available if the nodes that host resources are up; however, the cluster generally requires
more than half the nodes to be running, which is known as having quorum.

Quorum is designed to prevent split-brain scenarios which can happen when there is a
partition in the network and subsets of nodes cannot communicate with each other. This
can cause both subsets of nodes to try to own the workload and write to the same disk
which can lead to numerous problems. However, this is prevented with Failover
Clustering's concept of quorum which forces only one of these groups of nodes to
continue running, so only one of these groups will stay online.

Quorum determines the number of failures that the cluster can sustain while still
remaining online. Quorum is designed to handle the scenario when there is a problem
with communication between subsets of cluster nodes, so that multiple servers don't try
to simultaneously host a resource group and write to the same disk at the same time. By
having this concept of quorum, the cluster will force the cluster service to stop in one of
the subsets of nodes to ensure that there is only one true owner of a particular resource
group. Once nodes which have been stopped can once again communicate with the
main group of nodes, they will automatically rejoin the cluster and start their cluster
service.

In Azure Stack HCI and Windows Server 2019, there are two components of the system
that have their own quorum mechanisms:

Cluster Quorum: This operates at the cluster level (i.e. you can lose nodes and
have the cluster stay up)
Pool Quorum: This operates on the pool level (i.e. you can lose nodes and drives
and have the pool stay up). Storage pools were designed to be used in both
clustered and non-clustered scenarios, which is why they have a different quorum
mechanism.

Cluster quorum overview

The table below gives an overview of the cluster quorum outcomes per scenario:

Server Can survive one Can survive one server node Can survive two
nodes server node failure failure, then another simultaneous server node
failures

2 50/50 No No

2+ Yes No No
Witness

3 Yes 50/50 No

3+ Yes Yes No
Witness

4 Yes Yes 50/50

4+ Yes Yes Yes

Witness

5 and Yes Yes Yes

above

Cluster quorum recommendations

If you have two nodes, a witness is required.
If you have three or four nodes, witness is strongly recommended.
If you have five nodes or more, a witness isn't needed and doesn't provide
additional resiliency.
If you have internet access, use a cloud witness.
If you're in an IT environment with other machines and file shares, use a file share
witness.

How cluster quorum works

When nodes fail, or when some subset of nodes loses contact with another subset,
surviving nodes need to verify that they constitute the majority of the cluster to remain
online. If they can't verify that, they'll go offline.

But the concept of majority only works cleanly when the total number of nodes in the
cluster is odd (for example, three nodes in a five node cluster). So, what about clusters
with an even number of nodes (say, a four node cluster)?

There are two ways the cluster can make the total number of votes odd:
 1. First, it can go up one by adding a witness with an extra vote. This requires user
set-up.
2. Or, it can go down one by zeroing one unlucky node's vote (happens automatically
as needed).

Whenever surviving nodes successfully verify they are the majority, the definition of
majority is updated to be among just the survivors. This allows the cluster to lose one
node, then another, then another, and so forth. This concept of the total number of votes
adapting after successive failures is known as Dynamic quorum.

Dynamic witness
Dynamic witness toggles the vote of the witness to make sure that the total number of
votes is odd. If there are an odd number of votes, the witness doesn't have a vote. If
there is an even number of votes, the witness has a vote. Dynamic witness significantly
reduces the risk that the cluster will go down because of witness failure. The cluster
decides whether to use the witness vote based on the number of voting nodes that are
available in the cluster.

Dynamic quorum works with dynamic witness in the way described below.

Dynamic quorum behavior

If you have an even number of nodes and no witness, one node gets its vote zeroed.
For example, only three of the four nodes get votes, so the total number of votes is
three, and two survivors with votes are considered a majority.
If you have an odd number of nodes and no witness, they all get votes.
If you have an even number of nodes plus witness, the witness votes, so the total is
odd.
If you have an odd number of nodes plus witness, the witness doesn't vote.

Dynamic quorum enables the ability to assign a vote to a node dynamically to avoid
losing the majority of votes and to allow the cluster to run with one node (known as
last-man standing). Let's take a four-node cluster as an example. Assume that quorum
requires 3 votes.

In this case, the cluster would have gone down if you lost two nodes.
However, dynamic quorum prevents this from happening. The total number of votes
required for quorum is now determined based on the number of nodes available. So,
with dynamic quorum, the cluster will stay up even if you lose three nodes.

The above scenario applies to a general cluster that doesn't have Storage Spaces Direct
enabled. However, when Storage Spaces Direct is enabled, the cluster can only support
two node failures. This is explained more in the pool quorum section.

Examples

Two nodes without a witness

One node's vote is zeroed, so the majority vote is determined out of a total of 1 vote. If
the non-voting node goes down unexpectedly, the survivor has 1/1 and the cluster
survives. If the voting node goes down unexpectedly, the survivor has 0/1 and the
cluster goes down. If the voting node is gracefully powered down, the vote is transferred
to the other node, and the cluster survives. This is why it's critical to configure a
witness.

Can survive one server failure: Fifty percent chance.

Can survive one server failure, then another: No.
Can survive two server failures at once: No.

Two nodes with a witness

Both nodes vote, plus the witness votes, so the majority is determined out of a total of 3
votes. If either node goes down, the survivor has 2/3 and the cluster survives.

Can survive one server failure: Yes.

Can survive one server failure, then another: No.
Can survive two server failures at once: No.

Three nodes without a witness

All nodes vote, so the majority is determined out of a total of 3 votes. If any node goes
down, the survivors are 2/3 and the cluster survives. The cluster becomes two nodes
without a witness – at that point, you're in Scenario 1.
 Can survive one server failure: Yes.
Can survive one server failure, then another: Fifty percent chance.
Can survive two server failures at once: No.

Three nodes with a witness

All nodes vote, so the witness doesn't initially vote. The majority is determined out of a
total of 3 votes. After one failure, the cluster has two nodes with a witness – which is
back to Scenario 2. So, now the two nodes and the witness vote.

Can survive one server failure: Yes.

Can survive one server failure, then another: Yes.
Can survive two server failures at once: No.

Four nodes without a witness

One node's vote is zeroed, so the majority is determined out of a total of 3 votes. After
one failure, the cluster becomes three nodes, and you're in Scenario 3.

Can survive one server failure: Yes.

 Can survive one server failure, then another: Yes.
Can survive two server failures at once: Fifty percent chance.

Four nodes with a witness

All nodes votes and the witness votes, so the majority is determined out of a total of 5
votes. After one failure, you're in Scenario 4. After two simultaneous failures, you skip
down to Scenario 2.

Can survive one server failure: Yes.

Can survive one server failure, then another: Yes.
Can survive two server failures at once: Yes.

Five nodes and beyond

All nodes vote, or all but one vote, whatever makes the total odd. Storage Spaces Direct
cannot handle more than two nodes down anyway, so at this point, no witness is
needed or useful.

Can survive one server failure: Yes.

Can survive one server failure, then another: Yes.
Can survive two server failures at once: Yes.

Now that we understand how quorum works, let's look at the types of quorum
witnesses.

Quorum witness types

Failover Clustering supports three types of Quorum Witnesses:
 Cloud Witness - Blob storage in Azure accessible by all nodes of the cluster. It
maintains clustering information in a witness.log file, but doesn't store a copy of
the cluster database.
File Share Witness – A SMB file share that is configured on a file server running
Windows Server. It maintains clustering information in a witness.log file, but
doesn't store a copy of the cluster database.
Disk Witness - A small clustered disk which is in the Cluster Available Storage
group. This disk is highly-available and can failover between nodes. It contains a
copy of the cluster database. A Disk Witness isn't supported with Storage Spaces
Direct.

Pool quorum overview

We just talked about cluster quorum, which operates at the cluster level. Now, let's dive
into pool quorum, which operates on the pool level (i.e. you can lose nodes and drives
and have the pool stay up). Storage pools were designed to be used in both clustered
and non-clustered scenarios, which is why they have a different quorum mechanism.

The table below gives an overview of the pool quorum outcomes per scenario:

Server Can survive one Can survive one server node Can survive two
nodes server node failure failure, then another simultaneous server node
failures

2 No No No

2+ Yes No No
Witness

3 Yes No No

3+ Yes No No
Witness

4 Yes No No

4+ Yes Yes Yes

Witness

5 and Yes Yes Yes

above

How pool quorum works

When drives fail, or when some subset of drives loses contact with another subset,
surviving drives need to verify that they constitute the majority of the pool to remain
online. If they can't verify that, they'll go offline. The pool is the entity that goes offline
or stays online based on whether it has enough disks for quorum (50% + 1). The pool
resource owner (active cluster node) can be the +1.

But pool quorum works differently from cluster quorum in the following ways:

the pool uses one node in the cluster as a witness as a tie-breaker to survive half of
drives gone (this node that is the pool resource owner)
the pool does NOT have dynamic quorum
the pool does NOT implement its own version of removing a vote

Examples

Four nodes with a symmetrical layout

Each of the 16 drives has one vote and node two also has one vote (since it's the pool
resource owner). The majority is determined out of a total of 16 votes. If nodes three
and four go down, the surviving subset has 8 drives and the pool resource owner, which
is 9/16 votes. So, the pool survives.

Can survive one server failure: Yes.

Can survive one server failure, then another: Yes.
Can survive two server failures at once: Yes.

Four nodes with a symmetrical layout and drive failure

Each of the 16 drives has one vote and node 2 also has one vote (since it's the pool
resource owner). The majority is determined out of a total of 16 votes. First, drive 7 goes
down. If nodes three and four go down, the surviving subset has 7 drives and the pool
resource owner, which is 8/16 votes. So, the pool doesn't have majority and goes down.

Can survive one server failure: Yes.

Can survive one server failure, then another: No.
Can survive two server failures at once: No.

Four nodes with a non-symmetrical layout

Each of the 24 drives has one vote and node two also has one vote (since it's the pool
resource owner). The majority is determined out of a total of 24 votes. If nodes three
and four go down, the surviving subset has 8 drives and the pool resource owner, which
is 9/24 votes. So, the pool doesn't have majority and goes down.

Can survive one server failure: Yes.

Can survive one server failure, then another: Depends (cannot survive if both
nodes three and four go down, but can survive all other scenarios.
 Can survive two server failures at once: Depends (cannot survive if both nodes
three and four go down, but can survive all other scenarios.

Pool quorum recommendations

Ensure that each node in your cluster is symmetrical (each node has the same
number of drives)
Enable three-way mirror or dual parity so that you can tolerate a node failures and
keep the virtual disks online.
If more than two nodes are down, or two nodes and a disk on another node are
down, volumes may not have access to all three copies of their data, and therefore
be taken offline and be unavailable. It's recommended to bring the servers back or
replace the disks quickly to ensure the most resiliency for all the data in the
volume.

Next steps
For more information, see the following:

Configure and manage quorum

Set up a cluster witness
Deploy a cluster set
Article • 09/08/2022

Applies to: Windows Server 2019

This article provides information on how to deploy a cluster set for Windows Server
Failover Clusters using PowerShell. A cluster set is a group of multiple failover clusters
that are clustered together. By using a cluster set, you can increase the number of server
nodes in a single Software Defined Data Center (SDDC) cloud by orders of magnitude.

Cluster sets have been tested and supported up to 64 total cluster nodes. However,
cluster sets can scale to much larger limits and aren't hardcoded for a limit.

Benefits
Cluster sets offer the following benefits:

Significantly increases the supported SDDC cloud scale for running highly available
virtual machines (VMs) by combining multiple smaller clusters into a single large
fabric, while keeping the software fault boundary to a single cluster. You can easily
migrate VMs across the cluster set.

Increased resiliency. Having four 4-node clusters in a cluster set gives you better
resiliency than a single 16-node cluster in that multiple compute nodes can go
down and production remains intact.

Management of failover cluster lifecycle, including onboarding and retiring

clusters, without impacting tenant VM availability.

VM flexibility across individual clusters and a present a unified storage namespace.

Easily change the compute-to-storage workload ratio in your hyper-converged

environment.

Benefit from Azure-like Fault Domains and Availability sets across individual
clusters in initial VM placement and subsequent migration.

Can use even if compute and storage hardware between cluster nodes isn't
identical.

Live migration of VMs between clusters.

Azure-like availability sets and fault domains across multiple clusters.

 Moving of SQL Server VMs between clusters.

Requirements and limitations

There are a few requirements and limitations for using cluster sets:

All member clusters in a cluster set must be in the same Active Directory (AD)
forest.

Member servers in the set must run the same operating system version. Virtual
machines can't be live migrated between different operating systems. You can have
a cluster set that consists of any one, but not multiples, of the following options:
Windows Server 2019 Failover Cluster and Windows Server 2019 Failover Cluster
Windows Server 2019 Failover Cluster and Windows Server 2019 Storage Spaces
Direct
Windows Server 2019 Storage Spaces Direct and Windows Server 2019 Storage
Spaces Direct

Identical processor hardware is needed for all member servers for live migration
between member clusters to occur; otherwise, you must select CPU Processor
Compatibility in virtual machines settings.

Cluster set VMs must be manually live-migrated across clusters - they can't
automatically fail over.

Storage Replica must be used between member clusters to realize storage

resiliency to cluster failures. When using Storage Replica, bear in mind that
namespace storage UNC paths will not change automatically on Storage Replica
failover to the replica target cluster.

Storage Spaces Direct doesn't function across member clusters in a cluster set.
Rather, Storage Spaces Direct applies to a single cluster, with each cluster having
its own storage pool.

Architecture
The following diagram illustrates a cluster set at a high level:
 

The following provides a summary of each of the elements shown:

Management cluster
The management cluster hosts the highly-available management plane and the
namespace referral scale-out file server (SOFS) for the cluster set. A management cluster
is logically decoupled from individual member clusters that run VM workloads. This
makes the cluster set management plane resilient to any localized cluster-wide failures,
such as loss of power of a member cluster.

Cluster set namespace referral SOFS

A namespace for the cluster set is provided with an SOFS server role running on the
management cluster. This is similar to a Distributed File System Namespace (DFSN).
Unlike DFSN however, cluster set namespace referral metadata is auto-populated on all
cluster nodes without any intervention, so there's almost no performance overhead in
the storage access path. This lightweight referral mechanism doesn't participate in the
I/O path.

Each Server Message Block (SMB) referral share on the cluster set namespace referral
SOFS is of type SimpleReferral . This referral allows SMB clients access to the target SMB
share hosted on the member cluster SOFS. Referrals are cached perpetually on each of
the client nodes and the cluster set namespace dynamically updates the referrals as
needed automatically. Referral information is persistently cached in each cluster set
node, even during reboots.

Cluster set master

Communication between member clusters is loosely coupled and coordinated by the
cluster set master (CS-Master) resource. Like other cluster set resources, CS-Master is
highly available and resilient to individual member cluster failures or management
cluster node failures. Through a cluster set WMI provider, CS-Master provides the
management endpoint for all cluster set management actions.

Member cluster
A member cluster runs VM and Storage Spaces Direct workloads. Multiple member
clusters participate in a cluster set deployment, forming the larger SDDC cloud fabric.
Member clusters differ from the management cluster in two key aspects: member
clusters participate in fault domain and availability set constructs, and member clusters
are sized to host VM and Storage Spaces Direct workloads. VMs that move across
member clusters aren't hosted on the management cluster for this reason.

Cluster set worker

The CS-Master interacts with a cluster resource on member clusters called the cluster set
worker (CS-Worker). CS-Worker responds to requests by the CS-Master, including VM
placement and resource inventorying. There's one CS-Worker instance per member
cluster.

Fault domain
A fault domain is a group of hardware and software that could fail together. While you
could designate one or more clusters together as a fault domain, each node could
participate in a fault domain in an availability set. Fault domain boundaries are based on
data center topology, networking architecture, and other considerations.

Availability set
An availability set is used to configure the desired redundancy of clustered workloads
across fault domains by grouping and deploying workloads. For a two-tier application,
you should configure at least two VMs in an availability set for each tier, which ensures
that when a fault domain in an availability set goes down, your application will have at
least one VM in each tier hosted on a different fault domain.

Create a cluster set

Use PowerShell within the following example workflow to create a cluster set using two
clusters. The name of the cluster set here is CSMASTER.

Cluster name Infrastructure SOFS name

SET-CLUSTER SOFS-CLUSTERSET

CLUSTER1 SOFS-CLUSTER1

CLUSTER2 SOFS-CLUSTER2

1. Use a management client computer running Windows Server 2022 or Windows

Server 2019.

2. Install Failover Cluster tools on the management cluster server.

3. Create two cluster members and with at least two Cluster Shared Volumes (CSVs)
in each cluster.

4. Create a management cluster (physical or guest) that straddles the member

clusters. This ensures that the cluster set management plane continues to be
available despite possible member cluster failures.

5. To create the cluster set:

PowerShell

New-ClusterSet -Name CSMASTER -NamespaceRoot SOFS-CLUSTERSET -

CimSession SET-CLUSTER

７ Note

If you are using a static IP address, you must include -StaticAddress x.x.x.x on
the New-ClusterSet command.

6. To add cluster members to the cluster set:

PowerShell

Add-ClusterSetMember -ClusterName CLUSTER1 -CimSession CSMASTER -

InfraSOFSName SOFS-CLUSTER1
Add-ClusterSetMember -ClusterName CLUSTER2 -CimSession CSMASTER -
InfraSOFSName SOFS-CLUSTER2

7. To enumerate all member clusters in the cluster set:

 PowerShell

Get-ClusterSetMember -CimSession CSMASTER

8. To enumerate all the member clusters in the cluster set including the management
cluster nodes:

PowerShell

Get-ClusterSet -CimSession CSMASTER | Get-Cluster | Get-ClusterNode

9. To list all server nodes from all member clusters:

PowerShell

Get-ClusterSetNode -CimSession CSMASTER

10. To list all resource groups across the cluster set:

PowerShell

Get-ClusterSet -CimSession CSMASTER | Get-Cluster | Get-ClusterGroup

11. To verify the cluster set contains one SMB share ( ScopeName being the
Infrastructure File Server name) on the infrastructure SOFS for each cluster member
CSV volume:

PowerShell

Get-SmbShare -CimSession CSMASTER

12. Review the cluster set debug log files for the cluster set, the management cluster,
and each cluster member:

PowerShell

Get-ClusterSetLog -ClusterSetCimSession CSMASTER -IncludeClusterLog -

IncludeManagementClusterLog -DestinationFolderPath <path>

13. Configure Kerberos with constrained delegation between all cluster set
members.
 14. Configure the cross-cluster VM live migration authentication type to Kerberos on
each node in the cluster set:

PowerShell

foreach($h in $hosts){ Set-VMHost -

VirtualMachineMigrationAuthenticationType Kerberos -ComputerName $h }

15. Add the management cluster to the local administrators group on each cluster
member server node in the cluster set:

PowerShell

foreach($h in $hosts){ Invoke-Command -ComputerName $h -ScriptBlock

{Net localgroup administrators /add <management_cluster_name>$} }

Create cluster set VMs

After creating the cluster set, the next step is to create VMs. You should perform the
following checks beforehand:

Check available memory on each cluster server node

Check available disk space on each cluster server node
Check any specific VM storage requirements in terms of speed and performance

The Get-ClusterSetOptimalNodeForVM command identifies the optimal cluster and node

in the cluster set and then deploys the VM on it. In the following example, a new VM is
created with:

4 GB available
One virtual processor
10% minimum CPU available

PowerShell

# Identify the optimal node to create a new virtual machine

$memoryinMB=4096
$vpcount = 1
$targetnode = Get-ClusterSetOptimalNodeForVM -CimSession CSMASTER -VMMemory
$memoryinMB -VMVirtualCoreCount $vpcount -VMCpuReservation 10
$secure_string_pwd = convertto-securestring "<password>" -asplaintext -force
$cred = new-object -typename System.Management.Automation.PSCredential ("
<domain\account>",$secure_string_pwd)

# Deploy the virtual machine on the optimal node

 Invoke-Command -ComputerName $targetnode.name -scriptblock {
param([String]$storagepath); New-VM CSVM1 -MemoryStartupBytes 3072MB -path
$storagepath -NewVHDPath CSVM.vhdx -NewVHDSizeBytes 4194304 } -ArgumentList
@("\\SOFS-CLUSTER1\VOLUME1") -Credential $cred | Out-Null

Start-VM CSVM1 -ComputerName $targetnode.name | Out-Null

Get-VM CSVM1 -ComputerName $targetnode.name | fl State, ComputerName

When complete, you are shown which cluster node the VM was deployed on. For the
above example, it would show as:

State : Running
ComputerName : 1-S2D2

If there's not enough memory, CPU capacity, or disk space available to add the VM,
you'll receive the following error:

Get-ClusterSetOptimalNodeForVM : A cluster node isn't available for this

operation.

Once the VM is created, it is displayed in Hyper-V manager on the specific node

specified. To add it as a cluster set VM and add it to the cluster, use this command:

PowerShell

Register-ClusterSetVM -CimSession CSMASTER -MemberName $targetnode.Member -

VMName CSVM1

When complete, the output is:

Id VMName State MemberName PSComputerName

-- ------ ----- ---------- --------------
1 CSVM1 On CLUSTER1 CSMASTER

If you've created a cluster using existing VMs, the VMs need to be registered with the
cluster set. To register all VMs at once, use:

PowerShell
 Get-ClusterSetMember -Name CLUSTER3 -CimSession CSMASTER | Register-
ClusterSetVM -RegisterAll -CimSession CSMASTER

Next, add the VM path to the cluster set namespace.

As an example, suppose an existing cluster is added to the cluster set with pre-
configured VMs that reside on the local Cluster Shared Volume (CSV). The path for the
VHDX would be something similar to C:\ClusterStorage\Volume1\MYVM\Virtual Hard
Disks\MYVM.vhdx1 .

A storage migration is needed, as CSV paths are by design local to a single member
cluster and are therefore not accessible to the VM once they are live migrated across
member clusters.

In this example, CLUSTER3 is added to the cluster set using Add-ClusterSetMember with
the scale-out file server SOFS-CLUSTER3. To move the VM configuration and storage,
the command is:

PowerShell

Move-VMStorage -DestinationStoragePath \\SOFS-CLUSTER3\Volume1 -Name MyVM

Once complete, you may receive a warning:

WARNING: There were issues updating the virtual machine configuration that
may prevent the virtual machine from running. For more information view the
report file below.
WARNING: Report file location: C:\Windows\Cluster\Reports\Update-
ClusterVirtualMachineConfiguration '' on date at time.htm.

This warning may be ignored as there were no physical changes in the virtual machine
role storage configuration. The actual physical location doesn't change; only the
configuration paths do.

For more information on Move-VMStorage , see Move-VMStorage.

Live migrating a VM within a cluster set involves the following:

PowerShell

Set-VMHost -UseAnyNetworkForMigration $true

Then, to move a cluster set VM from CLUSTER1 to NODE2-CL3 on CLUSTER3 for
example, the command would be:

PowerShell

Move-ClusterSetVM -CimSession CSMASTER -VMName CSVM1 -Node NODE2-CL3

This command doesn't move the VM storage or configuration files and isn't necessary as
the path to the VM remains as \\SOFS-CLUSTER1\VOLUME1. Once a VM has been
registered with the infrastructure file server share path, the drives and VM don't require
being on the same node as the VM.

Create the infrastructure scale-out file server

There's one Infrastructure SOFS cluster role on a cluster. The Infrastructure SOFS role is
created by specifying the -Infrastructure switch parameter to the Add-
ClusterScaleOutFileServerRole cmdlet. For example:

PowerShell

Add-ClusterScaleoutFileServerRole -Name "my_infra_sofs_name" -Infrastructure

Each CSV volume created automatically triggers the creation of an SMB share with an
auto-generated name based on the CSV volume name. You can't directly create or
modify SMB shares under an SOFS role, other than by using CSV volume create and
modify operations.

In hyperconverged configurations, an Infrastructure SOFS allows an SMB client (Hyper-V

host) to communicate with continuous availability (CA) to the Infrastructure SOFS SMB
server. This hyper-converged SMB loopback CA is achieved by VMs accessing their
virtual disk (VHDX) files where the owning VM identity is forwarded between the client
and server. This identity forwarding allows the use of ACLs for VHDx files just as in
standard hyperconverged cluster configurations as before.

Once a cluster set is created, the cluster set namespace relies on an Infrastructure SOFS
on each of the member clusters, and additionally an Infrastructure SOFS in the
management cluster.

At the time a member cluster is added to a cluster set, you can specify the name of an
Infrastructure SOFS on that cluster if one already exists. If the Infrastructure SOFS
doesn't exist, a new Infrastructure SOFS role on the new member cluster is created. If an
Infrastructure SOFS role already exists on the member cluster, the Add operation
implicitly renames it to the specified name as needed. Any existing SMB servers, or non-
infrastructure SOFS roles on the member clusters, aren't used by the cluster set.

When the cluster set is created, you have the option to use an existing AD computer
object as the namespace root on the management cluster. Cluster set creation creates
the Infrastructure SOFS cluster role on the management cluster or renames the existing
Infrastructure SOFS role. The Infrastructure SOFS on the management cluster is used as
the cluster set namespace referral SOFS.

Create fault domains and availability sets

Azure-like fault domains and availability sets can be configured in a cluster set. This is
beneficial for initial VM placements and migrations between clusters.

The example below has four clusters in a cluster set. Within the set, one fault domain is
created with two of the clusters and a second fault domain is created with the other two
clusters. These two fault domains comprise the availability set.

In the example below, CLUSTER1 and CLUSTER2 are in the fault domain FD1 and
CLUSTER3 and CLUSTER4 are in the fault domain FD2. The availability set is CSMASTER-
AS.

To create the fault domains, the commands are:

PowerShell

New-ClusterSetFaultDomain -Name FD1 -FdType Logical -CimSession CSMASTER -

MemberCluster CLUSTER1,CLUSTER2 -Description "First fault domain"

New-ClusterSetFaultDomain -Name FD2 -FdType Logical -CimSession CSMASTER -

MemberCluster CLUSTER3,CLUSTER4 -Description "Second fault domain"

To ensure they've been created successfully, Get-ClusterSetFaultDomain can be run with

its output shown for FD1:

PowerShell

PS C:\> Get-ClusterSetFaultDomain -CimSession CSMASTER -FdName FD1 | fl *

PSShowComputerName : True
FaultDomainType : Logical
ClusterName : {CLUSTER1, CLUSTER2}
Description : First fault domain
FDName : FD1
Id : 1
PSComputerName : CSMASTER
Now that the fault domains have been created, the availability set is created:

PowerShell

New-ClusterSetAvailabilitySet -Name CSMASTER-AS -FdType Logical -CimSession

CSMASTER -ParticipantName FD1,FD2

To validate it has been created, use:

PowerShell

Get-ClusterSetAvailabilitySet -AvailabilitySetName CSMASTER-AS -CimSession

CSMASTER

When creating new VMs, use the -AvailabilitySet parameter to determine the optimal
node for placement. Here's an example:

PowerShell

# Identify the optimal node to create a new VM

$memoryinMB=4096
$vpcount = 1
$av = Get-ClusterSetAvailabilitySet -Name CSMASTER-AS -CimSession CSMASTER
$targetnode = Get-ClusterSetOptimalNodeForVM -CimSession CSMASTER -VMMemory
$memoryinMB -VMVirtualCoreCount $vpcount -VMCpuReservation 10 -
AvailabilitySet $av
$secure_string_pwd = convertto-securestring "<password>" -asplaintext -force
$cred = new-object -typename System.Management.Automation.PSCredential ("
<domain\account>",$secure_string_pwd)

Remove a cluster from a set

There are times when a cluster needs to be removed from a cluster set. As a best
practice, all cluster set VMs should be moved out of the cluster beforehand. This can be
done using the Move-ClusterSetVM and Move-VMStorage commands.

If the VMs aren't moved out of the cluster first, all remaining cluster set VMs hosted on
the cluster being removed will become highly available VMs bound to that cluster,
assuming they have access to their storage. Cluster sets also automatically update their
inventory by no longer tracking the health of a removed cluster and the VMs running on
it, and by removing the namespace and all references to shares hosted on the removed
cluster.
For example, the command to remove the CLUSTER1 cluster from a cluster set is:

PowerShell

Remove-ClusterSetMember -ClusterName CLUSTER1 -CimSession CSMASTER

System state backup

System state backup will back up the cluster state and metadata. Using Windows Server
Backup, you can restore just a node's cluster database if needed or do an authoritative
restore to roll back the entire cluster database across all nodes. For cluster sets, we
recommend doing an authoritative restore first for the member clusters and then for the
management cluster. For more information on system state backup, see Back up system
state and bare metal.

Next steps
Learn more about Storage Replica.
Storage Spaces Direct hardware
requirements
Article • 06/24/2022

Applies to: Azure Stack Hub, Azure Stack HCI version 21H2, Azure Stack HCI version
20H2, Windows Server 2022, Windows Server 2019, Windows Server 2016

This article describes minimum hardware requirements for Storage Spaces Direct. For
hardware requirements on Azure Stack HCI, our operating system designed for
hyperconverged deployments with a connection to the cloud, see Before you deploy
Azure Stack HCI: Determine hardware requirements.

For production, Microsoft recommends purchasing a validated hardware/software

solution from our partners, which include deployment tools and procedures. These
solutions are designed, assembled, and validated against our reference architecture to
ensure compatibility and reliability, so you get up and running quickly. For hardware
solutions, visit the Azure Stack HCI solutions website .

 Tip

Want to evaluate Storage Spaces Direct but don't have hardware? Use Hyper-V or
Azure virtual machines as described in Using Storage Spaces Direct in guest virtual
machine clusters.

Base requirements
Systems, components, devices, and drivers must be certified for the operating system
you’re using in the Windows Server Catalog . In addition, we recommend that servers
and network adapters have the Software-Defined Data Center (SDDC) Standard and/or
Software-Defined Data Center (SDDC) Premium additional qualifications (AQs), as
pictured below. There are over 1,000 components with the SDDC AQs.
The fully configured cluster (servers, networking, and storage) must pass all cluster
validation tests per the wizard in Failover Cluster Manager or with the Test-Cluster
cmdlet in PowerShell.

In addition, the following requirements apply:

Servers
Minimum of 2 servers, maximum of 16 servers
Recommended that all servers be the same manufacturer and model

CPU
Intel Nehalem or later compatible processor; or
AMD EPYC or later compatible processor

Memory
Memory for Windows Server, VMs, and other apps or workloads; plus
4 GB of RAM per terabyte (TB) of cache drive capacity on each server, for Storage
Spaces Direct metadata

Boot
Any boot device supported by Windows Server, which now includes SATADOM
RAID 1 mirror is not required, but is supported for boot
Recommended: 200 GB minimum size
Networking
Storage Spaces Direct requires a reliable high bandwidth, low latency network
connection between each node.

Minimum interconnect for small scale 2-3 node

10 Gbps network interface card (NIC), or faster

Two or more network connections from each node recommended for redundancy
and performance

Recommended interconnect for high performance, at scale, or deployments of 4+

NICs that are remote-direct memory access (RDMA) capable, iWARP

(recommended) or RoCE
Two or more network connections from each node recommended for redundancy
and performance
25 Gbps NIC or faster

Switched or switchless node interconnects

Switched: Network switches must be properly configured to handle the bandwidth

and networking type. If using RDMA that implements the RoCE protocol, network
device and switch configuration is even more important.
Switchless: Nodes can be interconnected using direct connections, avoiding using
a switch. It's required that every node has a direct connection with every other
node of the cluster.

Drives
Storage Spaces Direct works with direct-attached SATA, SAS, NVMe, or persistent
memory (PMem) drives that are physically attached to just one server each. For more
help choosing drives, see the Choosing drives and Understand and deploy persistent
memory articles.

SATA, SAS, persistent memory, and NVMe (M.2, U.2, and Add-In-Card) drives are all
supported
512n, 512e, and 4K native drives are all supported
Solid-state drives must provide power-loss protection
Same number and types of drives in every server – see Drive symmetry
considerations
Cache devices must be 32 GB or larger
Persistent memory devices are used in block storage mode
 When using persistent memory devices as cache devices, you must use NVMe or
SSD capacity devices (you can't use HDDs)
If you're using HDDs to provide storage capacity, you must use storage bus
caching. Storage bus caching isn't required when using all-flash deployments
NVMe driver is the Microsoft-provided one included in Windows (stornvme.sys)
Recommended: Number of capacity drives is a whole multiple of the number of
cache drives
Recommended: Cache drives should have high write endurance: at least 3 drive-
writes-per-day (DWPD) or at least 4 terabytes written (TBW) per day – see
Understanding drive writes per day (DWPD), terabytes written (TBW), and the
minimum recommended for Storage Spaces Direct

７ Note

When using all flash drives for storage capacity, the benefits of storage pool
caching will be limited. Learn more about the storage pool cache.

Here's how drives can be connected for Storage Spaces Direct:

Direct-attached SATA drives

Direct-attached NVMe drives
SAS host-bus adapter (HBA) with SAS drives
SAS host-bus adapter (HBA) with SATA drives
NOT SUPPORTED: RAID controller cards or SAN (Fibre Channel, iSCSI, FCoE)
storage. Host-bus adapter (HBA) cards must implement simple pass-through mode
for any storage devices used for Storage Spaces Direct.
Drives can be internal to the server, or in an external enclosure that is connected to just
one server. SCSI Enclosure Services (SES) is required for slot mapping and identification.
Each external enclosure must present a unique identifier (Unique ID).

Drives internal to the server

Drives in an external enclosure ("JBOD") connected to one server
NOT SUPPORTED: Shared SAS enclosures connected to multiple servers or any
form of multi-path IO (MPIO) where drives are accessible by multiple paths.
Minimum number of drives (excludes boot drive)
The minimum number of capacity drives you require varies with your deployment
scenario. If you're planning to use the storage pool cache, there must be at least 2 cache
devices per server.

You can deploy Storage Spaces Direct on a cluster of physical servers or on virtual
machine (VM) guest clusters. You can configure your Storage Spaces Direct design for
performance, capacity, or balanced scenarios based on the selection of physical or
virtual storage devices. Virtualized deployments take advantage of the private or public
cloud's underlying storage performance and resilience. Storage Spaces Direct deployed
on VM guest clusters allows you to use high availability solutions within virtual
environment.

The following sections describe the minimum drive requirements for physical and virtual
deployments.

Physical deployments
This table shows the minimum number of capacity drives by type for hardware
deployments such as Azure Stack HCI version 21H2 or later, and Windows Server.

Drive type present Minimum drives required Minimum drives required

(capacity only) (Windows Server) (Azure Stack HCI)

All persistent memory 4 persistent memory 2 persistent memory

(same model)
 Drive type present Minimum drives required Minimum drives required
(capacity only) (Windows Server) (Azure Stack HCI)

All NVMe (same model) 4 NVMe 2 NVMe

All SSD (same model) 4 SSD 2 SSD

If you're using the storage pool cache, there must be at least 2 more drives configured
for the cache. The table shows the minimum numbers of drives required for both
Windows Server and Azure Stack HCI deployments using 2 or more nodes.

Drive type present Minimum drives required

Persistent memory + NVMe or SSD 2 persistent memory + 4 NVMe or SSD

NVMe + SSD 2 NVMe + 4 SSD

NVMe + HDD 2 NVMe + 4 HDD

SSD + HDD 2 SSD + 4 HDD

） Important

The storage pool cache cannot be used with Azure Stack HCI in a single node
deployment.

Virtual deployment

This table shows the minimum number of drives by type for virtual deployments such as
Windows Server guest VMs or Windows Server Azure Edition.

Drive type present (capacity only) Minimum drives required

Virtual Hard Disk 2

 Tip

To boost the performance for guest VMs when running on Azure Stack HCI or
Windows Server, consider using the CSV in-memory read cache to cache
unbuffered read operations.

If you're using Storage Spaces Direct in a virtual environment, you must consider:
 Virtual disks aren't susceptible to failures like physical drives are, however you're
dependent on the performance and reliability of the public or private cloud
It's recommended to use a single tier of low latency / high performance storage
Virtual disks must be used for capacity only

Learn more about deploying Storage Spaces Direct using virtual machines and
virtualized storage.

Maximum capacity

Maximums Windows Server 2019 or later Windows Server 2016

Raw capacity per server 400 TB 100 TB

Pool capacity 4 PB (4,000 TB) 1 PB

Use the CSV in-memory read cache
Article • 04/20/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019, Windows Server 2016

This topic describes how to use system memory to boost the performance of Azure
Stack HCI and Windows Server by caching frequent reads. Writes cannot be cached in
memory.

Azure Stack HCI and Windows Server are compatible with the Cluster Shared Volume
(CSV) in-memory read cache. Using system memory to cache reads can improve
performance for applications like Hyper-V, which uses unbuffered I/O to access VHD or
VHDX files. (Unbuffered I/Os are any operations that are not cached by the Windows
Cache Manager.)

Because the in-memory cache is server-local, it improves data locality. Recent reads are
cached in memory on the same host where the virtual machine (VM) is running,
reducing how often reads go over the network. This results in lower latency and better
storage performance.

Note that the CSV in-memory read cache is different from the storage pool cache.

Planning considerations
The in-memory read cache is most effective for read-intensive workloads, such as Virtual
Desktop Infrastructure (VDI). Conversely, if the workload is extremely write-intensive, the
cache may introduce more overhead than value and should be disabled.

You can use up to 80% of total physical memory for the CSV in-memory read cache. Be
careful to leave enough memory for your VMs!

７ Note

Certain microbenchmarking tools like DISKSPD and VM Fleet may produce worse
results with the CSV in-memory read cache enabled than without it. By default VM
Fleet creates one 10 GiB VHDX per VM – approximately 1 TiB total for 100 VMs –
and then performs uniformly random reads and writes to them. Unlike real
workloads, the reads don't follow any predictable or repetitive pattern, so the in-
memory cache is not effective and just incurs overhead.
Configuring the in-memory read cache
The CSV in-memory read cache is available in Azure Stack HCI, Windows Server 2019,
and Windows Server 2016 with the same functionality. In Azure Stack HCI and Windows
Server 2019, it's on by default with 1 gibibyte (GiB) allocated. In Windows Server 2016,
it's off by default.

OS version Default CSV cache size

Azure Stack HCI 1 GiB

Windows Server 2019 1 GiB

Windows Server 2016 0 (disabled)

Configure the cache using Windows Admin Center

To configure the cache using Windows Admin Center, do the following:

1. In Windows Admin Center, connect to a cluster, and then select Settings from the
Tools pane on the left.
2. Select In-memory cache under Storage on the Settings pane.
3. In the right pane, a checkbox enables or disables the cache, and you can also
specify the maximum memory per server to be allocated to the cache.
4. When done, select Save.

Configure the cache using PowerShell

To see how much memory is allocated using PowerShell, run the following as
administrator:

PowerShell

(Get-Cluster).BlockCacheSize

The value returned is in mebibytes (MiB) per server. For example, 1024 represents 1 GiB.

To change how much memory is allocated, modify this value using PowerShell. For
example, to allocate 2 GiB per server, run:

PowerShell

(Get-Cluster).BlockCacheSize = 2048

For changes to take effect immediately, pause and then resume your CSV volumes, or
move them between servers. For example, use this PowerShell fragment to move each
CSV to another server node and back again:

PowerShell

Get-ClusterSharedVolume | ForEach {
$Owner = $_.OwnerNode
$_ | Move-ClusterSharedVolume
$_ | Move-ClusterSharedVolume -Node $Owner
}

Next steps
For related information, see also:

Understand the storage pool cache

Use Cluster Shared Volumes in a failover cluster
Choose drives for Azure Stack HCI and
Windows Server clusters
Article • 04/18/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019

This article provides guidance on how to choose drives to meet your performance and
capacity requirements.

Drive types
Storage Spaces Direct, the underlying storage virtualization technology behind Azure
Stack HCI and Windows Server currently works with four types of drives:

Type Description
of
drive

PMem refers to persistent memory, a new type of low latency, high-performance storage.

NVMe (Non-Volatile Memory Express) refers to solid-state drives that sit directly on the
PCIe bus. Common form factors are 2.5" U.2, PCIe Add-In-Card (AIC), and M.2. NVMe
offers higher IOPS and IO throughput with lower latency than any other type of drive we
support today except PMem.

SSD refers to solid-state drives, which connect via conventional SATA or SAS.

HDD refers to rotational, magnetic hard disk drives, which offer vast storage capacity.

７ Note

This article covers choosing drive configurations with NVMe, SSD, and HDD. For
more information on PMem, see Understand and deploy persistent memory.

７ Note
 Storage Bus Layer (SBL) cache is not supported in single server configuration. All
flat single storage type configurations (for example all-NVMe or all-SSD) is the only
supported storage type for single server.

Built-in cache
Storage Spaces Direct features a built-in server-side cache. It is a large, persistent, real-
time read and write cache. In deployments with multiple types of drives, it is configured
automatically to use all drives of the "fastest" type. The remaining drives are used for
capacity.

For more information, check out Understanding the storage pool cache.

Option 1 – Maximizing performance

To achieve predictable and uniform submillisecond latency across random reads and
writes to any data, or to achieve extremely high IOPS (we've done over 13 million !) or
IO throughput (we've done over 500 GB/sec reads), you should go "all-flash."

There are multiple ways to do so:

1. All NVMe. Using all NVMe provides unmatched performance, including the most
predictable low latency. If all your drives are the same model, there is no cache.
You can also mix higher-endurance and lower-endurance NVMe models, and
configure the former to cache writes for the latter (requires set-up).

2. NVMe + SSD. Using NVMe together with SSDs, the NVMe will automatically cache
writes to the SSDs. This allows writes to coalesce in cache and be de-staged only as
needed, to reduce wear on the SSDs. This provides NVMe-like write characteristics,
while reads are served directly from the also-fast SSDs.

3. All SSD. As with All-NVMe, there is no cache if all your drives are the same model.
If you mix higher-endurance and lower-endurance models, you can configure the
 former to cache writes for the latter (requires set-up).

７ Note

An advantage to using all-NVMe or all-SSD with no cache is that you get

usable storage capacity from every drive. There is no capacity "spent" on
caching, which may be appealing at smaller scale.

Option 2 – Balancing performance and capacity

For environments with a variety of applications and workloads, some with stringent
performance requirements and others requiring considerable storage capacity, you
should go "hybrid" with either NVMe or SSDs caching for larger HDDs.

1. NVMe + HDD. The NVMe drives will accelerate reads and writes by caching both.
Caching reads allows the HDDs to focus on writes. Caching writes absorbs bursts
and allows writes to coalesce and be de-staged only as needed, in an artificially
serialized manner that maximizes HDD IOPS and IO throughput. This provides
NVMe-like write characteristics, and for frequently or recently read data, NVMe-
like read characteristics too.

2. SSD + HDD. Similar to the above, the SSDs will accelerate reads and writes by
caching both. This provides SSD-like write characteristics, and SSD-like read
characteristics for frequently or recently read data.

There is one additional, rather exotic option: to use drives of all three types.

3. NVMe + SSD + HDD. With drives of all three types, the NVMe drives cache for
both the SSDs and HDDs. The appeal is that you can create volumes on the SSDs,
and volumes on the HDDs, side by side in the same cluster, all accelerated by
NVMe. The former are exactly as in an "all-flash" deployment, and the latter are
exactly as in the "hybrid" deployments described above. This is conceptually like
 having two pools, with largely independent capacity management, failure and
repair cycles, and so on.

） Important

We recommend using the SSD tier to place your most performance-sensitive

workloads on all-flash.

Option 3 – Maximizing capacity

For workloads that require vast capacity and write infrequently, such as archival, backup
targets, data warehouses or "cold" storage, you should combine a few SSDs for caching
with many larger HDDs for capacity.

1. SSD + HDD. The SSDs will cache reads and writes, to absorb bursts and provide
SSD-like write performance, with optimized de-staging later to the HDDs.

） Important

Configuration with HDDs only is not supported. High endurance SSDs caching to
low endurance SSDs is not advised.

Sizing considerations

Cache
Every server must have at least two cache drives (the minimum required for
redundancy). We recommend making the number of capacity drives a multiple of the
number of cache drives. For example, if you have 4 cache drives, you will experience
more consistent performance with 8 capacity drives (1:2 ratio) than with 7 or 9.

The cache should be sized to accommodate the working set of your applications and
workloads, that is, all the data they are actively reading and writing at any given time.
There is no cache size requirement beyond that. For deployments with HDDs, a fair
starting place is 10 percent of capacity – for example, if each server has 4 x 4 TB HDD =
16 TB of capacity, then 2 x 800 GB SSD = 1.6 TB of cache per server. For all-flash
deployments, especially with very high endurance SSDs, it may be fair to start closer
to 5 percent of capacity – for example, if each server has 24 x 1.2 TB SSD = 28.8 TB of
capacity, then 2 x 750 GB NVMe = 1.5 TB of cache per server. You can always add or
remove cache drives later to adjust.

General
We recommend limiting the total storage capacity per server to approximately 400
terabytes (TB). The more storage capacity per server, the longer the time required to
resync data after downtime or rebooting, such when applying software updates. The
current maximum size per storage pool is 4 petabytes (PB) (4,000 TB) (1 PB for Windows
Server 2016).

Next steps
For more information, see also:

Understand the storage pool cache

System requirements
Drive symmetry considerations
Plan volumes
Fault tolerance and storage efficiency
Understand and deploy persistent memory
Plan volumes on Azure Stack HCI and
Windows Server clusters
Article • 04/18/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019

This article provides guidance for how to plan cluster volumes to meet the performance
and capacity needs of your workloads, including choosing their filesystem, resiliency
type, and size.

７ Note

Storage Spaces Direct does not support a file server for general use. If you need to
run the file server or other generic services on Storage Space Direct, configure it on
the virtual machines.

Review: What are volumes

Volumes are where you put the files your workloads need, such as VHD or VHDX files for
Hyper-V virtual machines. Volumes combine the drives in the storage pool to introduce
the fault tolerance, scalability, and performance benefits of Storage Spaces Direct, the
software-defined storage technology behind Azure Stack HCI and Windows Server.

７ Note

We use term "volume" to refer jointly to the volume and the virtual disk under it,
including functionality provided by other built-in Windows features such as Cluster
Shared Volumes (CSV) and ReFS. Understanding these implementation-level
distinctions is not necessary to plan and deploy Storage Spaces Direct successfully.
All volumes are accessible by all servers in the cluster at the same time. Once created,
they show up at C:\ClusterStorage\ on all servers.

Choosing how many volumes to create

We recommend making the number of volumes a multiple of the number of servers in
your cluster. For example, if you have 4 servers, you will experience more consistent
performance with 4 total volumes than with 3 or 5. This allows the cluster to distribute
volume "ownership" (one server handles metadata orchestration for each volume)
evenly among servers.
We recommend limiting the total number of volumes to 64 volumes per cluster.

Choosing the filesystem

We recommend using the new Resilient File System (ReFS) for Storage Spaces Direct.
ReFS is the premier filesystem purpose-built for virtualization and offers many
advantages, including dramatic performance accelerations and built-in protection
against data corruption. It supports nearly all key NTFS features, including Data
Deduplication in Windows Server version 1709 and later. See the ReFS feature
comparison table for details.

If your workload requires a feature that ReFS doesn't support yet, you can use NTFS
instead.

 Tip

Volumes with different file systems can coexist in the same cluster.

Choosing the resiliency type

Volumes in Storage Spaces Direct provide resiliency to protect against hardware
problems, such as drive or server failures, and to enable continuous availability
throughout server maintenance, such as software updates.

７ Note

Which resiliency types you can choose is independent of which types of drives you
have.

With two servers

With two servers in the cluster, you can use two-way mirroring or you can use nested
resiliency.

Two-way mirroring keeps two copies of all data, one copy on the drives in each server.
Its storage efficiency is 50 percent; to write 1 TB of data, you need at least 2 TB of
physical storage capacity in the storage pool. Two-way mirroring can safely tolerate one
hardware failure at a time (one server or drive).
Nested resiliency provides data resiliency between servers with two-way mirroring, then
adds resiliency within a server with two-way mirroring or mirror-accelerated parity.
Nesting provides data resilience even when one server is restarting or unavailable. Its
storage efficiency is 25 percent with nested two-way mirroring and around 35-40
percent for nested mirror-accelerated parity. Nested resiliency can safely tolerate two
hardware failures at a time (two drives, or a server and a drive on the remaining server).
Because of this added data resilience, we recommend using nested resiliency on
production deployments of two-server clusters. For more info, see Nested resiliency.

With three servers

With three servers, you should use three-way mirroring for better fault tolerance and
performance. Three-way mirroring keeps three copies of all data, one copy on the drives
in each server. Its storage efficiency is 33.3 percent – to write 1 TB of data, you need at
least 3 TB of physical storage capacity in the storage pool. Three-way mirroring can
safely tolerate at least two hardware problems (drive or server) at a time. If 2 nodes
become unavailable the storage pool will lose quorum, since 2/3 of the disks are not
available, and the virtual disks will be unaccessible. However, a node can be down and
one or more disks on another node can fail and the virtual disks will remain online. For
example, if you're rebooting one server when suddenly another drive or server fails, all
data remains safe and continuously accessible.

With four or more servers

With four or more servers, you can choose for each volume whether to use three-way
mirroring, dual parity (often called "erasure coding"), or mix the two with mirror-
accelerated parity.

Dual parity provides the same fault tolerance as three-way mirroring but with better
storage efficiency. With four servers, its storage efficiency is 50.0 percent; to store 2 TB
of data, you need 4 TB of physical storage capacity in the storage pool. This increases to
66.7 percent storage efficiency with seven servers, and continues up to 80.0 percent
storage efficiency. The tradeoff is that parity encoding is more compute-intensive, which
can limit its performance.
Which resiliency type to use depends on the needs of your workload. Here's a table that
summarizes which workloads are a good fit for each resiliency type, as well as the
performance and storage efficiency of each resiliency type.

Resiliency Capacity efficiency Speed Workloads

type

Mirror Virtualized
Three-way mirror: 33% Highest performance workloads
Two-way-mirror: 50% Databases
Other high
performance
workloads

Mirror- Archival and

accelerated Depends on proportion Much slower than mirror, but up backup
parity of mirror and parity to twice as fast as dual-parity Virtualized
Best for large sequential writes desktop
and reads infrastructure

Dual-parity Archival and

4 servers: 50% Highest I/O latency & CPU usage backup
16 servers: up to 80% on writes Virtualized
Best for large sequential writes desktop
and reads infrastructure

When performance matters most

Workloads that have strict latency requirements or that need lots of mixed random
IOPS, such as SQL Server databases or performance-sensitive Hyper-V virtual machines,
should run on volumes that use mirroring to maximize performance.

 Tip
 Mirroring is faster than any other resiliency type. We use mirroring for nearly all our
performance examples.

When capacity matters most

Workloads that write infrequently, such as data warehouses or "cold" storage, should
run on volumes that use dual parity to maximize storage efficiency. Certain other
workloads, such as Scale-Out File Server (SoFS), virtual desktop infrastructure (VDI), or
others that don't create lots of fast-drifting random IO traffic and/or don't require the
best performance may also use dual parity, at your discretion. Parity inevitably increases
CPU utilization and IO latency, particularly on writes, compared to mirroring.

When data is written in bulk

Workloads that write in large, sequential passes, such as archival or backup targets, have
another option: one volume can mix mirroring and dual parity. Writes land first in the
mirrored portion and are gradually moved into the parity portion later. This accelerates
ingestion and reduces resource utilization when large writes arrive by allowing the
compute-intensive parity encoding to happen over a longer time. When sizing the
portions, consider that the quantity of writes that happen at once (such as one daily
backup) should comfortably fit in the mirror portion. For example, if you ingest 100 GB
once daily, consider using mirroring for 150 GB to 200 GB, and dual parity for the rest.

The resulting storage efficiency depends on the proportions you choose. See this
demo for some examples.

 Tip

If you observe an abrupt decrease in write performance partway through data

ingestion, it may indicate that the mirror portion is not large enough or that mirror-
accelerated parity isn't well suited for your use case. As an example, if write
performance decreases from 400 MB/s to 40 MB/s, consider expanding the mirror
portion or switching to three-way mirror.

About deployments with NVMe, SSD, and HDD

In deployments with two types of drives, the faster drives provide caching while the
slower drives provide capacity. This happens automatically – for more information, see
Understanding the cache in Storage Spaces Direct. In such deployments, all volumes
ultimately reside on the same type of drives – the capacity drives.

In deployments with all three types of drives, only the fastest drives (NVMe) provide
caching, leaving two types of drives (SSD and HDD) to provide capacity. For each
volume, you can choose whether it resides entirely on the SSD tier, entirely on the HDD
tier, or whether it spans the two.

） Important

We recommend using the SSD tier to place your most performance-sensitive

workloads on all-flash.

Choosing the size of volumes

We recommend limiting the size of each volume to 64 TB in Azure Stack HCI.

 Tip

If you use a backup solution that relies on the Volume Shadow Copy service (VSS)
and the Volsnap software provider—as is common with file server workloads—
limiting the volume size to 10 TB will improve performance and reliability. Backup
solutions that use the newer Hyper-V RCT API and/or ReFS block cloning and/or
the native SQL backup APIs perform well up to 32 TB and beyond.

Footprint
The size of a volume refers to its usable capacity, the amount of data it can store. This is
provided by the -Size parameter of the New-Volume cmdlet and then appears in the
Size property when you run the Get-Volume cmdlet.

Size is distinct from volume's footprint, the total physical storage capacity it occupies on
the storage pool. The footprint depends on its resiliency type. For example, volumes that
use three-way mirroring have a footprint three times their size.

The footprints of your volumes need to fit in the storage pool.

Reserve capacity
Leaving some capacity in the storage pool unallocated gives volumes space to repair
"in-place" after drives fail, improving data safety and performance. If there is sufficient
capacity, an immediate, in-place, parallel repair can restore volumes to full resiliency
even before the failed drives are replaced. This happens automatically.

We recommend reserving the equivalent of one capacity drive per server, up to 4 drives.
You may reserve more at your discretion, but this minimum recommendation
guarantees an immediate, in-place, parallel repair can succeed after the failure of any
drive.

For example, if you have 2 servers and you are using 1 TB capacity drives, set aside 2 x 1
= 2 TB of the pool as reserve. If you have 3 servers and 1 TB capacity drives, set aside 3 x
1 = 3 TB as reserve. If you have 4 or more servers and 1 TB capacity drives, set aside 4 x
1 = 4 TB as reserve.

７ Note

In clusters with drives of all three types (NVMe + SSD + HDD), we recommend
reserving the equivalent of one SSD plus one HDD per server, up to 4 drives of
each.

Example: Capacity planning

Consider one four-server cluster. Each server has some cache drives plus sixteen 2 TB
drives for capacity.

4 servers x 16 drives each x 2 TB each = 128 TB

From this 128 TB in the storage pool, we set aside four drives, or 8 TB, so that in-place
repairs can happen without any rush to replace drives after they fail. This leaves 120 TB
of physical storage capacity in the pool with which we can create volumes.

128 TB – (4 x 2 TB) = 120 TB

Suppose we need our deployment to host some highly active Hyper-V virtual machines,
but we also have lots of cold storage – old files and backups we need to retain. Because
we have four servers, let's create four volumes.

Let's put the virtual machines on the first two volumes, Volume1 and Volume2. We
choose ReFS as the filesystem (for the faster creation and checkpoints) and three-way
mirroring for resiliency to maximize performance. Let's put the cold storage on the other
two volumes, Volume 3 and Volume 4. We choose NTFS as the filesystem (for Data
Deduplication) and dual parity for resiliency to maximize capacity.

We aren't required to make all volumes the same size, but for simplicity, let's – for
example, we can make them all 12 TB.

Volume1 and Volume2 will each occupy 12 TB x 33.3 percent efficiency = 36 TB of

physical storage capacity.
Volume3 and Volume4 will each occupy 12 TB x 50.0 percent efficiency = 24 TB of
physical storage capacity.

36 TB + 36 TB + 24 TB + 24 TB = 120 TB

The four volumes fit exactly on the physical storage capacity available in our pool.
Perfect!

 Tip

You don't need to create all the volumes right away. You can always extend
volumes or create new volumes later.

For simplicity, this example uses decimal (base-10) units throughout, meaning 1 TB =
1,000,000,000,000 bytes. However, storage quantities in Windows appear in binary
(base-2) units. For example, each 2 TB drive would appear as 1.82 TiB in Windows.
Likewise, the 128 TB storage pool would appear as 116.41 TiB. This is expected.

Usage
See Creating volumes in Azure Stack HCI.

Next steps
For more information, see also:

Choosing drives for Storage Spaces Direct

Fault tolerance and storage efficiency
Using Storage Spaces Direct in guest
virtual machine clusters
Article • 06/24/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
versions 21H2 and 20H2

You can deploy Storage Spaces Direct on a cluster of physical servers or on virtual
machine (VM) guest clusters as discussed in this topic. This type of deployment delivers
virtual shared storage across a set of VMs on top of a private or public cloud. This allows
you to use application high availability solutions.

To instead use Azure Shared Disks for guest virtual machines, see Azure Shared Disks.

Deploying in Azure Iaas VM guest clusters

Azure templates have been published to decrease complexity, configure best
practices, and speed your Storage Spaces Direct deployments in an Azure Iaas VM. This
is the recommended solution for deploying in Azure.

Requirements for guest clusters

The following considerations apply when deploying Storage Spaces Direct in a
virtualized environment.
 Tip

Azure templates will automatically configure the following considerations for you
and they are the recommended solution when deploying in Azure IaaS VMs.

Minimum of two nodes and maximum of three nodes

Two-node deployments must configure a witness (Cloud Witness or File Share

Witness)

Three-node deployments can tolerate one node down and the loss of one or more
disks on another node. If two nodes shut down, then the virtual disks will be offline
until one of the nodes returns.

Configure the VMs to be deployed across fault domains

Azure – Configure the Availability Set

Hyper-V – Configure AntiAffinityClassNames on the VMs to separate the VMs

across nodes

VMware – Configure the VM-VM Anti-Affinity rule by creating a DRS Rule of

type "Separate Virtual Machines" to separate the VMs across ESX hosts. Disks
presented for use with Storage Spaces Direct should use the Paravirtual SCSI
(PVSCSI) adapter. For PVSCSI support with Windows Server, consult
https://kb.vmware.com/s/article/1010398 .

Use low latency / high performance storage such as Azure Premium SSD managed
disks or faster

Deploy a flat storage design with no caching devices configured

Use a minimum of two virtual data disks presented to each VM (VHD / VHDX /
VMDK)

This number is different than bare-metal deployments because the virtual disks
can be implemented as files that aren't susceptible to physical failures.

Disable the automatic drive replacement capabilities in the Health Service by

running the following PowerShell cmdlet:

PowerShell

Get-storagesubsystem clus* | set-storagehealthsetting -name

"System.Storage.PhysicalDisk.AutoReplace.Enabled" -value "False"
 To give greater resiliency to possible VHD / VHDX / VMDK storage latency in guest
clusters, increase the Storage Spaces I/O timeout value:

HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Services\\spaceport\\Parameters

\\HwTimeout

dword: 00007530

The decimal equivalent of Hexadecimal 7530 is 30000, which is 30 seconds. The

default value is 1770 Hexadecimal, or 6000 Decimal, which is 6 seconds.

Not supported
Host level virtual disk snapshot/restore

Instead use traditional guest level backup solutions to back up and restore the
data on the Storage Spaces Direct volumes.

Host level virtual disk size change

The virtual disks exposed through the VM must retain the same size and
characteristics. Adding more capacity to the storage pool can be accomplished by
adding more virtual disks to each of the VMs, and then adding them to the pool.
It's highly recommended to use virtual disks of the same size and characteristics as
the current virtual disks.

More references
Additional Azure Iaas VM templates for deploying Storage Spaces Direct, videos,
and step-by-step guides .

Additional Storage Spaces Direct Overview

Disaster recovery with Storage Spaces
Direct
Article • 02/15/2023

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

This topic provides scenarios on how hyper-converged infrastructure (HCI) can be

configured for disaster recovery.

Numerous companies are running hyper-converged solutions and planning for a

disaster gives the ability to remain in or get back to production quickly if a disaster were
to occur. There are several ways to configure HCI for disaster recovery and this
document explains the options that are available to you today.

When discussions of restoring availability if disaster occurs revolve around what's known
as Recovery Time Objective (RTO). This is the duration of time targeted where services
must be restored to avoid unacceptable consequences to the business. In some cases,
this process can occur automatically with production restored nearly immediately. In
other cases, manual administrator intervention must occur to restore services.

The disaster recovery options with a hyper-converged today are:

1. Multiple clusters utilizing Storage Replica

2. Hyper-V Replica between clusters
3. Backup and Restore

Multiple clusters utilizing Storage Replica

Storage Replica enables the replication of volumes and supports both synchronous and
asynchronous replication. When choosing between using either synchronous or
asynchronous replication, you should consider your Recovery Point Objective (RPO).
Recovery Point Objective is the amount of possible data loss you are willing to incur
before it is considered major loss. If you go with synchronous replication, it will
sequentially write to both ends at the same time. If you go with asynchronous, writes
will replicate very fast but could still be lost. You should consider the application or file
usage to see which best works for you.

Storage Replica is a block level copy mechanism versus file level; meaning, it does not
matter what types of data being replicated. This makes it a popular option for hyper-
converged infrastructure. Storage Replica also can utilize different types of drives
between the replication partners, so having all of one type storage on one HCI and
another type storage on the other is perfectly fine.

One important capability of Storage Replica is that it can be run in Azure as well as on-
premises. You can set up on-premises to on-premises, Azure to Azure, or even on-
premises to Azure (or vice versa).

In this scenario, there are two separate independent clusters. For configuring Storage
Replica between HCI, you can follow the steps in Cluster-to-cluster storage replication.

The following considerations apply when deploying Storage Replica.

1. Configuring replication is done outside of Failover Clustering.

2. Choosing the method of replication will be dependent upon your network latency
and RPO requirements. Synchronous replicates the data on low-latency networks
with crash consistency to ensure no data loss at a time of failure. Asynchronous
replicates the data over networks with higher latencies, but each site may not have
identical copies at a time of failure.
3. In the case of a disaster, failovers between the clusters are not automatic and need
to be orchestrated manually through the Storage Replica PowerShell cmdlets. In
the diagram above, ClusterA is the primary and ClusterB is the secondary. If
ClusterA goes down, you would need to manually set ClusterB as Primary before
you can bring the resources up. Once ClusterA is back up, you would need to make
it Secondary. Once all data has been synced up, make the change and swap the
roles back to the way they were originally set.
4. Since Storage Replica is only replicating the data, a new virtual machine or Scale
Out File Server (SOFS) utilizing this data would need to be created inside Failover
Cluster Manager on the replica partner.

Storage Replica can be used if you have virtual machines or an SOFS running on your
cluster. Bringing resources online in the replica HCI can be manual or automated
through the use of PowerShell scripting.

Hyper-V Replica
Hyper-V Replica provides virtual machine level replication for disaster recovery on
hyper-converged infrastructures. What Hyper-V Replica can do is to take a virtual
machine and replicate it to a secondary site or Azure (replica). Then from the secondary
site, Hyper-V Replica can replicate the virtual machine to a third (extended replica).

With Hyper-V Replica, the replication is taken care of by Hyper-V. When you first enable
a virtual machine for replication, there are three choices for how you wish the initial
copy to be sent to the corresponding replica cluster(s).

1. Send the initial copy over the network

2. Send the initial copy to external media so that it can be copied onto your server
manually
3. Use an existing virtual machine already created on the replica hosts

The other option is for when you wish this initial replication should take place.

1. Start the replication immediately

2. Schedule a time for when the initial replication takes place.

Other considerations you will need are:

What VHD/VHDX's you wish to replicate. You can choose to replicate all of them or
only one of them.
Number of recovery points you wish to save. If you wish to have several options
about what point in time you wish to restore, then you would want to specify how
many you want. If you only want one restore point, you can choose that as well.
 How often you wish to have the Volume Shadow Copy Service (VSS) replicate an
incremental shadow copy.
How often changes get replicated (30 seconds, 5 minutes, 15 minutes).

When HCI participate in Hyper-V Replica, you must have the Hyper-V Replica Broker
resource created in each cluster. This resource does several things:

1. Gives you a single namespace for each cluster for Hyper-V Replica to connect to.
2. Determines which node within that cluster the replica (or extended replica) will
reside on when it first receives the copy.
3. Keeps track of which node owns the replica (or extended replica) in case the virtual
machine moves to another node. It needs to track this so that when replication
takes place, it can send the information to the proper node.

Backup and restore

One traditional disaster recovery option that isn't talked about very much but is just as
important is the failure of the entire cluster or a node in the cluster. Either option with
this scenario makes use of Windows Server Backup.

It is always a recommendation to have periodic backups of the hyper-converged

infrastructure. While the Cluster Service is running, if you take a System State Backup,
the cluster registry database would be a part of that backup. Restoring the cluster or the
database has two different methods (Non-Authoritative and Authoritative).

Non-authoritative
A Non-Authoritative restore can be accomplished using Windows Server Backup and
equates to a full restore of just the cluster node itself. If you only need to restore a
cluster node (and the cluster registry database) and all current cluster information good,
you would restore using non-authoritative. Non-Authoritative restores can be done
through the Windows Server Backup interface or the command line WBADMIN.EXE.

Once you restore the node, let it join the cluster. What will happen is that it will go out
to the existing running cluster and update all of its information with what is currently
there.

Authoritative
An Authoritative restore of the cluster configuration, on the other hand, takes the cluster
configuration back in time. This type of restore should only be accomplished when the
cluster information itself has been lost and needs restored. For example, someone
accidentally deleted a File Server that contained over 1000 shares and you need them
back. Completing an Authoritative restore of the cluster requires that Backup be run
from the command line.

When an Authoritative restore is initiated on a cluster node, the cluster service is

stopped on all other nodes in the cluster view and the cluster configuration is frozen.
This is why it is critical that the cluster service on the node on which the restore was
executed be started first so the cluster is formed using the new copy of the cluster
configuration.

To run through an authoritative restore, the following steps can be accomplished.

1. Run WBADMIN.EXE from an administrative command prompt to get the latest

version of backups that you want to install and ensure that System State is one of
the components you can restore.

PowerShell

wbadmin get versions

2. Determine if the version backup you have has the cluster registry information in it
as a component. There are a couple items you will need from this command, the
version and the application/component for use in Step 3. For the version, for
example, say the backup was done January 3, 2018 at 2:04am and this is the one
you need restored.

PowerShell

wbadmin get items -backuptarget:\\backupserver\location

3. Start the authoritative restore to recover only the cluster registry version you need.

PowerShell

wbadmin start recovery -version:01/03/2018-02:04 -itemtype:app -

items:cluster

Once the restore has taken place, this node must be the one to start the Cluster Service
first and form the cluster. All other nodes would then need to be started and join the
cluster.
Summary
To sum this all up, hyper-converged disaster recovery is something that should be
planned out carefully. There are several scenarios that can best suits your needs and
should be thoroughly tested. One item to note is that if you are familiar with Failover
Clusters in the past, stretch clusters have been a very popular option over the years.
There was a bit of a design change with the hyper-converged solution and it is based on
resiliency. If you lose two nodes in a hyper-converged cluster, the entire cluster will go
down. With this being the case, in a hyper-converged environment, the stretch scenario
is not supported.
Deploy Storage Spaces Direct on
Windows Server
Article • 09/19/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

This topic provides step-by-step instructions to deploy Storage Spaces Direct on

Windows Server. To deploy Storage Spaces Direct as part of Azure Stack HCI, see What is
the deployment process for Azure Stack HCI?

 Tip

Looking to acquire hyperconverged infrastructure? Microsoft recommends

purchasing a validated hardware/software Azure Stack HCI solution from our
partners. These solutions are designed, assembled, and validated against our
reference architecture to ensure compatibility and reliability, so you get up and
running quickly. To peruse a catalog of hardware/software solutions that work with
Azure Stack HCI, see the Azure Stack HCI Catalog .

 Tip

You can use Hyper-V virtual machines, including in Microsoft Azure, to evaluate
Storage Spaces Direct without hardware. You may also want to review the handy
Windows Server rapid lab deployment scripts , which we use for training
purposes.

Before you start

Review the Storage Spaces Direct hardware requirements and skim this document to
familiarize yourself with the overall approach and important notes associated with some
steps.

Gather the following information:

Deployment option. Storage Spaces Direct supports two deployment options:

hyper-converged and converged, also known as disaggregated. Familiarize
yourself with the advantages of each to decide which is right for you. Steps 1-3
 below apply to both deployment options. Step 4 is only needed for converged
deployment.

Server names. Get familiar with your organization's naming policies for computers,
files, paths, and other resources. You'll need to provision several servers, each with
unique names.

Domain name. Get familiar with your organization's policies for domain naming
and domain joining. You'll be joining the servers to your domain, and you'll need
to specify the domain name.

RDMA networking. There are two types of RDMA protocols: iWarp and RoCE. Note
which one your network adapters use, and if RoCE, also note the version (v1 or v2).
For RoCE, also note the model of your top-of-rack switch.

VLAN ID. Note the VLAN ID to be used for management OS network adapters on
the servers, if any. You should be able to obtain this from your network
administrator.

Step 1: Deploy Windows Server

Step 1.1: Install the operating system

The first step is to install Windows Server on every server that will be in the cluster.
Storage Spaces Direct requires Windows Server Datacenter Edition. You can use the
Server Core installation option, or Server with Desktop Experience.

When you install Windows Server using the Setup wizard, you can choose between
Windows Server (referring to Server Core) and Windows Server (Server with Desktop
Experience), which is the equivalent of the Full installation option available in Windows
Server 2012 R2. If you don't choose, you'll get the Server Core installation option. For
more information, see Install Server Core.

Step 1.2: Connect to the servers

This guide focuses the Server Core installation option and deploying/managing
remotely from a separate management system, which must have:

A version of Windows Server or Windows 10 at least as new as the servers it's

managing, and with the latest updates
Network connectivity to the servers it's managing
Joined to the same domain or a fully trusted domain
 Remote Server Administration Tools (RSAT) and PowerShell modules for Hyper-V
and Failover Clustering. RSAT tools and PowerShell modules are available on
Windows Server and can be installed without installing other features. You can also
install the Remote Server Administration Tools on a Windows 10 management
PC.

On the Management system install the Failover Cluster and Hyper-V management tools.
This can be done through Server Manager using the Add Roles and Features wizard. On
the Features page, select Remote Server Administration Tools, and then select the tools
to install.

Enter the PS session and use either the server name or the IP address of the node you
want to connect to. You'll be prompted for a password after you execute this command,
enter the administrator password you specified when setting up Windows.

PowerShell

Enter-PSSession -ComputerName <myComputerName> -Credential

LocalHost\Administrator

Here's an example of doing the same thing in a way that is more useful in scripts, in case
you need to do this more than once:

PowerShell

$myServer1 = "myServer-1"
$user = "$myServer1\Administrator"

Enter-PSSession -ComputerName $myServer1 -Credential $user

 Tip

If you're deploying remotely from a management system, you might get an error
like WinRM cannot process the request. To fix this, use Windows PowerShell to add
each server to the Trusted Hosts list on your management computer:

Set-Item WSMAN:\Localhost\Client\TrustedHosts -Value Server01 -Force

Note: the trusted hosts list supports wildcards, like Server* .

To view your Trusted Hosts list, type Get-Item

WSMAN:\Localhost\Client\TrustedHosts .

To empty the list, type Clear-Item WSMAN:\Localhost\Client\TrustedHost .

Step 1.3: Join the domain and add domain accounts
So far you've configured the individual servers with the local administrator account,
<ComputerName>\Administrator .

To manage Storage Spaces Direct, you'll need to join the servers to a domain and use an
Active Directory Domain Services domain account that is in the Administrators group on
every server.

From the management system, open a PowerShell console with Administrator privileges.
Use Enter-PSSession to connect to each server and run the following cmdlet,
substituting your own computer name, domain name, and domain credentials:

PowerShell

Add-Computer -NewName "Server01" -DomainName "contoso.com" -Credential

"CONTOSO\User" -Restart -Force

If your storage administrator account isn't a member of the Domain Admins group, add
your storage administrator account to the local Administrators group on each node - or
better yet, add the group you use for storage administrators. You can use the following
command (or write a Windows PowerShell function to do so - see Use PowerShell to
Add Domain Users to a Local Group for more info):

Net localgroup Administrators <Domain\Account> /add

Step 1.4: Install roles and features

The next step is to install server roles on every server. You can do this by using Windows
Admin Center, Server Manager), or PowerShell. Here are the roles to install:

Failover Clustering
Hyper-V
File Server (if you want to host any file shares, such as for a converged
deployment)
Data-Center-Bridging (if you're using RoCEv2 instead of iWARP network adapters)
RSAT-Clustering-PowerShell
Hyper-V-PowerShell
To install via PowerShell, use the Install-WindowsFeature cmdlet. You can use it on a
single server like this:

PowerShell

Install-WindowsFeature -Name "Hyper-V", "Failover-Clustering", "Data-Center-

Bridging", "RSAT-Clustering-PowerShell", "Hyper-V-PowerShell", "FS-
FileServer"

To run the command on all servers in the cluster as the same time, use this little bit of
script, modifying the list of variables at the beginning of the script to fit your
environment.

PowerShell

# Fill in these variables with your values

$ServerList = "Server01", "Server02", "Server03", "Server04"
$FeatureList = "Hyper-V", "Failover-Clustering", "Data-Center-Bridging",
"RSAT-Clustering-PowerShell", "Hyper-V-PowerShell", "FS-FileServer"

# This part runs the Install-WindowsFeature cmdlet on all servers in

$ServerList, passing the list of features into the scriptblock with the
"Using" scope modifier so you don't have to hard-code them here.
Invoke-Command ($ServerList) {
Install-WindowsFeature -Name $Using:Featurelist
}

Step 2: Configure the network

If you're deploying Storage Spaces Direct inside virtual machines, skip this section.

Storage Spaces Direct requires high-bandwidth, low-latency networking between servers

in the cluster. At least 10 GbE networking is required and remote direct memory access
(RDMA) is recommended. You can use either iWARP or RoCE as long as it has the
Windows Server logo that matches your operating system version, but iWARP is usually
easier to set up.

） Important

Depending on your networking equipment, and especially with RoCE v2, some
configuration of the top-of-rack switch may be required. Correct switch
configuration is important to ensure reliability and performance of Storage Spaces
Direct.
Windows Server 2016 introduced switch-embedded teaming (SET) within the Hyper-V
virtual switch. This allows the same physical NIC ports to be used for all network traffic
while using RDMA, reducing the number of physical NIC ports required. Switch-
embedded teaming is recommended for Storage Spaces Direct.

Switched or switchless node interconnects

Switched: Network switches must be properly configured to handle the bandwidth

and networking type. If using RDMA that implements the RoCE protocol, network
device and switch configuration is even more important.
Switchless: Nodes can be interconnected using direct connections, avoiding using
a switch. It is required that every node have a direct connection with every other
node of the cluster.

For instructions to set up networking for Storage Spaces Direct, see the Windows Server
2016 and 2019 RDMA Deployment Guide .

Step 3: Configure Storage Spaces Direct

The following steps are done on a management system that is the same version as the
servers being configured. The following steps should NOT be run remotely using a
PowerShell session, but instead run in a local PowerShell session on the management
system, with administrative permissions.

Step 3.1: Clean drives

Before you enable Storage Spaces Direct, ensure your drives are empty: no old partitions
or other data. Run the following script, substituting your computer names, to remove all
any old partitions or other data.

２ Warning

This script will permanently remove any data on any drives other than the
operating system boot drive!

PowerShell

# Fill in these variables with your values

$ServerList = "Server01", "Server02", "Server03", "Server04"

Invoke-Command ($ServerList) {
Update-StorageProviderCache
Get-StoragePool | ? IsPrimordial -eq $false | Set-StoragePool -
 IsReadOnly:$false -ErrorAction SilentlyContinue
Get-StoragePool | ? IsPrimordial -eq $false | Get-VirtualDisk | Remove-
VirtualDisk -Confirm:$false -ErrorAction SilentlyContinue
Get-StoragePool | ? IsPrimordial -eq $false | Remove-StoragePool -
Confirm:$false -ErrorAction SilentlyContinue
Get-PhysicalDisk | Reset-PhysicalDisk -ErrorAction SilentlyContinue
Get-Disk | ? Number -ne $null | ? IsBoot -ne $true | ? IsSystem -ne
$true | ? PartitionStyle -ne RAW | % {
$_ | Set-Disk -isoffline:$false
$_ | Set-Disk -isreadonly:$false
$_ | Clear-Disk -RemoveData -RemoveOEM -Confirm:$false
$_ | Set-Disk -isreadonly:$true
$_ | Set-Disk -isoffline:$true
}
Get-Disk | Where Number -Ne $Null | Where IsBoot -Ne $True | Where
IsSystem -Ne $True | Where PartitionStyle -Eq RAW | Group -NoElement -
Property FriendlyName
} | Sort -Property PsComputerName, Count

The output will look like this, where Count is the number of drives of each model in
each server:

Count Name PSComputerName

----- ---- --------------
4 ATA SSDSC2BA800G4n Server01
10 ATA ST4000NM0033 Server01
4 ATA SSDSC2BA800G4n Server02
10 ATA ST4000NM0033 Server02
4 ATA SSDSC2BA800G4n Server03
10 ATA ST4000NM0033 Server03
4 ATA SSDSC2BA800G4n Server04
10 ATA ST4000NM0033 Server04

Step 3.2: Validate the cluster

In this step, you'll run the cluster validation tool to ensure that the server nodes are
configured correctly to create a cluster using Storage Spaces Direct. When cluster
validation ( Test-Cluster ) is run before the cluster is created, it runs the tests that verify
that the configuration appears suitable to successfully function as a failover cluster. The
example directly below uses the -Include parameter, and then the specific categories of
tests are specified. This ensures that the Storage Spaces Direct specific tests are included
in the validation.

Use the following PowerShell command to validate a set of servers for use as a Storage
Spaces Direct cluster.
 PowerShell

Test-Cluster -Node <MachineName1, MachineName2, MachineName3, MachineName4>

-Include "Storage Spaces Direct", "Inventory", "Network", "System
Configuration"

Step 3.3: Create the cluster

In this step, you'll create a cluster with the nodes that you have validated for cluster
creation in the preceding step using the following PowerShell cmdlet.

When creating the cluster, you'll get a warning that states - "There were issues while
creating the clustered role that may prevent it from starting. For more information, view
the report file below." You can safely ignore this warning. It's due to no disks being
available for the cluster quorum. Its recommended that a file share witness or cloud
witness is configured after creating the cluster.

７ Note

If the servers are using static IP addresses, modify the following command to reflect
the static IP address by adding the following parameter and specifying the IP
address:-StaticAddress <X.X.X.X>. In the following command the ClusterName
placeholder should be replaced with a netbios name that is unique and 15
characters or less.

PowerShell

New-Cluster -Name <ClusterName> -Node

<MachineName1,MachineName2,MachineName3,MachineName4> -NoStorage

After the cluster is created, it can take time for DNS entry for the cluster name to be
replicated. The time is dependent on the environment and DNS replication
configuration. If resolving the cluster isn't successful, in most cases you can be
successful with using the machine name of a node that is an active member of the
cluster may be used instead of the cluster name.

Step 3.4: Configure a cluster witness

We recommend that you configure a witness for the cluster, so clusters with three or
more servers can withstand two servers failing or being offline. A two-server
deployment requires a cluster witness, otherwise either server going offline causes the
other to become unavailable as well. With these systems, you can use a file share as a
witness, or use cloud witness.

For more info, see the following topics:

Configure and manage quorum

Deploy a Cloud Witness for a Failover Cluster

Step 3.5: Enable Storage Spaces Direct

After creating the cluster, use the Enable-ClusterStorageSpacesDirect PowerShell
cmdlet, which will put the storage system into the Storage Spaces Direct mode and do
the following automatically:

Create a pool: Creates a single large pool that has a name like "S2D on Cluster1".

Configures the Storage Spaces Direct caches: If there is more than one media
(drive) type available for Storage Spaces Direct use, it enables the fastest as cache
devices (read and write in most cases)

Tiers: Creates two tiers as default tiers. One is called "Capacity" and the other
called "Performance". The cmdlet analyzes the devices and configures each tier
with the mix of device types and resiliency.

From the management system, in a PowerShell command windows opened with

Administrator privileges, initiate the following command. The cluster name is the name
of the cluster that you created in the previous steps. If this command is run locally on
one of the nodes, the -CimSession parameter is not necessary.

PowerShell

Enable-ClusterStorageSpacesDirect -CimSession <ClusterName>

To enable Storage Spaces Direct using the above command, you can also use the node
name instead of the cluster name. Using the node name may be more reliable due to
DNS replication delays that may occur with the newly created cluster name.

When this command is finished, which may take several minutes, the system will be
ready for volumes to be created.

Step 3.6: Create volumes

We recommend using the New-Volume cmdlet as it provides the fastest and most
straightforward experience. This single cmdlet automatically creates the virtual disk,
partitions and formats it, creates the volume with matching name, and adds it to cluster
shared volumes – all in one easy step.

For more information, check out Creating volumes in Storage Spaces Direct.

Step 3.7: Optionally enable the CSV cache

You can optionally enable the cluster shared volume (CSV) cache to use system memory
(RAM) as a write-through block-level cache of read operations that aren't already
cached by the Windows cache manager. This can improve performance for applications
such as Hyper-V. The CSV cache can boost the performance of read requests and is also
useful for Scale-Out File Server scenarios.

Enabling the CSV cache reduces the amount of memory available to run VMs on a
hyper-converged cluster, so you'll have to balance storage performance with memory
available to VHDs.

To set the size of the CSV cache, open a PowerShell session on the management system
with an account that has administrator permissions on the storage cluster, and then use
this script, changing the $ClusterName and $CSVCacheSize variables as appropriate (this
example sets a 2 GB CSV cache per server):

PowerShell

$ClusterName = "StorageSpacesDirect1"
$CSVCacheSize = 2048 #Size in MB

Write-Output "Setting the CSV cache..."

(Get-Cluster $ClusterName).BlockCacheSize = $CSVCacheSize

$CSVCurrentCacheSize = (Get-Cluster $ClusterName).BlockCacheSize

Write-Output "$ClusterName CSV cache size: $CSVCurrentCacheSize MB"

For more info, see Using the CSV in-memory read cache.

Step 3.8: Deploy virtual machines for hyper-converged

deployments
If you're deploying a hyper-converged cluster, the last step is to provision virtual
machines on the Storage Spaces Direct cluster.
The virtual machine's files should be stored on the systems CSV namespace (example:
c:\ClusterStorage\Volume1) just like clustered VMs on failover clusters.

You can use in-box tools or other tools to manage the storage and virtual machines,
such as System Center Virtual Machine Manager.

Step 4: Deploy Scale-Out File Server for

converged solutions
If you're deploying a converged solution, the next step is to create a Scale-Out File
Server instance and setup some file shares. If you're deploying a hyper-converged
cluster - you're finished and don't need this section.

Step 4.1: Create the Scale-Out File Server role

The next step in setting up the cluster services for your file server is creating the
clustered file server role, which is when you create the Scale-Out File Server instance on
which your continuously available file shares are hosted.

To create a Scale-Out File Server role by using Server Manager

1. In Failover Cluster Manager, select the cluster, go to Roles, and then click
Configure Role….
The High Availability Wizard appears.

2. On the Select Role page, click File Server.

3. On the File Server Type page, click Scale-Out File Server for application data.

4. On the Client Access Point page, type a name for the Scale-Out File Server.

5. Verify that the role was successfully set up by going to Roles and confirming that
the Status column shows Running next to the clustered file server role you
created, as shown in Figure 1.
 Figure 1 Failover Cluster Manager showing the Scale-Out File Server with the
Running status

７ Note

After creating the clustered role, there might be some network propagation delays
that could prevent you from creating file shares on it for a few minutes, or
potentially longer.

To create a Scale-Out File Server role by using Windows PowerShell

In a Windows PowerShell session that's connected to the file server cluster, enter the
following commands to create the Scale-Out File Server role, changing FSCLUSTER to
match the name of your cluster, and SOFS to match the name you want to give the
Scale-Out File Server role:

PowerShell

Add-ClusterScaleOutFileServerRole -Name SOFS -Cluster FSCLUSTER

７ Note

After creating the clustered role, there might be some network propagation delays
that could prevent you from creating file shares on it for a few minutes, or
potentially longer. If the SOFS role fails immediately and won't start, it might be
because the cluster's computer object doesn't have permission to create a
computer account for the SOFS role. For help with that, see this blog post: Scale-
Out File Server Role Fails To Start With Event IDs 1205, 1069, and 1194 .
Step 4.2: Create file shares
After you've created your virtual disks and added them to CSVs, it's time to create file
shares on them - one file share per CSV per virtual disk. System Center Virtual Machine
Manager (VMM) is probably the handiest way to do this because it handles permissions
for you, but if you don't have it in your environment, you can use Windows PowerShell
to partially automate the deployment.

Use the scripts included in the SMB Share Configuration for Hyper-V Workloads script,
which partially automates the process of creating groups and shares. It's written for
Hyper-V workloads, so if you're deploying other workloads, you might have to modify
the settings or perform additional steps after you create the shares. For example, if
you're using Microsoft SQL Server, the SQL Server service account must be granted full
control on the share and the file system.

７ Note

You'll have to update the group membership when you add cluster nodes unless
you use System Center Virtual Machine Manager to create your shares.

To create file shares by using PowerShell scripts, do the following:

1. Download the scripts included in SMB Share Configuration for Hyper-V

Workloads to one of the nodes of the file server cluster.

2. Open a Windows PowerShell session with Domain Administrator credentials on the

management system, and then use the following script to create an Active
Directory group for the Hyper-V computer objects, changing the values for the
variables as appropriate for your environment:

PowerShell

# Replace the values of these variables

$HyperVClusterName = "Compute01"
$HyperVObjectADGroupSamName = "Hyper-VServerComputerAccounts" <#No
spaces#>
$ScriptFolder = "C:\Scripts\SetupSMBSharesWithHyperV"

# Start of script itself

CD $ScriptFolder
.\ADGroupSetup.ps1 -HyperVObjectADGroupSamName
$HyperVObjectADGroupSamName -HyperVClusterName $HyperVClusterName
 3. Open a Windows PowerShell session with Administrator credentials on one of the
storage nodes, and then use the following script to create shares for each CSV and
grant administrative permissions for the shares to the Domain Admins group and
the compute cluster.

PowerShell

# Replace the values of these variables

$StorageClusterName = "StorageSpacesDirect1"
$HyperVObjectADGroupSamName = "Hyper-VServerComputerAccounts" <#No
spaces#>
$SOFSName = "SOFS"
$SharePrefix = "Share"
$ScriptFolder = "C:\Scripts\SetupSMBSharesWithHyperV"

# Start of the script itself

CD $ScriptFolder
Get-ClusterSharedVolume -Cluster $StorageClusterName | ForEach-Object {
$ShareName = $SharePrefix +
$_.SharedVolumeInfo.friendlyvolumename.trimstart("C:\ClusterStorage\Vol
ume")
Write-host "Creating share $ShareName on "$_.name "on Volume: "
$_.SharedVolumeInfo.friendlyvolumename
.\FileShareSetup.ps1 -HyperVClusterName $StorageClusterName -
CSVVolumeNumber
$_.SharedVolumeInfo.friendlyvolumename.trimstart("C:\ClusterStorage\Vol
ume") -ScaleOutFSName $SOFSName -ShareName $ShareName -
HyperVObjectADGroupSamName $HyperVObjectADGroupSamName
}

Step 4.3 Enable Kerberos constrained delegation

To setup Kerberos constrained delegation for remote scenario management and
increased Live Migration security, from one of the storage cluster nodes, use the
KCDSetup.ps1 script included in SMB Share Configuration for Hyper-V Workloads .
Here's a little wrapper for the script:

PowerShell

$HyperVClusterName = "Compute01"
$ScaleOutFSName = "SOFS"
$ScriptFolder = "C:\Scripts\SetupSMBSharesWithHyperV"

CD $ScriptFolder
.\KCDSetup.ps1 -HyperVClusterName $HyperVClusterName -ScaleOutFSName
$ScaleOutFSName -EnableLM
Additional References
Storage Spaces Direct overview
Understand the cache in Storage Spaces Direct
Planning volumes in Storage Spaces Direct
Storage Spaces Fault Tolerance
Storage Spaces Direct Hardware Requirements
To RDMA, or not to RDMA – that is the question (TechNet blog)
Create volumes on Azure Stack HCI and Windows Server
clusters
Article • 04/17/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022, Windows Server 2019, Windows Server 2016

This article describes how to create volumes on a cluster by using Windows Admin Center and Windows PowerShell, how to work with files
on the volumes, and how to enable deduplication and compression, integrity checksums, or BitLocker encryption on volumes. To learn how
to create volumes and set up replication for stretched clusters, see Create stretched volumes.

 Tip

If you haven't already, check out Plan volumes first.

When creating volumes on a single-node cluster, you must use PowerShell. See Create volumes using PowerShell.

Create a two-way or three-way mirror volume

To create a two-way or three-way mirror volume using Windows Admin Center:

1. In Windows Admin Center, connect to a cluster, and then select Volumes from the Tools pane.

2. On the Volumes page, select the Inventory tab, and then select Create.

3. In the Create volume pane, enter a name for the volume.

4. In Resiliency, select Two-way mirror or Three-way mirror depending on the number of servers in your cluster.

5. In Size on HDD, specify the size of the volume. For example, 5 TB (terabytes).

6. Under More options, you can use the checkboxes to turn on deduplication and compression, integrity checksums, or BitLocker
encryption.

7. Select Create.

Depending on the size, creating the volume can take a few minutes. Notifications in the upper-right will let you know when the volume is
created. The new volume will then appear in the Inventory list.

Create a mirror-accelerated parity volume

Mirror-accelerated parity (MAP) reduces the footprint of the volume on the HDD. For example, a three-way mirror volume would mean that
for every 10 terabytes of size, you will need 30 terabytes as a footprint. To reduce the overhead in footprint, create a volume with mirror-
accelerated parity. This reduces the footprint from 30 terabytes to just 22 terabytes, even with only 4 servers, by mirroring the most active
20 percent of data and using parity, which is more space efficient, to store the rest. You can adjust this ratio of parity and mirror to make
the performance versus capacity tradeoff that's right for your workload. For example, 90 percent parity and 10 percent mirror yields less
performance but streamlines the footprint even further.

７ Note

Mirror-accelerated parity volumes require Resilient File System (ReFS).

To create a volume with mirror-accelerated parity in Windows Admin Center:

1. In Windows Admin Center, connect to a cluster, and then select Volumes from the Tools pane.
2. On the Volumes page, select the Inventory tab, and then select Create.
3. In the Create volume pane, enter a name for the volume.
4. In Resiliency, select Mirror-accelerated parity.
5. In Parity percentage, select the percentage of parity.
6. Under More options, you can use the checkboxes to turn on deduplication and compression, integrity checksums, or BitLocker
encryption.
7. Select Create.

Open volume and add files

To open a volume and add files to the volume in Windows Admin Center:

1. In Windows Admin Center, connect to a cluster, and then select Volumes from the Tools pane.

2. On the Volumes page, select the Inventory tab.

3. In the list of volumes, select the name of the volume that you want to open.

On the volume details page, you can see the path to the volume.

4. At the top of the page, select Open. This launches the Files tool in Windows Admin Center.

5. Navigate to the path of the volume. Here you can browse the files in the volume.

6. Select Upload, and then select a file to upload.

7. Use the browser Back button to go back to the Tools pane in Windows Admin Center.

Turn on deduplication and compression

Deduplication and compression is managed per volume. Deduplication and compression uses a post-processing model, which means that
you won't see savings until it runs. When it does, it'll work over all files, even those that were there from before.

To learn more, see Enable volume encryption, deduplication, and compression

Create volumes using Windows PowerShell

First, launch Windows PowerShell from the Windows start menu. We recommend using the New-Volume cmdlet to create volumes for
Azure Stack HCI. It provides the fastest and most straightforward experience. This single cmdlet automatically creates the virtual disk,
partitions and formats it, creates the volume with matching name, and adds it to cluster shared volumes – all in one easy step.

The New-Volume cmdlet has four parameters you'll always need to provide:

FriendlyName: Any string you want, for example "Volume1"

FileSystem: Either CSVFS_ReFS (recommended for all volumes; required for mirror-accelerated parity volumes) or CSVFS_NTFS

StoragePoolFriendlyName: The name of your storage pool, for example "S2D on ClusterName"

Size: The size of the volume, for example "10TB"

 ７ Note

Windows, including PowerShell, counts using binary (base-2) numbers, whereas drives are often labeled using decimal (base-10)
numbers. This explains why a "one terabyte" drive, defined as 1,000,000,000,000 bytes, appears in Windows as about "909 GB".
This is expected. When creating volumes using New-Volume, you should specify the Size parameter in binary (base-2) numbers.
For example, specifying "909GB" or "0.909495TB" will create a volume of approximately 1,000,000,000,000 bytes.

Example: With 1 to 3 servers

To make things easier, if your deployment has only one or two servers, Storage Spaces Direct will automatically use two-way mirroring for
resiliency. If your deployment has only three servers, it will automatically use three-way mirroring.

PowerShell

New-Volume -FriendlyName "Volume1" -FileSystem CSVFS_ReFS -StoragePoolFriendlyName S2D* -Size 1TB

Example: With 4+ servers

If you have four or more servers, you can use the optional ResiliencySettingName parameter to choose your resiliency type.

ResiliencySettingName: Either Mirror or Parity.

In the following example, "Volume2" uses three-way mirroring and "Volume3" uses dual parity (often called "erasure coding").

PowerShell

New-Volume -FriendlyName "Volume2" -FileSystem CSVFS_ReFS -StoragePoolFriendlyName S2D* -Size 1TB -ResiliencySettingName
Mirror
New-Volume -FriendlyName "Volume3" -FileSystem CSVFS_ReFS -StoragePoolFriendlyName S2D* -Size 1TB -ResiliencySettingName
Parity

Using storage tiers

In deployments with three types of drives, one volume can span the SSD and HDD tiers to reside partially on each. Likewise, in deployments
with four or more servers, one volume can mix mirroring and dual parity to reside partially on each.

To help you create such volumes, Azure Stack HCI provides default tier templates called MirrorOnMediaType and
NestedMirrorOnMediaType (for performance), and ParityOnMediaType and NestedParityOnMediaType (for capacity), where MediaType is
HDD or SSD. The templates represent storage tiers based on media types and encapsulate definitions for three-way mirroring on the faster
capacity drives (if applicable), and dual parity on the slower capacity drives (if applicable).

７ Note

Storage Bus Layer (SBL) cache is not supported in single server configuration. All flat single storage type configurations (for example
all-NVMe or all-SSD) is the only supported storage type for single server.

７ Note

On Storage Spaces Direct clusters running on earlier versions of Windows Server 2016, the default tier templates were simply called
Performance and Capacity.

You can see the storage tiers by running the Get-StorageTier cmdlet on any server in the cluster.

PowerShell

Get-StorageTier | Select FriendlyName, ResiliencySettingName, PhysicalDiskRedundancy

For example, if you have a two-node cluster with only HDD, your output might look something like this:

PowerShell
 FriendlyName ResiliencySettingName PhysicalDiskRedundancy
------------ --------------------- ----------------------
NestedParityOnHDD Parity 1
Capacity Mirror 1
NestedMirrorOnHDD Mirror 3
MirrorOnHDD Mirror 1

To create tiered volumes, reference these tier templates using the StorageTierFriendlyNames and StorageTierSizes parameters of the
New-Volume cmdlet. For example, the following cmdlet creates one volume which mixes three-way mirroring and dual parity in 30:70
proportions.

PowerShell

New-Volume -FriendlyName "Volume1" -FileSystem CSVFS_ReFS -StoragePoolFriendlyName S2D* -StorageTierFriendlyNames

MirrorOnHDD, Capacity -StorageTierSizes 300GB, 700GB

Repeat as needed to create more than one volume.

Storage tier summary table

The following tables summarize the storage tiers that are/can be created in Azure Stack HCI and Windows Server.

NumberOfNodes: 1

FriendlyName MediaType ResiliencySettingName NumberOfDataCopies PhysicalDiskRedundancy NumberOfGroups FaultDomainAwareness Co

MirrorOnHDD HDD Mirror 2 1 1 PhysicalDisk

MirrorOnSSD SSD Mirror 2 1 1 PhysicalDisk

MirrorOnSCM SCM Mirror 2 1 1 PhysicalDisk

ParityOnHDD HDD Parity 1 1 1 PhysicalDisk

ParityOnSSD SSD Parity 1 1 1 PhysicalDisk

ParityOnSCM SCM Parity 1 1 1 PhysicalDisk

NumberOfNodes: 2

FriendlyName MediaType ResiliencySettingName NumberOfDataCopies PhysicalDiskRedundancy NumberOfGroups FaultDomainAwarenes

MirrorOnHDD HDD Mirror 2 1 1 StorageScaleUnit

MirrorOnSSD SSD Mirror 2 1 1 StorageScaleUnit

MirrorOnSCM SCM Mirror 2 1 1 StorageScaleUnit

NestedMirrorOnHDD HDD Mirror 4 3 1 StorageScaleUnit

NestedMirrorOnSSD SSD Mirror 4 3 1 StorageScaleUnit

NestedMirrorOnSCM SCM Mirror 4 3 1 StorageScaleUnit

NestedParityOnHDD HDD Parity 2 1 1 StorageScaleUnit

NestedParityOnSSD SSD Parity 2 1 1 StorageScaleUnit

NestedParityOnSCM SCM Parity 2 1 1 StorageScaleUnit

NumberOfNodes: 3

FriendlyName MediaType ResiliencySettingName NumberOfDataCopies PhysicalDiskRedundancy NumberOfGroups FaultDomainAwareness Co

 FriendlyName MediaType ResiliencySettingName NumberOfDataCopies PhysicalDiskRedundancy NumberOfGroups FaultDomainAwareness Co

MirrorOnHDD HDD Mirror 3 2 1 StorageScaleUnit

MirrorOnSSD SSD Mirror 3 2 1 StorageScaleUnit

MirrorOnSCM SCM Mirror 3 2 1 StorageScaleUnit

NumberOfNodes: 4+

FriendlyName MediaType ResiliencySettingName NumberOfDataCopies PhysicalDiskRedundancy NumberOfGroups FaultDomainAwareness Co

MirrorOnHDD HDD Mirror 3 2 1 StorageScaleUnit

MirrorOnSSD SSD Mirror 3 2 1 StorageScaleUnit

MirrorOnSCM SCM Mirror 3 2 1 StorageScaleUnit

ParityOnHDD HDD Parity 1 2 Auto StorageScaleUnit St

ParityOnSSD SSD Parity 1 2 Auto StorageScaleUnit St

ParityOnSCM SCM Parity 1 2 Auto StorageScaleUnit St

Nested resiliency volumes

Nested resiliency only applies to two-server clusters running Azure Stack HCI or Windows Server 2022 or Windows Server 2019; you can't
use nested resiliency if your cluster has three or more servers, or if your cluster runs Windows Server 2016. Nested resiliency enables a two-
server cluster to withstand multiple hardware failures at the same time without loss of storage availability, allowing users, apps, and virtual
machines to continue to run without disruption. For more information, see Nested Resiliency for Storage Spaces Direct and Plan volumes:
choosing the resiliency type.

You can use familiar storage cmdlets in PowerShell to create volumes with nested resiliency, as described in the following section.

Step 1: Create storage tier templates (Windows Server 2019 only)

Windows Server 2019 requires you to create new storage tier templates using the New-StorageTier cmdlet before creating volumes. You
only need to do this once, and then every new volume you create can reference these templates.

７ Note

If you're running Windows Server 2022, Azure Stack HCI 21H2, or Azure Stack HCI 20H2, you can skip this step.

Specify the -MediaType of your capacity drives and, optionally, the -FriendlyName of your choice. Do not modify other parameters.

For example, if your capacity drives are hard disk drives (HDD), launch PowerShell as Administrator and run the following cmdlets.

To create a NestedMirror tier:

PowerShell

New-StorageTier -StoragePoolFriendlyName S2D* -FriendlyName NestedMirrorOnHDD -ResiliencySettingName Mirror -MediaType HDD

-NumberOfDataCopies 4

To create a NestedParity tier:

PowerShell

New-StorageTier -StoragePoolFriendlyName S2D* -FriendlyName NestedParityOnHDD -ResiliencySettingName Parity -MediaType HDD

-NumberOfDataCopies 2 -PhysicalDiskRedundancy 1 -NumberOfGroups 1 -FaultDomainAwareness StorageScaleUnit -ColumnIsolation
 PhysicalDisk

If your capacity drives are solid-state drives (SSD), set the -MediaType to SSD instead and change the -FriendlyName to *OnSSD . Do not
modify other parameters.

 Tip

Verify that Get-StorageTier created the tiers successfully.

Step 2: Create nested volumes

Create new volumes using the New-Volume cmdlet.

Nested two-way mirror

To use nested two-way mirror, reference the NestedMirror tier template and specify the size. For example:

PowerShell

New-Volume -StoragePoolFriendlyName S2D* -FriendlyName Volume01 -StorageTierFriendlyNames NestedMirrorOnHDD -

StorageTierSizes 500GB

If your capacity drives are solid-state drives (SSD), change -StorageTierFriendlyNames to *OnSSD .

Nested mirror-accelerated parity

To use nested mirror-accelerated parity, reference both the NestedMirror and NestedParity tier templates and specify two sizes, one
for each part of the volume (mirror first, parity second). For example, to create one 500 GB volume that's 20% nested two-way mirror
and 80% nested parity, run:

PowerShell

New-Volume -StoragePoolFriendlyName S2D* -FriendlyName Volume02 -StorageTierFriendlyNames NestedMirrorOnHDD,

NestedParityOnHDD -StorageTierSizes 100GB, 400GB

If your capacity drives are solid-state drives (SSD), change -StorageTierFriendlyNames to *OnSSD .

Step 3: Continue in Windows Admin Center

Volumes that use nested resiliency appear in Windows Admin Center with clear labeling, as in the screenshot below. Once they're created,
you can manage and monitor them using Windows Admin Center just like any other volume in Storage Spaces Direct.

Optional: Extend to cache drives

With its default settings, nested resiliency protects against the loss of multiple capacity drives at the same time, or one server and one
capacity drive at the same time. To extend this protection to cache drives, there's an additional consideration: because cache drives often
provide read and write caching for multiple capacity drives, the only way to ensure you can tolerate the loss of a cache drive when the
other server is down is to simply not cache writes, but that impacts performance.

To address this scenario, Storage Spaces Direct offers the option to automatically disable write caching when one server in a two-server
cluster is down, and then re-enable write caching once the server is back up. To allow routine restarts without performance impact, write
caching isn't disabled until the server has been down for 30 minutes. Once write caching is disabled, the contents of the write cache is
written to capacity devices. After this, the server can tolerate a failed cache device in the online server, though reads from the cache might
be delayed or fail if a cache device fails.

７ Note

For an all cache (single media type) physical system, you don't need to consider automatic disabling of write caching when one server
in a two-server cluster is down. You need to consider this only with the storage bus layer (SBL) cache, which is required only if you are
using HDDs.

(Optional) To automatically disable write caching when one server in a two-server cluster is down, launch PowerShell as Administrator and
run:

PowerShell

Get-StorageSubSystem Cluster* | Set-StorageHealthSetting -Name

"System.Storage.NestedResiliency.DisableWriteCacheOnNodeDown.Enabled" -Value "True"

Once set to True, the cache behavior is:

Situation Cache behavior Can tolerate cache drive loss?

Both servers up Cache reads and writes, full performance Yes

Server down, first 30 minutes Cache reads and writes, full performance No (temporarily)

After first 30 minutes Cache reads only, performance impacted Yes (after the cache has been written to capacity drives)

Next steps
For related topics and other storage management tasks, see also:

Storage Spaces Direct overview

Manage volumes
Nested resiliency for Storage Spaces
Direct
Article • 04/18/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022 and
Windows Server 2019

Nested resiliency is a capability of Storage Spaces Direct in Azure Stack HCI and
Windows Server. It enables a two-server cluster to withstand multiple hardware failures
at the same time without loss of storage availability, so users, apps, and virtual machines
continue to run without disruption. This article explains how nested resiliency works,
provides step-by-step instructions to get started, and answers the most frequently
asked questions.

Before you begin

Consider nested resiliency if:

Your cluster runs one of these operating systems: Azure Stack HCI, version 21H2,
Azure Stack HCI, version 20H2, Windows Server 2022, or Windows Server 2019; and
Your cluster has exactly two server nodes.

You can't use nested resiliency if:

Your cluster runs Windows Server 2016; or

Your cluster has only a single server node or has three or more server nodes.

Why nested resiliency

Volumes that use nested resiliency can stay online and accessible even if multiple
hardware failures happen at the same time, unlike classic two-way mirroring resiliency.
For example, if two drives fail at the same time, or if a server goes down and a drive fails,
volumes that use nested resiliency stay online and accessible. For hyper-converged
infrastructure, this increases uptime for apps and virtual machines; for file server
workloads, this means users have uninterrupted access to their files.
The trade-off is that nested resiliency has lower capacity efficiency than classic two-way
mirroring, meaning you get slightly less usable space. For details, see the Capacity
efficiency following section.

How it works
This section provides the background on nested resiliency for Storage Spaces Direct and
describes the two new resiliency options and their capacity efficiency.

Inspiration: RAID 5+1

RAID 5+1 is an established form of distributed storage resiliency that provides helpful
background for understanding nested resiliency. In RAID 5+1, within each server, local
resiliency is provided by RAID-5, or single parity, to protect against the loss of any single
drive. Then, further resiliency is provided by RAID-1, or two-way mirroring, between the
two servers to protect against the loss of either server.

Resiliency options
Storage Spaces Direct in Azure Stack HCI and Windows Server offers two resiliency
options implemented in software, without the need for specialized RAID hardware:

Nested two-way mirror. Within each server, local resiliency is provided by two-way
mirroring, and then further resiliency is provided by two-way mirroring between
the two servers. It's essentially a four-way mirror, with two copies on each server
that are located on different physical disks. Nested two-way mirroring provides
uncompromising performance: writes go to all copies and reads come from any
copy.

Nested mirror-accelerated parity. Combine nested two-way mirroring, from the

previous image, with nested parity. Within each server, local resiliency for most
data is provided by single bitwise parity arithmetic, except new recent writes that
use two-way mirroring. Then, further resiliency for all data is provided by two-way
mirroring between the servers. New writes to the volume go to the mirror part with
two copies on separate physical disks on each server. As the mirror part of the
volume fills on each server, the oldest data is converted and saved to the parity
part in the background. As a result, each server has the data for the volume either
in two-way mirror or in a single parity structure. This is similar to how mirror-
accelerated parity works—with the difference being that mirror-accelerated parity
requires four servers in the cluster and storage pool, and uses a different parity
algorithm.
Capacity efficiency
Capacity efficiency is the ratio of usable space to volume footprint. It describes the
capacity overhead attributable to resiliency, and depends on the resiliency option you
choose. As a simple example, storing data without resiliency is 100% capacity efficient (1
TB of data takes up 1 TB of physical storage capacity), while two-way mirroring is 50%
efficient (1 TB of data takes up 2 TB of physical storage capacity).

Nested two-way mirror writes four copies of everything. This means that to store 1
TB of data, you need 4 TB of physical storage capacity. Although its simplicity is
appealing, nested two-way mirror's capacity efficiency of 25% is the lowest of any
resiliency option in Storage Spaces Direct.

Nested mirror-accelerated parity achieves higher capacity efficiency, around

35%-40%, that depends on two factors: the number of capacity drives in each
server, and the mix of mirror and parity you specify for the volume. This table
provides a lookup for common configurations:

Capacity drives per server 10% mirror 20% mirror 30% mirror

4 35.7% 34.1% 32.6%

5 37.7% 35.7% 33.9%

 Capacity drives per server 10% mirror 20% mirror 30% mirror

6 39.1% 36.8% 34.7%

7+ 40.0% 37.5% 35.3%

The following is an example of the full math. Suppose we have six capacity drives
in each of two servers, and we want to create one 100 GB volume comprised of 10
GB of mirror and 90 GB of parity. Server-local two-way mirror is 50.0% efficient,
meaning the 10 GB of mirror data takes 20 GB to store on each server. Mirrored to
both servers, its total footprint is 40 GB. Server-local single parity, in this case, is
5/6 = 83.3% efficient, meaning the 90 GB of parity data takes 108 GB to store on
each server. Mirrored to both servers, its total footprint is 216 GB. The total
footprint is thus [(10 GB / 50.0%) + (90 GB / 83.3%)] × 2 = 256 GB, for 39.1%
overall capacity efficiency.

Notice that the capacity efficiency of classic two-way mirroring (about 50%) and
nested mirror-accelerated parity (up to 40%) aren't very different. Depending on
your requirements, the slightly lower capacity efficiency may be well worth the
significant increase in storage availability. You choose resiliency per-volume, so you
can mix nested resiliency volumes and classic two-way mirror volumes within the
same cluster.

Create nested resiliency volumes

You can use familiar storage cmdlets in PowerShell to create volumes with nested
resiliency, as described in the following section.

Step 1: Create storage tier templates (Windows Server

2019 only)
Windows Server 2019 requires you to create new storage tier templates using the New-
StorageTier cmdlet before creating volumes. You only need to do this once, and then

every new volume you create can reference these templates.

 ７ Note

If you're running Windows Server 2022, Azure Stack HCI 21H2, or Azure Stack HCI
20H2, you can skip this step.

Specify the -MediaType of your capacity drives and, optionally, the -FriendlyName of
your choice. Do not modify other parameters.

For example, if your capacity drives are hard disk drives (HDD), launch PowerShell as
Administrator and run the following cmdlets.

To create a NestedMirror tier:

PowerShell

New-StorageTier -StoragePoolFriendlyName S2D* -FriendlyName

NestedMirrorOnHDD -ResiliencySettingName Mirror -MediaType HDD -
NumberOfDataCopies 4

To create a NestedParity tier:

PowerShell

New-StorageTier -StoragePoolFriendlyName S2D* -FriendlyName

NestedParityOnHDD -ResiliencySettingName Parity -MediaType HDD -
NumberOfDataCopies 2 -PhysicalDiskRedundancy 1 -NumberOfGroups 1 -
FaultDomainAwareness StorageScaleUnit -ColumnIsolation PhysicalDisk

If your capacity drives are solid-state drives (SSD), set the -MediaType to SSD instead
and change the -FriendlyName to *OnSSD . Do not modify other parameters.

 Tip

Verify that Get-StorageTier created the tiers successfully.

Step 2: Create nested volumes

Create new volumes using the New-Volume cmdlet.

Nested two-way mirror

 To use nested two-way mirror, reference the NestedMirror tier template and
specify the size. For example:

PowerShell

New-Volume -StoragePoolFriendlyName S2D* -FriendlyName Volume01 -

StorageTierFriendlyNames NestedMirrorOnHDD -StorageTierSizes 500GB

If your capacity drives are solid-state drives (SSD), change -

StorageTierFriendlyNames to *OnSSD .

Nested mirror-accelerated parity

To use nested mirror-accelerated parity, reference both the NestedMirror and

NestedParity tier templates and specify two sizes, one for each part of the volume
(mirror first, parity second). For example, to create one 500 GB volume that's 20%
nested two-way mirror and 80% nested parity, run:

PowerShell

New-Volume -StoragePoolFriendlyName S2D* -FriendlyName Volume02 -

StorageTierFriendlyNames NestedMirrorOnHDD, NestedParityOnHDD -
StorageTierSizes 100GB, 400GB

If your capacity drives are solid-state drives (SSD), change -

StorageTierFriendlyNames to *OnSSD .

Step 3: Continue in Windows Admin Center

Volumes that use nested resiliency appear in Windows Admin Center with clear labeling,
as in the screenshot below. Once they're created, you can manage and monitor them
using Windows Admin Center just like any other volume in Storage Spaces Direct.

Optional: Extend to cache drives

With its default settings, nested resiliency protects against the loss of multiple capacity
drives at the same time, or one server and one capacity drive at the same time. To
extend this protection to cache drives, there's an additional consideration: because
cache drives often provide read and write caching for multiple capacity drives, the only
way to ensure you can tolerate the loss of a cache drive when the other server is down is
to simply not cache writes, but that impacts performance.

To address this scenario, Storage Spaces Direct offers the option to automatically disable
write caching when one server in a two-server cluster is down, and then re-enable write
caching once the server is back up. To allow routine restarts without performance
impact, write caching isn't disabled until the server has been down for 30 minutes. Once
write caching is disabled, the contents of the write cache is written to capacity devices.
After this, the server can tolerate a failed cache device in the online server, though reads
from the cache might be delayed or fail if a cache device fails.

７ Note

For an all cache (single media type) physical system, you don't need to consider
automatic disabling of write caching when one server in a two-server cluster is
down. You need to consider this only with the storage bus layer (SBL) cache, which
is required only if you are using HDDs.

(Optional) To automatically disable write caching when one server in a two-server cluster
is down, launch PowerShell as Administrator and run:

PowerShell

Get-StorageSubSystem Cluster* | Set-StorageHealthSetting -Name

"System.Storage.NestedResiliency.DisableWriteCacheOnNodeDown.Enabled" -Value
"True"

Once set to True, the cache behavior is:

Situation Cache behavior Can tolerate cache drive loss?

Both servers up Cache reads and writes, full Yes

performance

Server down, first 30 Cache reads and writes, full No (temporarily)

minutes performance

After first 30 minutes Cache reads only, Yes (after the cache has been written to
performance impacted capacity drives)
Frequently asked questions
Find answers to frequently asked questions about nested resiliency.

Can I convert an existing volume between two-way

mirror and nested resiliency?
No, volumes can't be converted between resiliency types. For new deployments on
Azure Stack HCI, Windows Server 2022, or Windows Server 2019, decide ahead of time
which resiliency type best fits your needs. If you're upgrading from Windows Server
2016, you can create new volumes with nested resiliency, migrate your data, and then
delete the older volumes.

Can I use nested resiliency with multiple types of capacity

drives?
Yes, just specify the -MediaType of each tier accordingly during step 1 above. For
example, with NVMe, SSD, and HDD in the same cluster, the NVMe provides cache while
the latter two provide capacity: set the NestedMirror tier to -MediaType SSD and the
NestedParity tier to -MediaType HDD . In this case, the parity capacity efficiency depends

on the number of HDD drives only, and you need at least 4 of them per server.

Can I use nested resiliency with three or more servers?

No, only use nested resiliency if your cluster has exactly two servers.

How many drives do I need to use nested resiliency?

The minimum number of drives required for Storage Spaces Direct is four capacity
drives per server node, plus two cache drives per server node (if any). This is unchanged
from Windows Server 2016. There's no other requirement for nested resiliency, and the
recommendation for reserve capacity is unchanged too.

Does nested resiliency change how drive replacement

works?
No.
Does nested resiliency change how server node
replacement works?
No. To replace a server node and its drives, follow this order:

1. Retire the drives in the outgoing server

2. Add the new server, with its drives, to the cluster
3. The storage pool will rebalance
4. Remove the outgoing server and its drives

For details see the Remove servers article.

Next steps
Storage Spaces Direct overview
Understand fault tolerance in Storage Spaces Direct
Plan volumes in Storage Spaces Direct
Create volumes in Storage Spaces Direct
Configure and manage quorum
Article • 06/06/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Azure Stack HCI, versions 21H2 and
20H2

This article provides background and steps to configure and manage the quorum in a
failover cluster.

For information about cluster and storage pool quorums in Storage Spaces Direct on
Azure Stack HCI and Windows Server clusters, see Understanding cluster and pool
quorum.

Understanding quorum
The quorum for a cluster is determined by the number of voting elements that must be
part of active cluster membership for that cluster to start properly or continue running.
For a more detailed explanation, see the understanding cluster and pool quorum doc.

Quorum configuration options

The quorum model in Windows Server is flexible. If you need to modify the quorum
configuration for your cluster, you can use the Configure Cluster Quorum Wizard or the
FailoverClusters Windows PowerShell cmdlets. For steps and considerations to configure
the quorum, see Configure the cluster quorum later in this topic.

The following table lists the three quorum configuration options that are available in the
Configure Cluster Quorum Wizard.

Option Description

Use typical The cluster automatically assigns a vote to each node and dynamically manages
settings the node votes. If it is suitable for your cluster, and there is cluster shared storage
available, the cluster selects a disk witness. This option is recommended in most
cases, because the cluster software automatically chooses a quorum and witness
configuration that provides the highest availability for your cluster.
 Option Description

Add or You can add, change, or remove a witness resource. You can configure a file share
change the or disk witness. The cluster automatically assigns a vote to each node and
quorum dynamically manages the node votes.
witness

Advanced You should select this option only when you have application-specific or site-
quorum specific requirements for configuring the quorum. You can modify the quorum
configuration witness, add or remove node votes, and choose whether the cluster dynamically
and witness manages node votes. By default, votes are assigned to all nodes, and the node
selection votes are dynamically managed.

Depending on the quorum configuration option that you choose and your specific
settings, the cluster will be configured in one of the following quorum modes:

Mode Description

Node Only nodes have votes. No quorum witness is configured. The cluster quorum is
majority (no the majority of voting nodes in the active cluster membership.
witness)

Node Nodes have votes. In addition, a quorum witness has a vote. The cluster quorum is
majority with the majority of voting nodes in the active cluster membership plus a witness vote.
witness (disk A quorum witness can be a designated disk witness or a designated file share
or file share) witness.

No majority No nodes have votes. Only a disk witness has a vote.

(disk witness The cluster quorum is determined by the state of the disk witness. Generally, this
only) mode is not recommended, and it should not be selected because it creates a
single point of failure for the cluster.

The following subsections will give you more information about advanced quorum
configuration settings.

Witness configuration
As a general rule when you configure a quorum, the voting elements in the cluster
should be an odd number. Therefore, if the cluster contains an even number of voting
nodes, you should configure a disk witness or a file share witness. The cluster will be
able to sustain one additional node down. In addition, adding a witness vote enables the
cluster to continue running if half the cluster nodes simultaneously go down or are
disconnected.

A disk witness is usually recommended if all nodes can see the disk. A file share witness
is recommended when you need to consider multisite disaster recovery with replicated
storage. Configuring a disk witness with replicated storage is possible only if the storage
vendor supports read-write access from all sites to the replicated storage. A Disk
Witness isn't supported with Storage Spaces Direct.

The following table provides additional information and considerations about the
quorum witness types.

Witness Description Requirements and recommendations

type

Disk Dedicated LUN Size of LUN must be at least 512 MB

witness that stores a Must be dedicated to cluster use and not assigned to a
copy of the clustered role
cluster Must be included in clustered storage and pass storage
database validation tests
Most useful for Cannot be a disk that is a Cluster Shared Volume (CSV)
clusters with Basic disk with a single volume
shared (not Does not need to have a drive letter
replicated) Can be formatted with NTFS or ReFS
storage Can be optionally configured with hardware RAID for
fault tolerance
Should be excluded from backups and antivirus scanning
A Disk witness isn't supported with Storage Spaces
Direct
Witness Description Requirements and recommendations
type

File SMB file share Must have a minimum of 5 MB of free space

share that is Must be dedicated to the single cluster and not used to
witness configured on store user or application data
a file server Must have write permissions enabled for the computer
running object for the cluster name
Windows
Server
Does not store The following are additional considerations for a file server that
a copy of the hosts the file share witness:
cluster
A single file server can be configured with file share
database
witnesses for multiple clusters.
Maintains
The file server must be on a site that is separate from the
cluster
cluster workload. This allows equal opportunity for any
information
cluster site to survive if site-to-site network
only in a
communication is lost. If the file server is on the same
witness.log file
site, that site becomes the primary site, and it is the only
Most useful for
site that can reach the file share.
multisite
The file server can run on a virtual machine if the virtual
clusters with
machine is not hosted on the same cluster that uses the
replicated
file share witness.
storage
For high availability, the file server can be configured on
a separate failover cluster.

Cloud A witness file See Deploy a cloud witness.

witness stored in Azure
Blob Storage
Does not store
a copy of the
cluster
database
Recommended
when all
servers in the
cluster have a
reliable
Internet
connection.

７ Note

If you configure a file share witness or a cloud witness then shutdown all nodes for
a maintenance or some reason, you have to make sure that start your cluster
 service from a last-man standing node since the latest cluster database is not
stored into those witnesses. See this also.

Node vote assignment

As an advanced quorum configuration option, you can choose to assign or remove
quorum votes on a per-node basis. By default, all nodes are assigned votes. Regardless
of vote assignment, all nodes continue to function in the cluster, receive cluster
database updates, and can host applications.

You might want to remove votes from nodes in certain disaster recovery configurations.
For example, in a multisite cluster, you could remove votes from the nodes in a backup
site so that those nodes do not affect quorum calculations. This configuration is
recommended only for manual failover across sites. For more information, see Quorum
considerations for disaster recovery configurations later in this topic.

The configured vote of a node can be verified by looking up the NodeWeight common
property of the cluster node by using the Get-ClusterNode Windows PowerShell cmdlet.
A value of 0 indicates that the node does not have a quorum vote configured. A value of
1 indicates that the quorum vote of the node is assigned, and it is managed by the
cluster. For more information about management of node votes, see Dynamic quorum
management later in this topic.

The vote assignment for all cluster nodes can be verified by using the Validate Cluster
Quorum validation test.

Additional considerations for node vote assignment

Node vote assignment is not recommended to enforce an odd number of voting

nodes. Instead, you should configure a disk witness or file share witness. For more
information, see Witness configuration later in this topic.
If dynamic quorum management is enabled, only the nodes that are configured to
have node votes assigned can have their votes assigned or removed dynamically.
For more information, see Dynamic quorum management later in this topic.

Dynamic quorum management

In Windows Server 2012, as an advanced quorum configuration option, you can choose
to enable dynamic quorum management by cluster. For more details on how dynamic
quorum works, see this explanation.
With dynamic quorum management, it is also possible for a cluster to run on the last
surviving cluster node. By dynamically adjusting the quorum majority requirement, the
cluster can sustain sequential node shutdowns to a single node.

The cluster-assigned dynamic vote of a node can be verified with the DynamicWeight
common property of the cluster node by using the Get-ClusterNode Windows
PowerShell cmdlet. A value of 0 indicates that the node does not have a quorum vote. A
value of 1 indicates that the node has a quorum vote.

The vote assignment for all cluster nodes can be verified by using the Validate Cluster
Quorum validation test.

Additional considerations for dynamic quorum management

Dynamic quorum management does not allow the cluster to sustain a

simultaneous failure of a majority of voting members. To continue running, the
cluster must always have a quorum majority at the time of a node shutdown or
failure.

If you have explicitly removed the vote of a node, the cluster cannot dynamically
add or remove that vote.

When Storage Spaces Direct is enabled, the cluster can only support two node
failures. This is explained more in the pool quorum section

General recommendations for quorum

configuration
The cluster software automatically configures the quorum for a new cluster, based on
the number of nodes configured and the availability of shared storage. This is usually
the most appropriate quorum configuration for that cluster. However, it is a good idea
to review the quorum configuration after the cluster is created, before placing the
cluster into production. To view the detailed cluster quorum configuration, you can use
the Validate a Configuration Wizard, or the Test-Cluster Windows PowerShell cmdlet, to
run the Validate Quorum Configuration test. In Failover Cluster Manager, the basic
quorum configuration is displayed in the summary information for the selected cluster,
or you can review the information about quorum resources that returns when you run
the Get-ClusterQuorum Windows PowerShell cmdlet.

At any time, you can run the Validate Quorum Configuration test to validate that the
quorum configuration is optimal for your cluster. The test output indicates if a change to
the quorum configuration is recommended and the settings that are optimal. If a
change is recommended, you can use the Configure Cluster Quorum Wizard to apply
the recommended settings.

After the cluster is in production, do not change the quorum configuration unless you
have determined that the change is appropriate for your cluster. You might want to
consider changing the quorum configuration in the following situations:

Adding or evicting nodes

Adding or removing storage
A long-term node or witness failure
Recovering a cluster in a multisite disaster recovery scenario

For more information about validating a failover cluster, see Validate Hardware for a
Failover Cluster.

Configure the cluster quorum

You can configure the cluster quorum settings by using Failover Cluster Manager or the
FailoverClusters Windows PowerShell cmdlets.

） Important

It is usually best to use the quorum configuration that is recommended by the

Configure Cluster Quorum Wizard. We recommend customizing the quorum
configuration only if you have determined that the change is appropriate for your
cluster. For more information, see General recommendations for quorum
configuration in this topic.

Configure the cluster quorum settings

Membership in the local Administrators group on each clustered server, or equivalent, is
the minimum permissions required to complete this procedure. Also, the account you
use must be a domain user account.

７ Note

You can change the cluster quorum configuration without stopping the cluster or
taking cluster resources offline.
Change the quorum configuration in a failover cluster by
using Failover Cluster Manager
1. In Failover Cluster Manager, select or specify the cluster that you want to change.

2. With the cluster selected, under Actions, select More Actions, and then select
Configure Cluster Quorum Settings. The Configure Cluster Quorum Wizard
appears. Select Next.

3. On the Select Quorum Configuration Option page, select one of the three
configuration options and complete the steps for that option. Before you configure
the quorum settings, you can review your choices. For more information about the
options, see Understanding quorum, earlier in this topic.

To allow the cluster to automatically reset the quorum settings that are
optimal for your current cluster configuration, select Use default quorum
configuration and then complete the wizard.

To add or change the quorum witness, select Select the quorum witness, and
then complete the following steps. For information and considerations about
configuring a quorum witness, see Witness configuration earlier in this topic.

a. On the Select Quorum Witness page, select an option to configure a disk

witness or a file share witness. The wizard indicates the witness selection
options that are recommended for your cluster.

７ Note

You can also select Do not configure a quorum witness and then
complete the wizard. If you have an even number of voting nodes in
your cluster, this may not be a recommended configuration.

b. If you select the option to configure a disk witness, on the Configure

Storage Witness page, select the storage volume that you want to assign
as the disk witness, and then complete the wizard.

c. If you select the option to configure a file share witness, on the Configure
File Share Witness page, type or browse to a file share that will be used as
the witness resource, and then complete the wizard.

d. If you select the option to configure a cloud witness, on the Configure

Cloud Witness page, enter your Azure storage account name, Azure
 storage account key and the Azure service endpoint, and then complete
the wizard.

７ Note

This option is available in Windows Server 2016 and above.

To configure quorum management settings and to add or change the

quorum witness, select Advanced quorum configuration, and then complete
the following steps. For information and considerations about the advanced
quorum configuration settings, see Node vote assignment and Dynamic
quorum management earlier in this topic.

a. On the Select Voting Configuration page, select an option to assign votes

to nodes. By default, all nodes are assigned a vote. However, for certain
scenarios, you can assign votes only to a subset of the nodes.

７ Note

You can also select No Nodes. This is generally not recommended,

because it does not allow nodes to participate in quorum voting, and
it requires configuring a disk witness. This disk witness becomes the
single point of failure for the cluster.

b. On the Configure Quorum Management page, you can enable or disable

the Allow cluster to dynamically manage the assignment of node votes
option. Selecting this option generally increases the availability of the
cluster. By default the option is enabled, and it is strongly recommended
to not disable this option. This option allows the cluster to continue
running in failure scenarios that are not possible when this option is
disabled.

７ Note

This option is not present in Windows Server 2016 and above.

c. On the Select Quorum Witness page, select an option to configure a disk

witness, file share witness or a cloud witness. The wizard indicates the
witness selection options that are recommended for your cluster.
 ７ Note

You can also select Do not configure a quorum witness, and then
complete the wizard. If you have an even number of voting nodes in
your cluster, this may not be a recommended configuration.

d. If you select the option to configure a disk witness, on the Configure

Storage Witness page, select the storage volume that you want to assign
as the disk witness, and then complete the wizard.

e. If you select the option to configure a file share witness, on the Configure
File Share Witness page, type or browse to a file share that will be used as
the witness resource, and then complete the wizard.

f. If you select the option to configure a cloud witness, on the Configure

Cloud Witness page, enter your Azure storage account name, Azure
storage account key and the Azure service endpoint, and then complete
the wizard.

７ Note

This option is available in Windows Server 2016 and above.

4. Select Next. Confirm your selections on the confirmation page that appears, and
then select Next.

After the wizard runs and the Summary page appears, if you want to view a report of
the tasks that the wizard performed, select View Report. The most recent report will
remain in the systemroot\Cluster\Reports folder with the name
QuorumConfiguration.mht.

７ Note

After you configure the cluster quorum, we recommend that you run the Validate
Quorum Configuration test to verify the updated quorum settings.

Windows PowerShell equivalent commands

The following examples show how to use the Set-ClusterQuorum cmdlet and other
Windows PowerShell cmdlets to configure the cluster quorum.
The following example changes the quorum configuration on cluster CONTOSO-FC1 to
a simple node majority configuration with no quorum witness.

PowerShell

Set-ClusterQuorum –Cluster CONTOSO-FC1 -NodeMajority

The following example changes the quorum configuration on the local cluster to a node
majority with witness configuration. The disk resource named Cluster Disk 2 is
configured as a disk witness.

PowerShell

Set-ClusterQuorum -NodeAndDiskMajority "Cluster Disk 2"

The following example changes the quorum configuration on the local cluster to a node
majority with witness configuration. The file share resource named \\CONTOSO-FS\fsw is
configured as a file share witness.

PowerShell

Set-ClusterQuorum -NodeAndFileShareMajority "\\fileserver\fsw"

The following example removes the quorum vote from node ContosoFCNode1 on the
local cluster.

PowerShell

(Get-ClusterNode ContosoFCNode1).NodeWeight=0

The following example adds the quorum vote to node ContosoFCNode1 on the local
cluster.

PowerShell

(Get-ClusterNode ContosoFCNode1).NodeWeight=1

The following example enables the DynamicQuorum property of the cluster CONTOSO-
FC1 (if it was previously disabled):

PowerShell

(Get-Cluster CONTOSO-FC1).DynamicQuorum=1
Recover a cluster by starting without quorum
A cluster that does not have enough quorum votes will not start. As a first step, you
should always confirm the cluster quorum configuration and investigate why the cluster
no longer has quorum. This might happen if you have nodes that stopped responding,
or if the primary site is not reachable in a multisite cluster. After you identify the root
cause for the cluster failure, you can use the recovery steps described in this section.

７ Note

If the Cluster service stops because quorum is lost, Event ID 1177 appears in
the system log.
It is always necessary to investigate why the cluster quorum was lost.
It is always preferable to bring a node or quorum witness to a healthy state
(join the cluster) rather than starting the cluster without quorum.

Force start cluster nodes

After you determine that you cannot recover your cluster by bringing the nodes or
quorum witness to a healthy state, forcing your cluster to start becomes necessary.
Forcing the cluster to start overrides your cluster quorum configuration settings and
starts the cluster in ForceQuorum mode.

Forcing a cluster to start when it does not have quorum may be especially useful in a
multisite cluster. Consider a disaster recovery scenario with a cluster that contains
separately located primary and backup sites, SiteA and SiteB. If there is a genuine
disaster at SiteA, it could take a significant amount of time for the site to come back
online. You would likely want to force SiteB to come online, even though it does not
have quorum.

When a cluster is started in ForceQuorum mode, and after it regains sufficient quorum
votes, the cluster automatically leaves the forced state, and it behaves normally. Hence,
it is not necessary to start the cluster again normally. If the cluster loses a node and it
loses quorum, it goes offline again because it is no longer in the forced state. To bring it
back online when it does not have quorum requires forcing the cluster to start without
quorum.

） Important
 After a cluster is force started, the administrator is in full control of the cluster.
The cluster uses the cluster configuration on the node where the cluster is
force started, and replicates it to all other nodes that are available.
If you force the cluster to start without quorum, all quorum configuration
settings are ignored while the cluster remains in ForceQuorum mode. This
includes specific node vote assignments and dynamic quorum management
settings.

Prevent quorum on remaining cluster nodes

After you have force started the cluster on a node, it is necessary to start any remaining
nodes in your cluster with a setting to prevent quorum. A node started with a setting
that prevents quorum indicates to the Cluster service to join an existing running cluster
instead of forming a new cluster instance. This prevents the remaining nodes from
forming a split cluster that contains two competing instances.

This becomes necessary when you need to recover your cluster in some multisite
disaster recovery scenarios after you have force started the cluster on your backup site,
SiteB. To join the force started cluster in SiteB, the nodes in your primary site, SiteA, need
to be started with the quorum prevented.

） Important

After a cluster is force started on a node, we recommend that you always start the
remaining nodes with the quorum prevented.

Here's how to recover the cluster with Failover Cluster Manager:

1. In Failover Cluster Manager, select or specify the cluster you want to recover.

2. With the cluster selected, under Actions, select Force Cluster Start.

Failover Cluster Manager force starts the cluster on all nodes that are reachable.
The cluster uses the current cluster configuration when starting.

７ Note

To force the cluster to start on a specific node that contains a cluster

configuration that you want to use, you must use the Windows PowerShell
cmdlets or equivalent command-line tools as presented after this procedure.
 If you use Failover Cluster Manager to connect to a cluster that is force
started, and you use the Start Cluster Service action to start a node, the node
is automatically started with the setting that prevents quorum.

Windows PowerShell equivalent commands (Start-Clusternode)

The following example shows how to use the Start-ClusterNode cmdlet to force start
the cluster on node ContosoFCNode1.

PowerShell

Start-ClusterNode –Node ContosoFCNode1 –FQ

Alternatively, you can type the following command locally on the node:

PowerShell

Net Start ClusSvc /FQ

The following example shows how to use the Start-ClusterNode cmdlet to start the
Cluster service with the quorum prevented on node ContosoFCNode1.

PowerShell

Start-ClusterNode –Node ContosoFCNode1 –PQ

Alternatively, you can type the following command locally on the node:

PowerShell

Net Start ClusSvc /PQ

Quorum considerations for disaster recovery

configurations
This section summarizes characteristics and quorum configurations for two multisite
cluster configurations in disaster recovery deployments. The quorum configuration
guidelines differ depending on if you need automatic failover or manual failover for
workloads between the sites. Your configuration usually depends on the service level
agreements (SLAs) that are in place in your organization to provide and support
clustered workloads in the event of a failure or disaster at a site.

Automatic failover
In this configuration, the cluster consists of two or more sites that can host clustered
roles. If a failure occurs at any site, the clustered roles are expected to automatically fail
over to the remaining sites. Therefore, the cluster quorum must be configured so that
any site can sustain a complete site failure.

The following table summarizes considerations and recommendations for this

configuration.

Item Description

Number of node votes Should be equal

per site

Node vote assignment Node votes should not be removed because all nodes are equally
important

Dynamic quorum Should be enabled

management

Witness configuration File share witness is recommended, configured in a site that is separate
from the cluster sites

Workloads Workloads can be configured on any of the sites

Additional considerations for automatic failover

Configuring the file share witness in a separate site is necessary to give each site
an equal opportunity to survive. For more information, see Witness configuration
earlier in this topic.

Manual failover
In this configuration, the cluster consists of a primary site, SiteA, and a backup (recovery)
site, SiteB. Clustered roles are hosted on SiteA. Because of the cluster quorum
configuration, if a failure occurs at all nodes in SiteA, the cluster stops functioning. In this
scenario the administrator must manually fail over the cluster services to SiteB and
perform additional steps to recover the cluster.
The following table summarizes considerations and recommendations for this
configuration.

Item Description

Number of Node votes should not be removed from nodes at the primary site, SiteA
node votes Node votes should be removed from nodes at the backup site, SiteB
per site If a long-term outage occurs at SiteA, votes must be assigned to nodes at
SiteB to enable a quorum majority at that site as part of recovery

Dynamic Should be enabled

quorum
management

Witness Configure a witness if there is an even number of nodes at SiteA

configuration If a witness is needed, configure either a file share witness or a disk witness
that is accessible only to nodes in SiteA (sometimes called an asymmetric
disk witness)

Workloads Use preferred owners to keep workloads running on nodes at SiteA

Additional considerations for manual failover

Only the nodes at SiteA are initially configured with quorum votes. This is
necessary to ensure that the state of nodes at SiteB does not affect the cluster
quorum.
Recovery steps can vary depending on if SiteA sustains a temporary failure or a
long-term failure.

More information
Failover Clustering
Failover Clusters Windows PowerShell cmdlets
Upgrade a Storage Spaces Direct cluster
to Windows Server 2019
Article • 01/12/2022

To upgrade a Storage Spaces Direct cluster from Windows Server 2016 to Windows
Server 2019, you have four options using the cluster OS rolling upgrade process. Two
options involve keeping the virtual machines (VMs) running, and two options involve
stopping all VMs. Each option has strengths and weaknesses, so select the option that
best suits the needs of your organization.

To read more about an upgrade option, select a link:

In-place upgrade while VMs are running on each server in the cluster. This option
incurs no VM downtime, but you must wait for storage jobs (mirror repair) to finish
after each server is upgraded.

Clean OS installation while VMs are running on each server in the cluster. This
option incurs no VM downtime, but you must wait for storage jobs (mirror repair)
to finish after each server is upgraded, and you must set up each server and all its
apps and roles again. We recommend this option over an in-place upgrade.

In-place upgrade while VMs are stopped on each server in the cluster. This option
incurs VM downtime, but you don't need to wait for storage jobs (mirror repair), so
it's faster.

Clean OS installation while VMs are stopped on each server in the cluster. This
option incurs VM downtime, but you don't need to wait for storage jobs (mirror
repair), so it's faster. We recommend this option over an in-place upgrade.

Prerequisites and limitations

Before you proceed with an upgrade:

Check that you have usable backups in case any issues occur during the upgrade
process.

Check that your hardware vendor has a BIOS, firmware, and drivers for your servers
to support Windows Server 2019.

It's important to be aware of some limitations with the upgrade process:

 To enable Storage Spaces Direct with Windows Server 2019, use build version
17763.292 or later. You can enable Storage Spaces Direct with Windows Server
2019 by ensuring that the latest Windows Update is applied.

Upgrading is fully supported on Resilient File System (ReFS) volumes, but in

Windows Server 2019, upgraded volumes don't benefit from ReFS enhancements.
Benefits from ReFS, such as improved performance for mirror-accelerated parity,
require a newly created Windows Server 2019 ReFS volume. To create a new
Windows Server 2019 ReFS volume, you must create new volumes by using the
New-Volume cmdlet or Server Manager. Here are some of the ReFS enhancements
in new volumes:

MAP log-bypass: A performance improvement in ReFS that applies only to

clustered (Storage Spaces Direct) systems and doesn't apply to standalone
storage pools.

Compaction: Efficiency improvements in Windows Server 2019 that are specific

to multi-resilient volumes.

Before you upgrade a Windows Server 2016 Storage Spaces Direct cluster server,
we recommend that you put the server in storage maintenance mode. For more
information, see the Event 5120 section of Troubleshoot Storage Spaces Direct.
Although this issue has been fixed in Windows Server 2016, we recommend that
you put each Storage Spaces Direct server in storage maintenance mode during
the upgrade.

A known issue occurs with Software Defined Networking environments that use
Switch Embedded Teaming (SET) switches. The issue involves Hyper-V VM live
migrations from Windows Server 2019 to Windows Server 2016 (live migration to
an earlier version of the operating system). To ensure successful live migrations, we
recommend that you change a VM network setting on VMs that you live-migrate
from Windows Server 2019 to Windows Server 2016. This issue is fixed for
Windows Server 2019 in build 17763.292 and later, but ensure that the latest
Windows Update is applied. For more information, see Microsoft Knowledge Base
article 4476976 .

Because of the known issues described here, some customers might consider building a
new Windows Server 2019 cluster and copying data from the old cluster instead of
upgrading their Windows Server 2016 clusters by using one of the four processes
described in the following sections.

In-place upgrade while VMs are running

This option incurs no VM downtime, but you must wait for storage jobs (mirror repair)
to complete after each server is upgraded. Although individual servers are restarted
sequentially during the upgrade process, the remaining servers in the cluster and all
VMs remain running.

1. Check that all servers in the cluster have installed the latest Windows Update. For
more information, see Windows 10 and Windows Server 2016 update history . At
a minimum, install Microsoft Knowledge Base article 4487006 (February 19,
2019). The build version should be 14393.2828 or later. You can check the build
version by using the ver command or the winver command.

2. If you're using Software Defined Networking with SET switches, open an elevated
PowerShell session and run the following command to disable VM live migration
verification checks on all VMs on the cluster:

PowerShell

Get-ClusterResourceType -Cluster {clusterName} -Name "Virtual Machine"

| `
Set-ClusterParameter -Create SkipMigrationDestinationCheck -Value 1

3. On one cluster server at a time, complete the following steps:

a. Use Hyper-V VM live migration to move running VMs off the server you're
about to upgrade.

b. Pause the cluster server by running the following PowerShell command. Some
internal groups are hidden. We recommend that you do this step with caution. If
you didn't already live-migrate VMs off the server, this cmdlet does that step for
you. In that case, you can skip the previous step, if you prefer.

PowerShell

Suspend-ClusterNode -Drain

c. Place the server in storage maintenance mode:

PowerShell

Get-StorageFaultDomain -type StorageScaleUnit | `

Where FriendlyName -Eq <ServerName> | `
Enable-StorageMaintenanceMode
 d. Run the following cmdlet to check that the OperationalStatus value is In
Maintenance Mode:

PowerShell

Get-PhysicalDisk

e. Perform an upgrade installation of Windows Server 2019 on the server by

running setup.exe and using the Keep personal files and apps option. When
installation is finished, the server remains in the cluster and the cluster service
starts automatically.

f. Check that the newly upgraded server has the latest Windows Server 2019
updates. For more information, see Windows 10 and Windows Server 2019
update history . The build number should be 17763.292 or later. You can check
the build number by using the ver command or the winver command.

g. Remove the server from storage maintenance mode:

PowerShell

Get-StorageFaultDomain -type StorageScaleUnit | `

Where FriendlyName -Eq <ServerName> | `
Disable-StorageMaintenanceMode

h. Resume the server:

PowerShell

Resume-ClusterNode

i. Wait for storage repair jobs to finish and for all disks to return to a healthy state.
The process might take considerable time depending on the number of VMs
running during the server upgrade. To check for a healthy state, run these
commands:

PowerShell

Get-StorageJob
Get-VirtualDisk

4. Upgrade the next server in the cluster.

 5. After all servers are upgraded to Windows Server 2019, use the following
PowerShell cmdlet to update the cluster functional level. After you update the
cluster functional level, you can't go back to the previous cluster functional level.
That is, after you update the cluster functional level, you can't add Windows Server
2016 nodes to the cluster. For more information, see Cluster operating system
rolling upgrade.

PowerShell

Update-ClusterFunctionalLevel

７ Note

Although you have up to four weeks to update the cluster functional level, we
recommend that you update the cluster functional level as soon as possible.

6. After you update the cluster functional level, use the following cmdlet to update
the storage pool. At this point, new cmdlets like Get-ClusterPerf are fully
operational on any server in the cluster.

PowerShell

Update-StoragePool

7. Optionally, upgrade VM configuration levels by stopping each VM, using the

Update-VMVersion cmdlet, and then starting the VMs again.

8. If you're using Software Defined Networking with SET switches and disabled VM
live migration checks as instructed earlier, use the following cmdlet to reenable VM
live verification checks:

PowerShell

Get-ClusterResourceType -Cluster {clusterName} -Name "Virtual Machine"

| `
Set-ClusterParameter SkipMigrationDestinationCheck -Value 0

9. Verify that the upgraded cluster works as expected. Roles should fail over correctly.
If VM live migration is used on the cluster, VMs should successfully live-migrate.

10. Validate the cluster by running cluster validation and examining the cluster
validation report. In an elevated PowerShell session, run the following command:
 PowerShell

Test-Cluster

Clean OS installation while VMs are running

This option incurs no VM downtime, but you must wait for storage jobs (mirror repair)
to complete after each server is upgraded. Although individual servers are restarted
sequentially during the upgrade process, the remaining servers in the cluster and all
VMs remain running.

1. Check that all servers in the cluster are running the latest updates. For more
information, see Windows 10 and Windows Server 2016 update history . At a
minimum, install Microsoft Knowledge Base article 4487006 (February 19, 2019).
The build version should be 14393.2828 or later. You can check the build version
by using the ver command or the winver command.

2. If you're using Software Defined Networking with SET switches, open an elevated
PowerShell session and run the following command to disable VM live migration
verification checks on all VMs on the cluster:

PowerShell

Get-ClusterResourceType -Cluster {clusterName} -Name "Virtual Machine"

| `
Set-ClusterParameter -Create SkipMigrationDestinationCheck -Value 1

3. On one cluster server at a time, complete the following steps:

a. Use Hyper-V VM live migration to move running VMs off the server you're
about to upgrade.

b. Pause the cluster server by running the following PowerShell command. Some
internal groups are hidden. We recommend that you do this step with caution. If
you didn't already live-migrate VMs off the server, this cmdlet does that step for
you. In that case, you can skip the previous step, if you prefer.

PowerShell

Suspend-ClusterNode -Drain

c. Place the server in storage maintenance mode:

 PowerShell

Get-StorageFaultDomain -type StorageScaleUnit | `

Where FriendlyName -Eq <ServerName> | `
Enable-StorageMaintenanceMode

d. Run the following cmdlet to check that the OperationalStatus value is In

Maintenance Mode:

PowerShell

Get-PhysicalDisk

e. Evict the server from the cluster:

PowerShell

Remove-ClusterNode <ServerName>

f. Perform a clean installation of Windows Server 2019 on the server. For a clean
installation, format the system drive, run setup.exe, and use the Nothing option.
You must configure the server identity, roles, features, and applications after
setup finishes and the server restarts.

g. Install the Hyper-V role and Failover-Clustering feature on the server (you can
use the Install-WindowsFeature cmdlet).

h. Install the latest storage and networking drivers for your hardware that are
approved by your server manufacturer to use with Storage Spaces Direct.

i. Check that the newly upgraded server has the latest Windows Server 2019
updates. For more information, see Windows 10 and Windows Server 2019
update history . The build version should be 17763.292 or later. You can check
the build number by using the ver command or the winver command.

j. Rejoin the server to the cluster:

PowerShell

Add-ClusterNode

k. Remove the server from storage maintenance mode:

PowerShell
 Get-StorageFaultDomain -type StorageScaleUnit | `
Where FriendlyName -Eq <ServerName> | `
Disable-StorageMaintenanceMode

l. Wait for storage repair jobs to finish and for all disks to return to a healthy state.
The process might take considerable time depending on the number of VMs
running during the server upgrade. To check for a healthy state, run these
commands:

PowerShell

Get-StorageJob
Get-VirtualDisk

4. Upgrade the next server in the cluster.

5. After all servers are upgraded to Windows Server 2019, use the following
PowerShell cmdlet to update the cluster functional level. After you update the
cluster functional level, you can't go back to the previous cluster functional level.
That is, after you update the cluster functional level, you can't add Windows Server
2016 nodes to the cluster. For more information, see Cluster operating system
rolling upgrade.

PowerShell

Update-ClusterFunctionalLevel

７ Note

Although you have up to four weeks to update the cluster functional level, we
recommend that you update the cluster functional level as soon as possible.

6. After you update the cluster functional level, use the following cmdlet to update
the storage pool. At this point, new cmdlets like Get-ClusterPerf are fully
operational on any server in the cluster.

PowerShell

Update-StoragePool

7. Optionally, upgrade VM configuration levels by stopping each VM, using the

Update-VMVersion cmdlet, and then starting the VMs again.
 8. If you're using Software Defined Networking with SET switches and disabled VM
live migration checks as instructed earlier, use the following cmdlet to reenable VM
live verification checks:

PowerShell

Get-ClusterResourceType -Cluster {clusterName} -Name "Virtual Machine"

| `
Set-ClusterParameter SkipMigrationDestinationCheck -Value 0

9. Verify that the upgraded cluster works as expected. Roles should fail over correctly.
If VM live migration is used on the cluster, VMs should successfully live-migrate.

10. Validate the cluster by running Cluster Validation and examining the cluster
validation report. In an elevated PowerShell session, run the following command:

PowerShell

Test-Cluster

In-place upgrade while VMs are stopped

This option incurs VM downtime, but it might take less time than if you kept the VMs
running during the upgrade because you don't need to wait for storage jobs (mirror
repair) to finish after each server is upgraded. Although individual servers are restarted
sequentially during the upgrade process, the remaining servers in the cluster remain
running.

1. Check that all servers in the cluster are running the latest updates. For more
information, see Windows 10 and Windows Server 2016 update history . At a
minimum, install Microsoft Knowledge Base article 4487006 (February 19, 2019).
The build version should be 14393.2828 or later. You can check the build version
by using the ver command or the winver command.

2. Stop the VMs that are running on the cluster.

3. On one cluster at a time, complete the following steps:

a. Pause the cluster server by opening an elevated PowerShell session and running
the following PowerShell command. Some internal groups are hidden. We
recommend that you do this step with caution.

PowerShell
 Suspend-ClusterNode -Drain

b. Place the server in storage maintenance mode:

PowerShell

Get-StorageFaultDomain -type StorageScaleUnit | `

Where FriendlyName -Eq <ServerName> | `
Enable-StorageMaintenanceMode

c. Run the following cmdlet to check that the OperationalStatus value is In

Maintenance Mode:

PowerShell

Get-PhysicalDisk

d. Perform an upgrade installation of Windows Server 2019 on the server by

running setup.exe and using the Keep personal files and apps option. When
installation is finished, the server remains in the cluster and the cluster service
starts automatically.

e. Check that the newly upgraded server has the latest Windows Server 2019
updates. For more information, see Windows 10 and Windows Server 2019
update history . The build version should be 17763.292 or later. You can check
the build version by using the ver command or the winver command.

f. Remove the server from storage maintenance mode:

PowerShell

Get-StorageFaultDomain -type StorageScaleUnit | `

Where FriendlyName -Eq <ServerName> | `
Disable-StorageMaintenanceMode

g. Resume the server:

PowerShell

Resume-ClusterNode

h. Wait for storage repair jobs to finish and for all disks to return to a healthy state.
The process should be relatively fast because VMs aren't running. Run these
commands to check for a healthy state:
 PowerShell

Get-StorageJob
Get-VirtualDisk

4. Upgrade the next server in the cluster.

5. After all servers are upgraded to Windows Server 2019, use the following
PowerShell cmdlet to update the cluster functional level. After you update the
cluster functional level, you can't go back to the previous cluster functional level.
That is, after you update the cluster functional level, you can't add Windows Server
2016 nodes to the cluster. For more information, see Cluster operating system
rolling upgrade.

PowerShell

Update-ClusterFunctionalLevel

７ Note

Although you have up to four weeks to update the cluster functional level, we
recommend that you update the cluster functional level as soon as possible.

6. After you update the cluster functional level, use the following cmdlet to update
the storage pool. At this point, new cmdlets like Get-ClusterPerf are fully
operational on any server in the cluster.

PowerShell

Update-StoragePool

7. Start the VMs on the cluster and check that they're working properly.

8. Optionally, upgrade VM configuration levels by stopping each VM, using the

Update-VMVersion cmdlet, and then starting the VMs again.

9. Verify that the upgraded cluster works as expected. Roles should fail over correctly.
If VM live migration is used on the cluster, VMs should successfully live-migrate.

10. Validate the cluster by running Cluster Validation and examining the cluster
validation report. In an elevated PowerShell session, run the following command:

PowerShell
 Test-Cluster

Clean OS installation while VMs are stopped

This option incurs VM downtime, but it might take less time than if you kept the VMs
running during the upgrade because you don't need to wait for storage jobs (mirror
repair) to finish after each server is upgraded. Although individual servers are restarted
sequentially during the upgrade process, the remaining servers in the cluster remain
running.

1. Check that all the servers in the cluster are running the latest updates. For more
info, see Windows 10 and Windows Server 2016 update history . At a minimum,
install Microsoft Knowledge Base article 4487006 (February 19, 2019). The build
version should be 14393.2828 or higher. You can check the build version by using
the ver command or the winver command.

2. Stop the VMs that are running on the cluster.

3. One cluster server at a time, complete the following steps:

a. Pause the cluster server by opening an elevated PowerShell session and running
the following PowerShell command. Some internal groups are hidden. We
recommend that you do this step with caution.

PowerShell

Suspend-ClusterNode -Drain

b. Place the server in storage maintenance mode:

PowerShell

Get-StorageFaultDomain -type StorageScaleUnit | `

Where FriendlyName -Eq <ServerName> | `
Enable-StorageMaintenanceMode

c. Run the following cmdlet to check that the OperationalStatus value is In

Maintenance Mode:

PowerShell

Get-PhysicalDisk
d. Evict the server from the cluster:

PowerShell

Remove-ClusterNode <ServerName>

e. Perform a clean installation of Windows Server 2019 on the server. For a clean
installation, format the system drive, run setup.exe, and use the Nothing option.
You must configure the server identity, roles, features, and applications after
setup finishes and the server restarts.

f. Install the Hyper-V role and Failover-Clustering feature on the server (you can
use the Install-WindowsFeature cmdlet).

g. Install the latest storage and networking drivers for your hardware that are
approved by your server manufacturer to use with Storage Spaces Direct.

h. Check that the newly upgraded server has the latest Windows Server 2019
updates. For more information, see Windows 10 and Windows Server 2019
update history . The build version should be 17763.292 or later. You can check
the build version by using the ver command or the winver command.

i. Rejoin the server to the cluster:

PowerShell

Add-ClusterNode

j. Remove the server from storage maintenance mode:

PowerShell

Get-StorageFaultDomain -type StorageScaleUnit | `

Where FriendlyName -Eq <ServerName> | `
Disable-StorageMaintenanceMode

k. Wait for storage repair jobs to finish and for all disks to return to a healthy state.
The process might take considerable time depending on the number of VMs
running during the server upgrade. To check for a healthy state, run these
commands:

PowerShell
 Get-StorageJob
Get-VirtualDisk

4. Upgrade the next server in the cluster.

5. After all servers are upgraded to Windows Server 2019, use the following
PowerShell cmdlet to update the cluster functional level. After you update the
cluster functional level, you can't go back to the previous cluster functional level.
That is, after you update the cluster functional level, you can't add Windows Server
2016 nodes to the cluster. For more information, see Cluster operating system
rolling upgrade.

PowerShell

Update-ClusterFunctionalLevel

７ Note

Although you have up to four weeks to update the cluster functional level, we
recommend that you update the cluster functional level as soon as possible.

6. After you update the cluster functional level, use the following cmdlet to update
the storage pool. At this point, new cmdlets like Get-ClusterPerf are fully
operational on any server in the cluster.

PowerShell

Update-StoragePool

7. Start the VMs on the cluster and check that they're working properly.

8. Optionally, upgrade VM configuration levels by stopping each VM, using the

Update-VMVersion cmdlet, and then starting the VMs again.

9. Verify that the upgraded cluster works as expected. Roles should fail over correctly.
If VM live migration is used on the cluster, VMs should successfully live-migrate.

10. Validate the cluster by running Cluster Validation and examining the cluster
validation report. In an elevated PowerShell session, run the following command:

PowerShell
Test-Cluster
Understand and deploy persistent
memory
Article • 04/18/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019, Windows Server 2016, Windows 10

Persistent memory (or PMem) is a new type of memory technology that retains its
content through power cycles and can be used as top-tier storage, which is why you
may hear people refer to PMem as "storage-class memory" or SCM. This article provides
background on persistent memory and explains how to deploy it as the top storage tier
in Azure Stack HCI and Windows Server.

What is persistent memory?

Persistent memory is a type of non-volatile media that fits in a standard DIMM
(memory) slot. It's slower than DRAM, but provides higher throughput than SSD and
NVMe. Compared to DRAM, persistent memory modules come in much larger capacities
and are less expensive per GB, however they are still more expensive than NVMe.
Memory contents remain even when system power goes down in the event of an
unexpected power loss, user initiated shutdown, or system crash. This means that you
can use persistent memory modules as ultra-fast, persistent storage.

Azure Stack HCI and Windows Server 2019 support using persistent memory as either a
cache or a capacity drive. However, given the pricing model, persistent memory provides
the most value as either a cache or as a small amount of dedicated storage for memory
mapping data. In most cases, persistent memory drives will be automatically used as
cache drives, and anything slower will be used as capacity drives. For more information
about how to set up cache and capacity drives, see Understanding the storage pool
cache and Plan volumes.

Persistent memory concepts

This section describes the basic concepts you'll need to understand in order to deploy
persistent memory in Windows Server and Azure Stack HCI environments to reduce I/O
bottlenecks and improve performance.

Access methods
There are two methods for accessing persistent memory. They are:

Block access, which operates like storage for app compatibility. In this
configuration, data flows through the file system and storage stacks as normal. You
can use this configuration in combination with NTFS and ReFS, and it is
recommended for most use cases.
Direct access (DAX), which operates like memory to get the lowest latency. You
can only use DAX in combination with NTFS. If you don't use DAX correctly, there
is potential for data loss. We strongly recommend that you use DAX with Block
translation table (BTT) turned on to mitigate the risk of torn writes. To learn more,
see Understand and configure DAX.

２ Warning

DAX isn't supported on Azure Stack HCI environments. Azure Stack HCI only
supports block access, with BTT turned on.

Regions
A region is a set of one or more persistent memory modules. Regions are often created
as interleaved sets in which multiple persistent memory modules appear as a single
logical virtual address space to increase throughput. To increase available bandwidth,
adjacent virtual addresses are spread across multiple persistent memory modules.
Regions can usually be created in a server platform's BIOS.

PmemDisks
To use persistent memory as storage, you must define at least one PmemDisk, which is a
virtual hard disk (VHD) on the host that enumerates as a PmemDisk inside a virtual
machine (VM). A PmemDisk is a contiguously addressed range of non-volatile memory
that you can think of like a hard disk partition or LUN. You can create multiple
PmemDisks using Windows PowerShell cmdlets to divide up the available raw capacity.
Each persistent memory module contains a Label Storage Area (LSA) that stores the
configuration metadata.

Block translation table

Unlike solid-state drives, persistent memory modules do not protect against "torn
writes" that can occur in the case of a power failure or system outage, putting data at
risk. BTT mitigates this risk by providing atomic sector update semantics for persistent
memory devices, essentially enabling block-like sector writes so that apps can avoid
mixing old and new data in a failure scenario. We strongly recommend turning on BTT in
nearly all cases. Because BTT is a property of the PmemDisk, it must be turned on when
the PmemDisk is created.

In block access mode, we recommend using BTT because all data will be using block
semantics. BTT is also useful in DAX mode because metadata operations still use block
semantics, even if the application's data operations don't. Even if all application
operations are using memory-mapped files with DAX semantics, torn writes could still
happen for the metadata operations; therefore, turning on BTT is still valuable.

Supported hardware
The following table shows supported persistent memory hardware for Azure Stack HCI
and Windows Server. Persistent memory is fully supported in Windows Server 2019,
including Storage Spaces Direct.

Persistent Memory Technology Windows Azure Stack HCI

Server 2016 v20H2/Windows Server 2019

NVDIMM-N in persistent mode Supported Supported

Intel Optane™ DC Persistent Memory in Not Supported Supported

App Direct Mode

Intel Optane™ DC Persistent Memory in Supported Supported

Memory Mode

Intel Optane DC Persistent Memory supports both Memory (volatile) and App Direct
(persistent) operating modes. To use persistent memory modules as storage, which is
the primary use case for server workloads, you must use App Direct mode. Memory
mode essentially uses persistent memory as slower RAM, which doesn't usually meet the
performance requirements of server workloads. Memory mode is distinct from DAX,
which is a persistent storage volume that can be accessed using memory-like semantics.

Operating mode is often preconfigured by the original device manufacturer.

７ Note

When you restart a system that has multiple Intel® Optane™ persistent memory
modules in App Direct mode that are divided into multiple PmemDisks, you might
lose access to some or all of the related logical storage disks. This issue occurs on
Windows Server 2019 versions that are older than version 1903.
 This loss of access occurs because a persistent memory module is untrained or
otherwise fails when the system starts. In such a case, all the PmemDisks on any
persistent memory module on the system fail, including those that do not
physically map to the failed module.

To restore access to all the PmemDisks, replace the failed module.

If a module fails on Windows Server 2019 version 1903 or newer versions, you lose
access only to PmemDisks that physically map to the affected module; others are
not affected.

Configure persistent memory

If you're using Intel Optane persistent memory, follow the instructions here . If you're
using persistent memory modules from another vendor, consult their documentation.

To create a PmemDisk that supports BTT, use the New-VHD cmdlet:

PowerShell

New-VHD E:\pmemtest.vhdpmem -Fixed -SizeBytes 1GB -AddressAbstractionType

BTT

The VHD extension must be "vhdpmem".

You can also convert a VHD that doesn't have BTT enabled into one that does (and vice-
versa) using the Convert-VHD cmdlet:

PowerShell

Convert-VHD .\pmemtest_nobtt.vhdpmem -AddressAbstractionType BTT -

DestinationPath pmemtest_btt.vhdpmem

After converting, the new VHD will have the same namespace GUID as the original one.
That can lead to problems, especially if they're both attached to the same VM. To create
a new namespace UUID for the converted VHD, use the Set-VHD cmdlet:

PowerShell

Set-VHD -ResetDiskIdentifier .\pmemtest_btt.vhdpmem

Understand interleaved sets

Interleaved sets can usually be created in a server platform's BIOS to make multiple
persistent memory devices appear as a single disk to the host operating system,
increasing throughput for that disk.

７ Note

Windows Server 2016 doesn't support interleaved sets of persistent memory

modules.

Recall that a persistent memory module resides in a standard DIMM (memory) slot,
which puts data closer to the processor. This configuration reduces latency and
improves fetch performance. To further increase throughput, two or more persistent
memory modules create an n-way interleaved set to stripe read/write operations. The
most common configurations are two-way or four-way interleaving.

You can use the Get-PmemDisk PowerShell cmdlet to review the configuration of such
logical disks, as follows:

PowerShell

Get-PmemDisk

DiskNumber Size HealthStatus AtomicityType CanBeRemoved PhysicalDeviceIds

UnsafeShutdownCount
---------- ---- ------------ ------------- ------------ -----------------
-------------------
2 252 GB Healthy None True {20, 120}
0
3 252 GB Healthy None True {1020, 1120}
0

We can see that the logical PMem disk 2 uses the physical devices Id20 and Id120, and
logical PMem disk 3 uses the physical devices Id1020 and Id1120.

To retrieve further information about the interleaved set that a logical drive uses, run the
Get-PmemPhysicalDevice cmdlet:

PowerShell

(Get-PmemDisk)[0] | Get-PmemPhysicalDevice

DeviceId DeviceType HealthStatus OperationalStatus

PhysicalLocation FirmwareRevision Persistent memory size Volatile memory
size
-------- ---------- ------------ ----------------- ---------------
- ---------------- ---------------------- --------------------
 20 Intel INVDIMM device Healthy {Ok} CPU1_DIMM_C1
102005310 126 GB 0 GB
120 Intel INVDIMM device Healthy {Ok} CPU1_DIMM_F1
102005310 126 GB 0 GB

Configure interleaved sets

To configure an interleaved set, run the Get-PmemUnusedRegion cmdlet to review all the
persistent memory regions that are not assigned to a logical persistent memory disk on
the system:

PowerShell

Get-PmemUnusedRegion

RegionId TotalSizeInBytes DeviceId

-------- ---------------- --------
1 270582939648 {20, 120}
3 270582939648 {1020, 1120}

To see all the PMem device information in the system, including device type, location,
health and operational status, and so on, run the Get-PmemPhysicalDevice cmdlet:

PowerShell

Get-PmemPhysicalDevice

DeviceId DeviceType HealthStatus OperationalStatus

PhysicalLocation FirmwareRevision Persistent memory size Volatile

memory size
-------- ---------- ------------ ----------------- ---------------
- ---------------- ---------------------- --------------
1020 Intel INVDIMM device Healthy {Ok} CPU2_DIMM_C1
102005310 126 GB 0 GB
1120 Intel INVDIMM device Healthy {Ok} CPU2_DIMM_F1
102005310 126 GB 0 GB
120 Intel INVDIMM device Healthy {Ok} CPU1_DIMM_F1
102005310 126 GB 0 GB
20 Intel INVDIMM device Healthy {Ok} CPU1_DIMM_C1
102005310 126 GB 0 GB

Because we have an available unused PMem region, we can create new persistent
memory disks. We can use the unused region to create multiple persistent memory disks
by running the following cmdlets:

PowerShell
 Get-PmemUnusedRegion | New-PmemDisk
Creating new persistent memory disk. This may take a few moments.

After this is done, we can see the results by running:

PowerShell

Get-PmemDisk

DiskNumber Size HealthStatus AtomicityType CanBeRemoved PhysicalDeviceIds

UnsafeShutdownCount
---------- ---- ------------ ------------- ------------ -----------------
-------------------
2 252 GB Healthy None True {20, 120}
0
3 252 GB Healthy None True {1020, 1120}
0

It is worth noting that we can run Get-PhysicalDisk | Where MediaType -eq SCM instead
of Get-PmemDisk to get the same results. The newly created persistent memory disk
corresponds one-to-one with drives that appear in PowerShell and in Windows Admin
Center.

Replace persistent memory

If you have to replace a failed module, you have to reprovision the PMem disk (refer to
the steps that we outlined previously).

When you troubleshoot, you might have to use Remove-PmemDisk . This cmdlet removes a
specific persistent memory disk. We can remove all current persistent memory disks by
running the following cmdlets:

PowerShell

Get-PmemDisk | Remove-PmemDisk

cmdlet Remove-PmemDisk at command pipeline position 1

Supply values for the following parameters:
DiskNumber: 2

This will remove the persistent memory disk(s) from the system and will
result in data loss.
Remove the persistent memory disk(s)?
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help
(default is "Y"): Y
Removing the persistent memory disk. This may take a few moments.
 ） Important

Removing a persistent memory disk causes data loss on that disk.

Another cmdlet you might need is Initialize-PmemPhysicalDevice . This cmdlet

initializes the label storage areas on the physical persistent memory devices, and can
clear corrupted label storage information on the devices.

PowerShell

Get-PmemPhysicalDevice | Initialize-PmemPhysicalDevice

This will initialize the label storage area on the physical persistent
memory device(s) and will result in data loss.
Initializes the physical persistent memory device(s)?
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help
(default is "Y"): A
Initializing the physical persistent memory device. This may take a few
moments.
Initializing the physical persistent memory device. This may take a few
moments.
Initializing the physical persistent memory device. This may take a few
moments.
Initializing the physical persistent memory device. This may take a few
moments.

） Important

Initialize-PmemPhysicalDevice causes data loss in persistent memory. Use it only

as a last resort to fix persistent memory-related issues.

Persistent memory in action at Microsoft Ignite

2018
To see some of the benefits of persistent memory, let's look at this video from
Microsoft Ignite 2018.

Any storage system that provides fault tolerance necessarily makes distributed copies of
writes. Such operations must traverse the network and amplify backend write traffic. For
this reason, the absolute largest IOPS benchmark numbers are typically achieved by
measuring reads only, especially if the storage system has common-sense optimizations
to read from the local copy whenever possible. Storage Spaces Direct is optimized to do
so.

When measured by using only read operations, the cluster delivered 13,798,674 IOPS.

If you watch the video closely, you'll notice that what's even more jaw-dropping is the
latency. Even at over 13.7 M IOPS, the file system in Windows is reporting latency that's
consistently less than 40 µs! (That's the symbol for microseconds, one-millionth of a
second.) This speed is an order of magnitude faster than what typical all-flash vendors
proudly advertise today.

Together, Storage Spaces Direct in Windows Server 2019 and Intel® Optane™ DC
persistent memory delivered breakthrough performance. This HCI benchmark of over
13.7M IOPS, accompanied by predictable and extremely low latency, is more than
double our previous industry-leading benchmark of 6.7M IOPS. What's more, this time
we needed only 12 server nodes—25 percent fewer than before.

The test hardware was a 12-server cluster that was configured to use three-way
mirroring and delimited ReFS volumes, 12 x Intel® S2600WFT, 384 GiB memory, 2 x 28-
core "CascadeLake," 1.5 TB Intel® Optane™ DC persistent memory as cache, 32 TB
NVMe (4 x 8 TB Intel® DC P4510) as capacity, 2 x Mellanox ConnectX-4 25 Gbps.

The following table shows the full performance numbers.

Benchmark Performance

4K 100% random read 13.8 million IOPS

4K 90/10% random read/write 9.45 million IOPS

2 MB sequential read 549 GB/s throughput

Next steps
For related information, see also:

Storage Spaces Direct overview

Persistent memory health management
Understanding the storage pool cache
Understand and configure DAX
Article • 05/19/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows 10

Direct access (DAX) treats persistent memory devices as byte-addressable memory to

get the lowest latency, providing direct access to byte-addressable memory rather than
following normal file system block I/O conventions. The app directly modifies the
persistent memory, bypassing the software overhead of the I/O stack. When used
properly by DAX-aware code (that is, by memory mapping data), this can provide
significant performance benefits. However, DAX has a number of issues, and it won’t
provide significant benefits without DAX-aware code.

In Windows Server 2019 and later, you can only create a DAX volume on a Storage
Spaces or Storage Spaces Direct configuration that uses a single persistent memory disk,
with no parity and no redundancy. You can't use DAX across more than one persistent
memory disk, and you can only use DAX with NTFS.

７ Note

DAX isn't supported on Azure Stack HCI environments.

DAX and block translation table (BTT)

If you don't use DAX correctly, there is potential for data loss. We strongly recommend
that you use DAX in conjunction with the block translation table (BTT) to protect against
"torn writes" that can occur in the case of a power failure or system outage. BTT
mitigates this risk by providing atomic sector update semantics for persistent memory
devices, essentially enabling block-like sector writes so that apps can avoid mixing old
and new data in a failure scenario.

Although we recommend enabling BTT on most DAX volumes to avoid subjecting the
NTFS metadata to torn write issues, the downside of BTT is that it can impact the usage
of "large" and "huge" memory pages on a DAX volume because remappings will occur
for metadata operations. If you want to use large and huge memory pages for your
memory mapped sections, do not turn on BTT.
Create a DAX volume by using Windows
PowerShell
Because DAX is a property of the file system, it must be specified when formatting an
NTFS volume.

After creating a volume, use the -IsDax switch with the Format-Volume cmdlet to format
the volume to be DAX-enabled.

PowerShell

Format-Volume -IsDax:$true

The following code snippet creates a DAX volume on a persistent memory disk.

PowerShell

# Here we use the first pmem disk to create the volume as an example
$disk = (Get-PmemDisk)[0] | Get-PhysicalDisk | Get-Disk
# Initialize the disk to GPT if it is not initialized
If ($disk.partitionstyle -eq "RAW") {$disk | Initialize-Disk -PartitionStyle
GPT}
# Create a partition with drive letter 'S' (can use any available drive
letter)
$disk | New-Partition -DriveLetter S -UseMaximumSize

DiskPath: \\?
\scmld#ven_8980&dev_097a&subsys_89804151&rev_0018#3&1b1819f6&0&03018089fb634
94db728d8418b3cbbf549997891#{53f56307-b6
bf-11d0-94f2-00a0c91efb8b}

PartitionNumber DriveLetter Offset

Size Type
--------------- ----------- ------
---- ----
2 S 16777216
251.98 GB Basic

# Format the volume with drive letter 'S' to DAX Volume

Format-Volume -FileSystem NTFS -IsDax:$true -DriveLetter S

DriveLetter FriendlyName FileSystemType DriveType HealthStatus

OperationalStatus SizeRemaining Size
----------- ------------ -------------- --------- ------------ -------------
---- ------------- ----
S NTFS Fixed Healthy OK
251.91 GB 251.98 GB

# Verify the volume is DAX enabled

Get-Partition -DriveLetter S | fl
 UniqueId : {00000000-0000-0000-0000-
000100000000}SCMLD\VEN_8980&DEV_097A&SUBSYS_89804151&REV_0018\3&1B1819F6&0&0
3018089F
B63494DB728D8418B3CBBF549997891:WIN-8KGI228ULGA
AccessPaths : {S:\, \\?\Volume{cf468ffa-ae17-4139-a575-
717547d4df09}\}
DiskNumber : 2
DiskPath : \\?
\scmld#ven_8980&dev_097a&subsys_89804151&rev_0018#3&1b1819f6&0&03018089fb634
94db728d8418b3cbbf549997891#{5
3f56307-b6bf-11d0-94f2-00a0c91efb8b}
DriveLetter : S
Guid : {cf468ffa-ae17-4139-a575-717547d4df09}
IsActive : False
IsBoot : False
IsHidden : False
IsOffline : False
IsReadOnly : False
IsShadowCopy : False
IsDAX : True # <- True: DAX enabled
IsSystem : False
NoDefaultDriveLetter : False
Offset : 16777216
OperationalStatus : Online
PartitionNumber : 2
Size : 251.98 GB
Type : Basic

Next steps
For related information, see also:

Understand and deploy persistent memory

Manage Azure Stack HCI
Article • 03/15/2023

Applies to: Windows Admin Center, Windows Admin Center Preview

What is Hyper-Converged Infrastructure

Hyper-Converged Infrastructure consolidates software-defined compute, storage, and
networking into one cluster to provide high-performance, cost-effective, and easily
scalable virtualization. This capability was introduced in Windows Server 2016 with
Storage Spaces Direct, Software Defined Networking and Hyper-V.

 Tip

Looking to acquire Hyper-Converged Infrastructure? Microsoft recommends these

Windows Server Software-Defined solutions from our partners. They are
designed, assembled, and validated against our reference architecture to ensure
compatibility and reliability, so you get up and running quickly.

） Important

Some of the features described in this article are only available in Windows Admin
Center Preview. How do I get this version?

What is Windows Admin Center?

Windows Admin Center is the next-generation management tool for Windows Server,
the successor to traditional "in-box" tools like Server Manager. It's free and can be
installed and used without an Internet connection. You can use Windows Admin Center
to manage and monitor Hyper-Converged Infrastructure running Windows Server 2016
or Windows Server 2019.
Key features
Highlights of Windows Admin Center for Hyper-Converged Infrastructure include:

Unified single-pane-of-glass for compute, storage, and soon networking. View

your virtual machines, host servers, volumes, drives, and more within one purpose-
built, consistent, interconnected experience.
Create and manage Storage Spaces and Hyper-V virtual machines. Radically
simple workflows to create, open, resize, and delete volumes, or to create, start,
connect to, and move virtual machines, and much more.
Powerful cluster-wide monitoring. The dashboard graphs memory and CPU
usage, storage capacity, IOPS, throughput, and latency in real-time, across every
server in the cluster, and with clear alerts when something's not right.
Software Defined Networking (SDN) support. Manage and monitor virtual
networks, subnets, connect virtual machines to virtual networks, and monitor SDN
infrastructure.

Windows Admin Center for Hyper-Converged Infrastructure is being actively developed

by Microsoft. It receives frequent updates that improve existing features and add new
features.

Before you start

To manage your cluster as Hyper-Converged Infrastructure in Windows Admin Center, it
needs to be running Windows Server 2016 or Windows Server 2019 and have Hyper-V
and Storage Spaces Direct enabled. Optionally, it can also have Software Defined
Networking enabled and managed through Windows Admin Center.
  Tip

Windows Admin Center also offers a general-purpose management experience for

any cluster supporting any workload, available for Windows Server 2012 and later.
If this sounds like a better fit, when you add your cluster to Windows Admin Center,
select Failover Cluster instead of Hyper-Converged Cluster.

Prepare your Windows Server 2016 cluster for Windows

Admin Center
Windows Admin Center for Hyper-Converged Infrastructure depends on management
APIs added after Windows Server 2016 was released. Before you can manage your
Windows Server 2016 cluster with Windows Admin Center, you'll need to perform these
two steps:

1. Verify that every server in the cluster has installed the 2018-05 Cumulative Update
for Windows Server 2016 (KB4103723) or later. To download and install this
update, go to Settings > Update & Security > Windows Update and select Check
online for updates from Microsoft Update.
2. Run the following PowerShell cmdlet as Administrator on the cluster:

PowerShell

Add-ClusterResourceType -Name "SDDC Management" -dll

"$env:SystemRoot\Cluster\sddcres.dll" -DisplayName "SDDC Management"

 Tip

You only need to run the cmdlet once, on any server in the cluster. You can run it
locally in Windows PowerShell or use Credential Security Service Provider (CredSSP)
to run it remotely. Depending on your configuration, you may not be able to run
this cmdlet from within Windows Admin Center.

Prepare your Windows Server 2019 cluster for Windows

Admin Center
If your cluster runs Windows Server 2019, the steps above are not necessary. Just add
the cluster to Windows Admin Center as described in the next section and you're good
to go!
Configure Software Defined Networking (Optional)
You can configure your Hyper-Converged Infrastructure running Windows Server 2016
or 2019 to use Software Defined Networking (SDN) with the following steps:

1. Prepare the VHD of the OS which is the same OS you installed on the hyper-
converged infrastructure hosts. This VHD will be used for all NC/SLB/GW VMs.
2. Download all the folder and files under SDN Express from
https://github.com/Microsoft/SDN/tree/master/SDNExpress .
3. Prepare a different VM using the deployment console. This VM should be able to
access the SDN hosts. Also, the VM should be have the RSAT Hyper-V tool
installed.
4. Copy everything you downloaded for SDN Express to the deployment console VM.
And share this SDNExpress folder. Make sure every host can access the
SDNExpress shared folder, as defined in the configuration file line 8:

\\$env:Computername\SDNExpress

5. Copy the VHD of the OS to the images folder under the SDNExpress folder on the
deployment console VM.
6. Modify the SDN Express configuration with your environment setup. Finish the
following two steps after you modify the SDN Express configuration based on your
environment information.
7. Run PowerShell with Admin privilege to deploy SDN:

PowerShell

.\SDNExpress.ps1 -ConfigurationDataFile .\your_fabricconfig.PSD1 -

verbose

The deployment will take around 30 – 45 minutes.

Get started
Once your Hyper-Converged Infrastructure is deployed, you can manage it using
Windows Admin Center.

Install Windows Admin Center

If you haven't already, download and install Windows Admin Center. The fastest way to
get up and running is to install it on your Windows 10 computer and manage your
servers remotely. This takes less than five minutes. Download now or learn more about
other installation options.

Add Hyper-Converged Cluster

To add your cluster to Windows Admin Center:

1. Click + Add under All Connections.

2. Choose to add a Hyper-Converged Cluster Connection.
3. Type the name of the cluster and, if prompted, the credentials to use.
4. Click Add to finish.

The cluster will be added to your connections list. Click it to launch the Dashboard.

Add SDN-enabled Hyper-Converged Cluster (Windows

Admin Center Preview)
The latest Windows Admin Center Preview supports Software Defined Networking
management for Hyper-Converged Infrastructure. By adding a Network Controller REST
URI to your Hyper-Converged cluster connection, you can use Hyper-converged Cluster
Manager to manage your SDN resources and monitor SDN infrastructure.

1. Click + Add under All Connections.

2. Choose to add a Hyper-Converged Cluster Connection.
3. Type the name of the cluster and, if prompted, the credentials to use.
 4. Check Configure the Network Controller to continue.
5. Enter the Network Controller URI and click Validate.
6. Click Add to finish.

The cluster will be added to your connections list. Click it to launch the Dashboard.

） Important

SDN environments with Kerberos authentication for Northbound communication

are currently not supported.

Frequently asked questions

Are there differences between managing Windows Server

2016 and Windows Server 2019?
Yes. Windows Admin Center for Hyper-Converged Infrastructure receives frequent
updates that improve the experience for both Windows Server 2016 and Windows
Server 2019. However, certain new features are only available for Windows Server 2019 –
for example, the toggle switch for deduplication and compression.

Can I use Windows Admin Center to manage Storage

Spaces Direct for other use cases (not hyper-converged),
such as converged Scale-Out File Server (SoFS) or
Microsoft SQL Server?
Windows Admin Center for Hyper-Converged Infrastructure does not provide
management or monitoring options specifically for other use cases of Storage Spaces
Direct – for example, it can't create file shares. However, the Dashboard and core
features, such as creating volumes or replacing drives, work for any Storage Spaces
Direct cluster.

What's the difference between a Failover Cluster and a

Hyper-Converged Cluster?
In general, the term "hyper-converged" refers to running Hyper-V and Storage Spaces
Direct on the same clustered servers to virtualize compute and storage resources. In the
context of Windows Admin Center, when you click + Add from the connections list, you
can choose between adding a Failover Cluster connection or a Hyper-Converged
Cluster connection:

The Failover Cluster connection is the successor to the Failover Cluster Manager
desktop app. It provides a familiar, general-purpose management experience for
any cluster supporting any workload, including Microsoft SQL Server. It is available
for Windows Server 2012 and later.

The Hyper-Converged Cluster connection is an all-new experience tailored for

Storage Spaces Direct and Hyper-V. It features the Dashboard and emphasizes
charts and alerts for monitoring. It is available for Windows Server 2016 and
Windows Server 2019.

Why do I need the latest cumulative update for Windows

Server 2016?
Windows Admin Center for Hyper-Converged Infrastructure depends on management
APIs developed since Windows Server 2016 was released. These APIs are added in the
2018-05 Cumulative Update for Windows Server 2016 (KB4103723) , available as of
May 8, 2018.

How much does it cost to use Windows Admin Center?

Windows Admin Center has no additional cost beyond Windows.

You can use Windows Admin Center (available as a separate download) with valid
licenses of Windows Server or Windows 10 at no additional cost - it's licensed under a
Windows Supplemental EULA.
Does Windows Admin Center require System Center?
No.

Does it require an Internet connection?

No.

Although Windows Admin Center offers powerful and convenient integration with the
Microsoft Azure cloud, the core management and monitoring experience for Hyper-
Converged Infrastructure is completely on-premises. It can be installed and used
without an Internet connection.

Things to try
If you're just getting started, here are some quick tutorials to help you learn how
Windows Admin Center for Hyper-Converged Infrastructure is organized and works.
Please exercise good judgment and be careful with production environments. These
videos were recorded with Windows Admin Center version 1804 and an Insider Preview
build of Windows Server 2019.

Manage Storage Spaces Direct volumes

(0:37) How to create a three-way mirror volume
(1:17) How to create a mirror-accelerated parity volume
(1:02) How to open a volume and add files
(0:51) How to turn on deduplication and compression
(0:47) How to expand a volume
(0:26) How to delete a volume

Create volume, three-way mirror Create volume, mirror-accelerated parity

How to create a three-w…

three-w… How to create a mirror-acce…
mirror-acce…

Open volume and add files Turn on deduplication and compression

 How to open a volume … How to turn on deduplicatio…
deduplicatio…

Expand volume Delete volume

How to expand a volum…

volum… How to delete a volume in …

Create a new virtual machine

1. Click the Virtual Machines tool from the left side navigation pane.
2. At the top of the Virtual Machines tool, choose the Inventory tab, then click New
to create a new virtual machine.
3. Enter the virtual machine name and choose between generation 1 and 2 virtual
machines.
4. You can then choose which host to initially create the virtual machine on or use the
recommended host.
5. Choose a path for the virtual machine files. Choose a volume from the dropdown
list or click Browse to choose a folder using the folder picker. The virtual machine
configuration files and virtual hard disk file will be saved in a single folder under
the \Hyper-V\[virtual machine name] path of the selected volume or path.
6. Choose the number of virtual processors, whether you want nested virtualization
enabled, configure memory settings, network adapters, virtual hard disks and
choose whether you want to install an operating system from an .iso image file or
from the network.
7. Click Create to create the virtual machine.
8. Once the virtual machine is created and appears in the virtual machine list, you can
start the virtual machine.
 9. Once the virtual machine is started, you can connect to the virtual machine's
console via "VMConnect" to install the operating system. Select the virtual machine
from the list, click More > Connect to download the .rdp file. Open the .rdp file in
the Remote Desktop Connection app. Since this is connecting to the virtual
machine's console, you will need to enter the Hyper-V host's admin credentials.

Learn more about virtual machine management with Windows Admin Center.

Pause and safely restart a server

1. From the Dashboard, select Servers from the navigation on the left side or by
clicking the VIEW SERVERS > link on the tile in the lower right corner of the
Dashboard.
2. At the top, switch from Summary to the Inventory tab.
3. Select a server by clicking its name to open the Server detail page.
4. Click Pause server for maintenance. If it's safe to proceed, this will move virtual
machines to other servers in the cluster. The server will have status Draining while
this happens. If you want, you can watch the virtual machines move on the Virtual
machines > Inventory page, where their host server is shown clearly in the grid.
When all virtual machines have moved, the server status will be Paused.
5. Click Manage server to access all the per-server management tools in Windows
Admin Center.
6. Click Restart, then Yes. You'll be kicked back to the connections list.
7. Back on the Dashboard, the server is colored red while it's down.
8. Once it's back up, navigate again the Server page and click Resume server from
maintenance to set the server status back to simply Up. In time, virtual machines
will move back – no user action is required.

Replace a failed drive

1. When a drive fails, an alert appears in the upper left Alerts area of the Dashboard.
2. You can also select Drives from the navigation on the left side or click the VIEW
DRIVES > link on the tile in the lower right corner to browse drives and see their
status for yourself. In the Inventory tab, the grid supports sorting, grouping, and
keyword search.
3. From the Dashboard, click the alert to see details, like the drive's physical location.
4. To learn more, click the Go to drive shortcut to the Drive detail page.
5. If your hardware supports it, you can click Turn light on/off to control the drive's
indicator light.
6. Storage Spaces Direct automatically retires and evacuates failed drives. When this
has happened, the drive status is Retired, and its storage capacity bar is empty.
 7. Remove the failed drive and insert its replacement.
8. In Drives > Inventory, the new drive will appear. In time, the alert will clear,
volumes will repair back to OK status, and storage will rebalance onto the new
drive – no user action is required.

Manage virtual networks (SDN-enabled HCI clusters

using Windows Admin Center Preview)
1. Select Virtual Networks from the navigation on the left side.
2. Click New to create a new virtual network and subnets, or choose an existing
virtual network and click Settings to modify its configuration.
3. Click an existing virtual network to view VM connections to the virtual network
subnets and access control lists applied to virtual network subnets.

Connect a virtual machine to a virtual network (SDN-

enabled HCI clusters using Windows Admin Center
Preview)
1. Select Virtual Machines from the navigation on the left side.
2. Choose an existing virtual machine > Click Settings > Open the Networks tab in
Settings.
3. Configure the Virtual Network and Virtual Subnet fields to connect the virtual
machine to a virtual network.

You can also configure the virtual network when creating a virtual machine.
Monitor Software Defined Networking infrastructure
(SDN-enabled HCI clusters using Windows Admin Center
Preview)
1. Select SDN Monitoring from the navigation on the left side.
2. View detailed information about the health of Network Controller, Software Load
Balancer, Virtual Gateway and monitor your Virtual Gateway Pool, Public and
Private IP Pool usage and SDN host status.

GPU management
1. Select GPUs from the navigation on the left side.
 2. View the available GPUs from your clustered VMs and provide GPU acceleration to
workloads running in the clustered VMs through Discrete Device Assignment
(DDA). Learn more about using GPUs with clustered VMs.

Security tool
1. Select Security from the navigation on the left side.
2. Select the Secured-core tab and enable or disable the available security features.

Give us feedback
It's all about your feedback! The most important benefit of frequent updates is to hear
what's working and what needs to be improved. Here are some ways to let us know
what you're thinking:
 Submit ideas for feature requests and provide feedback
Join the Windows Admin Center forum on Microsoft Tech Community
Tweet to @servermgmt

Additional References
Windows Admin Center
Storage Spaces Direct
Hyper-V
Software Defined Networking
Adding servers or drives to Storage
Spaces Direct
Article • 02/15/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

This topic describes how to add servers or drives to Storage Spaces Direct.

Adding servers
Adding servers, often called scaling out, adds storage capacity and can improve storage
performance and unlock better storage efficiency. If your deployment is hyper-
converged, adding servers also provides more compute resources for your workload.

Typical deployments are simple to scale out by adding servers. There are just two steps:

1. Run the cluster validation wizard using the Failover Cluster snap-in or with the
Test-Cluster cmdlet in PowerShell (run as Administrator). Include the new server
<NewNode> you wish to add.

PowerShell

Test-Cluster -Node <Node>, <Node>, <Node>, <NewNode> -Include "Storage

Spaces Direct", Inventory, Network, "System Configuration"

This confirms that the new server is running Windows Server 2016 Datacenter
Edition, has joined the same Active Directory Domain Services domain as the
 existing servers, has all the required roles and features, and has networking
properly configured.

） Important

If you are re-using drives that contain old data or metadata you no longer
need, clear them using Disk Management or the Reset-PhysicalDisk cmdlet. If
old data or metadata is detected, the drives aren't pooled.

2. Run the following cmdlet on the cluster to finish adding the server:

Add-ClusterNode -Name NewNode

７ Note

Automatic pooling depends on you having only one pool. If you've circumvented
the standard configuration to create multiple pools, you will need to add new
drives to your preferred pool yourself using Add-PhysicalDisk.

From 2 to 3 servers: unlocking three-way mirroring

With two servers, you can only create two-way mirrored volumes (compare with
distributed RAID-1). With three servers, you can create three-way mirrored volumes for
better fault tolerance. We recommend using three-way mirroring whenever possible.
Two-way mirrored volumes cannot be upgraded in-place to three-way mirroring.
Instead, you can create a new volume and migrate (copy, such as by using Storage
Replica) your data to it, and then remove the old volume.

To begin creating three-way mirrored volumes, you have several good options. You can
use whichever you prefer.

Option 1
Specify PhysicalDiskRedundancy = 2 on each new volume upon creation.

PowerShell

New-Volume -FriendlyName <Name> -FileSystem CSVFS_ReFS -

StoragePoolFriendlyName S2D* -Size <Size> -PhysicalDiskRedundancy 2

Option 2
Instead, you can set PhysicalDiskRedundancyDefault = 2 on the pool's
ResiliencySetting object named Mirror. Then, any new mirrored volumes will
automatically use three-way mirroring even if you don't specify it.

PowerShell

Get-StoragePool S2D* | Get-ResiliencySetting -Name Mirror | Set-

ResiliencySetting -PhysicalDiskRedundancyDefault 2

New-Volume -FriendlyName <Name> -FileSystem CSVFS_ReFS -

StoragePoolFriendlyName S2D* -Size <Size>

Option 3

Set PhysicalDiskRedundancy = 2 on the StorageTier template called Capacity, and then

create volumes by referencing the tier.

PowerShell

Set-StorageTier -FriendlyName Capacity -PhysicalDiskRedundancy 2

New-Volume -FriendlyName <Name> -FileSystem CSVFS_ReFS -

StoragePoolFriendlyName S2D* -StorageTierFriendlyNames Capacity -
StorageTierSizes <Size>
From 3 to 4 servers: unlocking dual parity

With four servers, you can use dual parity, also commonly called erasure coding
(compare to distributed RAID-6). This provides the same fault tolerance as three-way
mirroring, but with better storage efficiency. To learn more, see Fault tolerance and
storage efficiency.

If you're coming from a smaller deployment, you have several good options to begin
creating dual parity volumes. You can use whichever you prefer.

Option 1
Specify PhysicalDiskRedundancy = 2 and ResiliencySettingName = Parity on each new
volume upon creation.

PowerShell

New-Volume -FriendlyName <Name> -FileSystem CSVFS_ReFS -

StoragePoolFriendlyName S2D* -Size <Size> -PhysicalDiskRedundancy 2 -
ResiliencySettingName Parity

Option 2
Set PhysicalDiskRedundancy = 2 on the pool's ResiliencySetting object named Parity.
Then, any new parity volumes will automatically use dual parity even if you don't specify
it

PowerShell
 Get-StoragePool S2D* | Get-ResiliencySetting -Name Parity | Set-
ResiliencySetting -PhysicalDiskRedundancyDefault 2

New-Volume -FriendlyName <Name> -FileSystem CSVFS_ReFS -

StoragePoolFriendlyName S2D* -Size <Size> -ResiliencySettingName Parity

With four servers, you can also begin using mirror-accelerated parity, where an
individual volume is part mirror and part parity.

For this, you will need to update your StorageTier templates to have both Performance
and Capacity tiers, as they would be created if you had first run Enable-ClusterS2D at
four servers. Specifically, both tiers should have the MediaType of your capacity devices
(such as SSD or HDD) and PhysicalDiskRedundancy = 2. The Performance tier should be
ResiliencySettingName = Mirror, and the Capacity tier should be
ResiliencySettingName = Parity.

Option 3
You may find it easiest to simply remove the existing tier template and create the two
new ones. This will not affect any pre-existing volumes which were created by referring
the tier template: it's just a template.

PowerShell

Remove-StorageTier -FriendlyName Capacity

New-StorageTier -StoragePoolFriendlyName S2D* -MediaType HDD -

PhysicalDiskRedundancy 2 -ResiliencySettingName Mirror -FriendlyName
Performance
New-StorageTier -StoragePoolFriendlyName S2D* -MediaType HDD -
PhysicalDiskRedundancy 2 -ResiliencySettingName Parity -FriendlyName
Capacity

That's it! You are now ready to create mirror-accelerated parity volumes by referencing
these tier templates.

Example

PowerShell

New-Volume -FriendlyName "Sir-Mix-A-Lot" -FileSystem CSVFS_ReFS -

StoragePoolFriendlyName S2D* -StorageTierFriendlyNames Performance, Capacity
-StorageTierSizes <Size, Size>
Beyond 4 servers: greater parity efficiency
As you scale beyond four servers, new volumes can benefit from ever-greater parity
encoding efficiency. For example, between six and seven servers, efficiency improves
from 50.0% to 66.7% as it becomes possible to use Reed-Solomon 4+2 (rather than
2+2). There are no steps you need to take to begin enjoying this new efficiency; the best
possible encoding is determined automatically each time you create a volume.

However, any pre-existing volumes will not be "converted" to the new, wider encoding.
One good reason is that to do so would require a massive calculation affecting literally
every single bit in the entire deployment. If you would like pre-existing data to become
encoded at the higher efficiency, you can migrate it to new volume(s).

For more details, see Fault tolerance and storage efficiency.

Adding servers when using chassis or rack fault tolerance

If your deployment uses chassis or rack fault tolerance, you must specify the chassis or
rack of new servers before adding them to the cluster. This tells Storage Spaces Direct
how best to distribute data to maximize fault tolerance.

1. Create a temporary fault domain for the node by opening an elevated PowerShell
session and then using the following command, where <NewNode> is the name of
the new cluster node:

PowerShell

New-ClusterFaultDomain -Type Node -Name <NewNode>

2. Move this temporary fault-domain into the chassis or rack where the new server is
located in the real world, as specified by <ParentName>:

PowerShell

Set-ClusterFaultDomain -Name <NewNode> -Parent <ParentName>

For more information, see Fault domain awareness in Windows Server 2016.

3. Add the server to the cluster as described in Adding servers. When the new server
joins the cluster, it's automatically associated (using its name) with the placeholder
fault domain.
Adding drives
Adding drives, also known as scaling up, adds storage capacity and can improve
performance. If you have available slots, you can add drives to each server to expand
your storage capacity without adding servers. You can add cache drives or capacity
drives independently at any time.

） Important

We strongly recommend that all servers have identical storage configurations.

To scale up, connect the drives and verify that Windows discovers them. They should
appear in the output of the Get-PhysicalDisk cmdlet in PowerShell with their CanPool
property set to True. If they show as CanPool = False, you can see why by checking their
CannotPoolReason property.

PowerShell

Get-PhysicalDisk | Select SerialNumber, CanPool, CannotPoolReason

Within a short time, eligible drives will automatically be claimed by Storage Spaces
Direct, added to the storage pool, and volumes will automatically be redistributed
evenly across all the drives . At this point, you're finished and ready to extend your
volumes or create new ones.

If the drives don't appear, manually scan for hardware changes. This can be done using
Device Manager, under the Action menu. If they contain old data or metadata, consider
reformatting them. This can be done using Disk Management or with the Reset-
PhysicalDisk cmdlet.

７ Note

Automatic pooling depends on you having only one pool. If you've circumvented
the standard configuration to create multiple pools, you will need to add new
drives to your preferred pool yourself using Add-PhysicalDisk.

Optimizing drive usage after adding drives or

servers
Over time, as drives are added or removed, the distribution of data among the drives in
the pool can become uneven. In some cases, this can result in certain drives becoming
full while other drives in pool have much lower consumption.

To help keep drive allocation even across the pool, Storage Spaces Direct automatically
optimizes drive usage after you add drives or servers to the pool (this is a manual
process for Storage Spaces systems that use Shared SAS enclosures). Optimization starts
15 minutes after you add a new drive to the pool. Pool optimization runs as a low-
priority background operation, so it can take hours or days to complete, especially if
you're using large hard drives.

Optimization uses two jobs - one called Optimize and one called Rebalance - and you
can monitor their progress with the following command:

PowerShell

Get-StorageJob

You can manually optimize a storage pool with the Optimize-StoragePool cmdlet. Here's
an example:

PowerShell

Get-StoragePool <PoolName> | Optimize-StoragePool

Failover cluster maintenance procedures
Article • 04/17/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019, Windows Server 2016

This article assumes that you need to power down a physical server to perform
maintenance, or restart it for some other reason. To install updates on an Azure Stack
HCI cluster without taking servers offline, see Update Azure Stack HCI clusters.

Taking a server offline for maintenance requires taking portions of storage offline that
are shared across all servers in a failover cluster. This requires pausing the server that
you want to take offline, putting the server's disks in maintenance mode, moving
clustered roles and virtual machines (VMs) to other servers in the cluster, and verifying
that all data is available on the other servers in the cluster. This process ensures that the
data remains safe and accessible throughout the maintenance period.

You can use either Windows Admin Center or PowerShell to take a server offline for
maintenance. This topic covers both methods.

Take a server offline using Windows Admin

Center
The simplest way to prepare to take a server offline is by using Windows Admin Center.

Verify it's safe to take the server offline

1. Using Windows Admin Center, connect to the server you want to take offline.
Select Storage > Disks from the Tools menu, and verify that the Status column for
every virtual disk shows Online.

2. Then, select Storage > Volumes and verify that the Health column for every
volume shows Healthy and that the Status column for every volume shows OK.

Pause and drain the server

Before either shutting down or restarting a server, you should pause the server and
drain (move off) any clustered roles such as VMs running on it. Always pause and drain
clustered servers before taking them offline for maintenance.
 1. Using Windows Admin Center, connect to the cluster and then select Compute >
Servers from the Tools menu in Cluster Manager.

2. Select Inventory. Click on the name of the server you wish to pause and drain, and
select Pause. You should see the following prompt:

Pause server(s) for maintenance: Are you sure you want to pause server(s)? This
moves workloads, such as virtual machines, to other servers in the cluster.​

3. Select yes to pause the server and initiate the drain process. The server status will
show as In maintenance, Draining, and roles such as Hyper-V and VMs will
immediately begin live migrating to other server(s) in the cluster. This can take a
few minutes. No roles can be added to the server until it's resumed. When the
draining process is finished, the server status will show as In maintenance, Drain
completed. The operating system performs an automatic safety check to ensure it
is safe to proceed. If there are unhealthy volumes, it will stop and alert you that it's
not safe to proceed.

Shut down the server

Once the server has completed draining, you can safely shut it down for maintenance or
reboot it.

２ Warning

If the server is running Azure Stack HCI, version 20H2, Windows Server 2019, or
Windows Server 2016, you must put the disks in maintenance mode before
shutting down the server and take the disks out of maintenance mode before
resuming the server into the cluster.

Resume the server

When you are ready for the server to begin hosting clustered roles and VMs again,
simply turn the server on, wait for it to boot up, and resume the server using the
following steps.

1. In Cluster Manager, select Compute > Servers from the Tools menu at the left.

2. Select Inventory. Click on the name of the server you wish to resume, and then
click Resume.
Clustered roles and VMs will immediately begin live migrating back to the server. This
can take a few minutes.

Wait for storage to resync

When the server resumes, any new writes that happened while it was unavailable need
to resync. This happens automatically, using intelligent change tracking. It's not
necessary for all data to be scanned or synchronized; only the changes. This process is
throttled to mitigate impact to production workloads. Depending on how long the
server was paused and how much new data was written, it may take many minutes to
complete.

） Important

You must wait for re-syncing to complete before taking any other servers in the
cluster offline.

To check if storage resync is complete:

1. Connect to the cluster using Windows Admin Center and select Storage >
Volumes.
2. Select Inventory.
3. Check the Status column for every volume. If it shows OK, storage resync is
complete. It's now safe to take other servers in the cluster offline.

Take a server offline using PowerShell

Use the following procedures to properly pause, drain, and resume a server in a failover
cluster using PowerShell.

Verify it's safe to take the server offline

To verify that all your volumes are healthy, run the following cmdlet as an administrator:

PowerShell

Get-VirtualDisk

Here's an example of what the output might look like:

 FriendlyName ResiliencySettingName FaultDomainRedundancy
OperationalStatus HealthStatus Size FootprintOnPool StorageEfficiency
------------ --------------------- --------------------- ------
----------- ------------ ---- --------------- -----------------
Mirror II Mirror 1 OK
Healthy 4 TB 8.01 TB 49.99%
Mirror-accelerated parity OK
Healthy 1002 GB 1.96 TB 49.98%
Mirror Mirror 1 OK
Healthy 1 TB 2 TB 49.98%
ClusterPerformanceHistory Mirror 1 OK
Healthy 24 GB 49 GB 48.98%

Verify that the HealthStatus property for every volume is Healthy and the
OperationalStatus shows OK.

To do this using Failover Cluster Manager, go to Storage > Disks.

Pause and drain the server

Run the following cmdlet as an administrator to pause and drain the server:

PowerShell

Suspend-ClusterNode -Drain

To do this in Failover Cluster Manager, go to Nodes, right-click the node, and then select
Pause > Drain Roles.

If the server is running Azure Stack HCI, version 21H2 or Windows Server 2022, pausing
and draining the server will also put the server's disks into maintenance mode. If the
server is running Azure Stack HCI, version 20H2, Windows Server 2019, or Windows
Server 2016, you'll have to do this manually (see next step).

Put disks in maintenance mode

In Azure Stack HCI, version 20H2, Windows Server 2019, and Windows Server 2016,
putting the server's disks in maintenance mode gives Storage Spaces Direct an
opportunity to gracefully flush and commit data to ensure that the server shutdown
does not affect application state. As soon as a disk goes into maintenance mode, it will
no longer allow writes. To minimize storage resynch times, we recommend putting the
disks into maintenance mode right before the reboot and bringing them out of
maintenance mode as soon as the system is back up.
 ７ Note

If the server is running Azure Stack HCI, version 21H2 or Windows Server 2022, you
can skip this step because the disks are automatically put into maintenance mode
when the server is paused and drained. These operating systems have a granular
repair feature that makes resyncs faster and less impactful on system and network
resources, making it feasible to have server and storage maintenance done
together.

If the server is running Windows Server 2019 or Azure Stack HCI, version 20H2, run the
following cmdlet as administrator:

PowerShell

Get-StorageScaleUnit -FriendlyName "Server1" | Enable-StorageMaintenanceMode

If the server is running Windows Server 2016, use the following syntax instead:

PowerShell

Get-StorageFaultDomain -Type StorageScaleUnit | Where-Object

{$_.FriendlyName -eq "Server1"} | Enable-StorageMaintenanceMode

Shut down the server

Once the server has completed draining, it will show as Paused in PowerShell and
Failover Cluster Manager.

You can now safely shut the server down or restart it by using the Stop-Computer or
Restart-Computer PowerShell cmdlets, or by using Failover Cluster Manager.

７ Note

When running a Get-VirtualDisk command on servers that are shutting down or

starting/stopping the cluster service, the server's Operational Status may be
reported as incomplete or degraded, and the Health Status column may list a
warning. This is normal and should not cause concern. All your volumes remain
online and accessible.

Take disks out of maintenance mode

If the server is running Azure Stack HCI, version 20H2, Windows Server 2019, or
Windows Server 2016, you must disable storage maintenance mode on the disks before
resuming the server into the cluster. To minimize storage resynch times, we recommend
bringing them out of maintenance mode as soon as the system is back up.

７ Note

If the server is running Azure Stack HCI, version 21H2 or Windows Server 2022, you
can skip this step because the disks will automatically be taken out of maintenance
mode when the server is resumed.

If the server is running Windows Server 2019 or Azure Stack HCI, version 20H2, run the
following cmdlet as administrator to disable storage maintenance mode:

PowerShell

Get-StorageScaleUnit -FriendlyName "Server1" | Disable-

StorageMaintenanceMode

If the server is running Windows Server 2016, use the following syntax instead:

PowerShell

Get-StorageFaultDomain -Type StorageScaleUnit | Where-Object

{$_.FriendlyName -eq "Server1"} | Disable-StorageMaintenanceMode

Resume the server

Resume the server into the cluster. To return the clustered roles and VMs that were
previously running on the server, use the optional -Failback flag:

PowerShell

Resume-ClusterNode –Failback Immediate

To do this in Failover Cluster Manager, go to Nodes, right-click the node, and then select
Resume > Fail Roles Back.

Once the server has resumed, it will show as Up in PowerShell and Failover Cluster
Manager.

Wait for storage to resync

When the server resumes, you must wait for re-syncing to complete before taking any
other servers in the cluster offline.

Run the following cmdlet as administrator to monitor progress:

PowerShell

Get-StorageJob

If re-syncing has already completed, you won't get any output.

Here's some example output showing resync (repair) jobs still running:

Name IsBackgroundTask ElapsedTime JobState PercentComplete BytesProcessed

BytesTotal
---- ---------------- ----------- -------- --------------- --------------
----------
Repair True 00:06:23 Running 65 11477975040
17448304640
Repair True 00:06:40 Running 66 15987900416
23890755584
Repair True 00:06:52 Running 68 20104802841
22104819713

The BytesTotal column shows how much storage needs to resync. The PercentComplete
column displays progress.

２ Warning

It's not safe to take another server offline until these repair jobs finish.

During this time, under HealthStatus, your volumes will continue to show as Warning,
which is normal.

For example, if you use the Get-VirtualDisk cmdlet while storage is re-syncing, you
might see the following output:

FriendlyName ResiliencySettingName OperationalStatus HealthStatus

IsManualAttach Size
------------ --------------------- ----------------- ------------ ----------
---- ----
MyVolume1 Mirror InService Warning True
 1 TB
MyVolume2 Mirror InService Warning True
1 TB
MyVolume3 Mirror InService Warning True
1 TB

Once the jobs complete, verify that volumes show Healthy again by using the Get-
VirtualDisk cmdlet. Here's some example output:

FriendlyName ResiliencySettingName OperationalStatus HealthStatus

IsManualAttach Size
------------ --------------------- ----------------- ------------ ----------
---- ----
MyVolume1 Mirror OK Healthy True
1 TB
MyVolume2 Mirror OK Healthy True
1 TB
MyVolume3 Mirror OK Healthy True
1 TB

It's now safe to pause and restart other servers in the cluster.

Next steps
For related information, see also:

Storage Spaces Direct overview

Update Azure Stack HCI clusters
Removing servers in Storage Spaces
Direct
Article • 03/29/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

This topic describes how to remove servers in Storage Spaces Direct using PowerShell.

Remove a server but leave its drives

If you intend to add the server back into the cluster soon, or if you intend to keep its
drives by moving them to another server, you can remove the server from the cluster
without removing its drives from the storage pool. This is the default behavior if you use
Failover Cluster Manager to remove the server.

Use the Remove-ClusterNode cmdlet in PowerShell:

PowerShell

Remove-ClusterNode <Name>

This cmdlet succeeds quickly, regardless of any capacity considerations, because the
storage pool "remembers" the missing drives and expects them to come back. There is
no data movement away from the missing drives. While they remain missing, their
OperationalStatus will show as "Lost Communication", and your volumes will show
"Incomplete".

When the drives come back, they are automatically detected and re-associated with the
pool, even if they are now in a new server.

２ Warning

Do not distribute drives with pool data from one server into multiple other servers.
For example, if one server with ten drives fails (because its motherboard or boot
drive failed, for instance), you can move all ten drives into one new server, but you
cannot move each of them separately into different other servers.

Remove a server and its drives

If you want to permanently remove a server from the cluster (sometimes referred to as
scaling-in), you can remove the server from the cluster and remove its drives from the
storage pool.

Use the Remove-ClusterNode cmdlet with the optional -CleanUpDisks flag:

PowerShell

Remove-ClusterNode <Name> -CleanUpDisks

This cmdlet might take a long time (sometimes many hours) to run because Windows
must move all the data stored on that server to other servers in the cluster. Once this is
complete, the drives are permanently removed from the storage pool, returning affected
volumes to a healthy state.

Requirements
To permanently scale-in (remove a server and its drives), your cluster must meet the
following two requirements. If it doesn't, the Remove-ClusterNode -CleanUpDisks
cmdlet will return an error immediately, before it begins any data movement, to
minimize disruption.

Enough capacity
First, you must have enough storage capacity in the remaining servers to accommodate
all your volumes.

For example, if you have four servers, each with 10 x 1 TB drives, you have 40 TB of total
physical storage capacity. After removing one server and all its drives, you will have 30
TB of capacity left. If the footprints of your volumes are more than 30 TB together, they
won't fit in the remaining servers, so the cmdlet will return an error and not move any
data.

Enough fault domains

Second, you must have enough fault domains (typically servers) to provide the resiliency
of your volumes.

For example, if your volumes use three-way mirroring at the server level for resiliency,
they cannot be fully healthy unless you have at least three servers. If you have exactly
three servers, and then attempt to remove one and all its drives, the cmdlet will return
an error and not move any data.
This table shows the minimum number of fault domains required for each resiliency
type.

Resiliency Minimum required fault domains

Two-way mirror 2

Three-way mirror 3

Dual parity 4

７ Note

It is okay to briefly have fewer servers, such as during failures or maintenance.

However, in order for volumes to return to a fully healthy state, you must have the
minimum number of servers listed above.

Additional References
Storage Spaces Direct overview
Updating drive firmware
Article • 01/10/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows 10

Updating the firmware for drives has historically been a cumbersome task with a
potential for downtime, which is why we're making improvements to Storage Spaces,
Windows Server, and Windows 10, version 1703 and newer. If you have drives that
support the new firmware update mechanism included in Windows, you can update
drive firmware of in-production drives without downtime. However, if you're going to
update the firmware of a production drive, make sure to read our tips on how to
minimize the risk while using this powerful new functionality.

２ Warning

Firmware updates are a potentially risky maintenance operation and you should
only apply them after thorough testing of the new firmware image. It is possible
that new firmware on unsupported hardware could negatively affect reliability and
stability, or even cause data loss. Administrators should read the release notes a
given update comes with to determine its impact and applicability.

Drive compatibility
To use Windows Server to update drive firmware, you must have supported drives. To
ensure common device behavior, we began by defining new and - for Windows 10 and
Windows Server 2016 - optional Hardware Lab Kit (HLK) requirements for SAS, SATA,
and NVMe devices. These requirements outline which commands a SATA, SAS, or NVMe
device must support to be firmware-updatable using these new, Windows-native
PowerShell cmdlets. To support these requirements, there is a new HLK test to verify if
vendor products support the right commands and get them implemented in future
revisions.

Contact your solution vendor for info about whether your hardware supports Windows
updating the drive firmware. Here are links to the various requirements:

SATA: Device.Storage.Hd.Sata - in the [If Implemented] Firmware Download &

Activate section
 SAS: Device.Storage.Hd.Sas - in the [If Implemented] Firmware Download &
Activate section

NVMe: Device.Storage.ControllerDrive.NVMe - in sections 5.7 and 5.8.

PowerShell cmdlets
The two cmdlets added to Windows are:

Get-StorageFirmwareInformation
Update-StorageFirmware

The first cmdlet provides you with detailed information about the device's capabilities,
firmware images, and revisions. In this case, the machine only contains a single SATA
SSD with 1 firmware slot. Here's an example:

PowerShell

Get-PhysicalDisk | Get-StorageFirmwareInformation

SupportsUpdate : True
NumberOfSlots : 1
ActiveSlotNumber : 0
SlotNumber : {0}
IsSlotWritable : {True}
FirmwareVersionInSlot : {J3E16101}

Note that SAS devices always report "SupportsUpdate" as "True", since there is no way
of explicitly querying the device for support of these commands.

The second cmdlet, Update-StorageFirmware, enables administrators to update the

drive firmware with an image file, if the drive supports the new firmware update
mechanism. You should obtain this image file from the OEM or drive vendor directly.

７ Note

Before updating any production hardware, test the particular firmware image on
identical hardware in a lab setting.

The drive will first load the new firmware image to an internal staging area. While this
happens, I/O typically continues. The image activates after downloading. During this
time the drive will not be able to respond to I/O commands as an internal reset occurs.
This means that this drive serves no data during the activation. An application accessing
data on this drive would have to wait for a response until the firmware activation
completes. Here's an example of the cmdlet in action:

PowerShell

$pd | Update-StorageFirmware -ImagePath C:\Firmware\[email protected] -SlotNumber

0
$pd | Get-StorageFirmwareInformation

SupportsUpdate : True
NumberOfSlots : 1
ActiveSlotNumber : 0
SlotNumber : {0}
IsSlotWritable : {True}
FirmwareVersionInSlot : {J3E160@3}

Drives typically do not complete I/O requests when they activate a new firmware image.
How long a drive takes to activate depends on its design and the type of firmware you
update. We have observed update times range from fewer than 5 seconds to more than
30 seconds.

This drive performed the firmware update within ~5.8 seconds, as shown here:

PowerShell

Measure-Command {$pd | Update-StorageFirmware -ImagePath

C:\\Firmware\\J3E16101.enc -SlotNumber 0}

Days : 0
Hours : 0
Minutes : 0
Seconds : 5
Milliseconds : 791
Ticks : 57913910
TotalDays : 6.70299884259259E-05
TotalHours : 0.00160871972222222
TotalMinutes : 0.0965231833333333
TotalSeconds : 5.791391
TotalMilliseconds : 5791.391

Updating drives in production

Before placing a server into production, we highly recommend updating the firmware of
your drives to the firmware recommended by the hardware vendor or OEM that sold
and supports your solution (storage enclosures, drives, and servers).
Once a server is in production, it's a good idea to make as few changes to the server as
is practical. However, there may be times when your solution vendor advises you that
there is a critically important firmware update for your drives. If this occurs, here are a
few good practices to follow before applying any drive firmware updates:

1. Review the firmware release notes and confirm that the update addresses issues
that could affect your environment, and that the firmware doesn't contain any
known issues that could adversely affect you.

2. Install the firmware on a server in your lab that has identical drives (including the
revision of the drive if there are multiple revisions of the same drive), and test the
drive under load with the new firmware. For info about doing synthetic load
testing, see Test Storage Spaces Performance Using Synthetic Workloads.

Automated firmware updates with Storage

Spaces Direct
Windows Server 2016 includes a Health Service for Storage Spaces Direct deployments
(including Microsoft Azure Stack solutions). The main purpose of the Health Service is to
make monitoring and management of your hardware deployment easier. As part of its
management functions, it has the capability to roll-out drive firmware across an entire
cluster without taking any workloads offline or incurring downtime. This capability is
policy-driven, with the control in the admin's hands.

Using the Health Service to roll-out firmware across a cluster is very simple and involves
the following steps:

Identify what HDD and SSD drives you expect to be part of your Storage Spaces
Direct cluster, and whether the drives support Windows performing firmware
updates
List those drives in the Supported Components xml file
Identify the firmware versions you expect those drives to have in the Supported
Components xml (including location paths of the firmware images)
Upload the xml file to the cluster DB

At this point, the Health Service will inspect and parse the xml and identify any drives
that do not have the desired firmware version deployed. It will then proceed to re-direct
I/O away from the affected drives – going node-by-node – and updating the firmware
on them. A Storage Spaces Direct cluster achieves resiliency by spreading data across
multiple server nodes; it is possible for the health service to isolate an entire node worth
of drives for updates. Once a node updates, it will initiate a repair in Storage Spaces,
bringing all copies of data across the cluster back in sync with each other, before
moving on to the next node. It is expected and normal for Storage Spaces to transition
to a "degraded" mode of operation while firmware is rolled out.

To ensure a stable roll-out and sufficient validation time of a new firmware image, there
exists a significant delay between the updates of several servers. Per default, the Health
Service will wait 7 days before updating the 2nd server. Any subsequent server (3rd, 4th,
…) updates with a 1 day delay. Should an administrator find the firmware to be unstable
or otherwise undesirable, she can stop further roll-out by the health service at any time.
If the firmware has been previously validated and a quicker roll-out is desired, these
default values can be modified from days, to hours or minutes.

Here is an example of the supported components xml for a generic Storage Spaces
Direct cluster:

XML

<Components>
<Disks>
<Disk>
<Manufacturer>Contoso</Manufacturer>
<Model>XYZ9000</Model>
<AllowedFirmware>
<Version>2.0</Version>
<Version>2.1>/Version>
<Version>2.2</Version>
</AllowedFirmware>
<TargetFirmware>
<Version>2.2</Version>
<BinaryPath>\\path\to\image.bin</BinaryPath>
</TargetFirmware>
</Disk>
...
...
</Disks>
</Components>

To get the roll-out of the new firmware started in this Storage Spaces Direct cluster,
simply upload the .xml to the cluster DB:

PowerShell

$SpacesDirect = Get-StorageSubSystem Clus*

$CurrentDoc = $SpacesDirect | Get-StorageHealthSetting -Name

"System.Storage.SupportedComponents.Document"

$CurrentDoc.Value | Out-File <Path>

Edit the file in your favorite editor, such as Visual Studio Code or Notepad, then save it.

PowerShell

$NewDoc = Get-Content <Path> | Out-String

$SpacesDirect | Set-StorageHealthSetting -Name

"System.Storage.SupportedComponents.Document" -Value $NewDoc

Frequently asked questions

Also see Troubleshooting drive firmware updates.

Will this work on any storage device

This will work on storage devices that implement the correct commands in their
firmware. The Get-StorageFirmwareInformation cmdlet will show if a drive's firmware
indeed does support the correct commands (for SATA/NVMe) and the HLK test allows
vendors and OEMs to test this behavior.

After I update a SATA drive, it reports to no longer

support the update mechanism. Is something wrong with
the drive
No, the drive is fine, unless the new firmware doesn't allow updates anymore. You are
hitting a known issue whereby a cached version of the drive's capabilities is incorrect.
Running "Update-StorageProviderCache -DiscoveryLevel Full" will re-enumerate the
drive capabilities and update the cached copy. As a work-around, we recommend
running the above command once before initiating a firmware update or complete roll-
out on a Spaces Direct cluster.

Can I update firmware on my SAN through this

mechanism
No - SANs usually have their own utilities and interfaces for such maintenance
operations. This new mechanism is for directly attached storage, such as SATA, SAS, or
NVMe devices.

From where do I get the firmware image

You should always obtain any firmware directly from your OEM, solution vendor, or drive
vendor and not download it from other parties. Windows provides the mechanism to
get the image to the drive, but cannot verify its integrity.

Will this work on clustered drives

The cmdlets can perform their function on clustered drives as well, but keep in mind
that the Health Service orchestration mitigates the I/O impact on running workloads. If
the cmdlets are used directly on clustered drives, I/O is likely to stall. In general, it is a
best practice to perform drive firmware updates when there is no, or just a minimal
workload on the underlying drives.

What happens when I update firmware on Storage Spaces

On Windows Server 2016 with the Health Service deployed on Storage Spaces Direct,
you can perform this operation without taking your workloads offline, assuming the
drives support Windows Server updating the firmware.

What happens if the update fails

The update could fail for various reasons, some of them are: 1) The drive doesn't
support the correct commands for Windows to update its firmware. In this case the new
firmware image never activates and the drive continues functioning with the old image.
2) The image cannot download to or be applied to this drive (version mismatch, corrupt
image, …). In this case the drive fails the activate command. Again, the old firmware
image will continue function.

If the drive does not respond at all after a firmware update, you are likely hitting a bug
in the drive firmware itself. Test all firmware updates in a lab environment before putting
them in production. The only remediation may be to replace the drive.

For more info, see Troubleshooting drive firmware updates.

How do I stop an in-progress firmware roll-out

Disable the roll-out in PowerShell via:

PowerShell

Get-StorageSubSystem Cluster* | Set-StorageHealthSetting -Name

"System.Storage.PhysicalDisk.AutoFirmwareUpdate.RollOut.Enabled" -Value
false
I am seeing an access denied or path-not-found error
during roll out. How do I fix this
Ensure that the firmware image you would like to use for the update is accessible by all
cluster nodes. The easiest way to ensure this is to place it on a cluster shared volume.
Manage volumes in Azure Stack HCI and
Windows Server
Article • 05/02/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019, Windows Server 2016

This article describes how to expand, move, or delete volumes in Azure Stack HCI and
Windows Server by using either Windows Admin Center or PowerShell.

Prerequisites
Before you begin to manage volumes, make sure that:

You have administrator privileges to access the cluster.

You have access to a management computer that is in the same domain as your
cluster.

Here are a few additional prerequisites for moving volumes:

Make sure the volume is in a healthy state before you move it. To find out about
the health state of a volume, see Monitor volumes for Windows Admin Center and
Virtual disk state for PowerShell.
If using PowerShell to move volumes, make sure you have the Remote Server
Administration Tools (RSAT) cmdlets and PowerShell modules for Hyper-V and
Failover Clustering. If these aren't already available in your PowerShell session on
your management computer, add them using the following command: Add-
WindowsFeature RSAT-Clustering-PowerShell .

Expand volumes
This section describes how to expand volumes using Windows Admin Center or
PowerShell.

２ Warning

Not supported: resizing the underlying storage used by Storage Spaces Direct. If
you are running Storage Spaces Direct in a virtualized storage environment,
including in Azure, resizing or changing the characteristics of the storage devices
 used by the virtual machines isn't supported and will cause data to become
inaccessible. Instead, follow the instructions in the Add servers or drives section to
add additional capacity before expanding volumes.

Windows Admin Center

Follow these steps to expand volumes in Windows Admin Center:

1. In Windows Admin Center, connect to a cluster, and then select Volumes from
the Tools pane.

2. On the Volumes page, select the Inventory tab, and then select the volume
that you want to expand. On the volume detail page, the storage capacity for
the volume is indicated.

You can also open the volumes detail page directly from Dashboard. On the
Dashboard, in the Alerts section, select the alert, which notifies you if a
volume is running low on storage capacity, and then select Go To Volume.

3. At the top of the volumes detail page, select Settings.

4. Enter a new larger size in the right pane, and then select Save.

On the volumes detail page, the larger storage capacity for the volume is
indicated, and the alert on the Dashboard is cleared.

Move volumes
This section describes how to move Cluster Shared Volumes (CSV) from one cluster
node to another by using Windows Admin Center or PowerShell.

You may require to move volumes in several scenarios, including:

To balance out the storage capacity among different nodes in the cluster.

To troubleshoot an issue on an unhealthy cluster node.

To conform to system configuration rules, such as to have certain volumes on a

specific cluster node.

７ Note
 For stretched clusters, you can move a volume only to another server in the
same site.

Windows Admin Center

Follow these steps to move volumes using Windows Admin Center:

1. In Windows Admin Center, connect to a cluster, and then select Volumes from
the Tools pane on the left.
2. On the Volumes page, select the Inventory tab, and then select the volume
that you want to move.
3. At the top of the Volumes page, select Move.
4. In the right pane, select the Destination server where you want to move the
volume to and then select Move.

Delete volumes
This section describes how to delete volumes by using Windows Admin Center or
PowerShell.

Windows Admin Center

1. In Windows Admin Center, connect to a cluster, and then select Volumes from
the Tools pane on the left.
2. On the Volumes page, select the Inventory tab, and then select the volume
that you want to delete.
3. At the top of the volumes detail page, select Delete.
4. In the confirmations dialog, confirm that you want to delete the volume, and
select Delete.

Next steps
For step-by-step instructions on other essential storage management tasks, see also:

Plan volumes
Create volumes
Protect volumes
Performance history for Storage Spaces
Direct
Article • 03/29/2022

Applies to: Windows Server 2022, Windows Server 2019

Performance history is a new feature that gives Storage Spaces Direct administrators
easy access to historical compute, memory, network, and storage measurements across
host servers, drives, volumes, virtual machines, and more. Performance history is
collected automatically and stored on the cluster for up to one year.

） Important

This feature is new in Windows Server 2019. It is not available in Windows Server
2016.

Get started
Performance history is collected by default with Storage Spaces Direct in Windows
Server 2019. You do not need to install, configure, or start anything. An Internet
connection is not required, System Center is not required, and an external database is
not required.

To see your cluster's performance history graphically, use Windows Admin Center:

To query and process it programmatically, use the new Get-ClusterPerf cmdlet. See
Usage in PowerShell.
What's collected
Performance history is collected for 7 types of objects:

Each object type has many series: for example, ClusterNode.Cpu.Usage is collected for
each server.

For details of what's collected for each object type and how to interpret them, refer to
these sub-topics:

Object Series

Drives What's collected for drives

Network adapters What's collected for network adapters

Servers What's collected for servers

Virtual hard disks What's collected for virtual hard disks

Virtual machines What's collected for virtual machines

Volumes What's collected for volumes

Clusters What's collected for clusters

Many series are aggregated across peer objects to their parent: for example,
NetAdapter.Bandwidth.Inbound is collected for each network adapter separately and
aggregated to the overall server; likewise ClusterNode.Cpu.Usage is aggregated to the
overall cluster; and so on.

Timeframes
Performance history is stored for up to one year, with diminishing granularity. For the
most recent hour, measurements are available every ten seconds. Thereafter, they are
intelligently merged (by averaging or summing, as appropriate) into less granular series
that span more time. For the most recent day, measurements are available every five
minutes; for the most recent week, every fifteen minutes; and so on.

In Windows Admin Center, you can select the timeframe in the upper-right above the
chart.

In PowerShell, use the -TimeFrame parameter.

Here are the available timeframes:

Timeframe Measurement frequency Retained for

LastHour Every 10 secs 1 hour

LastDay Every 5 minutes 25 hours

LastWeek Every 15 minutes 8 days

LastMonth Every 1 hour 35 days

LastYear Every 1 day 400 days

Usage in PowerShell
Use the Get-ClusterPerformanceHistory cmdlet to query and process performance
history in PowerShell.

PowerShell

Get-ClusterPerformanceHistory
  Tip

Use the Get-ClusterPerf alias to save some keystrokes.

Example
Get the CPU usage of virtual machine MyVM for the last hour:

PowerShell

Get-VM "MyVM" | Get-ClusterPerf -VMSeriesName "VM.Cpu.Usage" -TimeFrame

LastHour

For more advanced examples, see the published sample scripts that provide starter code
to find peak values, calculate averages, plot trend lines, run outlier detection, and more.

Specify the object

You can specify the object you want by the pipeline. This works with 7 types of objects:

Object from pipeline Example

Get-PhysicalDisk Get-PhysicalDisk -SerialNumber "XYZ456" | Get-ClusterPerf

Get-NetAdapter Get-NetAdapter "Ethernet" | Get-ClusterPerf

Get-ClusterNode Get-ClusterNode "Server123" | Get-ClusterPerf

Get-VHD Get-VHD "C:\ClusterStorage\MyVolume\MyVHD.vhdx" | Get-ClusterPerf

Get-VM Get-VM "MyVM" | Get-ClusterPerf

Get-Volume Get-Volume -FriendlyName "MyVolume" | Get-ClusterPerf

Get-Cluster Get-Cluster "MyCluster" | Get-ClusterPerf

If you don't specify, performance history for the overall cluster is returned.

Specify the series

You can specify the series you want with these parameters:

Parameter Example List

 Parameter Example List

- "PhysicalDisk.Iops.Read" What's collected for drives

PhysicalDiskSeriesName

-NetAdapterSeriesName "NetAdapter.Bandwidth.Outbound" What's collected for network

adapters

-ClusterNodeSeriesName "ClusterNode.Cpu.Usage" What's collected for servers

-VHDSeriesName "Vhd.Size.Current" What's collected for virtual hard

disks

-VMSeriesName "Vm.Memory.Assigned" What's collected for virtual

machines

-VolumeSeriesName "Volume.Latency.Write" What's collected for volumes

-ClusterSeriesName "PhysicalDisk.Size.Total" What's collected for clusters

 Tip

Use tab completion to discover available series.

If you don't specify, every series available for the specified object is returned.

Specify the timeframe

You can specify the timeframe of history you want with the -TimeFrame parameter.

 Tip

Use tab completion to discover available timeframes.

If you don't specify, the MostRecent measurement is returned.

How it works

Performance history storage

Shortly after Storage Spaces Direct is enabled, an approximately 10 GB volume named
ClusterPerformanceHistory is created and an instance of the Extensible Storage Engine
(also known as Microsoft JET) is provisioned there. This lightweight database stores the
performance history without any Administrator involvement or management.

The volume is backed by Storage Spaces and uses either simple, two-way mirror, or
three-way mirror resiliency, depending on the number of nodes in the cluster. It is
repaired after drive or server failures just like any other volume in Storage Spaces Direct.

The volume uses ReFS but is not Cluster Shared Volume (CSV), so it only appears on the
Cluster Group owner node. Besides being automatically created, there is nothing special
about this volume: you can see it, browse it, resize it, or delete it (not recommended). If
something goes wrong, see Troubleshooting.

Object discovery and data collection

Performance history automatically discovers relevant objects, such as virtual machines,
anywhere in the cluster and begins streaming their performance counters. The counters
are aggregated, synchronized, and inserted into the database. Streaming runs
continuously and is optimized for minimal system impact.

Collection is handled by the Health Service, which is highly available: if the node where
it's running goes down, it will resume moments later on another node in the cluster.
Performance history may lapse briefly, but it will resume automatically. You can see the
Health Service and its owner node by running Get-ClusterResource Health in
PowerShell.

Handling measurement gaps

When measurements are merged into less granular series that span more time, as
described in Timeframes, periods of missing data are excluded. For example, if the
server was down for 30 minutes, then running at 50% CPU for the next 30 minutes, the
ClusterNode.Cpu.Usage average for the hour will be recorded correctly as 50% (not
25%).
Extensibility and customization
Performance history is scripting-friendly. Use PowerShell to pull any available history
directly from the database to build automated reporting or alerting, export history for
safekeeping, roll your own visualizations, etc. See the published sample scripts for
helpful starter code.

It's not possible to collect history for additional objects, timeframes, or series.

The measurement frequency and retention period are not currently configurable.

Start or stop performance history

How do I enable this feature?

Unless you Stop-ClusterPerformanceHistory , performance history is enabled by default.

To re-enable it, run this PowerShell cmdlet as Administrator:

PowerShell

Start-ClusterPerformanceHistory

How do I disable this feature?

To stop collecting performance history, run this PowerShell cmdlet as Administrator:

PowerShell

Stop-ClusterPerformanceHistory

To delete existing measurements, use the -DeleteHistory flag:

PowerShell

Stop-ClusterPerformanceHistory -DeleteHistory

 Tip

During initial deployment, you can prevent performance history from starting by
setting the -CollectPerformanceHistory parameter of Enable-
 ClusterStorageSpacesDirect to $False .

Troubleshooting

The cmdlet doesn't work

An error message like "The term 'Get-ClusterPerf' is not recognized as the name of a
cmdlet" means the feature is not available or installed. Verify that you have Windows
Server Insider Preview build 17692 or later, that you've installed Failover Clustering, and
that you're running Storage Spaces Direct.

７ Note

This feature is not available on Windows Server 2016 or earlier.

No data available
If a chart shows "No data available" as pictured, here's how to troubleshoot:

1. If the object was newly added or created, wait for it to be discovered (up to 15
minutes).

2. Refresh the page, or wait for the next background refresh (up to 30 seconds).

3. Certain special objects are excluded from performance history – for example,
virtual machines that aren't clustered, and volumes that don't use the Cluster
 Shared Volume (CSV) filesystem. Check the sub-topic for the object type, like
Performance history for volumes, for the fine print.

4. If the problem persists, open PowerShell as Administrator and run the Get-
ClusterPerf cmdlet. The cmdlet includes troubleshooting logic to identify common
problems, such as if the ClusterPerformanceHistory volume is missing, and
provides remediation instructions.

5. If the command in the previous step returns nothing, you can try restarting the
Health Service (which collects performance history) by running Stop-
ClusterResource Health ; Start-ClusterResource Health in PowerShell.

Additional References
Storage Spaces Direct overview
Performance history for drives
Article • 12/23/2021

Applies to: Windows Server 2022, Windows Server 2019

This sub-topic of Performance history for Storage Spaces Direct describes in detail the
performance history collected for drives. Performance history is available for every drive
in the cluster storage subsystem, regardless of bus or media type. However, it is not
available for OS boot drives.

７ Note

Performance history cannot be collected for drives in a server that is down.

Collection will resume automatically when the server comes back up.

Series names and units

These series are collected for every eligible drive:

Series Unit

physicaldisk.iops.read per second

physicaldisk.iops.write per second

physicaldisk.iops.total per second

physicaldisk.throughput.read bytes per second

physicaldisk.throughput.write bytes per second

physicaldisk.throughput.total bytes per second

physicaldisk.latency.read seconds

physicaldisk.latency.write seconds

physicaldisk.latency.average seconds

physicaldisk.size.total bytes

physicaldisk.size.used bytes
How to interpret
Series How to interpret

physicaldisk.iops.read Number of read operations per second completed by the drive.

physicaldisk.iops.write Number of write operations per second completed by the drive.

physicaldisk.iops.total Total number of read or write operations per second completed

by the drive.

physicaldisk.throughput.read Quantity of data read from the drive per second.

physicaldisk.throughput.write Quantity of data written to the drive per second.

physicaldisk.throughput.total Total quantity of data read from or written to the drive per
second.

physicaldisk.latency.read Average latency of read operations from the drive.

physicaldisk.latency.write Average latency of write operations to the drive.

physicaldisk.latency.average Average latency of all operations to or from the drive.

physicaldisk.size.total The total storage capacity of the drive.

physicaldisk.size.used The used storage capacity of the drive.

Where they come from

The iops.* , throughput.* , and latency.* series are collected from the Physical Disk
performance counter set on the server where the drive is connected, one instance per
drive. These counters are measured by partmgr.sys and do not include much of the
Windows software stack nor any network hops. They are representative of device
hardware performance.

Series Source counter

physicaldisk.iops.read Disk Reads/sec

physicaldisk.iops.write Disk Writes/sec

physicaldisk.iops.total Disk Transfers/sec

physicaldisk.throughput.read Disk Read Bytes/sec

physicaldisk.throughput.write Disk Write Bytes/sec

 Series Source counter

physicaldisk.throughput.total Disk Bytes/sec

physicaldisk.latency.read Avg. Disk sec/Read

physicaldisk.latency.write Avg. Disk sec/Writes

physicaldisk.latency.average Avg. Disk sec/Transfer

７ Note

Counters are measured over the entire interval, not sampled. For example, if the
drive is idle for 9 seconds but completes 30 IOs in the 10th second, its
physicaldisk.iops.total will be recorded as 3 IOs per second on average during
this 10-second interval. This ensures its performance history captures all activity
and is robust to noise.

The size.* series are collected from the MSFT_PhysicalDisk class in WMI, one instance
per drive.

Series Source property

physicaldisk.size.total Size

physicaldisk.size.used VirtualDiskFootprint

Usage in PowerShell
Use the Get-PhysicalDisk cmdlet:

PowerShell

Get-PhysicalDisk -SerialNumber <SerialNumber> | Get-ClusterPerf

Additional References
Performance history for Storage Spaces Direct
Performance history for network
adapters
Article • 12/23/2021

Applies to: Windows Server 2022, Windows Server 2019

This sub-topic of Performance history for Storage Spaces Direct describes in detail the
performance history collected for network adapters. Network adapter performance
history is available for every physical network adapter in every server in the cluster.
Remote Direct Memory Access (RDMA) performance history is available for every
physical network adapter with RDMA enabled.

７ Note

Performance history cannot be collected for network adapters in a server that is

down. Collection will resume automatically when the server comes back up.

Series names and units

These series are collected for every eligible network adapter:

Series Unit

netadapter.bandwidth.inbound bits per second

netadapter.bandwidth.outbound bits per second

netadapter.bandwidth.total bits per second

netadapter.bandwidth.rdma.inbound bits per second

netadapter.bandwidth.rdma.outbound bits per second

netadapter.bandwidth.rdma.total bits per second

７ Note

Network adapter performance history is recorded in bits per second, not bytes per
second. One 10 GbE network adapter can send and receive approximately
 1,000,000,000 bits = 125,000,000 bytes = 1.25 GB per second at its theoretical
maximum.

How to interpret
Series How to interpret

netadapter.bandwidth.inbound Rate of data received by the network adapter.

netadapter.bandwidth.outbound Rate of data sent by the network adapter.

netadapter.bandwidth.total Total rate of data received or sent by the network adapter.

netadapter.bandwidth.rdma.inbound Rate of data received over RDMA by the network adapter.

netadapter.bandwidth.rdma.outbound Rate of data sent over RDMA by the network adapter.

netadapter.bandwidth.rdma.total Total rate of data received or sent over RDMA by the

network adapter.

Where they come from

The bytes.* series are collected from the Network Adapter performance counter set on
the server where the network adapter is installed, one instance per network adapter.

Series Source counter

netadapter.bandwidth.inbound 8 × Bytes Received/sec

netadapter.bandwidth.outbound 8 × Bytes Sent/sec

netadapter.bandwidth.total 8 × Bytes Total/sec

The rdma.* series are collected from the RDMA Activity performance counter set on the
server where the network adapter is installed, one instance per network adapter with
RDMA enabled.

Series Source counter

netadapter.bandwidth.rdma.inbound 8 × Inbound bytes/sec

netadapter.bandwidth.rdma.outbound 8 × Outbound bytes/sec

netadapter.bandwidth.rdma.total 8 × sum of the above

 ７ Note

Counters are measured over the entire interval, not sampled. For example, if the
network adapter is idle for 9 seconds but transfers 200 bits in the 10th second, its
netadapter.bandwidth.total will be recorded as 20 bits per second on average

during this 10-second interval. This ensures its performance history captures all
activity and is robust to noise.

Usage in PowerShell
Use the Get-NetAdapter cmdlet:

PowerShell

Get-NetAdapter <Name> | Get-ClusterPerf

Additional References
Performance history for Storage Spaces Direct
Performance history for servers
Article • 12/23/2021

Applies to: Windows Server 2022, Windows Server 2019

This sub-topic of Performance history for Storage Spaces Direct describes in detail the
performance history collected for servers. Performance history is available for every
server in the cluster.

７ Note

Performance history cannot be collected for a server that is down. Collection will
resume automatically when the server comes back up.

Series names and units

These series are collected for every eligible server:

Series Unit

clusternode.cpu.usage percent

clusternode.cpu.usage.guest percent

clusternode.cpu.usage.host percent

clusternode.memory.total bytes

clusternode.memory.available bytes

clusternode.memory.usage bytes

clusternode.memory.usage.guest bytes

clusternode.memory.usage.host bytes

In addition, drive series such as physicaldisk.size.total are aggregated for all eligible
drives attached to the server, and network adapter series such as
networkadapter.bytes.total are aggregated for all eligible network adapters attached

to the server.
How to interpret
Series How to interpret

clusternode.cpu.usage Percentage of processor time that is not idle.

clusternode.cpu.usage.guest Percentage of processor time used for guest (virtual machine)

demand.

clusternode.cpu.usage.host Percentage of processor time used for host demand.

clusternode.memory.total The total physical memory of the server.

clusternode.memory.available The available memory of the server.

clusternode.memory.usage The allocated (not available) memory of the server.

clusternode.memory.usage.guest The memory allocated to guest (virtual machine) demand.

clusternode.memory.usage.host The memory allocated to host demand.

Where they come from

The cpu.* series are collected from different performance counters depending on
whether or not Hyper-V is enabled.

If Hyper-V is enabled:

Series Source counter

clusternode.cpu.usage Hyper-V Hypervisor Logical Processor > _Total > % Total Run
Time

clusternode.cpu.usage.guest Hyper-V Hypervisor Virtual Processor > _Total > % Total Run
Time

clusternode.cpu.usage.host Hyper-V Hypervisor Root Virtual Processor > _Total > % Total
Run Time

Using the % Total Run Time counters ensures that performance history attributes all
usage.

If Hyper-V is NOT enabled:

Series Source counter

clusternode.cpu.usage Processor > _Total > % Processor Time

 Series Source counter

clusternode.cpu.usage.guest zero

clusternode.cpu.usage.host same as total usage

Notwithstanding imperfect synchronization, clusternode.cpu.usage is always

clusternode.cpu.usage.host plus clusternode.cpu.usage.guest .

With the same caveat, clusternode.cpu.usage.guest is always the sum of vm.cpu.usage

for all virtual machines on the host server.

The memory.* series are (COMING SOON).

７ Note

Counters are measured over the entire interval, not sampled. For example, if the
server is idle for 9 seconds but spikes to 100% CPU in the 10th second, its
clusternode.cpu.usage will be recorded as 10% on average during this 10-second

interval. This ensures its performance history captures all activity and is robust to
noise.

Usage in PowerShell
Use the Get-ClusterNode cmdlet:

PowerShell

Get-ClusterNode <Name> | Get-ClusterPerf

Additional References
Performance history for Storage Spaces Direct
Performance history for virtual hard
disks
Article • 12/23/2021

Applies to: Windows Server 2022, Windows Server 2019

This sub-topic of Performance history for Storage Spaces Direct describes in detail the
performance history collected for virtual hard disk (VHD) files. Performance history is
available for every VHD attached to a running, clustered virtual machine. Performance
history is available for both VHD and VHDX formats, however it is not available for
Shared VHDX files.

７ Note

It may take several minutes for collection to begin for newly created or moved VHD
files.

Series names and units

These series are collected for every eligible virtual hard disk:

Series Unit

vhd.iops.read per second

vhd.iops.write per second

vhd.iops.total per second

vhd.throughput.read bytes per second

vhd.throughput.write bytes per second

vhd.throughput.total bytes per second

vhd.latency.average seconds

vhd.size.current bytes

vhd.size.maximum bytes
How to interpret
Series How to interpret

vhd.iops.read Number of read operations per second completed by the virtual hard
disk.

vhd.iops.write Number of write operations per second completed by the virtual hard
disk.

vhd.iops.total Total number of read or write operations per second completed by the
virtual hard disk.

vhd.throughput.read Quantity of data read from the virtual hard disk per second.

vhd.throughput.write Quantity of data written to the virtual hard disk per second.

vhd.throughput.total Total quantity of data read from or written to the virtual hard disk per
second.

vhd.latency.average Average latency of all operations to or from the virtual hard disk.

vhd.size.current The current file size of the virtual hard disk, if dynamically expanding. If
fixed, the series is not collected.

vhd.size.maximum The maximum size of the virtual hard disk, if dynamically expanding. If
fixed, the is the size.

Where they come from

The iops.* , throughput.* , and latency.* series are collected from the Hyper-V Virtual
Storage Device performance counter set on the server where the virtual machine is
running, one instance per VHD or VHDX.

Series Source counter

vhd.iops.read Read Operations/Sec

vhd.iops.write Write Operations/Sec

vhd.iops.total sum of the above

vhd.throughput.read Read Bytes/sec

vhd.throughput.write Write Bytes/sec

vhd.throughput.total sum of the above

 Series Source counter

vhd.latency.average Latency

７ Note

Counters are measured over the entire interval, not sampled. For example, if the
VHD is inactive for 9 seconds but completes 30 IOs in the 10th second, its
vhd.iops.total will be recorded as 3 IOs per second on average during this 10-
second interval. This ensures its performance history captures all activity and is
robust to noise.

Usage in PowerShell
Use the Get-VHD cmdlet:

PowerShell

Get-VHD <Path> | Get-ClusterPerf

To get the path of every VHD from the virtual machine:

PowerShell

(Get-VM <Name>).HardDrives | Select Path

７ Note

The Get-VHD cmdlet requires a file path to be provided. It does not support
enumeration.

Additional References
Performance history for Storage Spaces Direct
Performance history for virtual
machines
Article • 12/23/2021

Applies to: Windows Server 2022, Windows Server 2019

This sub-topic of Performance history for Storage Spaces Direct describes in detail the
performance history collected for virtual machines (VM). Performance history is available
for every running, clustered VM.

７ Note

It may take several minutes for collection to begin for newly created or renamed
VMs.

Series names and units

These series are collected for every eligible VM:

Series Unit

vm.cpu.usage percent

vm.memory.assigned bytes

vm.memory.available bytes

vm.memory.maximum bytes

vm.memory.minimum bytes

vm.memory.pressure -

vm.memory.startup bytes

vm.memory.total bytes

vmnetworkadapter.bandwidth.inbound bits per second

vmnetworkadapter.bandwidth.outbound bits per second

vmnetworkadapter.bandwidth.total bits per second

In addition, all virtual hard disk (VHD) series, such as vhd.iops.total , are aggregated for
every VHD attached to the VM.

How to interpret
Series Description

vm.cpu.usage Percentage the virtual machine is using of its host server's

processor(s).

vm.memory.assigned The quantity of memory assigned to the virtual machine.

vm.memory.available The quantity of memory that remains available, of the

amount assigned.

vm.memory.maximum If using dynamic memory, this is the maximum quantity

of memory that may be assigned to the virtual machine.

vm.memory.minimum If using dynamic memory, this is the minimum quantity of

memory that may be assigned to the virtual machine.

vm.memory.pressure The ratio of memory demanded by the virtual machine

over memory allocated to the virtual machine.

vm.memory.startup The quantity of memory required for the virtual machine

to start.

vm.memory.total Total memory.

vmnetworkadapter.bandwidth.inbound Rate of data received by the virtual machine across all its
virtual network adapters.

vmnetworkadapter.bandwidth.outbound Rate of data sent by the virtual machine across all its
virtual network adapters.

vmnetworkadapter.bandwidth.total Total rate of data received or sent by the virtual machine

across all its virtual network adapters.

７ Note

Counters are measured over the entire interval, not sampled. For example, if the
VM is idle for 9 seconds but spikes to use 50% of host CPU in the 10th second, its
vm.cpu.usage will be recorded as 5% on average during this 10-second interval.

This ensures its performance history captures all activity and is robust to noise.
Usage in PowerShell
Use the Get-VM cmdlet:

PowerShell

Get-VM <Name> | Get-ClusterPerf

７ Note

The Get-VM cmdlet only returns virtual machines on the local (or specified) server,
not across the cluster.

Additional References
Performance history for Storage Spaces Direct
Performance history for volumes
Article • 07/11/2022

Applies to: Windows Server 2022, Windows Server 2019

This sub-topic of Performance history for Storage Spaces Direct describes in detail the
performance history collected for volumes. Performance history is available for every
Cluster Shared Volume (CSV) in the cluster. However, it is not available for OS boot
volumes nor any other non-CSV storage.

７ Note

It may take several minutes for collection to begin for newly created or renamed
volumes.

Series names and units

These series are collected for every eligible volume:

Series Unit

volume.iops.read per second

volume.iops.write per second

volume.iops.total per second

volume.throughput.read bytes per second

volume.throughput.write bytes per second

volume.throughput.total bytes per second

volume.latency.read seconds

volume.latency.write seconds

volume.latency.average seconds

volume.size.total bytes

volume.size.available bytes
How to interpret
Series How to interpret

volume.iops.read Number of read operations per second completed by this volume.

volume.iops.write Number of write operations per second completed by this volume.

volume.iops.total Total number of read or write operations per second completed by

this volume.

volume.throughput.read Quantity of data read from this volume per second.

volume.throughput.write Quantity of data written to this volume per second.

volume.throughput.total Total quantity of data read from or written to this volume per second.

volume.latency.read Average latency of read operations from this volume.

volume.latency.write Average latency of write operations to this volume.

volume.latency.average Average latency of all operations to or from this volume.

volume.size.total The total storage capacity of the volume.

volume.size.available The available storage capacity of the volume.

Where they come from

The iops.* , throughput.* , and latency.* series are collected from the Cluster CSVFS
performance counter set. Every server in the cluster has an instance for every CSV
volume, regardless of ownership. The performance history recorded for volume
MyVolume is the aggregate of the MyVolume instances on every server in the cluster.

Series Source counter

volume.iops.read Reads/sec

volume.iops.write Writes/sec

volume.iops.total sum of the above

volume.throughput.read Read bytes/sec

volume.throughput.write Write bytes/sec

volume.throughput.total sum of the above

 Series Source counter

volume.latency.read Avg. sec/Read

volume.latency.write Avg. sec/Write

volume.latency.average average of the above

７ Note

Counters are measured over the entire interval, not sampled. For example, if the
volume is idle for 9 seconds but completes 30 IOs in the 10th second, its
volume.iops.total will be recorded as 3 IOs per second on average during this 10-

second interval. This ensures its performance history captures all activity and is
robust to noise.

 Tip

These are the same counters used by the popular VM Fleet benchmark
framework.

The size.* series are collected from the MSFT_Volume class in WMI, one instance per
volume.

Series Source property

volume.size.total Size

volume.size.available SizeRemaining

Usage in PowerShell
Use the Get-Volume cmdlet:

PowerShell

Get-Volume -FriendlyName <FriendlyName> | Get-ClusterPerf

Additional References
Performance history for Storage Spaces Direct
Performance history for clusters
Article • 12/23/2021

Applies to: Windows Server 2022, Windows Server 2019

This sub-topic of Performance history for Storage Spaces Direct describes the
performance history collected for clusters.

There are no series that originate at the cluster level. Instead, server series, such as
clusternode.cpu.usage , are aggregated for all servers in the cluster. Volume series, such

as volume.iops.total , are aggregated for all volumes in the cluster. And drive series,
such as physicaldisk.size.total , are aggregated for all drives in the cluster.

Usage in PowerShell
Use the Get-Cluster cmdlet:

PowerShell

Get-Cluster | Get-ClusterPerf

Additional References
Performance history for Storage Spaces Direct
Scripting with PowerShell and Storage
Spaces Direct performance history
Article • 03/29/2022

Applies to: Windows Server 2022, Windows Server 2019

In Windows Server 2019, Storage Spaces Direct records and stores extensive
performance history for virtual machines, servers, drives, volumes, network adapters, and
more. Performance history is easy to query and process in PowerShell so you can quickly
go from raw data to actual answers to questions like:

1. Were there any CPU spikes last week?

2. Is any physical disk exhibiting abnormal latency?
3. Which VMs are consuming the most storage IOPS right now?
4. Is my network bandwidth saturated?
5. When will this volume run out of free space?
6. In the past month, which VMs used the most memory?

The Get-ClusterPerf cmdlet is built for scripting. It accepts input from cmdlets like Get-
VM or Get-PhysicalDisk by the pipeline to handle association, and you can pipe its

output into utility cmdlets like Sort-Object , Where-Object , and Measure-Object to

quickly compose powerful queries.

This topic provides and explains 6 sample scripts that answer the 6 questions above.
They present patterns you can apply to find peaks, find averages, plot trend lines, run
outlier detection, and more, across a variety of data and timeframes. They are provided
as free starter code for you to copy, extend, and reuse.

７ Note

For brevity, the sample scripts omit things like error handling that you might expect
of high-quality PowerShell code. They are intended primarily for inspiration and
education rather than production use.

Sample 1: CPU, I see you!

This sample uses the ClusterNode.Cpu.Usage series from the LastWeek timeframe to
show the maximum ("high water mark"), minimum, and average CPU usage for every
server in the cluster. It also does simple quartile analysis to show how many hours CPU
usage was over 25%, 50%, and 75% in the last 8 days.

Screenshot
In the screenshot below, we see that Server-02 had an unexplained spike last week:

How it works
The output from Get-ClusterPerf pipes nicely into the built-in Measure-Object cmdlet,
we just specify the Value property. With its -Maximum , -Minimum , and -Average flags,
Measure-Object gives us the first three columns almost for free. To do the quartile
analysis, we can pipe to Where-Object and count how many values were -Gt (greater
than) 25, 50, or 75. The last step is to beautify with Format-Hours and Format-Percent
helper functions – certainly optional.

Script
Here's the script:

Function Format-Hours {
Param (
$RawValue
)
# Weekly timeframe has frequency 15 minutes = 4 points per hour
[Math]::Round($RawValue/4)
}

Function Format-Percent {
Param (
$RawValue
)
[String][Math]::Round($RawValue) + " " + "%"
}
 $Output = Get-ClusterNode | ForEach-Object {
$Data = $_ | Get-ClusterPerf -ClusterNodeSeriesName
"ClusterNode.Cpu.Usage" -TimeFrame "LastWeek"

$Measure = $Data | Measure-Object -Property Value -Minimum -Maximum -

Average
$Min = $Measure.Minimum
$Max = $Measure.Maximum
$Avg = $Measure.Average

[PsCustomObject]@{
"ClusterNode" = $_.Name
"MinCpuObserved" = Format-Percent $Min
"MaxCpuObserved" = Format-Percent $Max
"AvgCpuObserved" = Format-Percent $Avg
"HrsOver25%" = Format-Hours ($Data | Where-Object Value -Gt
25).Length
"HrsOver50%" = Format-Hours ($Data | Where-Object Value -Gt
50).Length
"HrsOver75%" = Format-Hours ($Data | Where-Object Value -Gt
75).Length
}
}

$Output | Sort-Object ClusterNode | Format-Table

Sample 2: Fire, fire, latency outlier

This sample uses the PhysicalDisk.Latency.Average series from the LastHour timeframe
to look for statistical outliers, defined as drives with an hourly average latency exceeding
+3σ (three standard deviations) above the population average.

） Important

For brevity, this script does not implement safeguards against low variance, does
not handle partial missing data, does not distinguish by model or firmware, etc.
Please exercise good judgement and do not rely on this script alone to determine
whether to replace a hard disk. It is presented here for educational purposes only.

Screenshot
In the screenshot below, we see there are no outliers:
How it works
First, we exclude idle or nearly idle drives by checking that PhysicalDisk.Iops.Total is
consistently -Gt 1 . For every active HDD, we pipe its LastHour timeframe, comprised of
360 measurements at 10 second intervals, to Measure-Object -Average to obtain its
average latency in the last hour. This sets up our population.

We implement the widely-known formula to find the mean μ and standard deviation
σ of the population. For every active HDD, we compare its average latency to the

population average and divide by the standard deviation. We keep the raw values, so we
can Sort-Object our results, but use Format-Latency and Format-StandardDeviation
helper functions to beautify what we'll show – certainly optional.

If any drive is more than +3σ, we Write-Host in red; if not, in green.

Script
Here's the script:

Function Format-Latency {
Param (
$RawValue
)
$i = 0 ; $Labels = ("s", "ms", "μs", "ns") # Petabits, just in case!
Do { $RawValue *= 1000 ; $i++ } While ( $RawValue -Lt 1 )
# Return
[String][Math]::Round($RawValue, 2) + " " + $Labels[$i]
}

Function Format-StandardDeviation {
Param (
$RawValue
)
If ($RawValue -Gt 0) {
$Sign = "+"
}
Else {
$Sign = "-"
}
# Return
$Sign + [String][Math]::Round([Math]::Abs($RawValue), 2) + "σ"
}

$HDD = Get-StorageSubSystem Cluster* | Get-PhysicalDisk | Where-Object

MediaType -Eq HDD

$Output = $HDD | ForEach-Object {

$Iops = $_ | Get-ClusterPerf -PhysicalDiskSeriesName

"PhysicalDisk.Iops.Total" -TimeFrame "LastHour"
$AvgIops = ($Iops | Measure-Object -Property Value -Average).Average

If ($AvgIops -Gt 1) { # Exclude idle or nearly idle drives

$Latency = $_ | Get-ClusterPerf -PhysicalDiskSeriesName

"PhysicalDisk.Latency.Average" -TimeFrame "LastHour"
$AvgLatency = ($Latency | Measure-Object -Property Value -
Average).Average

[PsCustomObject]@{
"FriendlyName" = $_.FriendlyName
"SerialNumber" = $_.SerialNumber
"MediaType" = $_.MediaType
"AvgLatencyPopulation" = $null # Set below
"AvgLatencyThisHDD" = Format-Latency $AvgLatency
"RawAvgLatencyThisHDD" = $AvgLatency
 "Deviation" = $null # Set below
"RawDeviation" = $null # Set below
}
}
}

If ($Output.Length -Ge 3) { # Minimum population requirement

# Find mean μ and standard deviation σ

$μ = ($Output | Measure-Object -Property RawAvgLatencyThisHDD -
Average).Average
$d = $Output | ForEach-Object { ($_.RawAvgLatencyThisHDD - $μ) *
($_.RawAvgLatencyThisHDD - $μ) }
$σ = [Math]::Sqrt(($d | Measure-Object -Sum).Sum / $Output.Length)

$FoundOutlier = $False

$Output | ForEach-Object {
$Deviation = ($_.RawAvgLatencyThisHDD - $μ) / $σ
$_.AvgLatencyPopulation = Format-Latency $μ
$_.Deviation = Format-StandardDeviation $Deviation
$_.RawDeviation = $Deviation
# If distribution is Normal, expect >99% within 3σ
If ($Deviation -Gt 3) {
$FoundOutlier = $True
}
}

If ($FoundOutlier) {
Write-Host -BackgroundColor Black -ForegroundColor Red "Oh no!
There's an HDD significantly slower than the others."
}
Else {
Write-Host -BackgroundColor Black -ForegroundColor Green "Good news!
No outlier found."
}

$Output | Sort-Object RawDeviation -Descending | Format-Table

FriendlyName, SerialNumber, MediaType, AvgLatencyPopulation,
AvgLatencyThisHDD, Deviation

}
Else {
Write-Warning "There aren't enough active drives to look for outliers
right now."
}

Sample 3: Noisy neighbor? That's write!

Performance history can answer questions about right now, too. New measurements are
available in real-time, every 10 seconds. This sample uses the VHD.Iops.Total series
from the MostRecent timeframe to identify the busiest (some might say "noisiest") virtual
machines consuming the most storage IOPS, across every host in the cluster, and show
the read/write breakdown of their activity.

Screenshot
In the screenshot below, we see the Top 10 virtual machines by storage activity:

How it works
Unlike Get-PhysicalDisk , the Get-VM cmdlet isn't cluster-aware – it only returns VMs on
the local server. To query from every server in parallel, we wrap our call in Invoke-
Command (Get-ClusterNode).Name { ... } . For every VM, we get the VHD.Iops.Total ,
VHD.Iops.Read , and VHD.Iops.Write measurements. By not specifying the -TimeFrame

parameter, we get the MostRecent single data point for each.

 Tip

These series reflect the sum of this VM's activity to all its VHD/VHDX files. This is an
example where performance history is being automatically aggregated for us. To
get the per-VHD/VHDX breakdown, you could pipe an individual Get-VHD into Get-
ClusterPerf instead of the VM.

The results from every server come together as $Output , which we can Sort-Object and
then Select-Object -First 10 . Notice that Invoke-Command decorates results with a
PsComputerName property indicating where they came from, which we can print to know
where the VM is running.
Script
Here's the script:

$Output = Invoke-Command (Get-ClusterNode).Name {

Function Format-Iops {
Param (
$RawValue
)
$i = 0 ; $Labels = (" ", "K", "M", "B", "T") # Thousands, millions,
billions, trillions...
Do { if($RawValue -Gt 1000){$RawValue /= 1000 ; $i++ } } While (
$RawValue -Gt 1000 )
# Return
[String][Math]::Round($RawValue) + " " + $Labels[$i]
}

Get-VM | ForEach-Object {
$IopsTotal = $_ | Get-ClusterPerf -VMSeriesName "VHD.Iops.Total"
$IopsRead = $_ | Get-ClusterPerf -VMSeriesName "VHD.Iops.Read"
$IopsWrite = $_ | Get-ClusterPerf -VMSeriesName "VHD.Iops.Write"
[PsCustomObject]@{
"VM" = $_.Name
"IopsTotal" = Format-Iops $IopsTotal.Value
"IopsRead" = Format-Iops $IopsRead.Value
"IopsWrite" = Format-Iops $IopsWrite.Value
"RawIopsTotal" = $IopsTotal.Value # For sorting...
}
}
}

$Output | Sort-Object RawIopsTotal -Descending | Select-Object -First 10 |

Format-Table PsComputerName, VM, IopsTotal, IopsRead, IopsWrite

Sample 4: As they say, "25-gig is the new 10-

gig"
This sample uses the NetAdapter.Bandwidth.Total series from the LastDay timeframe to
look for signs of network saturation, defined as >90% of theoretical maximum
bandwidth. For every network adapter in the cluster, it compares the highest observed
bandwidth usage in the last day to its stated link speed.

Screenshot
In the screenshot below, we see that one Fabrikam NX-4 Pro #2 peaked in the last day:
How it works
We repeat our Invoke-Command trick from above to Get-NetAdapter on every server and
pipe into Get-ClusterPerf . Along the way, we grab two relevant properties: its
LinkSpeed string like "10 Gbps", and its raw Speed integer like 10000000000. We use

Measure-Object to obtain the average and peak from the last day (reminder: each
measurement in the LastDay timeframe represents 5 minutes) and multiply by 8 bits per
byte to get an apples-to-apples comparison.

７ Note

Some vendors, like Chelsio, include remote-direct memory access (RDMA) activity
in their Network Adapter performance counters, so it's included in the
NetAdapter.Bandwidth.Total series. Others, like Mellanox, may not. If your vendor

doesn't, simply add the NetAdapter.Bandwidth.RDMA.Total series in your version of

this script.

Script
Here's the script:

$Output = Invoke-Command (Get-ClusterNode).Name {

Function Format-BitsPerSec {
Param (
$RawValue
)
 $i = 0 ; $Labels = ("bps", "kbps", "Mbps", "Gbps", "Tbps", "Pbps") #
Petabits, just in case!
Do { $RawValue /= 1000 ; $i++ } While ( $RawValue -Gt 1000 )
# Return
[String][Math]::Round($RawValue) + " " + $Labels[$i]
}

Get-NetAdapter | ForEach-Object {

$Inbound = $_ | Get-ClusterPerf -NetAdapterSeriesName

"NetAdapter.Bandwidth.Inbound" -TimeFrame "LastDay"
$Outbound = $_ | Get-ClusterPerf -NetAdapterSeriesName
"NetAdapter.Bandwidth.Outbound" -TimeFrame "LastDay"

If ($Inbound -Or $Outbound) {

$InterfaceDescription = $_.InterfaceDescription
$LinkSpeed = $_.LinkSpeed

$MeasureInbound = $Inbound | Measure-Object -Property Value -

Maximum
$MaxInbound = $MeasureInbound.Maximum * 8 # Multiply to bits/sec

$MeasureOutbound = $Outbound | Measure-Object -Property Value -

Maximum
$MaxOutbound = $MeasureOutbound.Maximum * 8 # Multiply to
bits/sec

$Saturated = $False

# Speed property is Int, e.g. 10000000000

If (($MaxInbound -Gt (0.90 * $_.Speed)) -Or ($MaxOutbound -Gt
(0.90 * $_.Speed))) {
$Saturated = $True
Write-Warning "In the last day, adapter
'$InterfaceDescription' on server '$Env:ComputerName' exceeded 90% of its
'$LinkSpeed' theoretical maximum bandwidth. In general, network saturation
leads to higher latency and diminished reliability. Not good!"
}

[PsCustomObject]@{
"NetAdapter" = $InterfaceDescription
"LinkSpeed" = $LinkSpeed
"MaxInbound" = Format-BitsPerSec $MaxInbound
"MaxOutbound" = Format-BitsPerSec $MaxOutbound
"Saturated" = $Saturated
}
}
}
}

$Output | Sort-Object PsComputerName, InterfaceDescription | Format-Table

PsComputerName, NetAdapter, LinkSpeed, MaxInbound, MaxOutbound, Saturated
Sample 5: Make storage trendy again!
To look at macro trends, performance history is retained for up to 1 year. This sample
uses the Volume.Size.Available series from the LastYear timeframe to determine the
rate that storage is filling up and estimate when it will be full.

Screenshot
In the screenshot below, we see the Backup volume is adding about 15 GB per day:

At this rate, it will reach its capacity in another 42 days.

How it works
The LastYear timeframe has one data point per day. Although you only strictly need
two points to fit a trend line, in practice it's better to require more, like 14 days. We use
Select-Object -Last 14 to set up an array of (x, y) points, for x in the range [1, 14]. With

these points, we implement the straightforward linear least squares algorithm to find
$A and $B that parameterize the line of best fit y = ax + b. Welcome to high school all
over again.

Dividing the volume's SizeRemaining property by the trend (the slope $A ) lets us
crudely estimate how many days, at the current rate of storage growth, until the volume
is full. The Format-Bytes , Format-Trend , and Format-Days helper functions beautify the
output.

） Important

This estimate is linear and based only on the most recent 14 daily measurements.
More sophisticated and accurate techniques exist. Please exercise good judgement
and do not rely on this script alone to determine whether to invest in expanding
your storage. It is presented here for educational purposes only.
Script
Here's the script:

Function Format-Bytes {
Param (
$RawValue
)
$i = 0 ; $Labels = ("B", "KB", "MB", "GB", "TB", "PB", "EB", "ZB", "YB")
Do { $RawValue /= 1024 ; $i++ } While ( $RawValue -Gt 1024 )
# Return
[String][Math]::Round($RawValue) + " " + $Labels[$i]
}

Function Format-Trend {
Param (
$RawValue
)
If ($RawValue -Eq 0) {
"0"
}
Else {
If ($RawValue -Gt 0) {
$Sign = "+"
}
Else {
$Sign = "-"
}
# Return
$Sign + $(Format-Bytes [Math]::Abs($RawValue)) + "/day"
}
}

Function Format-Days {
Param (
$RawValue
)
[Math]::Round($RawValue)
}

$CSV = Get-Volume | Where-Object FileSystem -Like "*CSV*"

$Output = $CSV | ForEach-Object {

$N = 14 # Require 14 days of history

$Data = $_ | Get-ClusterPerf -VolumeSeriesName "Volume.Size.Available" -

TimeFrame "LastYear" | Sort-Object Time | Select-Object -Last $N

If ($Data.Length -Ge $N) {

 # Last N days as (x, y) points
$PointsXY = @()
1..$N | ForEach-Object {
$PointsXY += [PsCustomObject]@{ "X" = $_ ; "Y" =
$Data[$_-1].Value }
}

# Linear (y = ax + b) least squares algorithm

$MeanX = ($PointsXY | Measure-Object -Property X -Average).Average
$MeanY = ($PointsXY | Measure-Object -Property Y -Average).Average
$XX = $PointsXY | ForEach-Object { $_.X * $_.X }
$XY = $PointsXY | ForEach-Object { $_.X * $_.Y }
$SSXX = ($XX | Measure-Object -Sum).Sum - $N * $MeanX * $MeanX
$SSXY = ($XY | Measure-Object -Sum).Sum - $N * $MeanX * $MeanY
$A = ($SSXY / $SSXX)
$B = ($MeanY - $A * $MeanX)
$RawTrend = -$A # Flip to get daily increase in Used (vs decrease in
Remaining)
$Trend = Format-Trend $RawTrend

If ($RawTrend -Gt 0) {
$DaysToFull = Format-Days ($_.SizeRemaining / $RawTrend)
}
Else {
$DaysToFull = "-"
}
}
Else {
$Trend = "InsufficientHistory"
$DaysToFull = "-"
}

[PsCustomObject]@{
"Volume" = $_.FileSystemLabel
"Size" = Format-Bytes ($_.Size)
"Used" = Format-Bytes ($_.Size - $_.SizeRemaining)
"Trend" = $Trend
"DaysToFull" = $DaysToFull
}
}

$Output | Format-Table

Sample 6: Memory hog, you can run but you

can't hide
Because performance history is collected and stored centrally for the whole cluster, you
never need to stitch together data from different machines, no matter how many times
VMs move between hosts. This sample uses the VM.Memory.Assigned series from the
LastMonth timeframe to identify the virtual machines consuming the most memory over

the last 35 days.

Screenshot
In the screenshot below, we see the Top 10 virtual machines by memory usage last
month:

How it works
We repeat our Invoke-Command trick, introduced above, to Get-VM on every server. We
use Measure-Object -Average to obtain the monthly average for every VM, then Sort-
Object followed by Select-Object -First 10 to obtain our leaderboard. (Or maybe it's
our Most Wanted list?)

Script
Here's the script:

$Output = Invoke-Command (Get-ClusterNode).Name {

Function Format-Bytes {
Param (
$RawValue
)
$i = 0 ; $Labels = ("B", "KB", "MB", "GB", "TB", "PB", "EB", "ZB",
"YB")
Do { if( $RawValue -Gt 1024 ){ $RawValue /= 1024 ; $i++ } } While (
$RawValue -Gt 1024 )
# Return
[String][Math]::Round($RawValue) + " " + $Labels[$i]
}
 Get-VM | ForEach-Object {
$Data = $_ | Get-ClusterPerf -VMSeriesName "VM.Memory.Assigned" -
TimeFrame "LastMonth"
If ($Data) {
$AvgMemoryUsage = ($Data | Measure-Object -Property Value -
Average).Average
[PsCustomObject]@{
"VM" = $_.Name
"AvgMemoryUsage" = Format-Bytes $AvgMemoryUsage.Value
"RawAvgMemoryUsage" = $AvgMemoryUsage.Value # For sorting...
}
}
}
}

$Output | Sort-Object RawAvgMemoryUsage -Descending | Select-Object -First

10 | Format-Table PsComputerName, VM, AvgMemoryUsage

That's it! Hopefully these samples inspire you and help you get started. With Storage
Spaces Direct performance history and the powerful, scripting-friendly Get-ClusterPerf
cmdlet, you are empowered to ask – and answer! – complex questions as you manage
and monitor your Windows Server 2019 infrastructure.

Additional References
Getting started with Windows PowerShell
Storage Spaces Direct overview
Performance history
Delimit the allocation of volumes in
Storage Spaces Direct
Article • 03/30/2022

Applies to: Windows Server 2022, Windows Server 2019

Windows Server 2019 introduces an option to manually delimit the allocation of

volumes in Storage Spaces Direct. Doing so can significantly increase fault tolerance
under certain conditions, but imposes some added management considerations and
complexity. This topic explains how it works and gives examples in PowerShell.

） Important

This feature is new in Windows Server 2019. It is not available in Windows Server
2016.

Prerequisites

Consider using this option if:

Your cluster has six or more servers; and
Your cluster uses only three-way mirror resiliency

Do not use this option if:

Your cluster has fewer than six servers; or
Your cluster uses parity or mirror-accelerated parity resiliency

Understand

Review: regular allocation

With regular three-way mirroring, the volume is divided into many small "slabs" that are
copied three times and distributed evenly across every drive in every server in the
cluster. For more details, read this deep dive blog .
This default allocation maximizes parallel reads and writes, leading to better
performance, and is appealing in its simplicity: every server is equally busy, every drive is
equally full, and all volumes stay online or go offline together. Every volume is
guaranteed to survive up to two concurrent failures, as these examples illustrate.

However, with this allocation, volumes can't survive three concurrent failures. If three
servers fail at once, or if drives in three servers fail at once, volumes become inaccessible
because at least some slabs were (with very high probability) allocated to the exact three
drives or servers that failed.

In the example below, servers 1, 3, and 5 fail at the same time. Although many slabs
have surviving copies, some do not:
The volume goes offline and becomes inaccessible until the servers are recovered.

New: delimited allocation

With delimited allocation, you specify a subset of servers to use (minimum four). The
volume is divided into slabs that are copied three times, like before, but instead of
allocating across every server, the slabs are allocated only to the subset of servers you
specify.

For example, if you have an 8 node cluster (nodes 1 through 8), you can specify a
volume to be located only on disks in nodes 1, 2, 3, 4.

Advantages
With the example allocation, the volume is likely to survive three concurrent failures. If
nodes 1, 2, and 6 go down, only 2 of the nodes that hold the 3 copies of data for the
volume are down and the volume will stay online.

Survival probability depends on the number of servers and other factors – see Analysis
for details.

Disadvantages

Delimited allocation imposes some added management considerations and complexity:

1. The administrator is responsible for delimiting the allocation of each volume to

balance storage utilization across servers and uphold high probability of survival,
as described in the Best practices section.

2. With delimited allocation, reserve the equivalent of one capacity drive per server
(with no maximum). This is more than the published recommendation for regular
allocation, which maxes out at four capacity drives total.

3. If a server fails and needs to be replaced, as described in Remove a server and its
drives, the administrator is responsible for updating the delimitation of affected
volumes by adding the new server and removing the failed one – example below.

Usage in PowerShell
You can use the New-Volume cmdlet to create volumes in Storage Spaces Direct.

For example, to create a regular three-way mirror volume:

 PowerShell

New-Volume -FriendlyName "MyRegularVolume" -Size 100GB

Create a volume and delimit its allocation

To create a three-way mirror volume and delimit its allocation:

1. First assign the servers in your cluster to the variable $Servers :

PowerShell

$Servers = Get-StorageFaultDomain -Type StorageScaleUnit | Sort

FriendlyName

 Tip

In Storage Spaces Direct, the term 'Storage Scale Unit' refers to all the raw
storage attached to one server, including direct-attached drives and direct-
attached external enclosures with drives. In this context, it's the same as
'server'.

2. Specify which servers to use with the new -StorageFaultDomainsToUse parameter

and by indexing into $Servers . For example, to delimit the allocation to the first,
second, third, and fourth servers (indices 0, 1, 2, and 3):

PowerShell

New-Volume -FriendlyName "MyVolume" -Size 100GB -

StorageFaultDomainsToUse $Servers[0,1,2,3]

See a delimited allocation

To see how MyVolume is allocated, use the Get-VirtualDiskFootprintBySSU.ps1 script in
Appendix:

PowerShell

PS C:\> .\Get-VirtualDiskFootprintBySSU.ps1

VirtualDiskFriendlyName TotalFootprint Server1 Server2 Server3 Server4

Server5 Server6
 ----------------------- -------------- ------- ------- ------- ------- -----
-- -------
MyVolume 300 GB 100 GB 100 GB 100 GB 100 GB 0
0

Note that only Server1, Server2, Server3, and Server4 contain slabs of MyVolume.

Change a delimited allocation

Use the new Add-StorageFaultDomain and Remove-StorageFaultDomain cmdlets to change
how the allocation is delimited.

For example, to move MyVolume over by one server:

1. Specify that the fifth server can store slabs of MyVolume:

PowerShell

Get-VirtualDisk MyVolume | Add-StorageFaultDomain -StorageFaultDomains

$Servers[4]

2. Specify that the first server cannot store slabs of MyVolume:

PowerShell

Get-VirtualDisk MyVolume | Remove-StorageFaultDomain -

StorageFaultDomains $Servers[0]

3. Rebalance the storage pool for the change to take effect:

PowerShell

Get-StoragePool S2D* | Optimize-StoragePool

You can monitor the progress of the rebalance with Get-StorageJob .

Once it is complete, verify that MyVolume has moved by running Get-

VirtualDiskFootprintBySSU.ps1 again.

PowerShell

PS C:\> .\Get-VirtualDiskFootprintBySSU.ps1

VirtualDiskFriendlyName TotalFootprint Server1 Server2 Server3 Server4

Server5 Server6
 ----------------------- -------------- ------- ------- ------- ------- -----
-- -------
MyVolume 300 GB 0 100 GB 100 GB 100 GB 100
GB 0

Note that Server1 does not contain slabs of MyVolume anymore – instead, Server5 does.

Best practices
Here are the best practices to follow when using delimited volume allocation:

Choose four servers

Delimit each three-way mirror volume to four servers, not more.

Balance storage
Balance how much storage is allocated to each server, accounting for volume size.

Stagger delimited allocation volumes

To maximize fault tolerance, make each volume's allocation unique, meaning it does not
share all its servers with another volume (some overlap is okay).

For example on an eight-node system: Volume 1: Servers 1, 2, 3, 4 Volume 2: Servers 5,

6, 7, 8 Volume 3: Servers 3, 4, 5, 6 Volume 4: Servers 1, 2, 7, 8

Analysis
This section derives the mathematical probability that a volume stays online and
accessible (or equivalently, the expected fraction of overall storage that stays online and
accessible) as a function of the number of failures and the cluster size.

７ Note

This section is optional reading. If you're keen to see the math, read on! But if not,
don't worry: Usage in PowerShell and Best practices is all you need to implement
delimited allocation successfully.

Up to two failures is always okay

Every three-way mirror volume can survive up to two failures at the same time,
regardless of its allocation. If two drives fail, or two servers fail, or one of each, every
three-way mirror volume stays online and accessible, even with regular allocation.

More than half the cluster failing is never okay

Conversely, in the extreme case that more than half of servers or drives in the cluster fail
at once, quorum is lost and every three-way mirror volume goes offline and becomes
inaccessible, regardless of its allocation.

What about in between?

If three or more failures occur at once, but at least half of the servers and the drives are
still up, volumes with delimited allocation may stay online and accessible, depending on
which servers have failures.

Frequently asked questions

Can I delimit some volumes but not others?

Yes. You can choose per-volume whether or not to delimit allocation.

Does delimited allocation change how drive replacement

works?
No, it's the same as with regular allocation.

Additional References
Storage Spaces Direct overview
Fault tolerance in Storage Spaces Direct

Appendix
This script helps you see how your volumes are allocated.

To use it as described above, copy/paste and save as Get-

VirtualDiskFootprintBySSU.ps1 .
PowerShell

Function ConvertTo-PrettyCapacity {
Param (
[Parameter(
Mandatory = $True,
ValueFromPipeline = $True
)
]
[Int64]$Bytes,
[Int64]$RoundTo = 0
)
If ($Bytes -Gt 0) {
$Base = 1024
$Labels = ("bytes", "KB", "MB", "GB", "TB", "PB", "EB", "ZB", "YB")
$Order = [Math]::Floor( [Math]::Log($Bytes, $Base) )
$Rounded = [Math]::Round($Bytes/( [Math]::Pow($Base, $Order) ),
$RoundTo)
[String]($Rounded) + " " + $Labels[$Order]
}
Else {
"0"
}
Return
}

Function Get-VirtualDiskFootprintByStorageFaultDomain {

################################################
### Step 1: Gather Configuration Information ###
################################################

Write-Progress -Activity "Get-VirtualDiskFootprintByStorageFaultDomain"

-CurrentOperation "Gathering configuration information..." -Status "Step
1/4" -PercentComplete 00

$ErrorCannotGetCluster = "Cannot proceed because 'Get-Cluster' failed."

$ErrorNotS2DEnabled = "Cannot proceed because the cluster is not running
Storage Spaces Direct."
$ErrorCannotGetClusterNode = "Cannot proceed because 'Get-ClusterNode'
failed."
$ErrorClusterNodeDown = "Cannot proceed because one or more cluster
nodes is not Up."
$ErrorCannotGetStoragePool = "Cannot proceed because 'Get-StoragePool'
failed."
$ErrorPhysicalDiskFaultDomainAwareness = "Cannot proceed because the
storage pool is set to 'PhysicalDisk' fault domain awareness. This cmdlet
only supports 'StorageScaleUnit', 'StorageChassis', or 'StorageRack' fault
domain awareness."

Try {
$GetCluster = Get-Cluster -ErrorAction Stop
}
Catch {
 throw $ErrorCannotGetCluster
}

If ($GetCluster.S2DEnabled -Ne 1) {
throw $ErrorNotS2DEnabled
}

Try {
$GetClusterNode = Get-ClusterNode -ErrorAction Stop
}
Catch {
throw $ErrorCannotGetClusterNode
}

If ($GetClusterNode | Where State -Ne Up) {

throw $ErrorClusterNodeDown
}

Try {
$GetStoragePool = Get-StoragePool -IsPrimordial $False -ErrorAction
Stop
}
Catch {
throw $ErrorCannotGetStoragePool
}

If ($GetStoragePool.FaultDomainAwarenessDefault -Eq "PhysicalDisk") {

throw $ErrorPhysicalDiskFaultDomainAwareness
}

###########################################################
### Step 2: Create SfdList[] and PhysicalDiskToSfdMap{} ###
###########################################################

Write-Progress -Activity "Get-VirtualDiskFootprintByStorageFaultDomain"

-CurrentOperation "Analyzing physical disk information..." -Status "Step
2/4" -PercentComplete 25

$SfdList = Get-StorageFaultDomain -Type

($GetStoragePool.FaultDomainAwarenessDefault) | Sort FriendlyName #
StorageScaleUnit, StorageChassis, or StorageRack

$PhysicalDiskToSfdMap = @{} # Map of PhysicalDisk.UniqueId ->

StorageFaultDomain.FriendlyName
$SfdList | ForEach {
$StorageFaultDomain = $_
$_ | Get-StorageFaultDomain -Type PhysicalDisk | ForEach {
$PhysicalDiskToSfdMap[$_.UniqueId] =
$StorageFaultDomain.FriendlyName
}
}

############################################################################
######################
 ### Step 3: Create VirtualDisk.FriendlyName -> {
StorageFaultDomain.FriendlyName -> Size } Map ###

############################################################################
######################

Write-Progress -Activity "Get-VirtualDiskFootprintByStorageFaultDomain"

-CurrentOperation "Analyzing virtual disk information..." -Status "Step 3/4"
-PercentComplete 50

$GetVirtualDisk = Get-VirtualDisk | Sort FriendlyName

$VirtualDiskMap = @{}

$GetVirtualDisk | ForEach {
# Map of PhysicalDisk.UniqueId -> Size for THIS virtual disk
$PhysicalDiskToSizeMap = @{}
$_ | Get-PhysicalExtent | ForEach {
$PhysicalDiskToSizeMap[$_.PhysicalDiskUniqueId] += $_.Size
}
# Map of StorageFaultDomain.FriendlyName -> Size for THIS virtual
disk
$SfdToSizeMap = @{}
$PhysicalDiskToSizeMap.keys | ForEach {
$SfdToSizeMap[$PhysicalDiskToSfdMap[$_]] +=
$PhysicalDiskToSizeMap[$_]
}
# Store
$VirtualDiskMap[$_.FriendlyName] = $SfdToSizeMap
}

#########################
### Step 4: Write-Out ###
#########################

Write-Progress -Activity "Get-VirtualDiskFootprintByStorageFaultDomain"

-CurrentOperation "Formatting output..." -Status "Step 4/4" -PercentComplete
75

$Output = $GetVirtualDisk | ForEach {

$Row = [PsCustomObject]@{}

$VirtualDiskFriendlyName = $_.FriendlyName
$Row | Add-Member -MemberType NoteProperty "VirtualDiskFriendlyName"
$VirtualDiskFriendlyName

$TotalFootprint = $_.FootprintOnPool | ConvertTo-PrettyCapacity

$Row | Add-Member -MemberType NoteProperty "TotalFootprint"
$TotalFootprint

$SfdList | ForEach {
$Size = $VirtualDiskMap[$VirtualDiskFriendlyName]
[$_.FriendlyName] | ConvertTo-PrettyCapacity
$Row | Add-Member -MemberType NoteProperty $_.FriendlyName $Size
}
 $Row
}

# Calculate width, in characters, required to Format-Table

$RequiredWindowWidth = ("TotalFootprint").length + 1 +
("VirtualDiskFriendlyName").length + 1
$SfdList | ForEach {
$RequiredWindowWidth += $_.FriendlyName.Length + 1
}

$ActualWindowWidth = (Get-Host).UI.RawUI.WindowSize.Width

If (!($ActualWindowWidth)) {
# Cannot get window width, probably ISE, Format-List
Write-Warning "Could not determine window width. For the best
experience, use a Powershell window instead of ISE"
$Output | Format-Table
}
ElseIf ($ActualWindowWidth -Lt $RequiredWindowWidth) {
# Narrower window, Format-List
Write-Warning "For the best experience, try making your PowerShell
window at least $RequiredWindowWidth characters wide. Current width is
$ActualWindowWidth characters."
$Output | Format-List
}
Else {
# Wider window, Format-Table
$Output | Format-Table
}
}

Get-VirtualDiskFootprintByStorageFaultDomain
Use Azure Monitor to send emails for
Health Service Faults
Article • 03/30/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

Azure Monitor maximizes the availability and performance of your applications by

delivering a comprehensive solution for collecting, analyzing, and acting on telemetry
from your cloud and on-premises environments. It helps you understand how your
applications are performing and proactively identifies issues affecting them and the
resources they depend on.

This is particularly helpful for your on-premises hyper-converged cluster. With Azure
Monitor integrated, you will be able to configure email, text (SMS), and other alerts to
ping you when something is wrong with your cluster (or when you want to flag some
other activity based on the data collected). Below, we will briefly explain how Azure
Monitor works, how to install Azure Monitor, and how to configure it to send you
notifications.

If you are using System Center, check out the Storage Spaces Direct management
pack that monitors both Windows Server 2019 and Windows Server 2016 Storage
Spaces Direct clusters.

This management pack includes:

Physical disk health and performance monitoring

Storage Node health and performance monitoring
Storage Pool health and performance monitoring
Volume resiliency type and Deduplication status

Understanding Azure Monitor

All data collected by Azure Monitor fits into one of two fundamental types: metrics and
logs.

1. Metrics are numerical values that describe some aspect of a system at a particular
point in time. They are lightweight and capable of supporting near real-time
scenarios. You'll see data collected by Azure Monitor right in their Overview page
in the Azure portal.
 2. Logs contain different kinds of data organized into records with different sets of
properties for each type. Telemetry such as events and traces are stored as logs in
addition to performance data so that it can all be combined for analysis. Log data
collected by Azure Monitor can be analyzed with queries to quickly retrieve,
consolidate, and analyze collected data. You can create and test queries using Log
Analytics in the Azure portal and then either directly analyze the data using these
tools or save queries for use with visualizations or alert rules.

We will have more details below on how to configure these alerts.

Onboarding your cluster using Windows Admin

Center
Using Windows Admin Center, you can onboard your cluster to Azure Monitor.
During this onboarding flow, the steps below are happening under the hood. We detail
how to configure them in detail in case you want to manually setup your cluster.

Configuring Health Service

The first thing that you need to do is configure your cluster. As you may know, the
Health Service improves the day-to-day monitoring and operational experience for
clusters running Storage Spaces Direct.

As we saw above, Azure Monitor collects logs from each node that it is running on in
your cluster. So, we have to configure the Health Service to write to an event channel,
which happens to be:

Event Channel: Microsoft-Windows-Health/Operational

Event ID: 8465

To configure the Health Service, you run:

PowerShell

get-storagesubsystem clus* | Set-StorageHealthSetting -Name

"Platform.ETW.MasTypes" -Value
"Microsoft.Health.EntityType.Subsystem,Microsoft.Health.EntityType.Server,Mi
crosoft.Health.EntityType.PhysicalDisk,Microsoft.Health.EntityType.StoragePo
ol,Microsoft.Health.EntityType.Volume,Microsoft.Health.EntityType.Cluster"
When you run the cmdlet above to set the Health Settings, you cause the events we
want to begin being written to the Microsoft-Windows-Health/Operational event
channel.

Configuring Log Analytics

Now that you have setup the proper logging on your cluster, the next step is to properly
configure log analytics.

To give an overview, Azure Log Analytics can collect data directly from your physical or
virtual Windows computers in your datacenter or other cloud environment into a single
repository for detailed analysis and correlation.

To understand the supported configuration, review supported Windows operating

systems and network firewall configuration.

If you don't have an Azure subscription, create a free account before you begin.

Login in to Azure Portal

Log in to the Azure portal at https://portal.azure.com .

Create a workspace

For more details on the steps listed below, see the Azure Monitor documentation.

1. In the Azure portal, click All services. In the list of resources, type Log Analytics. As
you begin typing, the list filters based on your input. Select Log Analytics.

2. Click Create, and then select choices for the following items:
 Provide a name for the new Log Analytics Workspace, such as
DefaultLAWorkspace.

Select a Subscription to link to by selecting from the drop-down list if the

default selected is not appropriate.

For Resource Group, select an existing resource group that contains one or
more Azure virtual machines.

3. After providing the required information on the Log Analytics Workspace pane,
click OK.

While the information is verified and the workspace is created, you can track its progress
under Notifications from the menu.

Obtain workspace ID and key

Before installing the Microsoft Monitoring Agent for Windows, you need the workspace
ID and key for your Log Analytics workspace. This information is required by the setup
wizard to properly configure the agent and ensure it can successfully communicate with
Log Analytics.

1. In the Azure portal, click All services found in the upper left-hand corner. In the list
of resources, type Log Analytics. As you begin typing, the list filters based on your
input. Select Log Analytics.
2. In your list of Log Analytics workspaces, select DefaultLAWorkspace created earlier.
3. Select Advanced settings.

4. Select Connected Sources, and then select Windows Servers.

5. The value to the right of Workspace ID and Primary Key. Save both temporarily -
copy and paste both into your favorite editor for the time being.

Installing the agent on Windows

The following steps install and configure the Microsoft Monitoring Agent. Be sure to
install this agent on each server in your cluster and indicate that you want the agent
to run at Windows Startup.

1. On the Windows Servers page, select the appropriate Download Windows Agent
version to download depending on the processor architecture of the Windows
operating system.
2. Run Setup to install the agent on your computer.
3. On the Welcome page, click Next.
4. On the License Terms page, read the license and then click I Agree.
5. On the Destination Folder page, change or keep the default installation folder and
then click Next.
6. On the Agent Setup Options page, choose to connect the agent to Azure Log
Analytics and then click Next.
7. On the Azure Log Analytics page, perform the following:
a. Paste the Workspace ID and Workspace Key (Primary Key) that you copied
earlier. a. If the computer needs to communicate through a proxy server to the
Log Analytics service, click Advanced and provide the URL and port number of
the proxy server. If your proxy server requires authentication, type the username
and password to authenticate with the proxy server and then click Next.
8. Click Next once you have completed providing the necessary configuration
settings.
 9. On the Ready to Install page, review your choices and then click Install.
10. On the Configuration completed successfully page, click Finish.

When complete, the Microsoft Monitoring Agent appears in Control Panel. You can
review your configuration and verify that the agent is connected to Log Analytics. When
connected, on the Azure Log Analytics tab, the agent displays a message stating: The
Microsoft Monitoring Agent has successfully connected to the Microsoft Log
Analytics service.
To understand the supported configuration, review supported Windows operating
systems and network firewall configuration.

Setting up alerts using Windows Admin Center

In Windows Admin Center, you can configure default alerts that will apply to all servers
in your Log Analytics workspace.
These are the alerts and their default conditions that you can opt into:

Alert Name Default Condition

CPU utilization Over 85% for 10 minutes

Disk capacity utilization Over 85% for 10 minutes

Memory utilization Available memory less than 100 MB for 10 minutes

Heartbeat Fewer than 2 beats for 5 minutes

System critical error Any critical alert in the cluster system event log

Health service alert Any health service fault on the cluster

Once you configure the alerts in Windows Admin Center, you can see the alerts in your
log analytics workspace in Azure.
During this onboarding flow, the steps below are happening under the hood. We detail
how to configure them in detail in case you want to manually setup your cluster.

Collecting event and performance data

Log Analytics can collect events from the Windows event log and performance counters
that you specify for longer term analysis and reporting, and take action when a
particular condition is detected. Follow these steps to configure collection of events
from the Windows event log, and several common performance counters to start with.

1. In the Azure portal, click More services found on the lower left-hand corner. In the
list of resources, type Log Analytics. As you begin typing, the list filters based on
your input. Select Log Analytics.
2. Select Advanced settings.
3. Select Data, and then select Windows Event Logs.
4. Here, add the Health Service event channel by typing in the name below and the
click the plus sign +.

Event Channel: Microsoft-Windows-Health/Operational

5. In the table, check the severities Error and Warning.

6. Click Save at the top of the page to save the configuration.
7. Select Windows Performance Counters to enable collection of performance
counters on a Windows computer.
8. When you first configure Windows Performance counters for a new Log Analytics
workspace, you are given the option to quickly create several common counters.
They are listed with a checkbox next to each.
 Click Add the selected performance counters. They are added and preset with a
ten second collection sample interval.
9. Click Save at the top of the page to save the configuration.

Creating alerts based on log data

If you've made it this far, your cluster should be sending your logs and performance
counters to Log Analytics. The next step is to create alert rules that automatically run log
searches at regular intervals. If results of the log search match particular criteria, then an
alert is fired that sends you an email or text notification. Let's explore this below.

Create a query
Start by opening the Log Search portal.

1. In the Azure portal, click All services. In the list of resources, type Monitor. As you
begin typing, the list filters based on your input. Select Monitor.
2. On the Monitor navigation menu, select Log Analytics and then select a
workspace.

The quickest way to retrieve some data to work with is a simple query that returns all
records in table. Type the following queries in the search box and click the search
button.

Event
Data is returned in the default list view, and you can see how many total records were
returned.

On the left side of the screen is the filter pane which allows you to add filtering to the
query without modifying it directly. Several record properties are displayed for that
record type, and you can select one or more property values to narrow your search
results.

Select the checkbox next to Error under EVENTLEVELNAME or type the following to
limit the results to error events.

Event | where (EventLevelName == "Error")

After you have the appropriate queries made for events you care about, save them for
the next step.

Create alerts
Now, let's walk through an example for creating an alert.

1. In the Azure portal, click All services. In the list of resources, type Log Analytics. As
you begin typing, the list filters based on your input. Select Log Analytics.

2. In the left-hand pane, select Alerts and then click New Alert Rule from the top of
the page to create a new alert.
3. For the first step, under the Create Alert section, you are going to select your Log
Analytics workspace as the resource, since this is a log based alert signal. Filter the
results by choosing the specific Subscription from the drop-down list if you have
more than one, which contains Log Analytics workspace created earlier. Filter the
Resource Type by selecting Log Analytics from the drop-down list. Finally, select
the Resource DefaultLAWorkspace and then click Done.

4. Under the section Alert Criteria, click Add Criteria to select your saved query and
then specify logic that the alert rule follows.

5. Configure the alert with the following information: a. From the Based on drop-
down list, select Metric measurement. A metric measurement will create an alert
for each object in the query with a value that exceeds our specified threshold. b.
For the Condition, select Greater than and specify a threshold. c. Then define
 when to trigger the alert. For example you could select Consecutive breaches and
from the drop-down list select Greater than a value of 3. d. Under Evaluation
based on section, modify the Period value to 30 minutes and Frequency to 5. The
rule will run every five minutes and return records that were created within the last
thirty minutes from the current time. Setting the time period to a wider window
accounts for the potential of data latency, and ensures the query returns data to
avoid a false negative where the alert never fires.

6. Click Done to complete the alert rule.

7. Now moving onto the second step, provide a name of your alert in the Alert rule
name field, such as Alert on all Error Events. Specify a Description detailing
specifics for the alert, and select Critical(Sev 0) for the Severity value from the
options provided.

8. To immediately activate the alert rule on creation, accept the default value for
Enable rule upon creation.

9. For the third and final step, you specify an Action Group, which ensures that the
same actions are taken each time an alert is triggered and can be used for each
rule you define. Configure a new action group with the following information: a.
Select New action group and the Add action group pane appears. b. For Action
group name, specify a name such as IT Operations - Notify and a Short name
such as itops-n. c. Verify the default values for Subscription and Resource group
are correct. If not, select the correct one from the drop-down list. d. Under the
Actions section, specify a name for the action, such as Send Email and under
 Action Type select Email/SMS/Push/Voice from the drop-down list. The
Email/SMS/Push/Voice properties pane will open to the right in order to provide
additional information. e. On the Email/SMS/Push/Voice pane, select and setup
your preference. For example, enable Email and provide a valid email SMTP
address to deliver the message to. f. Click OK to save your changes.

10. Click OK to complete the action group.

11. Click Create alert rule to complete the alert rule. It starts running immediately.

Example alert
For reference, this is what an example alert looks like in Azure.

Below is an example of the email that you will be send by Azure Monitor:
Additional References
Storage Spaces Direct overview
For more detailed information, read the Azure Monitor documentation.
Read this for an overview on how to connect to other Azure hybrid services.
Adjustable storage repair speed in
Azure Stack HCI and Windows Server
Article • 04/17/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022

User-adjustable storage repair speed offers more control over the data resync process
by allocating resources to either repair data copies (resiliency) or run active workloads
(performance). This helps improve availability and allows you to service your clusters
more flexibly and efficiently.

Storage repair speed settings

The storage speed repair settings are:

Setting Queue depth Resource allocation

Very low 1 Most resources to active workloads

Low 2 More resources to active workloads

Medium (default) 4 Balances workloads and repairs

High 8 More resources to resync and repairs

Very high 16 Most resources to resync and repairs

Very low to Low settings will allocate more resources to active workloads, allowing the
server to complete mission-critical tasks. Take caution when changing the setting to
Very low, as this may impact the time it takes for storage to regain full resiliency. High
to Very high settings will prioritize storage resync and repairs, allowing a patch and
update cycle to complete faster. The default setting is Medium, which ensures a balance
of workload and repair prioritization. It's recommended to keep the repair speed set to
Medium in most cases, unless you have a good reason for prioritizing either resiliency
or performance.

） Important

When storage repair speed is set to Very high, active workloads and cluster
performance can be impacted. Likewise, when repair speed is set to Very low, it will
take more time for storage to regain full resiliency.
Change storage repair speed using Windows
Admin Center
To change the storage repair speed for a cluster by using Windows Admin Center, do
the following:

1. In Windows Admin Center, connect to a cluster, and then select Settings from the
Tools pane.
2. In the Settings pane, select Storage Spaces and pools.
3. Use the Storage repair speed dropdown to select the desired speed.
4. Review the caution text if selecting Very low or Very high.
5. Select Save.

Change storage repair speed using PowerShell:

To check the current repair speed setting:

PowerShell

Get-StorageSubSystem -FriendlyName <name of cluster subsystem> | ft

FriendlyName,VirtualDiskRepairQueueDepth

To set the repair speed:

PowerShell
 Set-StorageSubSystem -FriendlyName <name of cluster subsystem> -
VirtualDiskRepairQueueDepth <value>

Next steps
For more information, see also:

Taking an Azure Stack HCI server offline for maintenance

Troubleshoot Storage Spaces Direct
Article • 04/28/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022, Windows Server 2019, Windows Server 2016

Use the following information to troubleshoot your Storage Spaces Direct deployment.

In general, start with the following steps:

1. Confirm the make/model of SSD is certified for Windows Server 2016 and Windows Server 2019 using the Windows Server Catalog.
Confirm with vendor that the drives are supported for Storage Spaces Direct.
2. Inspect the storage for any faulty drives. Use storage management software to check the status of the drives. If any of the drives are
faulty, work with your vendor.
3. Update storage and drive firmware if necessary. Ensure the latest Windows Updates are installed on all nodes. You can get the latest
updates for Windows Server 2016 from Windows 10 and Windows Server 2016 update history and for Windows Server 2019 from
Windows 10 and Windows Server 2019 update history .
4. Update network adapter drivers and firmware.
5. Run cluster validation and review the Storage Space Direct section, ensure the drives that are used for the cache are reported correctly
and with no errors.

If you're still having issues, review the following scenarios.

Virtual disk resources are in No Redundancy state

The nodes of a Storage Spaces Direct system restart unexpectedly because of a crash or power failure. Then, one or more of the virtual
disks may not come online, and you see the description "Not enough redundancy information."

FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size PSComputerName

Disk4 Mirror OK Healthy True 10 TB Node-01.conto...

Disk3 Mirror OK Healthy True 10 TB Node-01.conto...

Disk2 Mirror No Redundancy Unhealthy True 10 TB Node-01.conto...

Disk1 Mirror {No Redundancy, InService} Unhealthy True 10 TB Node-01.conto...

Additionally, after an attempt to bring the virtual disk online, the following information is logged in the Cluster log (DiskRecoveryAction).

[Verbose] 00002904.00001040::YYYY/MM/DD-12:03:44.891 INFO [RES] Physical Disk <DiskName>: OnlineThread: SuGetSpace returned
0.
[Verbose] 00002904.00001040:: YYYY/MM/DD -12:03:44.891 WARN [RES] Physical Disk < DiskName>: Underlying virtual disk is in
'no redundancy' state; its volume(s) may fail to mount.
[Verbose] 00002904.00001040:: YYYY/MM/DD -12:03:44.891 ERR [RES] Physical Disk <DiskName>: Failing online due to virtual
disk in 'no redundancy' state. If you would like to attempt to online the disk anyway, first set this resource's private
property 'DiskRecoveryAction' to 1. We will try to bring the disk online for recovery, but even if successful, its
volume(s) or CSV may be unavailable.

The No Redundancy Operational Status can occur if a disk failed or if the system is unable to access data on the virtual disk. This issue can
occur if a reboot occurs on a node during maintenance on the nodes.

To fix this issue, follow these steps:

1. Remove the affected Virtual Disks from CSV. This puts them in the "Available storage" group in the cluster and start showing as a
ResourceType of "Physical Disk."

PowerShell

Remove-ClusterSharedVolume -name "CSV Name"

2. On the node that owns the Available Storage group, run the following command on every disk that's in a No Redundancy state. To
identify which node the "Available Storage" group is on you can run the following command.

PowerShell
 Get-ClusterGroup

3. Set the disk recovery action and then start the disk(s).

PowerShell

Get-ClusterResource "Physical Disk Resource Name" | Set-ClusterParameter -Name DiskRecoveryAction -Value 1

Start-ClusterResource -Name "Physical Disk Resource Name"

4. A repair should automatically start. Wait for the repair to finish. It may go into a suspended state and start again. To monitor the
progress:

Run Get-StorageJob to monitor the status of the repair and to see when it's completed.
Run Get-VirtualDisk and verify that the Space returns a HealthStatus of Healthy.

5. After the repair finishes and the Virtual Disks are Healthy, change the Virtual Disk parameters back.

PowerShell

Get-ClusterResource "Physical Disk Resource Name" | Set-ClusterParameter -Name DiskRecoveryAction -Value 0

6. Take the disk(s) offline and then online again to have the DiskRecoveryAction take effect:

PowerShell

Stop-ClusterResource "Physical Disk Resource Name"

Start-ClusterResource "Physical Disk Resource Name"

7. Add the affected Virtual Disks back to CSV.

PowerShell

Add-ClusterSharedVolume -name "Physical Disk Resource Name"

DiskRecoveryAction is an override switch that enables attaching the Space volume in read-write mode without any checks. The property
enables you to do diagnostics into why a volume won't come online. It's similar to Maintenance Mode but you can invoke it on a resource
in a Failed state. It also lets you access the data, which can be helpful in situations such as "No Redundancy," where you can get access to
whatever data you can and copy it. The DiskRecoveryAction property was added in the February 22, 2018, update, KB 4077525.

Detached status in a cluster

When you run the Get-VirtualDisk cmdlet, the OperationalStatus for one or more Storage Spaces Direct virtual disks is Detached. However,
the HealthStatus reported by the Get-PhysicalDisk cmdlet indicates that all the physical disks are in a Healthy state.

The following is an example of the output from the Get-VirtualDisk cmdlet.

FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size PSComputerName

Disk4 Mirror OK Healthy True 10 TB Node-01.conto...

Disk3 Mirror OK Healthy True 10 TB Node-01.conto...

Disk2 Mirror Detached Unknown True 10 TB Node-01.conto...

Disk1 Mirror Detached Unknown True 10 TB Node-01.conto...

Additionally, the following events may be logged on the nodes:

Log Name: Microsoft-Windows-StorageSpaces-Driver/Operational

Source: Microsoft-Windows-StorageSpaces-Driver
Event ID: 311
Level: Error
User: SYSTEM
Computer: Node#.contoso.local
Description: Virtual disk {GUID} requires a data integrity scan.
 Data on the disk is out-of-sync and a data integrity scan is required.

To start the scan, run the following command:

Get-ScheduledTask -TaskName "Data Integrity Scan for Crash Recovery" | Start-ScheduledTask

Once you have resolved the condition listed above, you can online the disk by using the following commands in PowerShell:

Get-VirtualDisk | ?{ $_.ObjectId -Match "{GUID}" } | Get-Disk | Set-Disk -IsReadOnly $false

Get-VirtualDisk | ?{ $_.ObjectId -Match "{GUID}" } | Get-Disk | Set-Disk -IsOffline $false
------------------------------------------------------------

Log Name: System

Source: Microsoft-Windows-ReFS
Event ID: 134
Level: Error
User: SYSTEM
Computer: Node#.contoso.local
Description: The file system was unable to write metadata to the media backing volume <VolumeId>. A write failed with
status "A device which does not exist was specified." ReFS will take the volume offline. It may be mounted again
automatically.
------------------------------------------------------------
Log Name: Microsoft-Windows-ReFS/Operational
Source: Microsoft-Windows-ReFS
Event ID: 5
Level: Error
User: SYSTEM
Computer: Node#.contoso.local
Description: ReFS failed to mount the volume.
Context: 0xffffbb89f53f4180
Error: A device which does not exist was specified.
Volume GUID:{00000000-0000-0000-0000-000000000000}
DeviceName:
Volume Name:

The Detached Operational Status can occur if the dirty region tracking (DRT) log is full. Storage Spaces uses dirty region tracking (DRT) for
mirrored spaces to make sure that when a power failure occurs, any in-flight updates to metadata are logged to make sure that the storage
space can redo or undo operations to bring the storage space back into a flexible and consistent state when power is restored and the
system comes back up. If the DRT log is full, the virtual disk can't be brought online until the DRT metadata is synchronized and flushed.
This process requires running a full scan, which can take several hours to finish.

To fix this issue, follow these steps:

1. Remove the affected Virtual Disks from CSV.

PowerShell

Remove-ClusterSharedVolume -name "CSV Name"

2. Run the following commands on every disk that's not coming online.

PowerShell

Get-ClusterResource -Name "Physical Disk Resource Name" | Set-ClusterParameter DiskRunChkDsk 7

Start-ClusterResource -Name "Physical Disk Resource Name"

3. Run the following command on every node in which the detached volume is online.

PowerShell

Get-ScheduledTask -TaskName "Data Integrity Scan for Crash Recovery" | Start-ScheduledTask

This task should be initiated on all nodes on which the detached volume is online. A repair should automatically start. Wait for the
repair to finish. It may go into a suspended state and start again. To monitor the progress:

Run Get-StorageJob to monitor the status of the repair and to see when it's completed.
Run Get-VirtualDisk and verify the Space returns a HealthStatus of Healthy.

The "Data Integrity Scan for Crash Recovery" is a task that doesn't show as a storage job, and there's no progress indicator. If
the task is showing as running, it's running. When it completes, it shows completed.

Additionally, you can view the status of a running schedule task by using the following cmdlet:

PowerShell
 Get-ScheduledTask | ? State -eq running

4. As soon as the "Data Integrity Scan for Crash Recovery" is finished, the repair finishes and the Virtual Disks are Healthy, change the
Virtual Disk parameters back.

PowerShell

Get-ClusterResource -Name "Physical Disk Resource Name" | Set-ClusterParameter DiskRunChkDsk 0

5. Take the disk(s) offline and then online again to have the DiskRecoveryAction take effect:

PowerShell

Stop-ClusterResource "Physical Disk Resource Name"

Start-ClusterResource "Physical Disk Resource Name"

6. Add the affected Virtual Disks back to CSV.

PowerShell

Add-ClusterSharedVolume -name "Physical Disk Resource Name"

DiskRunChkdsk value 7 is used to attach the Space volume and have the partition in read-only mode. This enables Spaces to self-
discover and self-heal by triggering a repair. Repair runs automatically once mounted. It also allows you to access the data, which can
be helpful to get access to whatever data you can copy. For some fault conditions, such as a full DRT log, you need to run the Data
Integrity Scan for Crash Recovery scheduled task.

Data Integrity Scan for Crash Recovery task is used to synchronize and clear a full dirty region tracking (DRT) log. This task can take several
hours to complete. The "Data Integrity Scan for Crash Recovery" is a task that doesn't show as a storage job, and there is no progress
indicator. If the task is showing as running, it is running. When it completes, it shows as completed. If you cancel the task or restart a node
while this task is running, the task needs to start over from the beginning.

For more information, see Troubleshooting Storage Spaces Direct health and operational states.

Event 5120 with STATUS_IO_TIMEOUT c00000b5

） Important

For Windows Server 2016: To reduce the chance of experiencing these symptoms while applying the update with the fix, it's
recommended to use the Storage Maintenance Mode procedure below to install the October 18, 2018, cumulative update for
Windows Server 2016 or a later version when the nodes currently have installed a Windows Server 2016 cumulative update that was
released from May 8, 2018 to October 9, 2018 .

You might get event 5120 with STATUS_IO_TIMEOUT c00000b5 after you restart a node on Windows Server 2016 with cumulative update
that were released from May 8, 2018 KB 4103723 to October 9, 2018 KB 4462917 installed.

When you restart the node, Event 5120 is logged in the System event log and includes one of the following error codes:

Event Source: Microsoft-Windows-FailoverClustering

Event ID: 5120
Description: Cluster Shared Volume 'CSVName' ('Cluster Virtual Disk (CSVName)') has entered a paused state because of
'STATUS_IO_TIMEOUT(c00000b5)'. All I/O will temporarily be queued until a path to the volume is reestablished.

Cluster Shared Volume 'CSVName' ('Cluster Virtual Disk (CSVName)') has entered a paused state because of
'STATUS_CONNECTION_DISCONNECTED(c000020c)'. All I/O will temporarily be queued until a path to the volume is reestablished.

When an Event 5120 is logged, a live dump is generated to collect debugging information that may cause additional symptoms or have a
performance effect. Generating the live dump creates a brief pause to enable taking a snapshot of memory to write the dump file.
Systems that have lots of memory and are under stress may cause nodes to drop out of cluster membership and also cause the following
Event 1135 to be logged.
 Event source: Microsoft-Windows-FailoverClustering
Event ID: 1135
Description: Cluster node 'NODENAME'was removed from the active failover cluster membership. The Cluster service on this
node may have stopped. This could also be due to the node having lost communication with other active nodes in the failover
cluster. Run the Validate a Configuration wizard to check your network configuration. If the condition persists, check for
hardware or software errors related to the network adapters on this node. Also check for failures in any other network
components to which the node is connected such as hubs, switches, or bridges.

A change introduced in May 8, 2018 to Windows Server 2016, which was a cumulative update to add SMB Resilient Handles for the Storage
Spaces Direct intra-cluster SMB network sessions. This was done to improve resiliency to transient network failures and improve how RoCE
handles network congestion. These improvements also inadvertently increased timeouts when SMB connections try to reconnect and waits
to time out when a node is restarted. These issues can affect a system that is under stress. During unplanned downtime, IO pauses of up to
60 seconds have also been observed while the system waits for connections to time out. To fix this issue, install the October 18, 2018,
cumulative update for Windows Server 2016 or a later version.

Note This update aligns the CSV time-outs with SMB connection time-outs to fix this issue. It doesn't implement the changes to disable live
dump generation mentioned in the Workaround section.

Shutdown process flow

1. Run the Get-VirtualDisk cmdlet, and make sure that the HealthStatus value is Healthy.

2. Drain the node by running the following cmdlet:

PowerShell

Suspend-ClusterNode -Drain

3. Put the disks on that node in Storage Maintenance Mode by running the following cmdlet:

PowerShell

Get-StorageFaultDomain -type StorageScaleUnit | Where-Object {$_.FriendlyName -eq "<NodeName>"} | Enable-

StorageMaintenanceMode

4. Run the Get-PhysicalDisk cmdlet, and make sure that the OperationalStatus value is In Maintenance Mode.

5. Run the Restart-Computer cmdlet to restart the node.

6. After node restarts, remove the disks on that node from Storage Maintenance Mode by running the following cmdlet:

PowerShell

Get-StorageFaultDomain -type StorageScaleUnit | Where-Object {$_.FriendlyName -eq "<NodeName>"} | Disable-

StorageMaintenanceMode

7. Resume the node by running the following cmdlet:

PowerShell

Resume-ClusterNode

8. Check the status of the resync jobs by running the following cmdlet:

PowerShell

Get-StorageJob

Disabling live dumps

To mitigate the effect of live dump generation on systems that have lots of memory and are under stress, you may additionally want to
disable live dump generation. Three options are provided below.

Ｕ Caution
 This procedure can prevent the collection of diagnostic information that Microsoft Support may need to investigate this problem. A
Support agent may have to ask you to re-enable live dump generation based on specific troubleshooting scenarios.

The two methods to disable live dumps are as follows:

Method 1 (recommended in this scenario)

To completely disable all dumps, including live dumps system-wide, follow these steps:

1. Create the following registry key: HKLM\System\CurrentControlSet\Control\CrashControl\ForceDumpsDisabled

2. Under the new ForceDumpsDisabled key, create a REG_DWORD property as GuardedHost, and then set its value to 0x10000000.
3. Apply the new registry key to each cluster node.

７ Note

You have to restart the computer for the nregistry change to take effect.

After this registry key is set, live dump creation will fail and generate a "STATUS_NOT_SUPPORTED" error.

Method 2

By default, Windows Error Reporting allows only one LiveDump per report type per 7 days and only 1 LiveDump per machine per 5 days.
You can change that by setting the following registry keys to only allow one LiveDump on the machine forever.

reg add "HKLM\Software\Microsoft\Windows\Windows Error Reporting\FullLiveKernelReports" /v SystemThrottleThreshold /t

REG_DWORD /d 0xFFFFFFFF /f

reg add "HKLM\Software\Microsoft\Windows\Windows Error Reporting\FullLiveKernelReports" /v ComponentThrottleThreshold /t

REG_DWORD /d 0xFFFFFFFF /f

Note You have to restart the computer for the change to take effect.

Method 3
To disable cluster generation of live dumps (such as when an Event 5120 is logged), run the following cmdlet:

PowerShell

(Get-Cluster).DumpPolicy = ((Get-Cluster).DumpPolicy -band 0xFFFFFFFFFFFFFFFE)

This cmdlet has an immediate effect on all cluster nodes without a computer restart.

Slow IO performance
If you see slow IO performance, check if cache is enabled in your Storage Spaces Direct configuration.

There are two ways to check:

1. Using the cluster log. Open the cluster log in text editor of choice and search for "[=== SBL Disks ===]." This is a list of the disk on
the node the log was generated on.

Cache Enabled Disks Example: Note here that the state is CacheDiskStateInitializedAndBound and there's a GUID present here.

[=== SBL Disks ===]

{26e2e40f-a243-1196-49e3-8522f987df76},3,false,true,1,48,{1ff348f1-d10d-7a1a-d781-
4734f4440481},CacheDiskStateInitializedAndBound,1,8087,54,false,false,HGST ,HUH721010AL4200 , 7PG3N2ER,A21D,
{d5e27a3b-42fb-410a-81c6-9d8cc12da20c},[R/M 0 R/U 0 R/T 0 W/M 0 W/U 0 W/T 0],
 Cache Not Enabled: Here we can see there's no GUID present and the state is CacheDiskStateNonHybrid.

[=== SBL Disks ===]

{426f7f04-e975-fc9d-28fd-72a32f811b7d},12,false,true,1,24,{00000000-0000-0000-0000-
000000000000},CacheDiskStateNonHybrid,0,0,0,false,false,HGST ,HUH721010AL4200 , 7PGXXG6C,A21D,{d5e27a3b-
42fb-410a-81c6-9d8cc12da20c},[R/M 0 R/U 0 R/T 0 W/M 0 W/U 0 W/T 0],

Cache Not Enabled: When all disks are of the same type case is not enabled by default. Here we can see there's no GUID present and
the state is CacheDiskStateIneligibleDataPartition.

{d543f90c-798b-d2fe-7f0a-cb226c77eeed},10,false,false,1,20,{00000000-0000-0000-0000-
000000000000},CacheDiskStateIneligibleDataPartition,0,0,0,false,false,NVMe ,INTEL SSDPE7KX02,
PHLF7330004V2P0LGN,0170,{79b4d631-976f-4c94-a783-df950389fd38},[R/M 0 R/U 0 R/T 0 W/M 0 W/U 0 W/T 0],

2. Using Get-PhysicalDisk.xml from the SDDCDiagnosticInfo

a. Open the XML file using "$d = Import-Clixml GetPhysicalDisk.XML"
b. Run "ipmo storage"
c. run "$d". Note that Usage is Auto-Select, not Journal You see output like this:

FriendlyName SerialNumber MediaType CanPool OperationalStatus HealthStatus Usage Size

NVMe INTEL SSDPE7KX02 PHLF733000372P0LGN SSD False OK Healthy Auto-Select 1.82 TB

NVMe INTEL SSDPE7KX02 PHLF7504008J2P0LGN SSD False OK Healthy Auto-Select 1.82 TB

NVMe INTEL SSDPE7KX02 PHLF7504005F2P0LGN SSD False OK Healthy Auto-Select 1.82 TB

NVMe INTEL SSDPE7KX02 PHLF7504002A2P0LGN SSD False OK Healthy Auto-Select 1.82 TB

NVMe INTEL SSDPE7KX02 PHLF7504004T2P0LGN SSD False OK Healthy Auto-Select 1.82 TB

NVMe INTEL SSDPE7KX02 PHLF7504002E2P0LGN SSD False OK Healthy Auto-Select 1.82 TB

NVMe INTEL SSDPE7KX02 PHLF7330002Z2P0LGN SSD False OK Healthy Auto-Select 1.82 TB

NVMe INTEL SSDPE7KX02 PHLF733000272P0LGN SSD False OK Healthy Auto-Select 1.82 TB

NVMe INTEL SSDPE7KX02 PHLF7330001J2P0LGN SSD False OK Healthy Auto-Select 1.82 TB

NVMe INTEL SSDPE7KX02 PHLF733000302P0LGN SSD False OK Healthy Auto-Select 1.82 TB

NVMe INTEL SSDPE7KX02 PHLF7330004D2P0LGN SSD False OK Healthy Auto-Select 1.82 TB

How to destroy an existing cluster so you can use the same disks again
In a Storage Spaces Direct cluster, once you disable Storage Spaces Direct and use the clean-up process described in Clean drives, the
clustered storage pool still remains in an Offline state, and the Health Service is removed from cluster.

The next step is to remove the phantom storage pool:

PowerShell

Get-ClusterResource -Name "Cluster Pool 1" | Remove-ClusterResource

Now, if you run Get-PhysicalDisk on any of the nodes, you see all the disks that were in the pool. For example, in a lab with a 4-Node
cluster with 4 SAS disks, 100 GB each presented to each node. In that case, after Storage Space Direct is disabled, which removes the SBL
(Storage Bus Layer) but leaves the filter, if you run Get-PhysicalDisk, it should report 4 disks excluding the local OS disk. Instead it reported
16 instead. This is the same for all nodes in the cluster. When you run a Get-Disk command, you see the locally attached disks numbered as
0, 1, 2, and so on, as seen in this sample output:

Number Friendly Name Serial Number HealthStatus OperationalStatus Total Size Partition Style

0 Msft Virtu... Healthy Online 127 GB GPT

Msft Virtu... Healthy Offline 100 GB RAW

 Number Friendly Name Serial Number HealthStatus OperationalStatus Total Size Partition Style

Msft Virtu... Healthy Offline 100 GB RAW

Msft Virtu... Healthy Offline 100 GB RAW

Msft Virtu... Healthy Offline 100 GB RAW

1 Msft Virtu... Healthy Offline 100 GB RAW

Msft Virtu... Healthy Offline 100 GB RAW

2 Msft Virtu... Healthy Offline 100 GB RAW

Msft Virtu... Healthy Offline 100 GB RAW

Msft Virtu... Healthy Offline 100 GB RAW

Msft Virtu... Healthy Offline 100 GB RAW

Msft Virtu... Healthy Offline 100 GB RAW

4 Msft Virtu... Healthy Offline 100 GB RAW

3 Msft Virtu... Healthy Offline 100 GB RAW

Msft Virtu... Healthy Offline 100 GB RAW

Msft Virtu... Healthy Offline 100 GB RAW

Msft Virtu... Healthy Offline 100 GB RAW

Error message about "unsupported media type" when you create a Storage
Spaces Direct cluster using Enable-ClusterS2D
You might see errors similar to this when you run the Enable-ClusterS2D cmdlet:

To fix this issue, ensure the HBA adapter is configured in HBA mode. No HBA should be configured in RAID mode.

Enable-ClusterStorageSpacesDirect hangs at 'Waiting until SBL disks are

surfaced' or at 27%
You see the following information in the validation report:
Disk <identifier> connected to node <nodename> returned a SCSI Port Association and the corresponding enclosure device could not be
found. The hardware is not compatible with Storage Spaces Direct (S2D), contact the hardware vendor to verify support for SCSI Enclosure
Services (SES).

The issue is with the HPE SAS expander card that lies between the disks and the HBA card. The SAS expander creates a duplicate ID
between the first drive connected to the expander and the expander itself. This has been resolved in HPE Smart Array Controllers SAS
Expander Firmware: 4.02 .

Intel SSD DC P4600 series has a non-unique NGUID

You might see an issue where an Intel SSD DC P4600 series device seems to be reporting similar 16 byte NGUID for multiple namespaces
such as 0100000001000000E4D25C000014E214 or 0100000001000000E4D25C0000EEE214 in the following example.

uniqueid deviceid MediaType BusType serialnumber size canpool fri

5000CCA251D12E30 0 HDD SAS 7PKR197G 10000831348736 False HG

eui.0100000001000000E4D25C000014E214 4 SSD NVMe 0100_0000_0100_0000_E4D2_5C00_0014_E214. 1600321314816 True INT

eui.0100000001000000E4D25C000014E214 5 SSD NVMe 0100_0000_0100_0000_E4D2_5C00_0014_E214. 1600321314816 True INT

eui.0100000001000000E4D25C0000EEE214 6 SSD NVMe 0100_0000_0100_0000_E4D2_5C00_00EE_E214. 1600321314816 True INT

eui.0100000001000000E4D25C0000EEE214 7 SSD NVMe 0100_0000_0100_0000_E4D2_5C00_00EE_E214. 1600321314816 True INT

To fix this issue, update the firmware on the Intel drives to the latest version. Firmware version QDV101B1 from May 2018 is known to
resolve this issue.

The May 2018 release of the Intel SSD Data Center Tool includes a firmware update, QDV101B1, for the Intel SSD DC P4600 series.

Physical Disk "Healthy," and Operational Status is "Removing from Pool"

In a Windows Server 2016 Storage Spaces Direct cluster, you might see the HealthStatus for one or more physical disks as "Healthy," while
the OperationalStatus is "(Removing from Pool, OK)."

"Removing from Pool" is an intent set when Remove-PhysicalDisk is called but stored in Health to maintain state and allow recovery if the
remove operation fails. You can manually change the OperationalStatus to Healthy with one of the following methods:

Remove the physical disk from the pool, and then add it back.
Import-Module Clear-PhysicalDiskHealthData.ps1
Run the Clear-PhysicalDiskHealthData.ps1 script to clear the intent. (Available for download as a .TXT file. You need to save it as a
.PS1 file before you can run it.)

Here are some examples showing how to run the script:

Use the SerialNumber parameter to specify the disk you need to set to Healthy. You can get the serial number from WMI
MSFT_PhysicalDisk or Get-PhysicalDIsk. (We're just using 0s for the serial number below.)

PowerShell

Clear-PhysicalDiskHealthData -Intent -Policy -SerialNumber 000000000000000 -Verbose -Force

Use the UniqueId parameter to specify the disk (again from WMI MSFT_PhysicalDisk or Get-PhysicalDIsk).

PowerShell

Clear-PhysicalDiskHealthData -Intent -Policy -UniqueId 00000000000000000 -Verbose -Force

File copy is slow

You might see an issue using File Explorer to copy a large VHD to the virtual disk - the file copy is taking longer than expected.

Using File Explorer, Robocopy or Xcopy to copy a large VHD to the virtual disk is not a recommended method to as this results in slower
than expected performance. The copy process doesn't go through the Storage Spaces Direct stack, which sits lower on the storage stack,
and instead acts like a local copy process.
If you want to test Storage Spaces Direct performance, we recommend using VMFleet and Diskspd to load and stress test the servers to get
a base line and set expectations of the Storage Spaces Direct performance.

Expected events that you would see on rest of the nodes during the reboot
of a node.
It's safe to ignore these events:

Event ID 205: Windows lost communication with physical disk {XXXXXXXXXXXXXXXXXXXX }. This can occur if a cable failed or
was disconnected, or if the disk itself failed.

Event ID 203: Windows lost communication with physical disk {xxxxxxxxxxxxxxxxxxxxxxxx }. This can occur if a cable failed
or was disconnected, or if the disk itself failed.

If you're running Azure VMs, you can ignore this event: Event ID 32: The driver detected that the device \Device\Harddisk5\DR5 has its
write cache enabled. Data corruption may occur.

Slow performance or "Lost Communication," "IO Error," "Detached," or

"No Redundancy" errors for deployments that use Intel P3x00 NVMe
devices
We've identified a critical issue that affects some Storage Spaces Direct users who are using hardware based on the Intel P3x00 family of
NVM Express (NVMe) devices with firmware versions before "Maintenance Release 8."

７ Note

Individual OEMs may have devices that are based on the Intel P3x00 family of NVMe devices with unique firmware version strings.
Contact your OEM for more information of the latest firmware version.

If you're using hardware in your deployment based on the Intel P3x00 family of NVMe devices, we recommend that you immediately apply
the latest available firmware (at least Maintenance Release 8).
Troubleshoot Storage Spaces and
Storage Spaces Direct health and
operational states
Article • 04/28/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019, Windows Server 2016, Windows Server 2012 R2, Windows
Server 2012, Windows 10, Windows 8.1

This topic describes the health and operational states of storage pools, virtual disks
(which sit underneath volumes in Storage Spaces), and drives in Storage Spaces Direct
and Storage Spaces. These states can be invaluable when trying to troubleshoot various
issues such as why you can't delete a virtual disk because of a read-only configuration. It
also discusses why a drive can't be added to a pool (the CannotPoolReason).

Storage Spaces has three primary objects - physical disks (hard drives, SSDs, etc.) that
are added to a storage pool, virtualizing the storage so that you can create virtual disks
from free space in the pool, as shown here. Pool metadata is written to each drive in the
pool. Volumes are created on top of the virtual disks and store your files, but we're not
going to talk about volumes here.
You can view health and operational states in Server Manager, or with PowerShell. Here's
an example of a variety of (mostly bad) health and operational states on a Storage
Spaces Direct cluster that's missing most of its cluster nodes (right-click the column
headers to add Operational Status). This isn't a happy cluster.

Storage pool states

Every storage pool has a health status - Healthy, Warning, or Unknown/Unhealthy, as
well as one or more operational states.

To find out what state a pool is in, use the following PowerShell commands:

PowerShell

Get-StoragePool -IsPrimordial $False | Select-Object HealthStatus,

OperationalStatus, ReadOnlyReason

Here's an example output showing a storage pool in the Unknown health state with the
Read-only operational status:

FriendlyName OperationalStatus HealthStatus IsPrimordial

IsReadOnly
------------ ----------------- ------------ ------------ ----
------
S2D on StorageSpacesDirect1 Read-only Unknown False True
The following sections list the health and operational states.

Pool health state: Healthy

Operational state Description

OK The storage pool is healthy.

Pool health state: Warning

When the storage pool is in the Warning health state, it means that the pool is
accessible, but one or more drives failed or are missing. As a result, your storage pool
might have reduced resilience.

Operational Description
state

Degraded There are failed or missing drives in the storage pool. This condition occurs only
with drives hosting pool metadata.

Action: Check the state of your drives and replace any failed drives before there
are additional failures.

Pool health state: Unknown or Unhealthy

When a storage pool is in the Unknown or Unhealthy health state, it means that the
storage pool is read-only and can't be modified until the pool is returned to the
Warning or OK health states.

Operational Read-only Description

state reason
 Operational Read-only Description
state reason

Read-only Incomplete This can occur if the storage pool loses its quorum, which means
that most drives in the pool have failed or are offline for some
reason. When a pool loses its quorum, Storage Spaces automatically
sets the pool configuration to read-only until enough drives become
available again.

Action:
1. Reconnect any missing drives, and if you're using Storage Spaces
Direct, bring all servers online.
2. Set the pool back to read-write by opening a PowerShell session
with administrative permissions and then typing:

Get-StoragePool <PoolName> -IsPrimordial $False | Set-

StoragePool -IsReadOnly $false

Policy An administrator set the storage pool to read-only.

Action: To set a clustered storage pool to read-write access in

Failover Cluster Manager, go to Pools, right-click the pool and then
select Bring Online.

For other servers and PCs, open a PowerShell session with

administrative permissions and then type:

Get-StoragePool <PoolName> | Set-StoragePool -IsReadOnly $false

Starting Storage Spaces is starting or waiting for drives to be connected in

the pool. This should be a temporary state. Once completely started,
the pool should transition to a different operational state.

Action: If the pool stays in the Starting state, make sure that all
drives in the pool are connected properly.

See also, the Windows Server storage forum.

Virtual disk states

In Storage Spaces, volumes are placed on virtual disks (storage spaces) that are carved
out of free space in a pool. Every virtual disk has a health status - Healthy, Warning,
Unhealthy, or Unknown as well as one or more operational states.

To find out what state virtual disks are in, use the following PowerShell commands:
 PowerShell

Get-VirtualDisk | Select-Object FriendlyName,HealthStatus,

OperationalStatus, DetachedReason

Here's an example of output showing a detached virtual disk and a

degraded/incomplete virtual disk:

FriendlyName HealthStatus OperationalStatus DetachedReason

------------ ------------ ----------------- --------------
Volume1 Unknown Detached By Policy
Volume2 Warning {Degraded, Incomplete} None

The following sections list the health and operational states.

Virtual disk health state: Healthy

Operational Description
state

OK The virtual disk is healthy.

Suboptimal Data isn't written evenly across drives.

Action: Optimize drive usage in the storage pool by running the Optimize-
StoragePool cmdlet.

Virtual disk health state: Warning

When the virtual disk is in a Warning health state, it means that one or more copies of
your data are unavailable, but Storage Spaces can still read at least one copy of your
data.

Operational Description
state

In service Windows is repairing the virtual disk, such as after adding or removing a drive.
When the repair is complete, the virtual disk should return to the OK health state.
 Operational Description
state

Incomplete The resilience of the virtual disk is reduced because one or more drives failed or
are missing. However, the missing drives contain up-to-date copies of your data.

Action:
1. Reconnect any missing drives, replace any failed drives, and if you're using
Storage Spaces Direct, bring online any servers that are offline.
2. If you're not using Storage Spaces Direct, next repair the virtual disk using the
Repair-VirtualDisk cmdlet.
Storage Spaces Direct automatically starts a repair if needed after reconnecting or
replacing a drive.

Degraded The resilience of the virtual disk is reduced because one or more drives failed or
are missing, and there are outdated copies of your data on these drives.

Action:
1. Reconnect any missing drives, replace any failed drives, and if you're using
Storage Spaces Direct, bring online any servers that are offline.
2. If you're not using Storage Spaces Direct, next repair the virtual disk using the
Repair-VirtualDisk cmdlet.
Storage Spaces Direct automatically starts a repair if needed after reconnecting or
replacing a drive.

Virtual disk health state: Unhealthy

When a virtual disk is in an Unhealthy health state, some or all of the data on the virtual
disk is currently inaccessible.

Operational state Description

No redundancy The virtual disk has lost data because too many drives failed.

Action: Replace failed drives and then restore your data from backup.

Virtual disk health state: Information/Unknown

The virtual disk can also be in the Information health state (as shown in the Storage
Spaces Control Panel item) or Unknown health state (as shown in PowerShell) if an
administrator took the virtual disk offline or the virtual disk has become detached.

Operational Detached Description

state reason
Operational Detached Description
state reason

Detached By Policy An administrator took the virtual disk offline, or set the virtual disk to
require manual attachment, in which case you'll have to manually
attach the virtual disk every time Windows restarts.,

Action: Bring the virtual disk back online. To do so when the virtual
disk is in a clustered storage pool, in Failover Cluster Manager select
Storage > Pools > Virtual Disks, select the virtual disk that shows
the Offline status and then select Bring Online.

To bring a virtual disk back online when not in a cluster, open a

PowerShell session as an Administrator and then try using the
following command:

Get-VirtualDisk | Where-Object -Filter { $_.OperationalStatus -

eq "Detached" } | Connect-VirtualDisk

To automatically attach all non-clustered virtual disks after Windows

restarts, open a PowerShell session as an Administrator and then use
the following command:

Get-VirtualDisk | Set-VirtualDisk -ismanualattach $false

Majority Too many drives used by this virtual disk failed, are missing, or have
Disks stale data.
Unhealthy
Action:
1. Reconnect any missing drives, and if you're using Storage Spaces
Direct, bring online any servers that are offline.
2. After all drives and servers are online, replace any failed drives.
See Health Service for details.
Storage Spaces Direct automatically starts a repair if needed after
reconnecting or replacing a drive.
3. If you're not using Storage Spaces Direct, next repair the virtual
disk using the Repair-VirtualDisk cmdlet.

If more disks failed than you have copies of your data and the virtual
disk wasn't repaired in-between failures, all data on the virtual disk is
permanently lost. In this unfortunate case, delete the virtual disk,
create a new virtual disk, and then restore from a backup.
 Operational Detached Description
state reason

Incomplete Not enough drives are present to read the virtual disk.

Action:
1. Reconnect any missing drives, and if you're using Storage Spaces
Direct, bring online any servers that are offline.
2. After all drives and servers are online, replace any failed drives.
See Health Service for details.
Storage Spaces Direct automatically starts a repair if needed after
reconnecting or replacing a drive.
3. If you're not using Storage Spaces Direct, next repair the virtual
disk using the Repair-VirtualDisk cmdlet.

If more disks failed than you have copies of your data and the virtual
disk wasn't repaired in-between failures, all data on the virtual disk is
permanently lost. In this unfortunate case, delete the virtual disk,
create a new virtual disk, and then restore from a backup.

Timeout Attaching the virtual disk took too long

Action: This shouldn't happen often, so you might try see if the
condition passes in time. Or you can try disconnecting the virtual
disk with the Disconnect-VirtualDisk cmdlet, then using the Connect-
VirtualDisk cmdlet to reconnect it.

Drive (physical disk) states

The following sections describe the health states a drive can be in. Drives in a pool are
represented in PowerShell as physical disk objects.

Drive health state: Healthy

Operational Description
state

OK The drive is healthy.

In service The drive is performing some internal housekeeping operations. When the action
is complete, the drive should return to the OK health state.

Drive health state: Warning

A drive in the Warning state can read and write data successfully but has an issue.
Operational Description
state

Lost The drive is missing. If you're using Storage Spaces Direct, this could be because
communication a server is down.

Action: If you're using Storage Spaces Direct, bring all servers back online. If
that doesn't fix it, reconnect the drive, replace it, or try getting detailed
diagnostic info about this drive by following the steps in Troubleshooting using
Windows Error Reporting > Physical disk timed out.

Removing from Storage Spaces is in the process of removing the drive from its storage pool.
pool
This is a temporary state. After the removal is complete, if the drive is still
attached to the system, the drive transitions to another operational state
(usually OK) in a primordial pool.

Starting Storage Spaces is in the process of putting the drive in maintenance mode after
maintenance an administrator put the drive in maintenance mode. This is a temporary state -
mode the drive should soon be in the In maintenance mode state.

In maintenance An administrator placed the drive in maintenance mode, halting reads and
mode writes from the drive. This is usually done before updating drive firmware, or
when testing failures.

Action: To take the drive out of maintenance mode, use the Disable-
StorageMaintenanceMode cmdlet.

Stopping An administrator took the drive out of maintenance mode, and Storage Spaces
maintenance is in the process of bringing the drive back online. This is a temporary state -
mode the drive should soon be in another state - ideally Healthy.

Predictive The drive reported that it's close to failing.

failure
Action: Replace the drive.

IO error There was a temporary error accessing the drive.

Action:
1. If the drive doesn't transition back to the OK state, you can try using the
Reset-PhysicalDisk cmdlet to wipe the drive.
2. Use Repair-VirtualDisk to restore the resiliency of affected virtual disks.
3. If this keeps happening, replace the drive.
 Operational Description
state

Transient error There was a temporary error with the drive. This usually means the drive was
unresponsive, but it could also mean that the Storage Spaces protective
partition was inappropriately removed from the drive.

Action:
1. If the drive doesn't transition back to the OK state, you can try using the
Reset-PhysicalDisk cmdlet to wipe the drive.
2. Use Repair-VirtualDisk to restore the resiliency of affected virtual disks.
3. If this keeps happening, replace the drive, or try getting detailed diagnostic
info about this drive by following the steps in Troubleshooting using Windows
Error Reporting > Physical disk failed to come online.

Abnormal The drive is performing slowly, as measured by the Health Service in Storage
latency Spaces Direct.

Action: If this keeps happening, replace the drive so it doesn't reduce the
performance of Storage Spaces as a whole.

Drive health state: Unhealthy

A drive in the Unhealthy state can't currently be written to or accessed.

Operational Description
state

Not usable This drive can't be used by Storage Spaces. For more info see Storage Spaces
Direct hardware requirements; if you're not using Storage Spaces Direct, see
Storage Spaces overview.

Split The drive has become separated from the pool.

Action: Reset the drive, erasing all data from the drive and adding it back to the
pool as an empty drive. To do so, open a PowerShell session as an administrator,
run the Reset-PhysicalDisk cmdlet, and then run Repair-VirtualDisk.

To get detailed diagnostic info about this drive, follow the steps in
Troubleshooting using Windows Error Reporting > Physical disk failed to come
online.
 Operational Description
state

Stale Storage Spaces found old metadata on the drive.

metadata
Action: This should be a temporary state. If the drive doesn't transition back to
OK, you can run Repair-VirtualDisk to start a repair operation on affected virtual
disks. If that doesn't resolve the issue, you can reset the drive with the Reset-
PhysicalDisk cmdlet, wiping all data from the drive, and then run Repair-
VirtualDisk.

Unrecognized Storage Spaces found unrecognized metadata on the drive, which usually means
metadata that the drive has metadata from a different pool on it.

Action: To wipe the drive and add it to the current pool, reset the drive. To reset
the drive, open a PowerShell session as an administrator, run the Reset-
PhysicalDisk cmdlet, and then run Repair-VirtualDisk.

Failed media The drive failed and won't be used by Storage Spaces anymore.

Action: Replace the drive.

To get detailed diagnostic info about this drive, follow the steps in
Troubleshooting using Windows Error Reporting > Physical disk failed to come
online.

Device There was a hardware failure on this drive.

hardware
failure Action: Replace the drive.

Updating Windows is updating the firmware on the drive. This is a temporary state that
firmware usually lasts less than a minute and during which time other drives in the pool
handle all reads and writes. For more info, see Update drive firmware.

Starting The drive is getting ready for operation. This should be a temporary state - once
complete, the drive should transition to a different operational state.

Reasons a drive can't be pooled

Some drives just aren't ready to be in a storage pool. You can find out why a drive isn't
eligible for pooling by looking at the CannotPoolReason property of a physical disk.
Here's an example PowerShell script to display the CannotPoolReason property:

PowerShell

Get-PhysicalDisk | Format-Table
FriendlyName,MediaType,Size,CanPool,CannotPoolReason
Here's an example output:

FriendlyName MediaType Size CanPool CannotPoolReason

------------ --------- ---- ------- ----------------
ATA MZ7LM120HCFD00D3 SSD 120034123776 False Insufficient Capacity
Msft Virtual Disk SSD 10737418240 True
Generic Physical Disk SSD 119990648832 False In a Pool

The following table gives a little more detail on each of the reasons.

Reason Description

In a pool The drive already belongs to a storage pool.

Drives can belong to only a single storage pool at a time. To use this drive in
another storage pool, first remove the drive from its existing pool, which tells
Storage Spaces to move the data on the drive to other drives on the pool. Or reset
the drive if the drive has been disconnected from its pool without notifying Storage
Spaces.

To safely remove a drive from a storage pool, use Remove-PhysicalDisk, or go to

Server Manager > File and Storage Services > Storage Pools, > Physical Disks,
right-click the drive and then select Remove Disk.

To reset a drive, use Reset-PhysicalDisk.

Not The drive isn't in a healthy state and might need to be replaced.
healthy

Removable The drive is classified as a removable drive.

media
Storage Spaces doesn't support media that are recognized by Windows as
removable, such as Blu-Ray drives. Although many fixed drives are in removable
slots, in general, media that are classified by Windows as removable aren't suitable
for use with Storage Spaces.

In use by The drive is currently used by a Failover Cluster.

cluster
Reason Description

Offline The drive is offline.

To bring all offline drives online and set to read/write, open a PowerShell session as
an administrator and use the following scripts:

Get-Disk | Where-Object -Property OperationalStatus -EQ "Offline" | Set-Disk -

IsOffline $false

Get-Disk | Where-Object -Property IsReadOnly -EQ $true | Set-Disk -IsReadOnly

$false

Insufficient This typically occurs when there are partitions taking up the free space on the drive.
capacity
Action: Delete any volumes on the drive, erasing all data on the drive. One way to
do that is to use the Clear-Disk PowerShell cmdlet.

Verification The Health Service is checking to see if the drive or firmware on the drive is
in progress approved for use by the server administrator.

Verification The Health Service couldn't check to see if the drive or firmware on the drive is
failed approved for use by the server administrator.

Firmware The firmware on the physical drive isn't in the list of approved firmware revisions
not specified by the server administrator by using the Health Service.
compliant

Hardware The drive isn't in the list of approved storage models specified by the server
not administrator by using the Health Service.
compliant

Additional References
Storage Spaces Direct
Storage Spaces Direct hardware requirements
Understanding cluster and pool quorum
Collect diagnostic data for clusters
Article • 04/17/2023

Applies to: Azure Stack HCI, versions 22H2 and 21H2; Windows Server 2022,
Windows Server 2019, Windows Server 2016

There are various diagnostic tools in Storage Spaces Direct that you can use to collect
the data needed to troubleshoot Azure Stack HCI and Windows Server clusters. In this
article, we will focus on installing and using SDDC diagnostic tools to gather relevant
information to help you diagnose your cluster.

Because the logs and other information are dense, the information presented in this
article is helpful for troubleshooting advanced issues that have been escalated and that
may require data to be sent to Microsoft for triage.

Install and use diagnostic tools with Windows

Admin Center
You can use Windows Admin Center (version 1812 onwards) to:

Install SDDC diagnostic tools and keep them up to date

Schedule daily diagnostic runs (these have a low impact on your system, usually
take less than five minutes to run in the background, and won't take up more than
500MB on your cluster)
View previously collected diagnostic information if you need to give it to support
or analyze it yourself

To install SDDC diagnostic tools and begin collecting data, follow these steps:

1. Launch Windows Admin Center and select Tools > Diagnostics. If diagnostics tools
are not already installed, click the Install button.
 

2. To begin collecting diagnostic data, click Collect. You should see a message that
says "Collecting diagnostic information. This may take a few minutes." After the
initial data collection, if you want to automatically collect data every 24 hours,
change the slider to On.

3. Data collection is not complete until you see the screenshot below. To view
collected diagnostic information, choose Download (.zip) or Open in Files tool.


Installing Get-SDDCDiagnosticInfo with
PowerShell
You can use the Get-SDDCDiagnosticInfo PowerShell cmdlet (also known as Get-
PCStorageDiagnosticInfo , previously known as Test-StorageHealth ) to gather logs for
and perform health checks for Failover Clustering (cluster, resources, networks, nodes),
Storage Spaces (physical disks, enclosures, virtual disks), Cluster Shared Volumes, SMB
file shares, and Deduplication.

There are two methods of installing the script: PowerShell Gallery and GitHub. Both are
outlined below.

PowerShell Gallery
The PowerShell Gallery is a snapshot of the GitHub Repo. Note that installing items
from the PowerShell Gallery requires the latest version of the PowerShellGet module,
which is available in Windows 10, in Windows Management Framework (WMF) 5.0, or in
the MSI-based installer (for PowerShell 3 and 4).

We install the latest version of the Microsoft Networking Diagnostics tools during this
process as well since Get-SDDCDiagnosticInfo relies on this. This manifest module
contains network diagnostic and troubleshooting tool, which are maintained by the
Microsoft Core Networking Product Group at Microsoft.

You can install the module by running following command in PowerShell as

administrator:

PowerShell

Install-PackageProvider NuGet -Force

Install-Module PrivateCloud.DiagnosticInfo -Force
Import-Module PrivateCloud.DiagnosticInfo -Force
Install-Module -Name MSFT.Network.Diag

To update the module, run the following command in PowerShell:

PowerShell

Update-Module PrivateCloud.DiagnosticInfo

GitHub
The GitHub Repo is the most up-to-date version of the module, since we are
continually iterating here. To install the module from GitHub, download the latest
module from the archive and extract the PrivateCloud.DiagnosticInfo directory to the
correct PowerShell modules path pointed by $env:PSModulePath

PowerShell

# Allowing Tls12 and Tls11 -- e.g. github now requires Tls12

# If this is not set, the Invoke-WebRequest fails with "The request was
aborted: Could not create SSL/TLS secure channel."
[Net.ServicePointManager]::SecurityProtocol =
[Net.SecurityProtocolType]::Tls12
$module = 'PrivateCloud.DiagnosticInfo'
Invoke-WebRequest -Uri
https://github.com/PowerShell/$module/archive/master.zip -OutFile
$env:TEMP\master.zip
Expand-Archive -Path $env:TEMP\master.zip -DestinationPath $env:TEMP -Force
if (Test-Path
$env:SystemRoot\System32\WindowsPowerShell\v1.0\Modules\$module) {
rm -Recurse
$env:SystemRoot\System32\WindowsPowerShell\v1.0\Modules\$module -ErrorAction
Stop
Remove-Module $module -ErrorAction SilentlyContinue
} else {
Import-Module $module -ErrorAction SilentlyContinue
}
if (-not ($m = Get-Module $module -ErrorAction SilentlyContinue)) {
$md = "$env:ProgramFiles\WindowsPowerShell\Modules"
} else {
$md = (gi $m.ModuleBase -ErrorAction SilentlyContinue).PsParentPath
Remove-Module $module -ErrorAction SilentlyContinue
rm -Recurse $m.ModuleBase -ErrorAction Stop
}
cp -Recurse $env:TEMP\$module-master\$module $md -Force -ErrorAction Stop
rm -Recurse $env:TEMP\$module-master,$env:TEMP\master.zip
Import-Module $module -Force

If you need to get this module on an offline cluster, download the zip, move it to your
target server node, and install the module.

Gathering logs with PowerShell

After you have enabled event channels and completed the installation process, you can
use the Get-SDDCDiagnosticInfo PowerShell cmdlet in the module to get:

Reports on storage health, plus details on unhealthy components

Reports of storage capacity by pool, volume, and deduplicated volume
 Event logs from all cluster nodes and a summary error report

Assume that your storage cluster has the name "CLUS01".

To execute against a remote storage cluster:

PowerShell

Get-SDDCDiagnosticInfo -ClusterName CLUS01

To execute locally on clustered storage node:

PowerShell

Get-SDDCDiagnosticInfo

To save results to a specified folder:

PowerShell

Get-SDDCDiagnosticInfo -WriteToPath D:\Folder

Here is an example of how this looks on a real cluster:

PowerShell

New-Item -Name SDDCDiagTemp -Path d:\ -ItemType Directory -Force

Get-SddcDiagnosticInfo -ClusterName S2D-Cluster -WriteToPath d:\SDDCDiagTemp

As you can see, the script will also do validation of current cluster state:
All data is being written to SDDCDiagTemp folder:

After the script finishes, it will create a ZIP in your user directory:

Let's generate a report into a text file:

PowerShell

#find the latest diagnostic zip in UserProfile

$DiagZip=(get-childitem $env:USERPROFILE | where Name -like
HealthTest*.zip)
$LatestDiagPath=($DiagZip | sort lastwritetime | select -First
1).FullName
#expand to temp directory
New-Item -Name SDDCDiagTemp -Path d:\ -ItemType Directory -Force
Expand-Archive -Path $LatestDiagPath -DestinationPath D:\SDDCDiagTemp -
Force
 #generate report and save to text file
$report=Show-SddcDiagnosticReport -Path D:\SDDCDiagTemp
$report | out-file d:\SDDCReport.txt

For reference, here is a link to the sample report and sample zip .

Get-SDDCDiagnosticInfo output
The following are the files included in the zipped output of Get-SDDCDiagnosticInfo .

Health summary Report

The health summary report is saved as:

0_CloudHealthSummary.log

This file is generated after parsing all the data collected and is meant to provide a quick
summary of your system. It contains:

System information
Storage health overview (number of nodes up, resources online, cluster shared
volumes online, unhealthy components, etc.)
Details on unhealthy components (cluster resources that are offline, failed, or
online pending)
Firmware and driver information
Pool, physical disk, and volume details
Storage Performance (performance counters are collected)

This report is being continually updated to include more useful information. For the
latest information, see the GitHub README .

Logs and XML files

The script runs various log gathering scripts and saves the output as xml files. We collect
cluster and health logs, system information (MSInfo32), unfiltered event logs (failover
clustering, dis diagnostics, Hyper-V, storage spaces, and more), and storage diagnostics
information (operational logs). For the latest information on what information is
collected, see the GitHub README (what we collect) .
How to consume XML files from Get-
SDDCDiagnosticInfo
You can consume the data from the XML files provided in data collected by the Get-
SDDCDiagnosticInfo cmdlet. These files have information about the virtual disks, physical
disks, basic cluster information, and other PowerShell related outputs.

To see the results of these outputs, open a PowerShell window and run the following
steps.

PowerShell

ipmo storage
$d = import-clixml <filename>
$d

Next steps
Provide feedback on what you'd like to see by filing issues here . Also, feel free to
contribute helpful changes to the script by submitting a pull request .
Storage-class Memory (NVDIMM-N)
Health Management in Windows
Article • 01/10/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows 10

This article provides system administrators and IT Pros with information about error
handling and health management specific to storage-class memory (NVDIMM-N)
devices in Windows, highlighting the differences between storage-class memory and
traditional storage devices.

If you aren't familiar with Windows' support for storage-class memory devices, these
short videos provide an overview:

Using Non-volatile Memory (NVDIMM-N) as Block Storage in Windows Server

2016
Using Non-volatile Memory (NVDIMM-N) as Byte-Addressable Storage in
Windows Server 2016
Accelerating SQL Server 2016 performance with Persistent Memory in Windows
Server 2016

Also see Understand and deploy persistent memory in Storage Spaces Direct.

JEDEC-compliant NVDIMM-N storage-class memory devices are supported in Windows

with native drivers, starting in Windows Server 2016 and Windows 10 (version 1607).
While these devices behave similar to other disks (HDDs and SSDs), there are some
differences.

All conditions listed here are expected to be very rare occurrences, but depend on the
conditions in which the hardware is used.

The various cases below may refer to Storage Spaces configurations. The particular
configuration of interest is one where two NVDIMM-N devices are utilized as a mirrored
write-back cache in a storage space. To set up such a configuration, see Configuring
Storage Spaces with a NVDIMM-N write-back cache.

In Windows Server 2016, the Storage Spaces GUI shows NVDIMM-N bus type as
UNKNOWN. It doesn't have any fuctionality loss or inability in creation of Pool, Storage
VD. You can verify the bus type by running the following command:
 PowerShell

PS C:\>Get-PhysicalDisk | fl

The parameter BusType in output of cmdlet will correctly show bus type as "SCM"

Checking the health of storage-class memory

To query the health of storage-class memory, use the following commands in a
Windows PowerShell session.

PowerShell

PS C:\> Get-PhysicalDisk | where BusType -eq "SCM" | select SerialNumber,

HealthStatus, OperationalStatus, OperationalDetails

Doing so yields this example output:

SerialNumber HealthStatus OperationalStatus OperationalDetails

802c-01-1602- Healthy OK
117cb5fc

802c-01-1602- Warning Predictive Failure {Threshold Exceeded,NVDIMM_N

117cb64f Error}

７ Note

To find the Physical location of an NVDIMM-N device specified in an event, on the

Details tab of the event in Event Viewer, go to EventData > Location. Note that
Windows Server 2016 lists the incorrect location NVDIMM-N devices, but this is
fixed in Windows Server, version 1709.

For help understanding the various health conditions, see the following sections.

"Warning" Health Status

This condition is when you check the health of a storage-class memory device and see
that it's Health Status is listed as Warning, as shown in this example output:

SerialNumber HealthStatus OperationalStatus OperationalDetails

 SerialNumber HealthStatus OperationalStatus OperationalDetails

802c-01-1602- Healthy OK
117cb5fc

802c-01-1602- Warning Predictive Failure {Threshold Exceeded,NVDIMM_N

117cb64f Error}

The following table lists some info about this condition.

Heading Description

Likely NVDIMM-N Warning Threshold breached

condition

Root NVDIMM-N devices track various thresholds, such as temperature, NVM lifetime,
Cause and/or energy source lifetime. When one of those thresholds is exceeded, the
operating system is notified.

General Device remains fully operational. This is a warning, not an error.

behavior

Storage Device remains fully operational. This is a warning, not an error.

Spaces
behavior

More OperationalStatus field of the PhysicalDisk object. EventLog – Microsoft-Windows-

info ScmDisk0101/Operational

What to Depending on the warning threshold breached, it may be prudent to consider

do replacing the entire, or certain parts of the NVDIMM-N. For example, if the NVM
lifetime threshold is breached, replacing the NVDIMM-N may make sense.

Writes to an NVDIMM-N fail

This condition is when you check the health of a storage-class memory device and see
the Health Status listed as Unhealthy, and Operational Status mentions an IO Error, as
shown in this example output:

SerialNumber HealthStatus OperationalStatus OperationalDetails

802c-01-1602- Healthy OK
117cb5fc

802c-01-1602- Unhealthy {Stale Metadata, IO Error, {Lost Data Persistence, Lost

117cb64f Transient Error} Data, NV...}

The following table lists some info about this condition.

 Heading Description

Likely Loss of Persistence / Backup Power

condition

Root NVDIMM-N devices rely on a back-up power source for their persistence – usually a
Cause battery or super-cap. If this back-up power source is unavailable or the device cannot
perform a backup for any reason (Controller/Flash Error), data is at risk and Windows
will prevent any further writes to the affected devices. Reads are still possible to
evacuate data.

General The NTFS volume will be dismounted.

behavior The PhysicalDisk Health Status field will show "Unhealthy" for all affected NVDIMM-N
devices.

Storage Storage Space will remain operational as long as only one NVDIMM-N is affected. If
Spaces multiple devices are affected, writes to the Storage Space will fail.
behavior The PhysicalDisk Health Status field will show "Unhealthy" for all affected NVDIMM-N
devices.

More OperationalStatus field of the PhysicalDisk object.

info EventLog – Microsoft-Windows-ScmDisk0101/Operational

What to We recommended backing-up the affected NVDIMM-N's data. To gain read access,
do you can manually bring the disk online (it will surface as a read-only NTFS volume).

To fully clear this condition, the root cause must be resolved (i.e. service power supply
or replace NVDIMM-N, depending on issue) and the volume on the NVDIMM-N must
either be taken offline and brought online again, or the system must be restarted.

To make the NVDIMM-N usable in Storage Spaces again, use the Reset-PhysicalDisk
cmdlet, which re-integrates the device and starts the repair process.

NVDIMM-N is shown with a capacity of '0'

Bytes or as a "Generic Physical Disk"
This condition is when a storage-class memory device is shown with a capacity of 0
bytes and cannot be initialized, or is exposed as a "Generic Physical Disk" object with an
Operational Status of Lost Communication, as shown in this example output:

SerialNumber HealthStatus OperationalStatus OperationalDetails

802c-01-1602-117cb5fc Healthy OK

Warning Lost Communication

The following table lists some info about this condition.

 Heading Description

Likely BIOS Did Not Expose NVDIMM-N to OS

condition

Root NVDIMM-N devices are DRAM based. When a corrupt DRAM address is referenced,
Cause most CPUs will initiate a machine check and restart the server. Some server platforms
then un-map the NVDIMM, preventing the OS from accessing it and potentially
causing another machine check. This may also occur if the BIOS detects that the
NVDIMM-N has failed and needs to be replaced.

General NVDIMM-N is shown as uninitialized, with a capacity of 0 bytes and cannot be read or
behavior written.

Storage Storage Space remains operational (provided only 1 NVDIMM-N is affected).

Spaces NVDIMM-N PhysicalDisk object is shown with a Health Status of Warning and as a
behavior "General Physical Disk"

More OperationalStatus field of the PhysicalDisk object.

info EventLog – Microsoft-Windows-ScmDisk0101/Operational

What to The NVDIMM-N device must be replaced or sanitized, such that the server platform
do exposes it to the host OS again. Replacement of the device is recommended, as
additional uncorrectable errors could occur. Adding a replacement device to a storage
spaces configuration can be achieved with the Add-Physicaldisk cmdlet.

NVDIMM-N is shown as a RAW or empty disk

after a reboot
This condition is when you check the health of a storage-class memory device and see a
Health Status of Unhealthy and Operational Status of Unrecognized Metadata, as
shown in this example output:

SerialNumber HealthStatus OperationalStatus OperationalDetails

802c-01-1602- Healthy OK {Unknown}

117cb5fc

802c-01-1602- Unhealthy {Unrecognized Metadata, Stale {Unknown}

117cb64f Metadata}

The following table lists some info about this condition.

Heading Description

Likely Backup/Restore Failure

condition
 Heading Description

Root A failure in the backup or restore procedure will likely result in all data on the
Cause NVDIMM-N to be lost. When the operating system loads, it will appear as a brand
new NVDIMM-N without a partition or file system and surface as RAW, meaning it
doesn't have a file system.

General NVDIMM-N will be in read-only mode. Explicit user action is needed to begin using it
behavior again.

Storage Storage Spaces remains operational if only one NVDIMM is affected).

Spaces NVDIMM-N physical disk object will be shown with the Health Status "Unhealthy" and
behavior is not used by Storage Spaces.

More OperationalStatus field of the PhysicalDisk object.

info EventLog – Microsoft-Windows-ScmDisk0101/Operational

What to If the user doesn't want to replace the affected device, they can use the Reset-
do PhysicalDisk cmdlet to clear the read-only condition on the affected NVDIMM-N. In
Storage Spaces environments this will also attempt to re-integrate the NVDIMM-N
into Storage Space and start the repair process.

Interleaved Sets
Interleaved sets can often be created in a platform's BIOS to make multiple NVDIMM-N
devices appear as a single device to the host operating system.

Windows Server 2016 and Windows 10 Anniversary Edition do not support interleaved
sets of NVDIMM-Ns.

At the time of this writing, there is no mechanism for the host operating system to
correctly identify individual NVDIMM-Ns in such a set and clearly communicate to the
user which particular device may have caused an error or needs to be serviced.
Work Folders overview
Article • 03/29/2023

Applies To: Windows 11, Windows 10, Windows Server 2022, Windows Server 2019,
Windows Server 2016

Work Folders is a Windows Server role service for file servers. Work Folders provides a
consistent way for users to access their work files from their PCs and devices. For older
versions, see Other versions of Work Folders.

With Work Folders, users can store and access work files on personal computers and
devices, often referred to as bring-your-own device (BYOD), in addition to corporate
PCs. Users gain a convenient location to store work files, and they can access those files
from anywhere. Organizations use Work Folders to help maintain control over corporate
data. They can store files on centrally managed file servers and define user device
policies such as encryption and lock-screen passwords.

You can deploy Work Folders with existing deployments of Folder Redirection, Offline
Files, and home folders. Work Folders stores user files in a folder on the server called a
sync share. You can specify a folder that already contains user data, which enables you to
adopt Work Folders without migrating servers and data or immediately phasing out
your existing solution.

Common uses
Administrators can use Work Folders to provide users with access to their work files
while keeping centralized storage and control over the organization's data. Work Folders
can also accomplish these specific applications:

Provide a single point of access to work files from a user's work and personal
computers and devices.

Access work files while offline and then sync with the central file server when the
PC or device next has internet or intranet connectivity.

Deploy with existing deployments of Folder Redirection, Offline Files, and home
folders.

Use existing file server management technologies, such as file classification and
folder quotas, to manage user data.
 Specify security policies to instruct user's PCs and devices to encrypt Work Folders
and use a lock screen password.

Use Failover Clustering with Work Folders to provide a high-availability solution.

Features
Work Folders provides users with multiple ways to access, manage, and share work files.
The following table describes the primary features of Work Folders:

Features Availability Description

Work Folders Windows Server 2019, File and Storage Services provides a way to set up sync
role service Windows Server 2016, shares (folders that store user's work files), monitors
in Server or Windows Server Work Folders, and manages sync shares and user
Manager 2012 R2 access.

Work Folders Windows Server 2019, The SyncShare Windows PowerShell module contains
cmdlets Windows Server 2016, comprehensive cmdlets for managing Work Folders
or Windows Server servers.
2012 R2

Work Folders Windows 10 Work Folders provides the following functionality in

integration Windows 8.1 Windows computers:
with - A Control Panel item that sets up and monitors Work
Windows Windows RT 8.1 Folders.
- A File Explorer integration that enables easy access to
Windows 7 (download
files in Work Folders.
required)
- A sync engine that transfers files to and from a
central file server while maximizing battery life and
system performance.

Work Folders Android An app that allows popular devices to access files in
app for Apple iPhone and Work Folders.
devices iPad®

Feature changes
The following table describes some of the major changes in Work Folders across
different available versions and updates:

Features Availability Status Description

Features Availability Status Description

Improved Windows New to Event logs on the Work Folders server can be used to
logging Server 2019 version monitor sync activity and identify users that are failing
sync sessions. Use Event ID 4020 in the Microsoft-
Windows-SyncShare/Operational event log to identify
which users are failing sync sessions. Use Event ID 7000
and Event ID 7001 in the Microsoft-Windows-
SyncShare/Reporting event log to monitor users that
are successfully completing upload and download sync
sessions.

Performance Windows New to The following performance counters were added: Bytes
counters Server 2019 version downloaded/sec, Bytes uploaded/sec, Connected Users,
Files downloaded/sec, Files uploaded/sec, Users with
change detection, Incoming requests/sec and
Outstanding requests.

Improved Windows Updated Performance improvements were made to handle more

server Server 2019 users per server. The limit per server varies and is based
performance on the number of files and file churn. To determine the
limit per server, users should be added to the server in
phases.

On-demand Windows Added This feature enables you to see and access all of your
file access 10 version files. You control which files are stored on your PC and
1803 available offline. The rest of your files are always visible
and don’t take up any space on your PC, but you need
connectivity to the Work Folders file server to access
them.

Azure AD Windows Added Remote users can securely access their files on the Work
Application 10 version Folders server using Azure AD Application Proxy.
Proxy 1703,
support Android,
iOS

Faster Windows Updated For Windows Server 2012 R2, when file changes are
change 10 and synced to the Work Folders server, clients aren't notified
replication Windows of the change and wait up to 10 minutes to get the
Server 2016 update. When you use Windows Server 2016, the Work
Folders server immediately notifies Windows 10 clients,
and the file changes are synced immediately. This
capability is new in Windows Server 2016 and requires a
Windows 10 client. If you're using an older client or the
Work Folders server is Windows Server 2012 R2, the
client continues to poll every 10 minutes for changes.
 Features Availability Status Description

Integrated Windows Added If an administrator deploys WIP, Work Folders can

with 10 version enforce data protection by encrypting the data on the
Windows 1607 PC. The encryption uses a key associated with the
Information Enterprise ID, which can be remotely wiped by using a
Protection supported mobile device management package such as
(WIP) Microsoft Intune.

Software requirements
Work Folders requires specific software depending on the operating system and version
on which it's run. The following section lists the requirements and optional supporting
services.

Requirements for file servers and network infrastructure

Server requirement. A server running Windows Server 2019, Windows Server 2016,
or Windows Server 2012 R2 for hosting sync shares with user files.

Volume requirement. A volume formatted with the NTFS file system for storing
user files.

Password enforcement policy. To enforce password policies on Windows 7 PCs,

you must use Group Policy password policies. You also have to exclude the
Windows 7 PCs from Work Folders password policies (if you use them).

A server certificate. There must be a server certificate for each file server that will
host Work Folders. These certificates should be from a certification authority (CA)
that's trusted by your users—ideally a public CA.

(Optional) An Active Directory Domain Services forest. Use this forest with the
schema extensions in Windows Server 2012 R2 to support automatically referring
PCs and devices to the correct file server when you use multiple file servers.

Requirements for enabling users to sync across the

internet
Server accessibility. The ability to make a server accessible from the internet by
creating publishing rules in your organization's reverse proxy or network gateway.

(Optional) Domain name. A publicly registered domain name and the ability to
create more public DNS records for the domain.
 (Optional) Infrastructure. Active Directory Federation Services (AD FS)
infrastructure when you use AD FS authentication.

Requirements for client computers

PCs and devices must run one of the following operating systems:
Windows 10
Windows 8.1
Windows RT 8.1
Windows 7
Android 4.4 KitKat and later
iOS 10.2 and later

７ Note

The Work Folders application for Android and iOS is no longer being actively
developed and will remain on the respective app stores if the application is
functioning properly.

Windows 7 PCs must run one of the following editions of Windows:

Windows 7 Professional
Windows 7 Ultimate
Windows 7 Enterprise

Windows 7 PCs must be joined to your organization's domain. They can't be joined
to a workgroup.

Enough free space on a local, NTFS-formatted drive to store all the user's files in
Work Folders, plus 6 more GB of free space if Work Folders is located on the
system drive, as it is by default. Work Folders uses the following location by
default: %USERPROFILE%\Work Folders

However, users can change the location during setup. microSD cards and USB
drives formatted with the NTFS file system are supported locations. Sync stops if
the drives are removed.

The maximum size for individual files is 10 GB by default. There's no per-user

storage limit. However, administrators can use the quotas functionality of File
Server Resource Manager to implement quotas.

Work Folders doesn't support rolling back the virtual machine state of client virtual
machines. Instead perform backup and restore operations from inside the client
 virtual machine by using System Image Backup or another backup app.

Other versions of Work Folders

If you want to download or use Work Folders on Windows 10, Windows 7, or an Android
or iOS device, see:

Work Folders for Windows 10

Work Folders for Windows 7 (64-bit download)
Work Folders for Windows 7 (32-bit download)
Work Folders for iOS

７ Note

The Work Folders application for Android and iOS is no longer being actively
developed. The Work Folders application for Android is no longer available in the
Google Play store. The Work Folders application for iOS will remain in the Apple
App Store if the application is functioning properly.

Comparison with other sync technologies

The following table discusses how various Microsoft sync technologies are positioned
and when to use each.

Work Folders Offline Files OneDrive for Business OneDrive

Technology Syncs files Syncs files that are Syncs files that are stored Syncs
summary that are stored on a file in Microsoft 365 or in personal
stored on a server with PCs that SharePoint with PCs and files that are
file server have access to the devices inside or outside a stored in
with PCs and corporate network corporate network, and OneDrive
devices (can be replaced by provides document with PCs,
Work Folders) collaboration functionality Mac
computers,
and devices

Intended Yes Yes Yes No

to provide
user access
to work
files
 Work Folders Offline Files OneDrive for Business OneDrive

Cloud None None Microsoft 365 Microsoft

service OneDrive

Internal File servers File servers SharePoint server (optional) None

network running
servers Windows
Server 2012
R2, Windows
Server 2016,
and Windows
Server 2019

Supported PCs, iOS PCs in a corporate PCs, iOS, Android, PCs, Mac
clients network or Windows Phone computers,
connected through Windows
DirectAccess, VPNs, Phone, iOS,
or other remote Android
access technologies

７ Note

In addition to the sync technologies listed in the previous table, Microsoft offers
other replication technologies, including DFS Replication, which is designed for
server-to-server replication, and BranchCache, which is designed as a branch office
WAN acceleration technology. For more information, see the DFS Namespaces
overview, DFS Replication overview, and BranchCache Overview.

Server Manager
Work Folders is part of the File and Storage Services role. You can install Work Folders by
using the Add Roles and Features Wizard or the Install-WindowsFeature cmdlet. Both
methods accomplish the following tasks:

Add the Work Folders page to File and Storage Services in Server Manager.

Install the Windows Sync Shares service, which is used by Windows Server to host
sync shares.

Install the SyncShare Windows PowerShell module to manage Work Folders on the
server.
Interoperability with Azure virtual machines
You can run this Windows Server role service on a virtual machine in Azure. This scenario
has been tested with Windows Server 2012 R2, Windows Server 2016, and Windows
Server 2019.

To learn how to get started, see Virtual machines in Azure.

Related links
Content type References

Product - Work Folders for Android – Released (blog post)

evaluation - Work Folders for iOS – iPad App Release (blog post)
- Introducing Work Folders on Windows Server 2012 R2 (blog post)
- Work Folders Test Lab Deployment (blog post)
- Work Folders for Windows 7 (blog post)

Deployment - Planning a Work Folders deployment

- Deploying Work Folders
- Deploy Work Folders with AD FS and Web Application Proxy: Overview
- Deploying Work Folders with Azure AD Application Proxy
- Offline Files (CSC) to Work Folders Migration Guide
- Performance Considerations for Work Folders Deployments
- Work Folders for Windows 7 (64-bit download)
- Work Folders for Windows 7 (32-bit download)

Operations - Work Folders iPad app: FAQ (for users)

- Work Folders Certificate Management (blog post)
- Monitoring Windows Server 2012 R2 Work Folders Deployments (blog
post)
- SyncShare (Work Folders) Cmdlets in Windows PowerShell
- Storage and File Services PowerShell Cmdlets Quick Reference Card For
Windows Server 2012 R2 Preview Edition

Troubleshooting - Windows Server 2012 R2 – Resolving Port Conflict with IIS Websites and
Work Folders (blog post)
- Common Errors in Work Folders

Community - File Services and Storage Forum

resources - Storage at Microsoft Blogs
- Ask the Directory Services Team Blog
Content type References

Related - Storage in Windows Server 2019 and Windows Server 2016

technologies - File and Storage Services
- File Server Resource Manager
- Folder Redirection, Offline Files, and Roaming User Profiles
- BranchCache Overview
- DFS Namespaces overview
- DFS Replication overview
Planning a Work Folders deployment
Article • 11/16/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows 10, Windows 8.1, Windows 7

This topic explains the design process for a Work Folders implementation, and assumes
that you have the following background:

Have a basic understanding of Work Folders (as described in Work Folders)

Have a basic understanding of Active Directory Domain Services (AD DS) concepts

Have a basic understanding of Windows file sharing and related technologies

Have a basic understanding of SSL certificate usage

Have a basic understanding of enabling web access to internal resources via a web
reverse proxy

The following sections will help you design your Work Folders implementation.
Deploying Work Folders is discussed in the next topic, Deploying Work Folders.

Software requirements
Work Folders has the following software requirements for file servers and your network
infrastructure:

A server running Windows Server 2012 R2 or Windows Server 2016 for hosting
sync shares with user files

A volume formatted with the NTFS file system for storing user files

To enforce password policies on Windows 7 PCs, you must use Group Policy
password policies. You also have to exclude the Windows 7 PCs from Work Folders
password policies (if you use them).

A server certificate for each file server that will host Work Folders. These certificates
should be from a certification authority (CA) that is trusted by your users—ideally a
public CA.

(Optional) An Active Directory Domain Services forest with schema extensions in

Windows Server 2012 R2 to support automatically referring PCs and devices to the
 correct file server when using multiple file servers.

To enable users to sync across the Internet, there are additional requirements:

The ability to make a server accessible from the Internet by creating publishing
rules in your organization's reverse proxy or network gateway

(Optional) A publicly registered domain name and the ability to create additional
public DNS records for the domain

(Optional) Active Directory Federation Services (AD FS) infrastructure when using
AD FS authentication

Work Folders has the following software requirements for client computers:

Computers must be running one of the following operating systems:

Windows 10

Windows 8.1

Windows RT 8.1

Windows 7

Android 4.4 KitKat and later

iOS 10.2 and later

Windows 7 PCs must be running one of the following editions of Windows:

Windows 7 Professional

Windows 7 Ultimate

Windows 7 Enterprise

Windows 7 PCs must be joined to your organization's domain (they can't be joined
to a workgroup).

Enough free space on a local, NTFS-formatted drive to store all the user's files in
Work Folders, plus an additional 6 GB of free space if Work Folders is located on
the system drive, as it is by default. Work Folders uses the following location by
default: %USERPROFILE%\Work Folders

However, users can change the location during setup (microSD cards and USB
drives formatted with the NTFS file system are supported locations, though sync
will stop if the drives are removed).
 The maximum size for individual files is 10 GB by default. There is no per-user
storage limit, although administrators can use the quotas functionality of File
Server Resource Manager to implement quotas.

Work Folders doesn't support rolling back the virtual machine state of client virtual
machines. Instead perform backup and restore operations from inside the client
virtual machine by using System Image Backup or another backup app.

７ Note

Make sure to install the Windows 8.1 and Windows Server 2012 R2 General
Availability update rollup on all Work Folders servers and any client computers
running Windows 8.1 or Windows Server 2012 R2. For more information, see article
2883200 in the Microsoft Knowledge Base.

Deployment scenarios
Work Folders can be implemented on any number of file servers within a customer
environment. This allows Work Folders implementations to scale based on customer
needs and can result in highly individualized deployments. However, most deployments
will fall into one of the following three basic scenarios.

Single-Site Deployment
In a single-site deployment, file servers are hosted within a central site in the customer
infrastructure. This deployment type is seen most often in customers with a highly
centralized infrastructure or with smaller branch offices that do not maintain local file
servers. This deployment model can be easier for IT staff to administer, since all server
assets are local, and internet ingress/egress is likely centralized at this location as well.
However, this deployment model also relies on good WAN connectivity between the
central site and any branch offices, and users in branch offices are vulnerable to an
interruption of service due to network conditions.

Multiple-Site Deployment
In a multiple-site deployment, file servers are hosted in multiple locations within the
customer's infrastructure. This could mean multiple datacenters or it could mean that
branch offices maintain individual file servers. This deployment type is seen most often
in larger customer environments or in customers that have several larger branch offices
that maintain local server assets. This deployment model is more complex for IT
personnel to administer, and relies on careful coordination of data storage and
maintenance of Active Directory Domain Services (AD DS) to ensure that users are using
the correct sync server for Work Folders.

Hosted Deployment
In a hosted deployment, sync servers are deployed in an IAAS (Infrastructure-as-a-
Service) solution such as Windows Azure VM. This deployment method has the
advantage of making the availability of file servers less dependent on WAN connectivity
within a customer's business. If a device is able to connect to the Internet, it can get to
its sync server. However, the servers deployed in the hosted environment still need to be
able to reach the organization's Active Directory domain to authenticate users, and the
customer trades infrastructure requirements on-premises for additional complexity in
maintaining that connection.

Deployment technologies
Work Folders deployments consist of a number of technologies that work together to
provide service to devices on both the internal and external networks. Before designing
a Work Folders deployment, customers should be familiar with the requirements of each
of the following technologies.

Active Directory Domain Services

AD DS provides two important services in a Work Folders deployment. First, as the back-
end for Windows authentication, AD DS provides the security and authentication
services that are used to grant access to user data. If a domain controller cannot be
reached, a file server will be unable to authenticate an incoming request and the device
will not be able to access any data stored in that file server's sync share.

Second, AD DS (with the Windows Server 2012 R2 schema update) maintains the msDS-
SyncServerURL attribute on each user, which is used to automatically direct users to the
appropriate sync server.

File Servers
File servers running Windows Server 2012 R2 or Windows Server 2016 host the Work
Folders role service, and host the sync shares that store users' Work Folders data. File
servers can also host data stored by other technologies operating on the internal
network (such as file shares), and can be clustered to provide fault tolerance for user
data.

Group Policy
If you have Windows 7 PCs in your environment, we recommend the following:

Use Group Policy to control password policies for all domain-joined PCs that use
Work Folders.

Use the Work Folders Automatically lock screen, and require a password policy
on PCs that aren't joined to your domain.

You can also use Group Policy to specify a Work Folders server to domain-joined
PCs. This simplifies Work Folders setup a little bit– users would otherwise need to
enter their work email address to lookup the settings (assuming that Work Folders
is set up properly), or enter the Work Folders URL that you explicitly provided them
via email or another means of communication.

You can also use Group Policy to forcibly set up Work Folders on a per-user or per-
computer basis, though doing so causes Work Folders to sync on every PC a user
signs in to (when using the per-user policy setting), and prevents users from
specifying an alternate location for Work Folders on their PC (such as on a microSD
card to conserve space on the primary drive). We suggest carefully evaluating your
user's needs before forcing automatic setup.

Windows Intune
Windows Intune also provides a layer of security and manageability for non-domain-
joined devices that would not otherwise be present. You can use Windows Intune to
configure and manage users' personal devices such as tablets that connect to Work
Folders from across the Internet. Windows Intune can provide devices with the sync
server URL to use – otherwise users must enter their work email address to lookup the
settings (if you publish a public Work Folders URL in the form of
https://workfolders.contoso.com), or enter the sync server URL directly.

Without a Windows Intune deployment, users must configure external devices manually,
which can result in increased demands on a customer's help desk staff.

You can also use Windows Intune to selectively wipe the data from Work Folders on a
user's device without affecting the rest of their data – handy for if a user leaves your
organization or has their device stolen.
Web Application Proxy/Azure AD Application Proxy
Work Folders is built around the concept of allowing Internet-connected devices to
retrieve business data securely from the internal network, which allows users to "take
their data with them" on their tablets and devices that would not normally be able to
access work files. To do this, a reverse proxy must be used to publish sync server URLs
and make them available to Internet clients.

Work Folders supports using Web Application Proxy, Azure AD Application Proxy or 3rd
party reverse proxy solutions:

Web Application Proxy is an on-premises reverse proxy solution. To learn more,

see Web Application Proxy in Windows Server 2016.

Azure AD Application Proxy is a cloud reverse proxy solution. To learn more, see
How to provide secure remote access to on-premises applications

Additional design considerations

In addition to understanding each of the components noted above, customers need to
spend time in their design thinking about the number of sync servers and shares to
operate, and whether or not to leverage failover clustering to provide fault tolerance on
those sync servers

Number of Sync Servers

It is possible for a customer to operate multiple sync servers in an environment. This can
be a desirable configuration for several reasons:

Geographic distribution of users – for example, branch office files servers or

regional datacenters

Data storage requirements – certain business departments might have specific

data storage or handling requirements that are easier with a dedicated server

Load balancing – in large environments, storing user data on multiple servers can
increase server performance and uptime.

For information on Work Folders server scaling and performance, see Performance
Considerations for Work Folders Deployments .

７ Note
 When using multiple sync servers, we recommend setting up automatic server
discovery for users. This process relies upon the configuration of an attribute on
each user account in AD DS. The attribute is named msDS-SyncServerURL and
becomes available on user accounts after a Windows Server 2012 R2 domain
controller is added to the domain or the Active Directory schema updates are
applied. This attribute should be set for each user to ensure that users connect to
the appropriate sync server. By using automatic server discovery, organizations can
publish Work Folders behind a "friendly" URL such as
https://workfolders.contoso.com , regardless of the number of sync servers in
operation.

Number of Sync Shares

Individual sync servers can maintain multiple sync shares. This can be useful for the
following reasons:

Auditing and security requirements – If data used by a certain department must be

more heavily audited or retained for a longer period of time, separate sync shares
can help administrators keep user folders with differing audit levels separated.

Differing quotas or file screens – If you want to set different storage quotas or
limits on which file types are allowed in Work Folders (file screens) for different
groups of users, separate sync shares can help.

Departmental control – If administrative duties are distributed by department,

utilizing separate shares for different departments can aid administrators in
enforcing quotas or other policies.

Differing device policies –If an organization needs to maintain multiple device

policies (such as encrypting Work Folders) for different groups of users, using
multiple shares enables this.

Storage capacity –If a file server has multiple volumes, additional shares can be
used to take advantage of these additional volumes. An individual share has access
to only the volume that it is hosted on, and is unable to take advantage of
additional volumes on a file server.

Access to Sync Shares

While the sync server that a user accesses is determined by the URL entered at their
client (or the URL published for that user in AD DS when using server automatic
discovery), access to individual sync shares is determined by the permissions present on
the share.

As a result, if a customer is hosting multiple sync shares on the same server, care must
be taken to ensure that individual users have permissions to access only one of those
shares. Otherwise, when users connect to the server, their client may connect to the
wrong share. This can be accomplished by creating a separate security group for each
sync share.

Further, access to an individual user's folder inside a sync share is determined by

ownership rights on the folder. When creating a sync share, Work Folders by default
grants users exclusive access to their files (disabling inheritance and making them the
owner of their individual folders).

Design checklist
The following set of design questions is intended to aid customers in designing a Work
Folders implementation that best serves their environment. Customers should work
through this checklist prior to attempting to deploy servers.

Intended Users

Which users will use Work Folders?

How are users organized? (Geographically, by office, by department, etc)

Do any users have special requirements for data storage, security, or retention?

Do any users have specific device policy requirements, such as encryption?

Which client computers and devices do you need to support? (Windows 8.1,
Windows RT 8.1, Windows 7)

If you're supporting Windows 7 PCs and want to use password policies, exclude
the domain storing their computer accounts from the Work Folders password
policy, and instead use Group Policy password policies for domain-joined PCs in
that domain.

Do you need to interoperate with or migrate from other user data management
solutions such as Folder Redirection?

Do users from multiple domains need to sync across the Internet with a single
server?
 Do you need to support users who aren't members of the Local Administrators
group on their domain-joined PCs? (If so, you'll need to exclude the relevant
domains from Work Folders device policies such as encryption and password
policies)

Infrastructure and Capacity Planning

In what sites should sync servers be located on the network?

Will any sync servers be hosted by an Infrastructure as a Service (IaaS) provider

such as in an Azure VM?

Will dedicated servers be needed for any specific user groups, and if so, how
many users for each dedicated server?

Where are the Internet ingress/egress points on the network?

Will sync servers be clustered for fault-tolerance?

Will sync servers need to maintain multiple data volumes to host user data?

Data Security

Will multiple sync shares need to be created on any sync servers?

What groups will be used to provide access to sync shares?

If you're using multiple sync servers, what security group will you create for the
delegated ability to modify the msDS-SyncServerURL property of user objects?

Are there any special security or auditing requirements for individual sync
shares?

Is multi-factor authentication (MFA) required?

Do you need the ability to remotely wipe Work Folders data from PCs and
devices?

Device Access

What URL will be used to provide access for Internet-based devices (the default
URL that is required for email-based automatic server discovery is
workfolders.domainname)?

How will the URL be published to the Internet?

Will automatic server discovery be used?

 Will Group Policy be used to configure domain-joined PCs?

Will Windows Intune be used to configure external devices?

Will Device Registration be required for devices to connect?

Next steps
After designing your Work Folders implementation, it's time to deploy Work Folders. For
more information, see Deploying Work Folders.

Additional References
For additional related information, see the following resources.

Content type References

Product evaluation - Work Folders

- Work Folders for Windows 7 (blog post)

Deployment - Designing a Work Folders Implementation

- Deploying Work Folders
- Deploying Work Folders with AD FS and Web Application Proxy (WAP)
- Deploying Work Folders with Azure AD Application Proxy
- Performance Considerations for Work Folders Deployments
- Work Folders for Windows 7 (64 bit download)
- Work Folders for Windows 7 (32 bit download)
- Work Folders Test Lab Deployment (blog post)
Deploying Work Folders
Article • 11/16/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016,
Windows Server 2012 R2, Windows 10, Windows 8.1, Windows 7

This topic discusses the steps needed to deploy Work Folders. It assumes that you've
already read Planning a Work Folders deployment.

To deploy Work Folders, a process that can involve multiple servers and technologies,
use the following steps.

 Tip

The simplest Work Folders deployment is a single file server (often called a sync
server) without support for syncing over the Internet, which can be a useful
deployment for a test lab or as a sync solution for domain-joined client computers.
To create a simple deployment, these are minimum steps to follow:

Step 1: Obtain SSL certificates

Step 2: Create DNS records
Step 3: Install Work Folders on file servers
Step 4: Binding the SSL certificate on the sync servers
Step 5: Create security groups for Work Folders
Step 7: Create sync shares for user data

Step 1: Obtain SSL certificates

Work Folders uses HTTPS to securely synchronize files between the Work Folders clients
and the Work Folders server. The requirements for SSL certificates used by Work Folders
are as follows:

The certificate must be issued by a trusted certification authority. For most Work
Folders implementations, a publicly trusted CA is recommended, since certificates
will be used by non-domain-joined, Internet-based devices.

The certificate must be valid.

 The private key of the certificate must be exportable (as you will need to install the
certificate on multiple servers).

The subject name of the certificate must contain the public Work Folders URL used
for discovering the Work Folders service from across the Internet – this must be in
the format of workfolders. <domain_name>.

Subject alternative names (SANs) must be present on the certificate listing the
server name for each sync server in use.

The Work Folders Certificate Management blog provides additional information

on using certificates with Work Folders.

Step 2: Create DNS records

To allow users to sync across the Internet, you must create a Host (A) record in public
DNS to allow Internet clients to resolve your Work Folders URL. This DNS record should
resolve to the external interface of the reverse proxy server.

On your internal network, create a CNAME record in DNS named workfolders which
resolves to the FDQN of a Work Folders server. When Work Folders clients use auto
discovery, the URL used to discover the Work Folders server is
https://workfolders.domain.com. If you plan to use auto discovery, the workfolders
CNAME record must exist in DNS.

Step 3: Install Work Folders on file servers

You can install Work Folders on a domain-joined server by using Server Manager or by
using Windows PowerShell, locally or remotely across a network. This is useful if you are
configuring multiple sync servers across your network.

To deploy the role in Server Manager, do the following:

1. Start the Add Roles and Features Wizard.

2. On the Select installation type page, choose Role-based or feature-based

deployment.

3. On the Select destination server page, select the server on which you want to
install Work Folders.

4. On the Select server roles page, expand File and Storage Services, expand File
and iSCSI Services, and then select Work Folders.
 5. When asked if you want to install IIS Hostable Web Core, click Ok to install the
minimal version of Internet Information Services (IIS) required by Work Folders.

6. Click Next until you have completed the wizard.

To deploy the role by using Windows PowerShell, use the following cmdlet:

PowerShell

Add-WindowsFeature FS-SyncShareService

Step 4: Binding the SSL certificate on the sync

servers
Work Folders installs the IIS Hostable Web Core, which is an IIS component designed to
enable web services without requiring a full installation of IIS. After installing the IIS
Hostable Web Core, you should bind the SSL certificate for the server to the Default
Web Site on the file server. However, the IIS Hostable Web Core does not install the IIS
Management console.

There are two options for binding the certificate to the Default Web Interface. To use
either option you must have installed the private key for the certificate into the
computer's personal store.

Utilize the IIS management console on a server that has it installed. From within
the console, connect to the file server you want to manage, and then select the
Default Web Site for that server. The Default Web Site will appear disabled, but you
can still edit the bindings for the site and select the certificate to bind it to that
web site.

Use the netsh command to bind the certificate to the Default Web Site https
interface. The command is as follows:

netsh http add sslcert ipport=<IP address>:443 certhash=<Cert

thumbprint> appid={CE66697B-3AA0-49D1-BDBD-A25C8359FD5D}
certstorename=MY

Step 5: Create security groups for Work Folders

Before creating sync shares, a member of the Domain Admins or Enterprise Admins
groups needs to create some security groups in Active Directory Domain Services (AD
DS) for Work Folders (they might also want to delegate some control as described in
Step 6). Here are the groups you need:

One group per sync share to specify which users are allowed to sync with the sync
share

One group for all Work Folders administrators so that they can edit an attribute on
each user object that links the user to the correct sync server (if you're going to
use multiple sync servers)

Groups should follow a standard naming convention and should be used only for
Work Folders to avoid potential conflicts with other security requirements.

To create the appropriate security groups, use the following procedure multiple
times – once for each sync share, and once to optionally create a group for file
server administrators.

To create security groups for Work Folders

1. Open Server Manager on a Windows Server 2012 R2 or Windows Server 2016

computer with Active Directory Administration Center installed.

2. On the Tools menu, click Active Directory Administration Center. Active Directory
Administration Center appears.

3. Right-click the container where you want to create the new group (for example,
the Users container of the appropriate domain or OU), click New, and then click
Group.

4. In the Create Group window, in the Group section, specify the following settings:

In Group name, type the name of the security group, for example: HR Sync
Share Users, or Work Folders Administrators.

In Group scope, click Security, and then click Global.

5. In the Members section, click Add. The Select Users, Contacts, Computers, Service
Accounts or Groups dialog box appears.

6. Type the names of the users or groups to which you grant access to a particular
sync share (if you're creating a group to control access to a sync share), or type the
names of the Work Folders administrators (if you're going to configure user
 accounts to automatically discover the appropriate sync server), click OK, and then
click OK again.

To create a security group by using Windows PowerShell, use the following cmdlets:

PowerShell

$GroupName = "Work Folders Administrators"

$DC = "DC1.contoso.com"
$ADGroupPath = "CN=Users,DC=contoso,DC=com"
$Members = "CN=Maya Bender,CN=Users,DC=contoso,DC=com","CN=Irwin
Hume,CN=Users,DC=contoso,DC=com"

New-ADGroup -GroupCategory:"Security" -GroupScope:"Global" -Name:$GroupName

-Path:$ADGroupPath -SamAccountName:$GroupName -Server:$DC
Set-ADGroup -Add:@{'Member'=$Members} -Identity:$GroupName -Server:$DC

Step 6: Optionally delegate user attribute

control to Work Folders administrators
If you are deploying multiple sync servers and want to automatically direct users to the
correct sync server, you'll need to update an attribute on each user account in AD DS.
However, this normally requires getting a member of the Domain Admins or Enterprise
Admins groups to update the attributes, which can quickly become tiresome if you need
to frequently add users or move them between sync servers.

For this reason, a member of the Domain Admins or Enterprise Admins groups might
want to delegate the ability to modify the msDS-SyncServerURL property of user objects
to the Work Folders Administrators group you created in Step 5, as described in the
following procedure.

Delegate the ability to edit the msDS-SyncServerURL property on

user objects in AD DS

1. Open Server Manager on a Windows Server 2012 R2 or Windows Server 2016

computer with Active Directory Users and Computers installed.

2. On the Tools menu, click Active Directory Users and Computers. Active Directory
Users and Computers appears.

3. Right-click the OU under which all user objects exist for Work Folders (if users are
stored in multiple OUs or domains, right-click the container that is common to all
 of the users), and then click Delegate Control…. The Delegation of Control Wizard
appears.

4. On the Users or Groups page, click Add… and then specify the group you created
for Work Folders administrators (for example, Work Folders Administrators).

5. On the Tasks to Delegate page, click Create a custom task to delegate.

6. On the Active Directory Object Type page, click Only the following objects in the
folder, and then select the User objects checkbox.

7. On the Permissions page, clear the General checkbox, select the Property-specific
checkbox, and then select the Read msDS-SyncServerUrl, and Write msDS-
SyncServerUrl checkboxes.

To delegate the ability to edit the msDS-SyncServerURL property on user objects by

using Windows PowerShell, use the following example script that makes use of the
DsAcls command.

PowerShell

$GroupName = "Contoso\Work Folders Administrators"

$ADGroupPath = "CN=Users,dc=contoso,dc=com"

DsAcls $ADGroupPath /I:S /G ""$GroupName":RPWP;msDS-SyncServerUrl;user"

７ Note

The delegation operation might take a while to run in domains with a large number
of users.

Step 7: Create sync shares for user data

At this point, you're ready to designate a folder on the sync server to store your user's
files. This folder is called a sync share, and you can use the following procedure to create
one.

1. If you don't already have an NTFS volume with free space for the sync share and
the user files it will contain, create a new volume and format it with the NTFS file
system.

2. In Server Manager, click File and Storage Services, and then click Work Folders.
3. A list of any existing sync shares is visible at the top of the details pane. To create a
new sync share, from the Tasks menu choose New Sync Share…. The New Sync
Share Wizard appears.

4. On the Select the server and path page, specify where to store the sync share. If
you already have a file share created for this user data, you can choose that share.
Alternatively you can create a new folder.

７ Note

By default, sync shares aren't directly accessible via a file share (unless you
pick an existing file share). If you want to make a sync share accessible via a
file share, use the Shares tile of Server Manager or the New-SmbShare
cmdlet to create a file share, preferably with access-based enumeration
enabled.

5. On the Specify the structure for user folders page, choose a naming convention
for user folders within the sync share. There are two options available:

User alias creates user folders that don't include a domain name. If you are
using a file share that is already in use with Folder Redirection or another user
data solution, select this naming convention. You can optionally select the
Sync only the following subfolder checkbox to sync only a specific subfolder,
such as the Documents folder.

User alias@domain creates user folders that include a domain name. If you
aren't using a file share already in use with Folder Redirection or another user
data solution, select this naming convention to eliminate folder naming
conflicts when multiple users of the share have identical aliases (which can
happen if the users belong to different domains).

6. On the Enter the sync share name page, specify a name and a description for the
sync share. This is not advertised on the network but is visible in Server Manager
and Windows PowerShell to help distinguish sync shares from each other.

7. On the Grant sync access to groups page, specify the group that you created that
lists the users allowed to use this sync share.

） Important

To improve performance and security, grant access to groups instead of

individual users and be as specific as possible, avoiding generic groups such
 as Authenticated Users and Domain Users. Granting access to groups with
large numbers of users increases the time it takes Work Folders to query AD
DS. If you have a large number of users, create multiple sync shares to help
disperse the load.

8. On the Specify device policies page, specify whether to request any security
restrictions on client PCs and devices. There are two device policies that can be
individually selected:

Encrypt Work Folders Requests that Work Folders be encrypted on client PCs
and devices

Automatically lock screen, and require a password Requests that client PCs
and devices automatically lock their screens after 15 minutes, require a six-
character or longer password to unlock the screen, and activate a device
lockout mode after 10 failed retries

） Important

To enforce password policies for Windows 7 PCs and for non-

administrators on domain-joined PCs, use Group Policy password
policies for the computer domains and exclude these domains from the
Work Folders password policies. You can exclude domains by using the
Set-Syncshare -PasswordAutoExcludeDomain cmdlet after creating
the sync share. For information about setting Group Policy password
policies, see Password Policy.

9. Review your selections and complete the wizard to create the sync share.

You can create sync shares using Windows PowerShell by using the New-SyncShare
cmdlet. Below is an example of this method:

PowerShell

New-SyncShare "HR Sync Share" K:\Share-1 –User "HR Sync Share Users"

The example above creates a new sync share named Share01 with the path K:\Share-1,
and access granted to the group named HR Sync Share Users

 Tip
 After you create sync shares you can use File Server Resource Manager functionality
to manage the data in the shares. For example, you can use the Quota tile inside
the Work Folders page in Server Manager to set quotas on the user folders. You can
also use File Screening Management to control the types of files that Work Folders
will sync, or you can use the scenarios described in Dynamic Access Control for
more sophisticated file classification tasks.

Step 8: Optionally specify a tech support email

address
After installing Work Folders on a file server, you probably want to specify an
administrative contact email address for the server. To add an email address, use the
following procedure:

Specifying an administrative contact email

1. In Server Manager, click File and Storage Services, and then click Servers.

2. Right-click the sync server, and then click Work Folders Settings. The Work Folders
Settings window appears.

3. In the navigation pane, click Support Email and then type the email address or
addresses that users should use when emailing for help with Work Folders. Click
OK when you're finished.

Work Folders users can click a link in the Work Folders Control Panel item that
sends an email containing diagnostic information about the client PC to the
address(es) you specify here.

Step 9: Optionally set up server automatic

discovery
If you are hosting multiple sync servers in your environment, you should configure
server automatic discovery by populating the msDS-SyncServerURL property on user
accounts in AD DS.

７ Note
 The msDS-SyncServerURL property in Active Directory should not be defined for
remote users that are accessing Work Folders through a reverse proxy solution
such as Web Application Proxy or Azure AD Application Proxy. If the msDS-
SyncServerURL property is defined, the Work Folders client will try to access an
internal URL that's not accessible through the reverse proxy solution. When using
Web Application Proxy or Azure AD Application Proxy, you need to create unique
proxy applications for each Work Folders server. For more details, see Deploying
Work Folders with AD FS and Web Application Proxy: Overview or Deploying
Work Folders with Azure AD Application Proxy .

Before you can do so, you must install a Windows Server 2012 R2 domain controller or
update the forest and domain schemas by using the Adprep /forestprep and Adprep
/domainprep commands. For information on how to safely run these commands, see
Running Adprep.

You probably also want to create a security group for file server administrators and give
them delegated permissions to modify this particular user attribute, as described in Step
5 and Step 6. Without these steps you would need to get a member of the Domain
Admins or Enterprise Admins group to configure automatic discovery for each user.

To specify the sync server for users

1. Open Server Manager on a computer with Active Directory Administration Tools

installed.

2. On the Tools menu, click Active Directory Administration Center. Active Directory
Administration Center appears.

3. Navigate to the Users container in the appropriate domain, right-click the user you
want to assign to a sync share, and then click Properties.

4. In the Navigation pane, click Extensions.

5. Click the Attribute Editor tab, select msDS-SyncServerUrl and then click Edit. The
Multi-valued String Editor dialog box appears.

6. In the Value to add box, type the URL of the sync server with which you want this
user to sync, click Add, click OK, and then click OK again.

７ Note
 The sync server URL is simply https:// or http:// (depending on whether
you want to require a secure connection) followed by the fully qualified
domain name of the sync server. For example, https://sync1.contoso.com.

To populate the attribute for multiple users, use Active Directory PowerShell. Below is an
example that populates the attribute for all members of the HR Sync Share Users group,
discussed in Step 5.

PowerShell

$SyncServerURL = "https://sync1.contoso.com"
$GroupName = "HR Sync Share Users"

Get-ADGroupMember -Identity $GroupName |

Set-ADUser –Add @{"msDS-SyncServerURL"=$SyncServerURL}

Step 10: Optionally configure Web Application

Proxy, Azure AD Application Proxy, or another
reverse proxy
To enable remote users to access their files, you need to publish the Work Folders server
through a reverse proxy, making Work Folders available externally on the Internet. You
can use Web Application Proxy, Azure Active Directory Application Proxy, or another
reverse proxy solution.

To configure Work Folders access using AD FS and Web Application Proxy, see
Deploying Work Folders with AD FS and Web Application Proxy (WAP). For background
information about Web Application Proxy, see Web Application Proxy in Windows Server
2016. For details on publishing applications such as Work Folders on the Internet using
Web Application Proxy, see Publishing Applications using AD FS Preauthentication.

To configure Work Folders access using Azure Active Directory Application Proxy, see
Enable remote access to Work Folders using Azure Active Directory Application Proxy

Step 11: Optionally use Group Policy to

configure domain-joined PCs
If you have a large number of domain-joined PCs to which you want to deploy Work
Folders, you can use Group Policy to do the following client PC configuration tasks:
 Specify which sync server users should sync with

Force Work Folders to be set up automatically, using default settings (review the
Group Policy discussion in Designing a Work Folders Implementation before doing
this)

To control these settings, create a new Group Policy object (GPO) for Work Folders
and then configure the following Group Policy settings as appropriate:

"Specify Work Folders settings" policy setting in User

Configuration\Policies\Administrative Templates\Windows
Components\WorkFolders

"Force automatic setup for all users" policy setting in Computer

Configuration\Policies\Administrative Templates\Windows
Components\WorkFolders

７ Note

These policy settings are available only when editing Group Policy from a computer
running Group Policy Management on Windows 8.1, Windows Server 2012 R2 or
later. Versions of Group Policy Management from earlier operating systems do not
have this setting available. These policy settings do apply to Windows 7 PCs on
which the Work Folders for Windows 7 app has been installed.

See also
For additional related information, see the following resources.

Content type References

Understanding - Work Folders

Planning - Designing a Work Folders Implementation

Deployment - Deploying Work Folders with AD FS and Web Application Proxy (WAP)
- Work Folders Test Lab Deployment (blog post)
- A new user attribute for Work Folders server Url (blog post)

Technical - [Interactive logon: Machine account lockout threshold](/previous-

Reference versions/windows/it-pro/windows-server-2012-R2-and-2012/jj966264
- Sync Share Cmdlets
Deploy Work Folders with AD FS and
Web Application Proxy: Overview
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

The topics in this section provide instructions for deploying Work Folders with Active
Directory Federation Services (AD FS) and Web Application Proxy. The instructions are
designed to help you create a complete functioning Work Folders setup with client
machines that are ready to start using Work Folders either on-premises or over the
Internet.

Work Folders is a component introduced in Windows Server 2012 R2 that allows

information workers to sync work files between their devices. For more information
about Work Folders, see Work Folders Overview.

To enable users to sync their Work Folders across the Internet, you need to publish Work
Folders through a reverse proxy, making Work Folders available externally on the
Internet. Web Application Proxy, which is included in AD FS, is one option that you can
use to provide reverse proxy functionality. Web Application Proxy pre-authenticates
access to the Work Folders web application by using AD FS, so that users on any device
can access Work Folders from outside the corporate network.

７ Note

The instructions covered in this section are for a Windows Server 2016
environment. If you're using Windows Server 2012 R2, follow the Windows Server
2012 R2 instructions.

These topics provide the following:

Step-by-step instructions for setting up and deploying Work Folders with AD FS

and Web Application Proxy via the Windows Server user interface. The instructions
describe how to set up a simple test environment with self-signed certificates. You
can then use the test example as a guide to help you create a production
environment that uses publicly trusted certificates.

Prerequisites
To follow the procedures and examples in these topics, you need to have the following
components ready:

An Active Directory® Domain Services forest with schema extensions in Windows

Server 2012 R2 to support automatic referral of PCs and devices to the correct file
server when you are using multiple file servers. It is preferable that DNS be
enabled in the forest, but this is not required.

A domain controller: A server that has the AD DS role enabled, and is configured
with a domain (for the test example, contoso.com).

A domain controller running at least Windows Server 2012 R2 is needed in order to

support device registration for Workplace Join. If you don't want to use Workplace
Join, you can run Windows Server 2012 on the domain controller.

Two servers that are joined to the domain (e.g., contoso.com) and that are running
Windows Server 2016. One server will be for used for AD FS, and the other will be
used for Work Folders.

One server that is not domain joined and that is running Windows Server 2016.
This server will run Web Application Proxy, and it must have one network card for
the network domain (e.g., contoso.com) and another network card for the external
network.

One domain-joined client computer that is running Windows 7 or later.

One non-domain-joined client computer that is running Windows 7 or later.

For the test environment that we're covering in this guide, you should have the
topology that is shown in the following diagram. The computers can be physical
machines or virtual machines (VMs).
Deployment overview
In this group of topics, you'll walk through a step-by-step example of setting up AD FS,
Web Application Proxy, and Work Folders in a test environment. The components will be
set up in this order:

1. AD FS

2. Work Folders

3. Web Application Proxy

4. The domain-joined workstation and non-domain-joined workstation

You will also use a Windows PowerShell Script to create self-signed certificates.

Deployment steps
To perform the deployment by using the Windows Server user interface, follow the steps
in these topics:

Deploy Work Folders with AD FS and Web Application Proxy: Step 1, Set Up AD FS

Deploy Work Folders with AD FS and Web Application Proxy: Step 2, AD FS Post-
Configuration Work

Deploy Work Folders with AD FS and Web Application Proxy: Step 3, Set Up Work
Folders

Deploy Work Folders with AD FS and Web Application Proxy: Step 4, Set Up Web
Application Proxy

Deploy Work Folders with AD FS and Web Application Proxy: Step 5, Set Up Clients

See Also
Work Folders Overview Designing a Work Folders Implementation Deploying Work
Folders
Deploy Work Folders with AD FS and
Web Application Proxy: Step 1, Set-up
AD FS
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

This topic describes the first step in deploying Work Folders with Active Directory
Federation Services (AD FS) and Web Application Proxy. You can find the other steps in
this process in these topics:

Deploy Work Folders with AD FS and Web Application Proxy: Overview

Deploy Work Folders with AD FS and Web Application Proxy: Step 2, AD FS Post-
Configuration Work

Deploy Work Folders with AD FS and Web Application Proxy: Step 3, Set Up Work
Folders

Deploy Work Folders with AD FS and Web Application Proxy: Step 4, Set Up Web
Application Proxy

Deploy Work Folders with AD FS and Web Application Proxy: Step 5, Set Up Clients

７ Note

The instructions covered in this section are for a Windows Server 2019 or Windows
Server 2016 environment. If you're using Windows Server 2012 R2, follow the
Windows Server 2012 R2 instructions.

To set up AD FS for use with Work Folders, use the following procedures.

Pre-installment work
If you intend to convert the test environment that you're setting up with these
instructions to production, there are two things that you might want to do before you
start:
 Set up an Active Directory domain administrator account to use to run the AD FS
service.

Obtain a Secure Sockets Layer (SSL) subject alternative name (SAN) certificate for
server authentication. For the test example, you will use a self-signed certificate
but for production you should use a publicly trusted certificate.

Obtaining these items can take some time, depending on your company's policies, so it
can be beneficial to start the request process for the items before you begin to create
the test environment.

There are many commercial certificate authorities (CAs) from which you can purchase
the certificate. You can find a list of the CAs that are trusted by Microsoft in KB article
931125 . Another alternative is to get a certificate from your company's enterprise CA.

For the test environment, you will use a self-signed certificate that is created by one of
the provided scripts.

７ Note

AD FS does not support Cryptography Next Generation (CNG) certificates, which

means that you cannot create the self-signed certificate by using the Windows
PowerShell cmdlet New-SelfSignedCertificate. You can, however, use the
makecert.ps1 script included in the Deploying Work Folders with AD FS and Web
Application Proxy blog post. This script creates a self-signed certificated that
works with AD FS and prompts for the SAN names that will be needed to create the
certificate.

Next, do the additional pre-installment work described in the following sections.

Create an AD FS self-signed certificate

To create an AD FS self-signed certificate, follow these steps:

1. Download the scripts provided in the Deploying Work Folders with AD FS and Web
Application Proxy blog post and then copy the file makecert.ps1 to the AD FS
machine.

2. Open a Windows PowerShell window with admin privileges.

3. Set the execution policy to unrestricted:

PowerShell
 Set-ExecutionPolicy –ExecutionPolicy Unrestricted

4. Change to the directory where you copied the script.

5. Execute the makecert script:

PowerShell

.\makecert.ps1

6. When you are prompted to change the subject certificate, enter the new value for
the subject. In this example, the value is blueadfs.contoso.com.

7. When you are prompted to enter SAN names, press Y and then enter the SAN
names, one at a time.

For this example, type blueadfs.contoso.com and press Enter, then type 2016-
adfs.contoso.com and press Enter, then type enterpriseregistration.contoso.com
and press Enter.

When all of the SAN names have been entered, press Enter on an empty line.

8. When you are prompted to install the certificates to the Trusted Root Certification
Authority store, press Y.

The AD FS certificate must be a SAN certificate with the following values:

AD FS service name.domain

enterpriseregistration.domain

AD FS server name.domain

In the test example, the values are:

blueadfs.contoso.com

enterpriseregistration.contoso.com

2016-adfs.contoso.com

The enterpriseregistration SAN is needed for Workplace Join.

Set the server IP address

Change your server IP address to a static IP address. For the test example, use IP class A,
which is 192.168.0.160 / subnet mask: 255.255.0.0 / Default Gateway: 192.168.0.1 /
Preferred DNS: 192.168.0.150 (the IP address of your domain controller).

Install the AD FS role service

To install AD FS, follow these steps:

1. Log on to the physical or virtual machine on which you plan to install AD FS, open
Server Manager, and start the Add Roles and Features Wizard.

2. On the Server Roles page, select the Active Directory Federation Services role,
and then click Next.

3. On the Active Directory Federation Services (AD FS) page, you will see a message
that states that the Web Application Proxy role cannot be installed on the same
computer as AD FS. Click Next.

4. Click Install on the confirmation page.

To accomplish the equivalent installation of AD FS via Windows PowerShell, use these

commands:

PowerShell

Add-WindowsFeature RSAT-AD-Tools
Add-WindowsFeature ADFS-Federation –IncludeManagementTools

Configure AD FS
Next, configure AD FS by using either Server Manager or Windows PowerShell.

Configure AD FS by using Server Manager

To configure AD FS by using Server Manager, follow these steps:

1. Open Server Manager.

2. Click the Notifications flag at the top of the Server Manager window, and then
click Configure the federation service on this server.

3. The Active Directory Federation Services Configuration Wizard launches. On the

Connect to AD DS page, enter the domain administrator account that you want to
 use as the AD FS account, and click Next.

4. On the Specify Service Properties page, enter the subject name of the SSL
certificate to use for AD FS communication. In the test example, this is
blueadfs.contoso.com.

5. Enter the Federation Service name. In the test example, this is

blueadfs.contoso.com. Click Next.

７ Note

The Federation Service name must not use the name of an existing server in
the environment. If you do use the name of an existing server, the AD FS
installation will fail and must be restarted.

6. On the Specify Service Account page, enter the name that you would like to use
for the managed service account. For the test example, select Create a Group
Managed Service Account, and in Account Name, enter ADFSService. Click Next.

7. On the Specify Configuration Database page, select Create a database on this

server using Windows Internal Database, and click Next.

8. The Review Options page shows you an overview of the options you have
selected. Click Next.

9. The Pre-requisite Checks page indicates whether all the prerequisite checks
passed successfully. If there are no issues, click Configure.

７ Note

If you used the name of the AD FS server or any other existing machine for
the Federation Service Name, an error message is displayed. You must start
the installation over and choose a name other than the name of an existing
machine.

10. When the configuration completes successfully, the Results page confirms that AD
FS was successfully configured.

Configure AD FS by using PowerShell

To accomplish the equivalent configuration of AD FS via Windows PowerShell, use the
following commands.
To install AD FS:

PowerShell

Add-WindowsFeature RSAT-AD-Tools
Add-WindowsFeature ADFS-Federation -IncludeManagementTools

To create the managed service account:

PowerShell

New-ADServiceAccount "ADFSService"-Server 2016-DC.contoso.com -Path

"CN=Managed Service Accounts,DC=Contoso,DC=COM" -DNSHostName 2016-
ADFS.contoso.com -ServicePrincipalNames HTTP/2016-ADFS,HTTP/2016-
ADFS.contoso.com

After you configure AD FS, you must set up an AD FS farm by using the managed
service account that you created in the previous step and the certificate you created in
the pre-configuration steps.

To set up an AD FS farm:

PowerShell

$cert = Get-ChildItem CERT:\LocalMachine\My |where {$_.Subject -match

blueadfs.contoso.com} | sort $_.NotAfter -Descending | select -first 1
$thumbprint = $cert.Thumbprint
Install-ADFSFarm -CertificateThumbprint $thumbprint -
FederationServiceDisplayName "Contoso Corporation" –FederationServiceName
blueadfs.contoso.com -GroupServiceAccountIdentifier contoso\ADFSService$ -
OverwriteConfiguration -ErrorAction Stop

Next step: Deploy Work Folders with AD FS and Web Application Proxy: Step 2, AD FS
Post-Configuration Work

See Also
Work Folders Overview
Deploy Work Folders with AD FS and
Web Application Proxy: Step 2, AD FS
Post-Configuration Work
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

This topic describes the second step in deploying Work Folders with Active Directory
Federation Services (AD FS) and Web Application Proxy. You can find the other steps in
this process in these topics:

Deploy Work Folders with AD FS and Web Application Proxy: Overview

Deploy Work Folders with AD FS and Web Application Proxy: Step 1, Set Up AD FS

Deploy Work Folders with AD FS and Web Application Proxy: Step 3, Set Up Work
Folders

Deploy Work Folders with AD FS and Web Application Proxy: Step 4, Set Up Web
Application Proxy

Deploy Work Folders with AD FS and Web Application Proxy: Step 5, Set Up Clients

７ Note

The instructions covered in this section are for a Windows Server 2019 or Windows
Server 2016 environment. If you're using Windows Server 2012 R2, follow the
Windows Server 2012 R2 instructions.

In step 1, you installed and configured AD FS. Now, you need to perform the following
post-configuration steps for AD FS.

Configure DNS entries

You must create two DNS entries for AD FS. These are the same two entries that were
used in the pre-installation steps when you created the subject alternative name (SAN)
certificate.

The DNS entries are in the form:

 AD FS service name.domain

enterpriseregistration.domain

AD FS server name.domain (DNS entry should already exist. e.g., 2016-

ADFS.contoso.com)

In the test example, the values are:

blueadfs.contoso.com

enterpriseregistration.contoso.com

Create the A and CNAME records for AD FS

To create A and CNAME records for AD FS, follow these steps:

1. On your domain controller, open DNS Manager.

2. Expand the Forward Lookup Zones folder, right-click on your domain, and select
New Host (A).

3. The New Host window opens. In the Name field, enter the alias for the AD FS
service name. In the test example, this is blueadfs.

The alias must be the same as the subject in the certificate that was used for AD FS.
For example, if the subject was adfs.contoso.com, then the alias entered here
would be adfs.

） Important

When you set up AD FS by using the Windows Server user interface (UI)
instead of Windows PowerShell, you must create an A record instead of a
CNAME record for AD FS. The reason is that the service principal name (SPN)
that is created via the UI contains only the alias that is used to set up the AD
FS service as the host.

4. In IP address, enter the IP address for the AD FS server. In the test example, this is
192.168.0.160. Click Add Host.

5. In the Forward Lookup Zones folder, right-click on your domain again, and select
New Alias (CNAME).
 6. In the New Resource Record window, add the alias name enterpriseregistration
and enter the FQDN for the AD FS server. This alias is used for Device Join and
must be called enterpriseregistration.

7. Click OK.

To accomplish the equivalent steps via Windows PowerShell, use the following
command. The command must be executed on the domain controller.

Powershell

Add-DnsServerResourceRecord -ZoneName "contoso.com" -Name blueadfs -A -

IPv4Address 192.168.0.160
Add-DnsServerResourceRecord -ZoneName "contoso.com" -Name
enterpriseregistration -CName -HostNameAlias 2016-ADFS.contoso.com

Set up the AD FS relying party trust for Work

Folders
You can set up and configure the relying party trust for Work Folders, even though Work
Folders hasn't been set up yet. The relying party trust must be set up to enable Work
Folders to use AD FS. Because you're in the process of setting up AD FS, now is a good
time to do this step.

To set up the relying party trust:

1. Open Server Manager, on the Tools menu, select AD FS Management.

2. In the right-hand pane, under Actions, click Add Relying Party Trust.

3. On the Welcome page, select Claims aware and click Start.

4. On the Select Data Source page, select Enter data about the relying party
manually, and then click Next.

5. In the Display name field, enter WorkFolders, and then click Next.

6. On the Configure Certificate page, click Next. The token encryption certificates are
optional, and are not needed for the test configuration.

7. On the Configure URL page, click Next.

8. On the Configure Identifiers page, add the following identifier: https://windows-

server-work-folders/V1 . This identifier is a hard-coded value used by Work
 Folders, and is sent by the Work Folders service when it is communicating with AD
FS. Click Next.

9. On the Choose Access Control Policy page, select Permit Everyone, and then click
Next.

10. On the Ready to Add Trust page, click Next.

11. After the configuration is finished, the last page of the wizard indicates that the
configuration was successful. Select the checkbox for editing the claims rules, and
click Close.

12. In the AD FS snap-in, select the WorkFolders relying party trust and click Edit
Claim Issuance Policy under Actions.

13. The Edit Claim Issuance Policy for WorkFolders window opens. Click Add rule.

14. In the Claim rule template drop-down list, select Send LDAP Attributes as Claims,
and click Next.

15. On the Configure Claim Rule page, in the Claim rule name field, enter
WorkFolders.

16. In the Attribute store drop-down list, select Active Directory.

17. In the mapping table, enter these values:

User-Principal-Name: UPN

Display Name: Name

Surname: Surname

Given-Name: Given Name

18. Click Finish. You'll see the WorkFolders rule listed on the Issuance Transform Rules
tab and click OK.

Set relying part trust options

After the relying party trust has been set up for AD FS, you must finish the configuration
by running five commands in Windows PowerShell. These commands set options that
are needed for Work Folders to communicate successfully with AD FS, and can't be set
through the UI. These options are:

Enable the use of JSON web tokens (JWTs)

 Disable encrypted claims

Enable auto-update

Set the issuing of Oauth refresh tokens to All Devices.

Grant clients access to the relying party trust

To set these options, use the following commands:

PowerShell

Set-ADFSRelyingPartyTrust -TargetIdentifier "https://windows-server-work-

folders/V1" -EnableJWT $true
Set-ADFSRelyingPartyTrust -TargetIdentifier "https://windows-server-work-
folders/V1" -Encryptclaims $false
Set-ADFSRelyingPartyTrust -TargetIdentifier "https://windows-server-work-
folders/V1" -AutoupdateEnabled $true
Set-ADFSRelyingPartyTrust -TargetIdentifier "https://windows-server-work-
folders/V1" -IssueOAuthRefreshTokensTo AllDevices
Grant-AdfsApplicationPermission -ServerRoleIdentifier "https://windows-
server-work-folders/V1" -AllowAllRegisteredClients -ScopeNames
openid,profile

Enable Workplace Join

Enabling Workplace Join is optional, but can be useful when you want users to be able
to use their personal devices to access workplace resources.

To enable device registration for Workplace Join, you must run the following Windows
PowerShell commands, which will configure device registration and set the global
authentication policy:

PowerShell

Initialize-ADDeviceRegistration -ServiceAccountName <your AD FS service

account>
Example: Initialize-ADDeviceRegistration -ServiceAccountName
contoso\adfsservice$
Set-ADFSGlobalAuthenticationPolicy -DeviceAuthenticationEnabled $true

Export the AD FS certificate

Next, export the self-signed AD FS certificate so that it can be installed on the following
machines in the test environment:
 The server that is used for Work Folders

The server that is used for Web Application Proxy

The domain-joined Windows client

The non-domain-joined Windows client

To export the certificate, follow these steps:

1. Click Start, and then click Run.

2. Type MMC.

3. On the File menu, click Add/Remove Snap-in.

4. In the Available snap-ins list, select Certificates, and then click Add. The
Certificates Snap-in Wizard starts.

5. Select Computer account, and then click Next.

6. Select Local computer: (the computer this console is running on), and then click
Finish.

7. Click OK.

8. Expand the folder Console Root\Certificates(Local

Computer)\Personal\Certificates.

9. Right-click the AD FS certificate, click All Tasks, and then click Export....

10. The Certificate Export Wizard opens. Select Yes, export the private key.

11. On the Export File Format page, leave the default options selected, and click Next.

12. Create a password for the certificate. This is the password that you'll use later when
you import the certificate to other devices. Click Next.

13. Enter a location and name for the certificate, and then click Finish.

Installation of the certificate is covered later in the deployment procedure.

Manage the private key setting

You must give the AD FS service account permission to access the private key of the new
certificate. You will need to grant this permission again when you replace the
communication certificate after it expires. To grant permission, follow these steps:
 1. Click Start, and then click Run.

2. Type MMC.

3. On the File menu, click Add/Remove Snap-in.

4. In the Available snap-ins list, select Certificates, and then click Add. The
Certificates Snap-in Wizard starts.

5. Select Computer account, and then click Next.

6. Select Local computer: (the computer this console is running on), and then click
Finish.

7. Click OK.

8. Expand the folder Console Root\Certificates(Local

Computer)\Personal\Certificates.

9. Right-click the AD FS certificate, click All Tasks, and then click Manage Private
Keys.

10. In the Permissions window, click Add.

11. In the Object Types window, select Service Accounts, and then click OK.

12. Type the name of the account that is running AD FS. In the test example, this is
ADFSService. Click OK.

13. In the Permissions window, give the account at least read permissions, and click
OK.

If you don't have the option to manage private keys, you might need to run the
following command: certutil -repairstore my *

Verify that AD FS is operational

To verify that AD FS is operational, open a browser window and go to
https://blueadfs.contoso.com/federationmetadata/2007-06/federationmetadata.xml ,
changing the URL to match your environment.

The browser window will display the federation server metadata without any formatting.
If you can see the data without any SSL errors or warnings, your federation server is
operational.
Next step: Deploy Work Folders with AD FS and Web Application Proxy: Step 3, Set Up
Work Folders

See Also
Work Folders Overview
Deploy Work Folders with AD FS and
Web Application Proxy: Step 3, Set-up
Work Folders
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

This topic describes the third step in deploying Work Folders with Active Directory
Federation Services (AD FS) and Web Application Proxy. You can find the other steps in
this process in these topics:

Deploy Work Folders with AD FS and Web Application Proxy: Overview

Deploy Work Folders with AD FS and Web Application Proxy: Step 1, Set Up AD FS

Deploy Work Folders with AD FS and Web Application Proxy: Step 2, AD FS Post-
Configuration Work

Deploy Work Folders with AD FS and Web Application Proxy: Step 4, Set Up Web
Application Proxy

Deploy Work Folders with AD FS and Web Application Proxy: Step 5, Set Up Clients

７ Note

The instructions covered in this section are for a Windows Server 2019 or Windows
Server 2016 environment. If you're using Windows Server 2012 R2, follow the
Windows Server 2012 R2 instructions.

To set up Work Folders, use the following procedures.

Pre-installment work
In order to install Work Folders, you must have a server that is joined to the domain and
running Windows Server 2016. The server must have a valid network configuration.

For the test example, join the machine that will run Work Folders to the Contoso domain
and set up the network interface as described in the following sections.
Set the server IP address
Change your server IP address to a static IP address. For the test example, use IP class A,
which is 192.168.0.170 / subnet mask: 255.255.0.0 / Default Gateway: 192.168.0.1 /
Preferred DNS: 192.168.0.150 (the IP address of your domain controller).

Create the CNAME record for Work Folders

To create the CNAME record for Work Folders, follow these steps:

1. On your domain controller, open DNS Manager.

2. Expand the Forward Lookup Zones folder, right-click on your domain, and click
New Alias (CNAME).

3. In the New Resource Record window, in the Alias name field, enter the alias for
Work Folders. In the test example, this is workfolders.

4. In the Fully qualified domain name field, the value should be

workfolders.contoso.com.

5. In the Fully qualified domain name for target host field, enter the FQDN for the
Work Folders server. In the test example, this is 2016-WF.contoso.com.

6. Click OK.

To accomplish the equivalent steps via Windows PowerShell, use the following
command. The command must be executed on the domain controller.

PowerShell

Add-DnsServerResourceRecord -ZoneName "contoso.com" -Name workfolders -

CName -HostNameAlias 2016-wf.contoso.com

Install the AD FS certificate

Install the AD FS certificate that was created during AD FS setup into the local computer
certificate store, using these steps:

1. Click Start, and then click Run.

2. Type MMC.

3. On the File menu, click Add/Remove Snap-in.

 4. In the Available snap-ins list, select Certificates, and then click Add. The
Certificates Snap-in Wizard starts.

5. Select Computer account, and then click Next.

6. Select Local computer: (the computer this console is running on), and then click
Finish.

7. Click OK.

8. Expand the folder Console Root\Certificates(Local

Computer)\Personal\Certificates.

9. Right-click Certificates, click All Tasks, and then click Import.

10. Browse to the folder that contains the AD FS certificate, and follow the instructions
in the wizard to import the file and place it in the certificate store.

11. Expand the folder Console Root\Certificates(Local Computer)\Trusted Root

Certification Authorities\Certificates.

12. Right-click Certificates, click All Tasks, and then click Import.

13. Browse to the folder that contains the AD FS certificate, and follow the instructions
in the wizard to import the file and place it in the Trusted Root Certification
Authorities store.

Create the Work Folders self-signed certificate

To create the Work Folders self-signed certificate, follow these steps:

1. Download the scripts provided in the Deploying Work Folders with AD FS and Web
Application Proxy blog post and then copy the file makecert.ps1 to the Work
Folders machine.

2. Open a Windows PowerShell window with admin privileges.

3. Set the execution policy to unrestricted:

PowerShell

PS C:\temp\scripts> Set-ExecutionPolicy -ExecutionPolicy Unrestricted

4. Change to the directory where you copied the script.

5. Execute the makeCert script:

 PowerShell

PS C:\temp\scripts> .\makecert.ps1

6. When you are prompted to change the subject certificate, enter the new value for
the subject. In this example, the value is workfolders.contoso.com.

7. When you are prompted to enter subject alternative name (SAN) names, press Y
and then enter the SAN names, one at a time.

For this example, type workfolders.contoso.com, and press Enter. Then type 2016-
WF.contoso.com and press Enter.

When all of the SAN names have been entered, press Enter on an empty line.

8. When you are prompted to install the certificates to the Trusted Root Certification
Authority store, press Y.

The Work Folders certificate must be a SAN certificate with the following values:

workfolders.domain

machine name.domain

In the test example, the values are:

workfolders.contoso.com

2016-WF.contoso.com

Install Work Folders

To install the Work Folders role, follow these steps:

1. Open Server Manager, click Add roles and features, and click Next.

2. On the Installation Type page, select Role-based or feature-based installation,

and click Next.

3. On the Server Selection page, select the current server, and click Next.

4. On the Server Roles page, expand File and Storage Services, expand File and
iSCSI Services, and then select Work Folders.

5. On the Add Roles and Feature Wizard page, click Add Features, and click Next.
 6. On the Features page, click Next.

7. On the Confirmation page, click Install.

Configure Work Folders

To configure Work Folders, follow these steps:

1. Open Server Manager.

2. Select File and Storage Services, and then select Work Folders.

3. On the Work Folders page, start the New Sync Share Wizard, and click Next.

4. On the Server and Path page, select the server where the sync share will be
created, enter a local path where the Work Folders data will be stored, and click
Next.

If the path doesn't exist, you'll be prompted to create it. Click OK.

5. On the User Folder Structure page, select User alias, and then click Next.

6. On the Sync Share Name page, enter the name for the sync share. For the test
example, this is WorkFolders. Click Next.

7. On the Sync Access page, add the users or groups that will have access to the new
sync share. For the test example, grant access to all domain users. Click Next.

8. On the PC Security Policies page, select Encrypt work folders and Automatically
lock page and require a password. Click Next.

9. On the Confirmation page, click Create to finish the configuration process.

Work Folders post-configuration work

To finish setting up Work Folders, complete these additional steps:

Bind the Work Folders certificate to the SSL port

Configure Work Folders to use AD FS authentication

Export the Work Folders certificate (if you are using a self-signed certificate)

Bind the certificate

Work Folders communicates only over SSL and must have the self-signed certificate that
you created earlier (or that your certificate authority issued) bound to the port.

There are two methods that you can use to bind the certificate to the port via Windows
PowerShell: IIS cmdlets and netsh.

Bind the certificate by using netsh

To use the netsh command-line scripting utility in Windows PowerShell, you must pipe
the command to netsh. The following example script finds the certificate with the
subject workfolders.contoso.com and binds it to port 443 by using netsh:

PowerShell

$subject = "workfolders.contoso.com"
Try
{
#In case there are multiple certificates with the same subject, get the
latest version
$cert = Get-ChildItem CERT:\LocalMachine\My |where {$_.Subject -match
$subject} | sort $_.NotAfter -Descending | select -first 1
$thumbprint = $cert.Thumbprint
$Command = "http add sslcert ipport=0.0.0.0:443 certhash=$thumbprint appid=
{CE66697B-3AA0-49D1-BDBD-A25C8359FD5D} certstorename=MY"
$Command | netsh
}
Catch
{
" Error: unable to locate certificate for $($subject)"
Exit
}

Bind the certificate by using IIS cmdlets

You can also bind the certificate to the port by using IIS management cmdlets, which are
available if you installed the IIS management tools and scripts.

７ Note

Installation of the IIS management tools doesn't enable the full version of Internet
Information Services (IIS) on the Work Folders machine; it only enables the
management cmdlets. There are some possible benefits to this setup. For example,
if you're looking for cmdlets to provide the functionality that you get from netsh.
When the certificate is bound to the port via the New-WebBinding cmdlet, the
binding is not dependent on IIS in any way. After you do the binding, you can even
 remove the Web-Mgmt-Console feature, and the certificate will still be bound to
the port. You can verify the binding via netsh by typing netsh http show sslcert.

The following example uses the New-WebBinding cmdlet to find the certificate with the
subject workfolders.contoso.com and bind it to port 443:

PowerShell

$subject = "workfolders.contoso.com"
Try
{
#In case there are multiple certificates with the same subject, get the
latest version
$cert =Get-ChildItem CERT:\LocalMachine\My |where {$_.Subject -match
$subject } | sort $_.NotAfter -Descending | select -first 1
$thumbprint = $cert.Thumbprint
New-WebBinding -Name "Default Web Site" -IP * -Port 443 -Protocol https
#The default IIS website name must be used for the binding. Because Work
Folders uses Hostable Web Core and its own configuration file, its website
name, 'ECSsite', will not work with the cmdlet. The workaround is to use the
default IIS website name, even though IIS is not enabled, because the
NewWebBinding cmdlet looks for a site in the default IIS configuration file.
Push-Location IIS:\SslBindings
Get-Item cert:\LocalMachine\MY\$thumbprint | new-item *!443
Pop-Location
}
Catch
{
" Error: unable to locate certificate for $($subject)"
Exit
}

Set up AD FS authentication
To configure Work Folders to use AD FS for authentication, follow these steps:

1. Open Server Manager.

2. Click Servers, and then select your Work Folders server in the list.

3. Right-click the server name, and click Work Folders Settings.

4. In the Work Folder Settings window, select Active Directory Federation Services,
and type in the Federation Service URL. Click Apply.

In the test example, the URL is https://blueadfs.contoso.com .

The cmdlet to accomplish the same task via Windows PowerShell is:
 PowerShell

Set-SyncServerSetting -ADFSUrl "https://blueadfs.contoso.com"

If you're setting up AD FS with self-signed certificates, you might receive an error

message that says the Federation Service URL is incorrect, unreachable, or a relying
party trust has not been set up.

This error can also happen if the AD FS certificate was not installed on the Work Folders
server or if the CNAME for AD FS was not set up correctly. You must correct these issues
before proceeding.

Export the Work Folders certificate

The self-signed Work Folders certificate must be exported so that you can later install it
on the following machines in the test environment:

The server that is used for Web Application Proxy

The domain-joined Windows client

The non-domain-joined Windows client

To export the certificate, follow the same steps you used to export the AD FS certificate
earlier, as described in Deploy Work Folders with AD FS and Web Application Proxy: Step
2, AD FS Post-Configuration Work, Export the AD FS certificate.

Next step: Deploy Work Folders with AD FS and Web Application Proxy: Step 4, Set Up
Web Application Proxy

See Also
Work Folders Overview
Deploy Work Folders with AD FS and
Web Application Proxy: Step 4, Set-up
Web Application Proxy
Article • 07/29/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

This topic describes the fourth step in deploying Work Folders with Active Directory
Federation Services (AD FS) and Web Application Proxy. You can find the other steps in
this process in these topics:

Deploy Work Folders with AD FS and Web Application Proxy: Overview

Deploy Work Folders with AD FS and Web Application Proxy: Step 1, Set Up AD FS

Deploy Work Folders with AD FS and Web Application Proxy: Step 2, AD FS Post-
Configuration Work

Deploy Work Folders with AD FS and Web Application Proxy: Step 3, Set Up Work
Folders

Deploy Work Folders with AD FS and Web Application Proxy: Step 5, Set Up Clients

７ Note

The instructions covered in this section are for a Windows Server 2019 or Windows
Server 2016 environment. If you're using Windows Server 2012 R2, follow the
Windows Server 2012 R2 instructions.

To set up Web Application Proxy for use with Work Folders, use the following
procedures.

Install the AD FS and Work Folder certificates

You must install the AD FS and Work Folders certificates that you created earlier (in step
1, Set up AD FS, and step 3, Set up Work Folders) into the local computer certificate
store on the machine where the Web Application Proxy role will be installed.
Because you're installing self-signed certificates that can't be traced back to a publisher
in the Trusted Root Certification Authorities certificate store, you must also copy the
certificates to that store.

To install the certificates, follow these steps:

1. Click Start, and then click Run.

2. Type MMC.

3. On the File menu, click Add/Remove Snap-in.

4. In the Available snap-ins list, select Certificates, and then click Add. The
Certificates Snap-in Wizard starts.

5. Select Computer account, and then click Next.

6. Select Local computer: (the computer this console is running on), and then click
Finish.

7. Click OK.

8. Expand the folder Console Root\Certificates(Local

Computer)\Personal\Certificates.

9. Right-click Certificates, click All Tasks, and then click Import.

10. Browse to the folder that contains the AD FS certificate, and follow the instructions
in the wizard to import the file and place it in the certificate store.

11. Repeat steps 9 and 10, this time browsing to the Work Folders certificate and
importing it.

12. Expand the folder Console Root\Certificates(Local Computer)\Trusted Root

Certification Authorities\Certificates.

13. Right-click Certificates, click All Tasks, and then click Import.

14. Browse to the folder that contains the AD FS certificate, and follow the instructions
in the wizard to import the file and place it in the Trusted Root Certification
Authorities store.

15. Repeat steps 13 and 14, this time browsing to the Work Folders certificate and
importing it.

Install Web Application Proxy

To install Web Application Proxy, follow these steps:

1. On the server where you plan to install the Web Application Proxy, open Server
Manager and start the Add Roles and Features Wizard.

2. Click Next on the first and second pages of the wizard.

3. On the Server Selection page, select your server, and then click Next.

4. On the Server Role page, select the Remote Access role, and then click Next.

5. On the Features page and Remote Access page, click Next.

6. On the Role Services page, select Web Application Proxy, click Add Features, and
then click Next.

7. On the Confirm installation selections page, click Install.

Configure Web Application Proxy

To configure Web Application Proxy, follow these steps:

1. Click the warning flag at the top of Server Manager, and then click the link to open
the Web Application Proxy Configuration Wizard.

2. On the Welcome page, press Next.

3. On the Federation Server page, enter the Federation Service name. In the test
example, this is blueadfs.contoso.com.

4. Enter the credentials of a local administrator account on the federation servers. Do

not enter in domain credentials (for example, contoso\administrator), but local
credentials (for example, administrator).

5. On the AD FS Proxy Certificate page, select the AD FS certificate that you

imported earlier. In the test case, this is blueadfs.contoso.com. Click Next.

6. The confirmation page shows the Windows PowerShell command that will execute
to configure the service. Click Configure.

Publish the Work Folders web application

The next step is to publish a web application that will make Work Folders available to
clients. To publish the Work Folders web application, follow these steps:
 1. Open Server Manager, and on the Tools menu, click Remote Access Management
to open the Remote Access Management Console.

2. Under Configuration, click Web Application Proxy.

3. Under Tasks, click Publish. The Publish New Application Wizard opens.

4. On the Welcome page, click Next.

5. On the Preauthentication page, select Active Directory Federation Services (AD

FS), and click Next.

6. On the Support Clients page, select OAuth2, and click Next.

7. On the Relying Party page, select Work Folders, and then click Next. This list is
published to the Web Application Proxy from AD FS.

8. On the Publishing Settings page, enter the following and then click Next:

The name you want to use for the web application

The external URL for Work Folders

The name of the Work Folders certificate

The back-end URL for Work Folders

By default, the wizard makes the back-end URL the same as the external URL.

For the test example, use these values:

Name: WorkFolders

External URL: https://workfolders.contoso.com

External certificate: The Work Folders certificate that you installed earlier

Backend server URL: https://workfolders.contoso.com

9. The confirmation page shows the Windows PowerShell command that will execute
to publish the application. Click Publish.

10. On the Results page, you should see the application was published successfully.

７ Note

If you have multiple Work Folders servers, you need to publish a Work Folders
web application for each Work Folders server (repeat steps 1-10).
Next step: Deploy Work Folders with AD FS and Web Application Proxy: Step 5, Set Up
Clients

See Also
Work Folders Overview
Deploy Work Folders with AD FS and
Web Application Proxy: Step 5, Set-up
Clients
Article • 09/24/2021

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

This topic describes the fifth step in deploying Work Folders with Active Directory
Federation Services (AD FS) and Web Application Proxy. You can find the other steps in
this process in these topics:

Deploy Work Folders with AD FS and Web Application Proxy: Overview

Deploy Work Folders with AD FS and Web Application Proxy: Step 1, Set Up AD FS

Deploy Work Folders with AD FS and Web Application Proxy: Step 2, AD FS Post-
Configuration Work

Deploy Work Folders with AD FS and Web Application Proxy: Step 3, Set Up Work
Folders

Deploy Work Folders with AD FS and Web Application Proxy: Step 4, Set Up Web
Application Proxy

Use the following procedures to set up the domain-joined and non-domain joined
Windows clients. You can use these clients to test whether files are syncing correctly
between the clients' Work Folders.

Set up a domain-joined client

Install the AD FS and Work Folder certificates

You must install the AD FS and Work Folders certificates that you created earlier into the
local computer certificate store on the domain-joined client machine.

Because you are installing self-signed certificates that can't be traced back to a publisher
in the Trusted Root Certification Authorities certificate store, you must also copy the
certificates to that store.

To install the certificates, follow these steps:

 1. Click Start, and then click Run.

2. Type MMC.

3. On the File menu, click Add/Remove Snap-in.

4. In the Available snap-ins list, select Certificates, and then click Add. The
Certificates Snap-in Wizard starts.

5. Select Computer account, and then click Next.

6. Select Local computer: (the computer this console is running on), and then click
Finish.

7. Click OK.

8. Expand the folder Console Root\Certificates(Local Computer)\Personal\Certificates.

9. Right-click Certificates, click All Tasks, and then click Import.

10. Browse to the folder that contains the AD FS certificate, and follow the instructions
in the wizard to import the file and place it in the certificate store.

11. Repeat steps 9 and 10, this time browsing to the Work Folders certificate and
importing it.

12. Expand the folder Console Root\Certificates(Local Computer)\Trusted Root

Certification Authorities\Certificates.

13. Right-click Certificates, click All Tasks, and then click Import.

14. Browse to the folder that contains the AD FS certificate, and follow the instructions
in the wizard to import the file and place it in the Trusted Root Certification
Authorities store.

15. Repeat steps 13 and 14, this time browsing to the Work Folders certificate and
importing it.

Configure Work Folders on the client

To configure Work Folders on the client machine, follow these steps:

1. On the client machine, open Control Panel and click Work Folders.

2. Click Set up Work Folders.

 3. On the Enter your work email address page, enter either the user's email address
(for example, [email protected]) or the Work Folders URL (in the test example,
https://workfolders.contoso.com), and then click Next.

4. If the user is connected to the corporate network, the authentication is performed

by Windows Integrated Authentication. If the user is not connected to the
corporate network, the authentication is performed by AD FS (OAuth) and the user
will be prompted for credentials. Enter your credentials and click OK.

5. After you have authenticated, the Introducing Work Folders page is displayed,
where you can optionally change the Work Folders directory location. Click Next.

6. The Security Policies page lists the security policies that you set up for Work
Folders. Click Next.

7. A message is displayed stating that Work Folders has started syncing with your PC.
Click Close.

8. The Manage Work Folders page shows the amount of space available on the
server, sync status, and so on. If necessary, you can re-enter your credentials here.
Close the window.

9. Your Work Folders folder opens automatically. You can add content to this folder
to sync between your devices.

For the purpose of the test example, add a test file to this Work Folders folder.
After you set up Work Folders on the non-domain-joined machine, you will be able
to sync files between the Work Folders on each machine.

Set up a non-domain-joined client

Install the AD FS and Work Folder certificates

Install the AD FS and Work Folders certificates on the non-domain-joined machine,
using the same procedure that you used for the domain-joined machine.

Update the hosts file

The hosts file on the non-domain-joined client must be updated for the test
environment, because no public DNS records were created for Work Folders. Add these
entries to the hosts file:

workfolders.domain
 AD FS service name.domain

enterpriseregistration.domain

For the test example, use these values:

10.0.0.10 workfolders.contoso.com

10.0.0.10 blueadfs.contoso.com

10.0.0.10 enterpriseregistration.contoso.com

Configure Work Folders on the client

Configure Work Folders on the non-domain-joined machine by using the same
procedure that you used for the domain-joined machine.

When the new Work Folders folder opens on this client, you can see that the file from
the domain-joined machine has already synced to the non-domain-joined machine. You
can start adding content to the folder to sync between your devices.

This concludes the procedure for deploying Work Folders, AD FS and Web Application
Proxy via the Windows Server UI.

See Also
Work Folders Overview
Storage Quality of Service
Article • 03/29/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

Storage Quality of Service (QoS) in Windows Server 2016 provides a way to centrally
monitor and manage storage performance for virtual machines using Hyper-V and the
Scale-Out File Server roles. The feature automatically improves storage resource fairness
between multiple virtual machines using the same file server cluster and allows policy-
based minimum and maximum performance goals to be configured in units of
normalized IOPS.

You can use Storage QoS in Windows Server 2016 to accomplish the following:

Mitigate noisy neighbor issues. By default, Storage QoS ensures that a single
virtual machine cannot consume all storage resources and starve other virtual
machines of storage bandwidth.

Monitor end to end storage performance. As soon as virtual machines stored on

a Scale-Out File Server are started, their performance is monitored. Performance
details of all running virtual machines and the configuration of the Scale-Out File
Server cluster can be viewed from a single location

Manage Storage I/O per workload business needs Storage QoS policies define
performance minimums and maximums for virtual machines and ensures that they
are met. This provides consistent performance to virtual machines, even in dense
and overprovisioned environments. If policies cannot be met, alerts are available to
track when VMs are out of policy or have invalid policies assigned.

This document outlines how your business can benefit from the new Storage QoS
functionality. It assumes that you have a previous working knowledge of Windows
Server, Windows Server Failover Clustering, Scale-Out File Server, Hyper-V, and Windows
PowerShell.

Overview
This section describes the requirements for using Storage QoS, an overview of a
software-defined solution using Storage QoS, and a list of Storage QoS related
terminologies.
Storage QoS Requirements
Storage QoS supports two deployment scenarios:

Hyper-V using a Scale-Out File Server This scenario requires both of the following:

Storage cluster that is a Scale-Out File Server cluster

Compute cluster that has least one server with the Hyper-V role enabled

For Storage QoS, the Failover Cluster is required on Storage servers, but the
compute servers are not required to be in a failover cluster. All servers (used for
both Storage and Compute) must be running Windows Server 2016.

If you do not have a Scale-Out File Server cluster deployed for evaluation
purposes, for step by step instructions to build one using either existing servers or
virtual machines, see Windows Server 2012 R2 Storage: Step-by-step with Storage
Spaces, SMB Scale-Out and Shared VHDX (Physical).

Hyper-V using Cluster Shared Volumes. This scenario requires both of the
following:

Compute cluster with the Hyper-V role enabled

Hyper-V using Cluster Shared Volumes (CSV) for storage

Failover Cluster is required. All servers must be running the same version of Windows
Server 2016.

Using Storage QoS in a software-defined storage solution

Storage Quality of Service is built into the Microsoft software-defined storage solution
provided by Scale-Out File Server and Hyper-V. The Scale-Out File Server exposes file
shares to the Hyper-V servers using the SMB3 protocol. A new Policy Manager has been
added to the File Server cluster, which provides the central storage performance
monitoring.
Figure 1: Using Storage QoS in a software-defined storage solution in Scale-Out File
Server

As Hyper-V servers launch virtual machines, they are monitored by the Policy Manager.
The Policy Manager communicates the Storage QoS policy and any limits or reservations
back to the Hyper-V server, which controls the performance of the virtual machine as
appropriate.

When there are changes to Storage QoS policies or to the performance demands by
virtual machines, the Policy Manager notifies the Hyper-V servers to adjust their
behavior. This feedback loop ensures that all virtual machines VHDs perform
consistently according to the Storage QoS policies as defined.

Glossary

Term Description

Normalized All of the storage usage is measured in "Normalized IOPS." This is a count of the
IOPS storage input/output operations per second. Any IO that is 8KB or smaller is
considered as one normalized IO. Any IO that is larger than 8KB is treated as
multiple normalized IOs. For example, a 256KB request is treated as 32
normalized IOPS.
Windows Server 2016 includes the ability to specify the size used to normalize
IOs. On the storage cluster, the normalized size can be specified and take effect
on the normalization calculations cluster wide. The default remains 8 KB. This
setting affects all virtual machines. (The virtual machines created on local
volumes are also affected.)
 Term Description

Flow Each file handle opened by a Hyper-V server to a VHD or VHDX file is considered
a "flow". If a virtual machine has two virtual hard disks attached, it will have 1
flow to the file server cluster per file. If a VHDX is shared with multiple virtual
machines, it will have 1 flow per virtual machine.

InitiatorName Name of the virtual machine that is reported to the Scale-Out File Server for
each flow.

InitiatorID An identifier matching the virtual machine ID. This can always be used to
uniquely identify individual flows virtual machines even if the virtual machines
have the same InitiatorName.

Policy Storage QoS policies are stored in the cluster database, and have the following
properties: PolicyId, MinimumIOPS, MaximumIOPS, ParentPolicy, and PolicyType.

PolicyId Unique identifier for a policy. It is generated by default, but can be specified if
desired.

MinimumIOPS Minimum normalized IOPS that will be provided by a policy. Also known as
"Reservation".

MaximumIOPS Maximum normalized IOPS that will be limited by a policy. Also known as
"Limit".

Aggregated A policy type where the specified MinimumIOPS & MaximumIOPS and
Bandwidth are shared among all flows assigned to the policy. All VHD's assigned
the policy on that storage system have a single allocation of I/O bandwidth for
them to all share.

Dedicated A policy type where the specified Minimum & MaximumIOPS and Bandwidth are
managed for individual VHD/VHDx.

How to set up Storage QoS and monitor basic

performance
This section describes how to enable the new Storage QoS feature and how to monitor
storage performance without applying custom policies.

Set up Storage QoS on a Storage Cluster

This section discusses how to enable Storage QoS on either a new or an existing Failover
Cluster and Scale-Out File Server that is running Windows Server 2016.

Set up Storage QoS on a new installation

If you have configured a new Failover Cluster and configured a Cluster Shared
Volume(CSV) on Windows Server 2016, then the Storage QoS feature will be set up
automatically.

Verify Storage QoS installation

After you have created a Failover Cluster and configured a CSV disk, , Storage QoS
Resource is displayed as a Cluster Core Resource and visible in both Failover Cluster
Manager and Windows PowerShell. The intent is that the failover cluster system will
manage this resource and you should not have to do any actions against this resource.
We display it in both Failover Cluster Manager and PowerShell to be consistent with the
other failover cluster system resources like the new Health Service.

Figure 2: Storage QoS Resource displayed as a Cluster Core Resource in Failover

Cluster Manager

Use the following PowerShell cmdlet to view the status of Storage QoS Resource.

PowerShell

PS C:\> Get-ClusterResource -Name "Storage Qos Resource"

Name State OwnerGroup ResourceType

---- ----- ---------- ------------
Storage Qos Resource Online Cluster Group Storage QoS Policy
Manager

Set up Storage QoS on a Compute Cluster

The Hyper-V role in Windows Server 2016 has built-in support for Storage QoS and is
enabled by default.
Install Remote Administration Tools to manage Storage QoS
policies from remote computers

You can manage Storage QoS policies and monitor flows from compute hosts using the
Remote Server Administration Tools. These are available as optional features on all
Windows Server 2016 installations, and can be downloaded separately for Windows 10
at the Microsoft Download Center website.

The RSAT-Clustering optional feature includes the Windows PowerShell module for
remote management of Failover Clustering, including Storage QoS.

Windows PowerShell: Add-WindowsFeature RSAT-Clustering

The RSAT-Hyper-V-Tools optional feature includes the Windows PowerShell module for
remote management of Hyper-V.

Windows PowerShell: Add-WindowsFeature RSAT-Hyper-V-Tools

Deploy virtual machines to run workloads for testing

You will need some virtual machines stored on the Scale-Out File Server with relevant
workloads. For some tips in how to simulate load and do some stress testing, see the
following page for a recommended tool (DiskSpd) and some example usage: DiskSpd,
PowerShell and storage performance: measuring IOPS, throughput and latency for both
local disks and SMB file shares.

The example scenarios shown in this guide includes five virtual machines. BuildVM1,
BuildVM2, BuildVM3 and BuildVM4 are running a desktop workload with low to
moderate storage demands. TestVm1 is running an online transaction processing
benchmark with high storage demand.

View current storage performance metrics

This section includes:

How to query flows using the Get-StorageQosFlow cmdlet.

How to view performance for a volume using the Get-StorageQosVolume cmdlet.

Query flows using the Get-StorageQosFlow cmdlet

The Get-StorageQosFlow cmdlet shows all current flows initiated by Hyper-V servers. All
data is collected by the Scale-Out File Server cluster, hence the cmdlet can be used on
any node in the Scale-Out File Server cluster, or against a remote server using the -
CimSession parameter.

The following sample command shows how to view all files opened by Hyper-V on
server using Get-StorageQoSFlow.

PowerShell

PS C:\> Get-StorageQosFlow

InitiatorName InitiatorNodeNam StorageNodeName FilePath Status

e
------------- ---------------- --------------- -------- ------
plang-fs3.pla... C:\ClusterSt... Ok
plang-fs2.pla... C:\ClusterSt... Ok
plang-fs1.pla... C:\ClusterSt... Ok
plang-fs3.pla... C:\ClusterSt... Ok
plang-fs2.pla... C:\ClusterSt... Ok
plang-fs1.pla... C:\ClusterSt... Ok
TR20-VMM plang-z400.pl... plang-fs1.pla... C:\ClusterSt... Ok
BuildVM4 plang-c2.plan... plang-fs1.pla... C:\ClusterSt... Ok
WinOltp1 plang-c1.plan... plang-fs1.pla... C:\ClusterSt... Ok
BuildVM3 plang-c2.plan... plang-fs1.pla... C:\ClusterSt... Ok
BuildVM1 plang-c2.plan... plang-fs1.pla... C:\ClusterSt... Ok
TR20-VMM plang-z400.pl... plang-fs1.pla... C:\ClusterSt... Ok
BuildVM2 plang-c2.plan... plang-fs1.pla... C:\ClusterSt... Ok
TR20-VMM plang-z400.pl... plang-fs1.pla... C:\ClusterSt... Ok
plang-fs3.pla... C:\ClusterSt... Ok
plang-fs2.pla... C:\ClusterSt... Ok
BuildVM4 plang-c2.plan... plang-fs2.pla... C:\ClusterSt... Ok
WinOltp1 plang-c1.plan... plang-fs2.pla... C:\ClusterSt... Ok
BuildVM3 plang-c2.plan... plang-fs2.pla... C:\ClusterSt... Ok
WinOltp1 plang-c1.plan... plang-fs2.pla... C:\ClusterSt... Ok
plang-fs1.pla... C:\ClusterSt... Ok

The following sample command is formatted to show virtual machine name, Hyper-V
host name, IOPS, and VHD file name, sorted by IOPS.

PowerShell

PS C:\> Get-StorageQosFlow | Sort-Object StorageNodeIOPS -Descending | ft

InitiatorName, @{Expression=
{$_.InitiatorNodeName.Substring(0,$_.InitiatorNodeName.IndexOf('.'))};Label=
"InitiatorNodeName"}, StorageNodeIOPS, Status, @{Expression=
{$_.FilePath.Substring($_.FilePath.LastIndexOf('\')+1)};Label="File"} -
AutoSize

InitiatorName InitiatorNodeName StorageNodeIOPS Status File

------------- ----------------- --------------- ------ ----
WinOltp1 plang-c1 3482 Ok IOMETER.VHDX
BuildVM2 plang-c2 544 Ok BUILDVM2.VHDX
 BuildVM1 plang-c2 497 Ok BUILDVM1.VHDX
BuildVM4 plang-c2 316 Ok BUILDVM4.VHDX
BuildVM3 plang-c2 296 Ok BUILDVM3.VHDX
BuildVM4 plang-c2 195 Ok
WIN8RTM_ENTERPRISE_VL_BU...
TR20-VMM plang-z400 156 Ok DATA1.VHDX
BuildVM3 plang-c2 81 Ok
WIN8RTM_ENTERPRISE_VL_BU...
WinOltp1 plang-c1 65 Ok BOOT.VHDX
18 Ok DefaultFlow
12 Ok DefaultFlow
WinOltp1 plang-c1 4 Ok
9914.0.AMD64FRE.WINMAIN....
TR20-VMM plang-z400 4 Ok DATA2.VHDX
TR20-VMM plang-z400 3 Ok BOOT.VHDX
0 Ok DefaultFlow
0 Ok DefaultFlow
0 Ok DefaultFlow
0 Ok DefaultFlow
0 Ok DefaultFlow
0 Ok DefaultFlow
0 Ok DefaultFlow

The following sample command shows how to filter flows based on InitiatorName to
easily find the storage performance and settings for a specific virtual machine.

PowerShell

PS C:\> Get-StorageQosFlow -InitiatorName BuildVm1 | Format-List

FilePath :
C:\ClusterStorage\Volume2\SHARES\TWO\BUILDWORKLOAD\BUILDVM1.V
HDX
FlowId : ebfecb54-e47a-5a2d-8ec0-0940994ff21c
InitiatorId : ae4e3dd0-3bde-42ef-b035-9064309e6fec
InitiatorIOPS : 464
InitiatorLatency : 26.2684
InitiatorName : BuildVM1
InitiatorNodeName : plang-c2.plang.nttest.microsoft.com
Interval : 300000
Limit : 500
PolicyId : b145e63a-3c9e-48a8-87f8-1dfc2abfe5f4
Reservation : 500
Status : Ok
StorageNodeIOPS : 475
StorageNodeLatency : 7.9725
StorageNodeName : plang-fs1.plang.nttest.microsoft.com
TimeStamp : 2/12/2015 2:58:49 PM
VolumeId : 4d91fc3a-1a1e-4917-86f6-54853b2a6787
PSComputerName :
MaximumIops : 500
MinimumIops : 500
The data returned by the Get-StorageQosFlow cmdlet includes:

The Hyper-V hostname (InitiatorNodeName).

The virtual machine's name and its Id (InitiatorName and InitiatorId)

Recent average performance as observed by the Hyper-V host for the virtual disk
(InitiatorIOPS, InitiatorLatency)

Recent average performance as observed by the Storage cluster for the virtual disk
(StorageNodeIOPS, StorageNodeLatency)

Current policy being applied to the file, if any, and the resulting configuration
(PolicyId, Reservation, Limit)

Status of the policy

Ok - No issues exist

InsufficientThroughput- A policy is applied, but the Minimum IOPS cannot be

delivered. This can happen if the minimum for a VM, or all VMs together, are
more than the storage volume can deliver.

UnknownPolicyId - A policy was assigned to the virtual machine on the Hyper-

V host, but is missing from the file server. This policy should be removed from
the virtual machine configuration, or a matching policy should be created on
the file server cluster.

View performance for a volume using Get-StorageQosVolume

Storage performance metrics are also collected on a per-storage volume level, in
addition to the per-flow performance metrics. This makes it easy to see the average
total utilization in normalized IOPS, latency, and aggregate limits and reservations
applied to a volume.

PowerShell

PS C:\> Get-StorageQosVolume | Format-List

Interval : 300000
IOPS : 0
Latency : 0
Limit : 0
Reservation : 0
Status : Ok
TimeStamp : 2/12/2015 2:59:38 PM
VolumeId : 434f561f-88ae-46c0-a152-8c6641561504
 PSComputerName :
MaximumIops : 0
MinimumIops : 0

Interval : 300000
IOPS : 1097
Latency : 3.1524
Limit : 0
Reservation : 1568
Status : Ok
TimeStamp : 2/12/2015 2:59:38 PM
VolumeId : 4d91fc3a-1a1e-4917-86f6-54853b2a6787
PSComputerName :
MaximumIops : 0
MinimumIops : 1568

Interval : 300000
IOPS : 5354
Latency : 6.5084
Limit : 0
Reservation : 781
Status : Ok
TimeStamp : 2/12/2015 2:59:38 PM
VolumeId : 0d2fd367-8d74-4146-9934-306726913dda
PSComputerName :
MaximumIops : 0
MinimumIops : 781

How to create and monitor Storage QoS

Policies
This section describes how to create Storage QoS policies, apply these policies to virtual
machines, and monitor a storage cluster after policies are applied.

Create Storage QoS policies

Storage QoS policies are defined and managed in the Scale-Out File Server cluster. You
can create as many policies as needed for flexible deployments (up to 10,000 per
storage cluster).

Each VHD/VHDX file assigned to a virtual machine may be configured with a policy.
Different files and virtual machines can use the same policy or they can each be
configured with separate policies. If multiple VHD/VHDX files or multiple virtual
machines are configured with the same policy, they will be aggregated together and will
share the MinimumIOPS and MaximumIOPS fairly. If you use separate policies for
multiple VHD/VHDX files or virtual machines, the minimum and maximums are tracked
separately for each.

If you create multiple similar policies for different virtual machines and the virtual
machines have equal storage demand, they will receive a similar share of IOPS. If one
VM demands more and the other less, then IOPS will follow that demand.

Types of Storage QoS Policies

There are two types of policies: Aggregated (previously known as SingleInstance) and
Dedicated (previously known as MultiInstance). Aggregated policies apply maximums
and minimum for the combined set of VHD/VHDX files and virtual machines where they
apply. In effect, they share a specified set of IOPS and bandwidth. Dedicated policies
apply the minimum and maximum values for each VHD/VHDx, separately. This makes it
easy to create a single policy that applies similar limits to multiple VHD/VHDx files.

For instance, if you create a Aggregated policy with a minimum of 300 IOPS and a
maximum of 500 IOPS. If you apply this policy to 5 different VHD/VHDx files, you are
making sure that the 5 VHD/VHDx files combined will be guaranteed at least 300 IOPS
(if there is demand and the storage system can provide that performance) and no more
than 500 IOPS. If the VHD/VHDx files have similar high demand for IOPS and the
storage system can keep up, each VHD/VHDx files will get about 100 IOPS.

However, if you create a Dedicated policy with similar limits and apply it to VHD/VHDx
files on 5 different virtual machines, each virtual machine will get at least 300 IOPS and
no more than 500 IOPS. If the virtual machines have similar high demand for IOPS and
the storage system can keep up, each virtual machine will get about 500 IOPS. . If one of
the virtual machines has multiple VHD/VHDx files with the same MulitInstance policy
configured, they will share the limit so that the total IO from the VM from files with that
policy will not exceed the limits.

Hence, if you have a group of VHD/VHDx files that you want to exhibit the same
performance characteristics and you don't want the trouble of creating multiple, similar
policies, you can use a single Dedicated policy and apply to the files of each virtual
machine.

Keep the number of VHD/VHDx files assigned to a single Aggregated policy to 20 or

less. This policy type was meant to do aggregation with a few VMs on a cluster.

Create and apply a Dedicated policy

First, use the New-StorageQosPolicy cmdlet to create a policy on the Scale-Out File
Server as shown in the following example:

PowerShell

$desktopVmPolicy = New-StorageQosPolicy -Name Desktop -PolicyType Dedicated

-MinimumIops 100 -MaximumIops 200

Next, apply it to the appropriate virtual machines' hard disk drives on the Hyper-V
server. Note the PolicyId from the previous step or store it in a variable in your scripts.

On the Scale-Out File Server, using PowerShell, create a Storage QoS policy and get its
Policy ID as shown in the following example:

PowerShell

PS C:\> $desktopVmPolicy = New-StorageQosPolicy -Name Desktop -PolicyType

Dedicated -MinimumIops 100 -MaximumIops 200

C:\> $desktopVmPolicy.PolicyId

Guid
----
cd6e6b87-fb13-492b-9103-41c6f631f8e0

On the Hyper-V server, using PowerShell, set the Storage QoS Policy using the Policy ID
as shown in the following example:

PowerShell

Get-VM -Name Build* | Get-VMHardDiskDrive | Set-VMHardDiskDrive -QoSPolicyID

cd6e6b87-fb13-492b-9103-41c6f631f8e0

Confirm that the policies are applied

Use Get-StorageQosFlow PowerShell cmdlet to confirm that the MinimumIOPS and
MaximumIOPS have been applied to the appropriate flows as shown in the following
example.

PowerShell

PS C:\> Get-StorageQoSflow | Sort-Object InitiatorName |

ft InitiatorName, Status, MinimumIOPS, MaximumIOPS, StorageNodeIOPS,
Status, @{Expression=
{$_.FilePath.Substring($_.FilePath.LastIndexOf('\')+1)};Label="File"} -
 AutoSize

InitiatorName Status MinimumIops MaximumIops StorageNodeIOPS Status File

------------- ------ ----------- ----------- --------------- ------ ----
BuildVM1 Ok 100 200 250 Ok
BUILDVM1.VHDX
BuildVM2 Ok 100 200 251 Ok
BUILDVM2.VHDX
BuildVM3 Ok 100 200 252 Ok
BUILDVM3.VHDX
BuildVM4 Ok 100 200 233 Ok
BUILDVM4.VHDX
TR20-VMM Ok 33 666 1 Ok
DATA2.VHDX
TR20-VMM Ok 33 666 5 Ok
DATA1.VHDX
TR20-VMM Ok 33 666 4 Ok
BOOT.VHDX
WinOltp1 Ok 0 0 0 Ok
9914.0.AMD6...
WinOltp1 Ok 0 0 5166 Ok
IOMETER.VHDX
WinOltp1 Ok 0 0 0 Ok
BOOT.VHDX

On the Hyper-V server, you can also use the provided script Get-
VMHardDiskDrivePolicy.ps1 to see what policy is applied to a virtual hard disk drive.

PowerShell

PS C:\> Get-VM -Name BuildVM1 | Get-VMHardDiskDrive | Format-List

Path : \\plang-
fs.plang.nttest.microsoft.com\two\BuildWorkload
\BuildVM1.vhdx
DiskNumber :
MaximumIOPS : 0
MinimumIOPS : 0
QoSPolicyID : cd6e6b87-fb13-492b-9103-41c6f631f8e0
SupportPersistentReservations : False
ControllerLocation : 0
ControllerNumber : 0
ControllerType : IDE
PoolName : Primordial
Name : Hard Drive
Id : Microsoft:AE4E3DD0-3BDE-42EF-B035-
9064309E6FEC\83F8638B
-8DCA-4152-9EDA-2CA8B33039B4\0\0\D
VMId : ae4e3dd0-3bde-42ef-b035-9064309e6fec
VMName : BuildVM1
VMSnapshotId : 00000000-0000-0000-0000-000000000000
VMSnapshotName :
 ComputerName : PLANG-C2
IsDeleted : False

Query for Storage QoS Policies

Get-StorageQosPolicy lists all configured policies and their status on a Scale-Out File
Server.

PowerShell

PS C:\> Get-StorageQosPolicy

Name MinimumIops MaximumIops Status

---- ----------- ----------- ------
Default 0 0 Ok
Limit500 0 500 Ok
SilverVm 500 500 Ok
Desktop 100 200 Ok
Limit500 0 0 Ok
VMM 100 2000 Ok
Vdi 1 100 Ok

Status can change over time based on how the system is performing.

Ok - All flows using that policy are receiving their requested MinimumIOPS.

InsufficientThroughput - One or more of the flows using this policy are not
receiving the Minimum IOPS

You can also pipe a policy to Get-StorageQosPolicy to get the status of all flows
configured to use the policy as follows:

PowerShell

PS C:\> Get-StorageQosPolicy -Name Desktop | Get-StorageQosFlow | ft

InitiatorName, *IOPS, Status, FilePath -AutoSize

InitiatorName MaximumIops MinimumIops InitiatorIOPS StorageNodeIOPS Status

FilePat
h
------------- ----------- ----------- ------------- --------------- ------ -
------
BuildVM4 100 50 187 17 Ok
C:\C...
BuildVM3 100 50 194 25 Ok
C:\C...
BuildVM1 200 100 195 196 Ok
C:\C...
 BuildVM2 200 100 193 192 Ok
C:\C...
BuildVM4 200 100 187 169 Ok
C:\C...
BuildVM3 200 100 194 169 Ok
C:\C...

Create an Aggregated Policy

Aggregated policies may be used if you want multiple virtual hard disks to share a single
pool of IOPS and bandwidth. For example, if you apply the same Aggregated policy to
hard disks from two virtual machines, the minimum will be split between them
according to demand. Both disks will be guaranteed a combined minimum, and
together they will not exceed the specified maximum IOPS or bandwidth.

The same approach could also be used to provide a single allocation to all VHD/VHDx
files for the virtual machines comprising a service or belonging to a tenant in a
multihosted environment.

There is no difference in the process to create Dedicated and Aggregated policies other
than the PolicyType that is specified.

The following example shows how to create an Aggregated Storage QoS Policy and get
its policyID on a Scale-Out File Server:

PowerShell

PS C:\> $highPerf = New-StorageQosPolicy -Name SqlWorkload -MinimumIops 1000

-MaximumIops 5000 -PolicyType Aggregated
[plang-fs]: PS C:\Users\plang\Documents> $highPerf.PolicyId

Guid
----
7e2f3e73-1ae4-4710-8219-0769a4aba072

The following example shows how to apply the Storage QoS Policy on Hyper-V server
using the policyID obtained in the preceding example:

PowerShell

PS C:\> Get-VM -Name WinOltp1 | Get-VMHardDiskDrive | Set-VMHardDiskDrive -

QoSPolicyID 7e2f3e73-1ae4-4710-8219-0769a4aba072

The following example shows how to viewing effects of the Storage QoS policy from file
server:
PowerShell

PS C:\> Get-StorageQosFlow -InitiatorName WinOltp1 | format-list

InitiatorName, PolicyId, MinimumIOPS, MaximumIOPS, StorageNodeIOPS, FilePath

InitiatorName : WinOltp1
PolicyId : 7e2f3e73-1ae4-4710-8219-0769a4aba072
MinimumIops : 250
MaximumIops : 1250
StorageNodeIOPS : 0
FilePath :
C:\ClusterStorage\Volume2\SHARES\TWO\BASEVHD\9914.0.AMD64FRE.WIN
MAIN.141218-1718_SERVER_SERVERDATACENTER_EN-US.VHDX

InitiatorName : WinOltp1
PolicyId : 7e2f3e73-1ae4-4710-8219-0769a4aba072
MinimumIops : 250
MaximumIops : 1250
StorageNodeIOPS : 0
FilePath : C:\ClusterStorage\Volume3\SHARES\THREE\WINOLTP1\BOOT.VHDX

InitiatorName : WinOltp1
PolicyId : 7e2f3e73-1ae4-4710-8219-0769a4aba072
MinimumIops : 1000
MaximumIops : 5000
StorageNodeIOPS : 4550
FilePath :
C:\ClusterStorage\Volume3\SHARES\THREE\WINOLTP1\IOMETER.VHDX
PS C:\> Get-StorageQosFlow -InitiatorName WinOltp1 | for
mat-list InitiatorName, PolicyId, MinimumIOPS, MaximumIOPS, StorageNodeIOPS,
FilePath

InitiatorName : WinOltp1
PolicyId : 7e2f3e73-1ae4-4710-8219-0769a4aba072
MinimumIops : 250
MaximumIops : 1250
StorageNodeIOPS : 0
FilePath :
C:\ClusterStorage\Volume2\SHARES\TWO\BASEVHD\9914.0.AMD64FRE.WIN
MAIN.141218-1718_SERVER_SERVERDATACENTER_EN-US.VHDX

InitiatorName : WinOltp1
PolicyId : 7e2f3e73-1ae4-4710-8219-0769a4aba072
MinimumIops : 250
MaximumIops : 1250
StorageNodeIOPS : 0
FilePath : C:\ClusterStorage\Volume3\SHARES\THREE\WINOLTP1\BOOT.VHDX

InitiatorName : WinOltp1
PolicyId : 7e2f3e73-1ae4-4710-8219-0769a4aba072
MinimumIops : 1000
MaximumIops : 5000
StorageNodeIOPS : 4550
 FilePath :
C:\ClusterStorage\Volume3\SHARES\THREE\WINOLTP1\IOMETER.VHDX

Each virtual hard disk will have the MinimumIOPS and MaximumIOPS and
MaximumIobandwidth value adjusted based on its load. This ensures that the total
amount of bandwidth used for the group of disks stays within the range defined by
policy. In the example above, the first two disks are idle, and the third one is allowed to
use up to the maximum IOPS. If the first two disks start issuing IO again, then the
maximum IOPS of the third disk will be lowered automatically.

Modify an existing policy

The properties of Name, MinimumIOPS, MaximumIOPS, and MaximumIoBandwidthcan
be changed after a policy is created. However, the Policy Type (Aggregated/Dedicated)
cannot be changed once the policy is created.

The following Windows PowerShell cmdlet shows how to change the MaximumIOPS
property for an existing policy:

PowerShell

[DBG]: PS C:\demoscripts>> Get-StorageQosPolicy -Name SqlWorkload | Set-

StorageQosPolicy -MaximumIops 6000

The following cmdlet verifies the change:

PowerShell

PS C:\> Get-StorageQosPolicy -Name SqlWorkload

Name MinimumIops MaximumIops Status

---- ----------- ----------- ------
SqlWorkload 1000 6000 Ok

[plang-fs1]: PS C:\Users\plang\Documents> Get-StorageQosPolicy -Name

SqlWorkload | Get-Storag
eQosFlow | Format-Table InitiatorName, PolicyId, MaximumIops, MinimumIops,
StorageNodeIops -A
utoSize

InitiatorName PolicyId MaximumIops MinimumIops

StorageNodeIops
------------- -------- ----------- ----------- -
--------------
WinOltp1 7e2f3e73-1ae4-4710-8219-0769a4aba072 1500 250
0
WinOltp1 7e2f3e73-1ae4-4710-8219-0769a4aba072 1500 250
 0
WinOltp1 7e2f3e73-1ae4-4710-8219-0769a4aba072 6000 1000
4507

How to identify and address common issues

This section describes how to find virtual machines with invalid Storage QoS policies,
how to recreate a matching policy, how to remove a policy from a virtual machine, and
how to identify virtual machines that do not meet the Storage QoS policy requirements.

Identify virtual machines with invalid policies

If a policy is deleted from the file server before it's removed from a virtual machine, the
virtual machine will keep running as if no policy were applied.

PowerShell

PS C:\> Get-StorageQosPolicy -Name SqlWorkload | Remove-StorageQosPolicy

Confirm
Are you sure you want to perform this action?
Performing the operation "DeletePolicy" on target "MSFT_StorageQoSPolicy
(PolicyId =
"7e2f3e73-1ae4-4710-8219-0769a4aba072")".
[Y] Yes [A] Yes to All [N] No [L] No to All [?] Help (default is "Y"):

The status for the flows will now show "UnknownPolicyId"

PowerShell

PS C:\> Get-StorageQoSflow | Sort-Object InitiatorName | ft InitiatorName,

Status, MinimumIOPS, MaximumIOPS, StorageNodeIOPS, Status, @{Expression=
{$_.FilePath.Substring($_.FilePath.LastIndexOf('\')+1)};Label="File"} -
AutoSize

InitiatorName Status MinimumIops MaximumIops StorageNodeIOPS

Status File
------------- ------ ----------- ----------- ---------------
------ ----
Ok 0 0 0
Ok Def...
Ok 0 0 10
Ok Def...
Ok 0 0 13
Ok Def...
Ok 0 0 0
Ok Def...
 Ok 0 0 0
Ok Def...
Ok 0 0 0
Ok Def...
Ok 0 0 0
Ok Def...
Ok 0 0 0
Ok Def...
Ok 0 0 0
Ok Def...
BuildVM1 Ok 100 200 193
Ok BUI...
BuildVM2 Ok 100 200 196
Ok BUI...
BuildVM3 Ok 50 64 17
Ok WIN...
BuildVM3 Ok 50 136 179
Ok BUI...
BuildVM4 Ok 50 100 23
Ok WIN...
BuildVM4 Ok 100 200 173
Ok BUI...
TR20-VMM Ok 33 666 2
Ok DAT...
TR20-VMM Ok 25 975 3
Ok DAT...
TR20-VMM Ok 75 1025 12
Ok BOO...
WinOltp1 UnknownPolicyId 0 0 0
UnknownPolicyId 991...
WinOltp1 UnknownPolicyId 0 0 4926
UnknownPolicyId IOM...
WinOltp1 UnknownPolicyId 0 0 0
UnknownPolicyId BOO...

Recreate a matching Storage QoS policy

If a policy was unintentionally removed, you can create a new one using the old PolicyId.
First, get the needed PolicyId

PowerShell

PS C:\> Get-StorageQosFlow -Status UnknownPolicyId | ft InitiatorName,

PolicyId -AutoSize

InitiatorName PolicyId
------------- --------
WinOltp1 7e2f3e73-1ae4-4710-8219-0769a4aba072
WinOltp1 7e2f3e73-1ae4-4710-8219-0769a4aba072
WinOltp1 7e2f3e73-1ae4-4710-8219-0769a4aba072
Next, create a new policy using that PolicyId

PowerShell

PS C:\> New-StorageQosPolicy -PolicyId 7e2f3e73-1ae4-4710-8219-0769a4aba072

-PolicyType Aggregated -Name RestoredPolicy -MinimumIops 100 -MaximumIops
2000

Name MinimumIops MaximumIops Status

---- ----------- ----------- ------
RestoredPolicy 100 2000 Ok

Finally, verify that it was applied.

PowerShell

PS C:\> Get-StorageQoSflow | Sort-Object InitiatorName | ft InitiatorName,

Status, MinimumIOPS, MaximumIOPS, StorageNodeIOPS, Status, @{Expression=
{$_.FilePath.Substring($_.FilePath.LastIndexOf('\')+1)};Label="File"} -
AutoSize

InitiatorName Status MinimumIops MaximumIops StorageNodeIOPS Status File

------------- ------ ----------- ----------- --------------- ------ ----
Ok 0 0 0 Ok
DefaultFlow
Ok 0 0 8 Ok
DefaultFlow
Ok 0 0 9 Ok
DefaultFlow
Ok 0 0 0 Ok
DefaultFlow
Ok 0 0 0 Ok
DefaultFlow
Ok 0 0 0 Ok
DefaultFlow
Ok 0 0 0 Ok
DefaultFlow
Ok 0 0 0 Ok
DefaultFlow
Ok 0 0 0 Ok
DefaultFlow
BuildVM1 Ok 100 200 192 Ok
BUILDVM1.VHDX
BuildVM2 Ok 100 200 193 Ok
BUILDVM2.VHDX
BuildVM3 Ok 50 100 24 Ok
WIN8RTM_ENTERPRISE_VL...
BuildVM3 Ok 100 200 166 Ok
BUILDVM3.VHDX
BuildVM4 Ok 50 100 12 Ok
WIN8RTM_ENTERPRISE_VL...
BuildVM4 Ok 100 200 178 Ok
 BUILDVM4.VHDX
TR20-VMM Ok 33 666 2 Ok
DATA2.VHDX
TR20-VMM Ok 33 666 2 Ok
DATA1.VHDX
TR20-VMM Ok 33 666 10 Ok
BOOT.VHDX
WinOltp1 Ok 25 500 0 Ok
9914.0.AMD64FRE.WINMA...

Remove Storage QoS Policies

If the policy was removed intentionally, or if a VM was imported with a policy that you
don't need, it may be removed.

PowerShell

PS C:\> Get-VM -Name WinOltp1 | Get-VMHardDiskDrive | Set-VMHardDiskDrive -

QoSPolicyID $null

Once the PolicyId is removed from the virtual hard disk settings, the status will be "Ok"
and no minimum or maximum will be applied.

PowerShell

PS C:\> Get-StorageQoSflow | Sort-Object InitiatorName | ft InitiatorName,

MinimumIOPS, MaximumIOPS, StorageNodeIOPS, Status, @{Expression=
{$_.FilePath.Substring($_.FilePath.LastIndexOf('\')+1)};Label="File"} -
AutoSize

InitiatorName MinimumIops MaximumIops StorageNodeIOPS Status File

------------- ----------- ----------- --------------- ------ ----
0 0 0 Ok DefaultFlow
0 0 16 Ok DefaultFlow
0 0 12 Ok DefaultFlow
0 0 0 Ok DefaultFlow
0 0 0 Ok DefaultFlow
0 0 0 Ok DefaultFlow
0 0 0 Ok DefaultFlow
0 0 0 Ok DefaultFlow
0 0 0 Ok DefaultFlow
BuildVM1 100 200 197 Ok BUILDVM1.VHDX
BuildVM2 100 200 192 Ok BUILDVM2.VHDX
BuildVM3 9 9 23 Ok
WIN8RTM_ENTERPRISE_VL_BUILDW...
BuildVM3 91 191 171 Ok BUILDVM3.VHDX
BuildVM4 8 8 18 Ok
WIN8RTM_ENTERPRISE_VL_BUILDW...
BuildVM4 92 192 163 Ok BUILDVM4.VHDX
TR20-VMM 33 666 2 Ok DATA2.VHDX
 TR20-VMM 33 666 1 Ok DATA1.VHDX
TR20-VMM 33 666 5 Ok BOOT.VHDX
WinOltp1 0 0 0 Ok
9914.0.AMD64FRE.WINMAIN.1412...
WinOltp1 0 0 1811 Ok IOMETER.VHDX
WinOltp1 0 0 0 Ok BOOT.VHDX

Find virtual machines that are not meeting Storage QoS

Policies
The InsufficientThroughput status is assigned to any flows that:

Have a minimum defined IOPS set by policy; and

Are initiating IO at a rate meeting or exceeding the minimum; and

Are not achieving minimum IOP rate

PowerShell

PS C:\> Get-StorageQoSflow | Sort-Object InitiatorName | ft InitiatorName,

MinimumIOPS, MaximumIOPS, StorageNodeIOPS, Status, @{Expression=
{$_.FilePath.Substring($_.FilePath.LastIndexOf('\')+1)};Label="File"} -
AutoSize

InitiatorName MinimumIops MaximumIops StorageNodeIOPS Status

File
------------- ----------- ----------- --------------- ------
----
0 0 0 Ok
DefaultFlow
0 0 0 Ok
DefaultFlow
0 0 15 Ok
DefaultFlow
0 0 0 Ok
DefaultFlow
0 0 0 Ok
DefaultFlow
0 0 0 Ok
DefaultFlow
0 0 0 Ok
DefaultFlow
0 0 0 Ok
DefaultFlow
0 0 0 Ok
DefaultFlow
BuildVM3 50 100 20 Ok
WIN8RTM_ENTE...
BuildVM3 100 200 174 Ok
BUILDVM3.VHDX
 BuildVM4 50 100 11 Ok
WIN8RTM_ENTE...
BuildVM4 100 200 188 Ok
BUILDVM4.VHDX
TR20-VMM 33 666 3 Ok
DATA1.VHDX
TR20-VMM 78 1032 180 Ok
BOOT.VHDX
TR20-VMM 22 968 4 Ok
DATA2.VHDX
WinOltp1 3750 5000 0 Ok
9914.0.AMD64...
WinOltp1 15000 20000 11679 InsufficientThroughput
IOMETER.VHDX
WinOltp1 3750 5000 0 Ok
BOOT.VHDX

You can determine flows for any status, including InsufficientThroughput as shown in
the following example:

PowerShell

PS C:\> Get-StorageQosFlow -Status InsufficientThroughput | fl

FilePath :
C:\ClusterStorage\Volume3\SHARES\THREE\WINOLTP1\IOMETER.VHDX
FlowId : 1ca356ff-fd33-5b5d-b60a-2c8659dc803e
InitiatorId : 2ceabcef-2eba-4f1b-9e66-10f960b50bbf
InitiatorIOPS : 12168
InitiatorLatency : 22.983
InitiatorName : WinOltp1
InitiatorNodeName : plang-c1.plang.nttest.microsoft.com
Interval : 300000
Limit : 20000
PolicyId : 5d1bf221-c8f0-4368-abcf-aa139e8a7c72
Reservation : 15000
Status : InsufficientThroughput
StorageNodeIOPS : 12181
StorageNodeLatency : 22.0514
StorageNodeName : plang-fs2.plang.nttest.microsoft.com
TimeStamp : 2/13/2015 12:07:30 PM
VolumeId : 0d2fd367-8d74-4146-9934-306726913dda
PSComputerName :
MaximumIops : 20000
MinimumIops : 15000

Monitor Health using Storage QoS

The new Health Service simplifies the monitoring of the Storage Cluster, providing a
single place to check for any actionable events in any of the nodes. This section
describes how monitor the health of your storage cluster using the debug-
storagesubsystem cmdlet.

View Storage Status with Debug-StorageSubSystem

Clustered Storage Spaces also provide information on the health of the storage cluster
in a single location. This can help administrators quickly identify current problems in
storage deployments and monitor as issues arrive or are dismissed.

VM with invalid policy

VMs with invalid policies are also reported through the storage subsystem health
monitoring. Here is an example from the same state as described in Finding VMs with
invalid policies section of this document.

PowerShell

C:\> Get-StorageSubSystem -FriendlyName Clustered* | Debug-StorageSubSystem

EventTime :
FaultId : 0d16d034-9f15-4920-a305-f9852abf47c3
FaultingObject :
FaultingObjectDescription : Storage QoS Policy 5d1bf221-c8f0-4368-abcf-
aa139e8a7c72
FaultingObjectLocation :
FaultType : Storage QoS policy used by consumer does not
exist.
PerceivedSeverity : Minor
Reason : One or more storage consumers (usually Virtual
Machines) are
using a non-existent policy with id
5d1bf221-c8f0-4368-abcf-aa139e8a7c72. Consumer
details:

Flow ID: 1ca356ff-fd33-5b5d-b60a-2c8659dc803e

Initiator ID: 2ceabcef-2eba-4f1b-9e66-
10f960b50bbf
Initiator Name: WinOltp1
Initiator Node: plang-
c1.plang.nttest.microsoft.com
File Path:

C:\ClusterStorage\Volume3\SHARES\THREE\WINOLTP1\IOMETER.VHDX
RecommendedActions : {Reconfigure the storage consumers (usually
Virtual Machines)
to use a valid policy., Recreate any missing
Storage QoS
policies.}
PSComputerName :
Lost redundancy for a storage spaces virtual disk
In this example, a Clustered Storage Space has a virtual disk created as a three-way
mirror. A failed disk was removed from the system, but a replacement disk was not
added. The storage subsystem is reporting a loss of redundancy with HealthStatus
Warning, but OperationalStatus "OK because the volume is still online.

PowerShell

PS C:\> Get-StorageSubSystem -FriendlyName Clustered*

FriendlyName HealthStatus
OperationalStatus
------------ ------------ --------------
---
Clustered Windows Storage o... Warning OK

[plang-fs1]: PS C:\Users\plang\Documents> Get-StorageSubSystem -FriendlyName

Clustered* | Deb
ug-StorageSubSystem

EventTime :
FaultId : dfb4b672-22a6-4229-b2ed-c29d7485bede
FaultingObject :
FaultingObjectDescription : Virtual disk 'Two'
FaultingObjectLocation :
FaultType : VirtualDiskDegradedFaultType
PerceivedSeverity : Minor
Reason : Virtual disk 'Two' does not have enough
redundancy remaining to
successfully repair or regenerate its data.
RecommendedActions : {Rebalance the pool, replace failed physical
disks, or add new
physical disks to the storage pool, then repair
the virtual
disk.}
PSComputerName :

Sample script for continuous monitoring of Storage QoS

This section includes a sample script showing how common failures can be monitored
using WMI script. It's designed as a starting part for developers to retrieve health events
in real time.

Example script:

PowerShell
 param($cimSession)
# Register and display events
Register-CimIndicationEvent -Namespace root\microsoft\windows\storage -
ClassName msft_storagefaultevent -CimSession $cimSession

while ($true)
{
$e = (Wait-Event)
$e.SourceEventArgs.NewEvent
Remove-Event $e.SourceIdentifier
}

Frequently Asked Questions

How do I retain a Storage QoS policy being enforced for

my virtual machine if I move its VHD/VHDx files to
another storage cluster
The setting on the VHD/VHDx file that specifies the policy is the GUID of a policy ID.
When a policy is created, the GUID can be specified using the PolicyID parameter. If that
parameter is not specified, a random GUID is created. Therefore, you can get the
PolicyID on the storage cluster where the VMs currently store their VHD/VHDx files and
create an identical policy on the destination storage cluster and then specify that it be
created with the same GUID. When the VMs files are moved to the new storage clusters,
the policy with the same GUID will be in effect.

System Center Virtual Machine Manager can be used to apply policies across multiple
storage clusters, which makes this scenario much easier.

If I change the Storage QoS Policy, why don't I see it take

effect immediately when I run Get-StorageQoSFlow
If you have a flow that is hitting a maximum of a policy and you change the policy to
either make it higher or lower, and then you immediately determine the
latency/IOPS/BandWidth of the flows using the PowerShell cmdlets, it will take up to 5
minutes to see the full effects of the policy change on the flows. The new limits will be in
effect within a few seconds, but the Get-StorgeQoSFlow PowerShell cmdlet uses an
average of each counter using a 5 minute sliding window. Otherwise, if it was showing a
current value and you ran the PowerShell cmdlet multiple times in a row, you may see
vastly different values because values for IOPS and latencies can fluctuate significantly
from one second to another.
What new functionality was added in Windows Server
2016
In Windows Server 2016 the Storage QoS Policy type names were renamed. The Multi-
instance policy type is renamed as Dedicated and Single-instance was renamed as
Aggregated. The management behavior of Dedicated policies is also modified -
VHD/VHDX files within the same virtual machine that have the same Dedicated policy
applied to them will not share I/O allocations.

There are two new Storage QoS features Windows Server 2016:

Maximum Bandwidth

Storage QoS in Windows Server 2016 introduces the ability to specify the
maximum bandwidth that the flows assigned to the policy may consume. The
parameter when specifying it in the StorageQosPolicy cmdlets is
MaximumIOBandwidth and the output is expressed in bytes per second. If both
MaximimIops and MaximumIOBandwidth are set in a policy, they will both be in
effect and the first one to be reached by the flow(s) will limit the I/O of the flows.

IOPS normalization is configurable

Storage QoSin uses normalization of IOPS. The default is to use a normalization

size of 8K. Storage QoS in Windows Server 2016 introduces the ability to specify a
different normalization size for the storage cluster. This normalization size effects
all flows on the storage cluster and takes effect immediately (within a few seconds)
once it is changed. The minimum is 1KB and the maximum is 4GB (recommend not
setting more than 4MB since it's unusual to have more than 4MB IOs).

Something to consider is that the same IO pattern/throughput shows up with

different IOPS numbers in the Storage QoS output when you change the IOPS
normalization due to the change in normalization calculation. If you are comparing
IOPS between storage clusters, you may also want to verify what normalization
value each is using since that will effect the normalized IOPS reported.

Example 1: Creating a new policy and viewing the maximum

bandwidth on the storage cluster

In PowerShell, you can specify the units that a number is expressed in. In the following
example, 10MB is used as the maximum bandwidth value. Storage QoS will convert this
and save it as bytes per second Hence, 10MB is converted into 10485760 bytes per
second.
 PowerShell

PS C:\Windows\system32> New-StorageQosPolicy -Name HR_VMs -MaximumIops 1000

-MinimumIops 20 -MaximumIOBandwidth 10MB

Name MinimumIops MaximumIops MaximumIOBandwidth Status

---- ----------- ----------- ------------------ ------
HR_VMs 20 1000 10485760 Ok

PS C:\Windows\system32> Get-StorageQosPolicy

Name MinimumIops MaximumIops MaximumIOBandwidth Status

---- ----------- ----------- ------------------ ------
Default 0 0 0 Ok
HR_VMs 20 1000 10485760 Ok

PS C:\Windows\system32> Get-StorageQoSFlow | fL
InitiatorName,FilePath,InitiatorIOPS,InitiatorLatency,InitiatorBandwidth

InitiatorName : testsQoS
FilePath : C:\ClusterStorage\Volume2\TESTSQOS\VIRTUAL HARD
DISKS\TESTSQOS.VHDX
InitiatorIOPS : 5
InitiatorLatency : 1.5455
InitiatorBandwidth : 37888

Example 2: Get IOPS normalization settings and specify a new value

The following example demonstrates how to get the storage clusters IOPS normalization
settings (default of 8KB), then set it to 32KB, and then show it again. Note, in this
example, specify "32KB", since PowerShell allows specifying the unit instead of requiring
the conversion to bytes. The output does show the value in bytes per second. This
setting affects all virtual machines. (The virtual machines created on local volumes are
also affected.)

PowerShell

PS C:\Windows\system32> Get-StorageQosPolicyStore

IOPSNormalizationSize
---------------------
8192

PS C:\Windows\system32> Set-StorageQosPolicyStore -IOPSNormalizationSize

32KB
PS C:\Windows\system32> Get-StorageQosPolicyStore

IOPSNormalizationSize
---------------------
32768
See Also
Windows Server 2016
Storage Replica in Windows Server 2016
Storage Spaces Direct in Windows Server 2016
Change history for storage topics in
Windows Server
Article • 06/30/2022

Applies to: Windows Server 2022, Windows Server 2019, Windows Server 2016

This topic lists new and updated topics in the Storage documentation for Windows
Server.

If you're looking for update history for Windows Server, see Windows 10 and
Windows Server 2019 update history or Windows Server 2016 update history .

January 2020
New or changed topic Description

Understand and deploy persistent memory Added known hardware issue.

December 2019
New or changed topic Description

Troubleshooting Disk Edited to further refine the guidance, based on customer

Management requests.

Extend a volume in Disk Added guidance in response to customer feedback.

Management

Change a dynamic disk back to Fixed an error in the command line and added some info based
a basic disk on customer feedback.

August 2019
New or changed topic Description

Storage Migration Service FAQ Updated to reflect new support for Linux sources.

June 2019
New or changed topic Description

Disk cleanup New (Migrated from the Previous Versions)

Storage Migration Service FAQ Added performance optimization info.

May 2019
New or changed Description
topic

Delete volumes New

Create volumes Added steps and videos for creating a volume in Windows Admin
Center.

Extend volumes Added steps and video for resizing a volume in Windows Admin Center.

March 2019
New or changed topic Description

Monitor with Azure Monitor New

Understand and deploy persistent memory New

Upgrade a Storage Spaces Direct cluster to Windows New

Server 2019

DFS Replication Migrated from the Previous Versions

library

February 2019
New or changed topic Description

Storage Migration Service known issues Added an issue

January 2019
New or changed topic Description

Understand and monitor storage resync New topic

December 2018
New or changed topic Description

Use Storage Migration Service to migrate a server Added some clarification on how we
transfer files

Cluster to Cluster Storage Replica cross region in Added validation steps

Azure

Cluster to Cluster Storage Replica within the same Added validation steps
region in Azure

Storage Replica frequently asked questions Added support statement for Data
Deduplication

November 2018
New or changed topic Description

Nested resiliency New topic

Storage Migration Service known issues New topic

DFS Replication: Frequently Asked Questions (FAQ) Migrated from the Previous Versions library

Migrate SYSVOL replication to DFS Replication Migrated from the Previous Versions library

SMB: File and printer sharing ports should be open Migrated from the Previous Versions library

Volume Shadow Copy Service Migrated from the Previous Versions library

October 2018
New or changed topic Description

What's new in Storage Updated to cover what's new in Windows Server 2019

Storage Replica known issues Added info about a new update.

September 2018
New or changed topic Description

Storage Migration Service overview New topic

New or changed topic Description

Use Storage Migration Service to migrate a server New topic

Storage Migration Service frequently asked questions New topic

(FAQ)

iSCSI Target Server Migrated from the Previous Versions

library.

iSCSI Target Server scalability limits Migrated from the Previous Versions
library.

June 2018
New or changed topic Description

Server-to-server storage replication Added info on using Azure VMs, including ExpressRoute.

Cluster sets New topic

May 2018
New or changed topic Description

NFS overview Migrated from the Previous Versions library.

Deploy NFS Migrated from the Previous Versions library.

Deploy Storage Spaces on a stand- Migrated from the Previous Versions library.
alone server

NTFS Overview Migrated from the Previous Versions library.

Use Robocopy to Preseed Files for Migrated from the Previous Versions library.
DFS Replication

Vssadmin - Previous Versions Migrated from the Previous Versions library.

command line tool

File Server Resource Manager Added info about a new registry setting in Windows
overview Server 2016, version 1803.

Server-to-server storage replication Added info on using Windows Admin Center.

Storage Replica known issues Added new information.

April 2018
New or changed topic Description

Collect data in Storage Spaces Direct New topic.

Storage Spaces overview New topic.

Folder Redirection, Offline Files, and Roaming User Migrated multiple topics from the Previous
Profiles overview Versions library.

File sharing using the SMB 3 protocol Migrated from the Previous Versions library.

Improve performance of a file server with SMB Migrated from the Previous Versions library.
Direct

SMB security enhancements Migrated from the Previous Versions library.

March 2018
New or changed topic Description

Disaster recovery with Storage New topic.

Spaces Direct

Understanding Quorum in Storage New topic.

Spaces Direct

Deploying Storage Spaces Direct Heavily revised to include both converged and hyper-
converged scenarios.

Deploying Roaming User Profiles Moved from Previous Versions library and updated.

Storage Replica frequently asked Added Is CSV required to replicate in a stretch cluster or
questions between clusters?.

February 2018
New or changed topic Description

Storage Spaces health and operational states New topic.

Using Storage Spaces Direct with the CSV in-memory read cache New topic.

January 2018
New or changed topic Description

Drive symmetry considerations in Storage Spaces Direct New topic.

Using Storage Replica with Project Honolulu New topic.

December 2017
New or changed Description
topic

Change a drive New topic.

letter

Troubleshooting Rewrote the A disk's status is Not Initialized or the disk is missing entirely
Disk Management section to add extensive troubleshooting steps, based on customer
requests.

Initialize new disks Rewrote to attempt to make it easier to understand and address customer
questions.

Planning volumes in Added a table summarizing the resiliency types available on four-node and
Storage Spaces larger clusters.
Direct

ReFS overview Clarified recommended workloads for mirror-accelerated parity and

corrected the supported file and volume sizes for ReFS and NTFS.

Mirror-accelerated Clarified recommendation to place write-heavy files in separate directories.

parity

Storage Replica Added new information.

known issues

November 2017
New or Description
changed topic

What's new in Added info about what's new in Windows Server, version 1709.
storage

Add servers or Added information about how Storage Spaces Direct automatically optimizes
drives drive usage after adding drives.
October 2017
New or changed topic Description

Deploying Storage Spaces Direct in a virtual New topic.

machine guest cluster

Overview of Disk Management Published 13 new topics for Windows and

Windows Server.

Storage Replica overview Added what's new info for Windows Server,
version 1709.

Storage Replica known issues Added new information.

Cluster to cluster storage replication Revised the number of supported cluster nodes
for Storage Spaces Direct.

Storage Spaces Direct hardware Added a note about a specific line of NVMe
requirements devices.

July 2017
New or changed topic Description

DFS Namespaces Published 20 new topics for Windows Server 2016.

File Server Resource Manager Published 33 new topics for Windows Server 2016.

Understanding the cache in Storage Added a Storage Spaces Direct design

Spaces Direct considerations video.

Storage Replica frequently asked questions Added more best practices around log volumes.

June 2017
New or changed topic Description

Planning a Work Folders Added info about Azure AD Application Proxy support & updated
deployment requirements.

Work Folders Added info about Azure AD Application Proxy support & updated
requirements.

Deploying Storage Spaces Removed Nano Server from supported installation options.
Direct
New or changed topic Description

File Server Resource Manager New topic for Windows Server 2016.

May 2017
New or changed topic Description

Data Deduplication overview Updated the system requirements to include a newer software
and update.
Install Data Deduplication

Deploying Work Folders Added info about Azure AD Application Proxy support &
updated required steps.

Deploying Storage Spaces Added step 1.3 with required features and fixed an obsolete
Direct parameter in Enable-NetAdapterQos.

Storage Replica frequently Added info on how to choose between different replication
asked questions topologies.

Storage Spaces Direct Changed drive endurance requirements for cache devices.
hardware requirements

April 2017
New or changed topic Description

Troubleshooting drive firmware updates New topic.

Work Folders New topic.

Planning a Work Folders deployment New topic.

Deploying Work Folders New topic.

Deploying Work Folders with AD FS and New topic.

Web Application Proxy (WAP)

Deploying Storage Spaces Direct Removed a reference to an obsolete software update

and fixed a typo in the sample output.

Storage Replica known issues Added new information.

March 2017
New or changed topic Description

Taking a Storage Spaces Direct server offline for maintenance New topic.

February 2017
New or changed topic Description

Removing servers in Storage Spaces Direct New topic.

Adding server or drives to Storage Spaces Revamped with new images and updated
Direct content.

Storage Spaces Direct hardware requirements Updated with latest requirements.

January 2017
New or changed Description
topic

Planning volumes New topic.

Creating volumes New topic.

Extending volumes New topic.

in Storage Spaces
Direct

ReFS Overview New topic.

Understanding New list of links.

Storage Spaces
Direct

Planning Storage New list of links.

Spaces Direct

Deploying Storage New list of links.

Spaces Direct

Managing Storage New topic.

Spaces Direct

Storage Replica Updated port requirements and clarified how extending replicated
frequently asked volumes works.
questions
New or changed Description
topic

Storage Replica Added info about a fix in the December 9, 2016 Cumulative Update and
known issues added info about how to resolve an error when extending a replicated
volume.

Storage Spaces Added visually-oriented Understand/Plan/Deploy/Manage section to serve

Direct overview as a learning map for our topics.

Deploying Storage Removed some obsolete content and added new links.
Spaces Direct