Planning for an Azure File Sync deployment
Use Azure File Sync to centralize your organization's file shares in Azure Files, while keeping the flexibility, performance, and compatibility of an on-premises file server. Azure File Sync transforms Windows Server into a quick cache of your Azure file share. You can use any protocol that's available on Windows Server to access your data locally, including SMB, NFS, and FTPS. You can have as many caches as you need across the world.
This article describes important considerations for an Azure File Sync deployment. We recommend that you also read Planning for an Azure Files deployment.
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure PowerShell.
Azure File Sync terminology
Before getting into the details of planning for an Azure File Sync deployment, it's important to understand the terminology.
Storage Sync Service
The Storage Sync Service is the top-level Azure resource for Azure File Sync. The Storage Sync Service resource is a peer of the storage account resource, and can similarly be deployed to Azure resource groups. A distinct top-level resource from the storage account resource is required because the Storage Sync Service can create sync relationships with multiple storage accounts via multiple sync groups. A subscription can have multiple Storage Sync Service resources deployed.
A sync group defines the sync topology for a set of files. Endpoints within a sync group are kept in sync with each other. If for example, you have two distinct sets of files that you want to manage with Azure File Sync, you would create two sync groups and add different endpoints to each sync group. A Storage Sync Service can host as many sync groups as you need.
The registered server object represents a trust relationship between your server (or cluster) and the Storage Sync Service. You can register as many servers to a Storage Sync Service instance as you want. However, a server (or cluster) can be registered with only one Storage Sync Service at a time.
Azure File Sync agent
The Azure File Sync agent is a downloadable package that enables Windows Server to be synced with an Azure file share. The Azure File Sync agent has three main components:
- FileSyncSvc.exe: The background service that is responsible for monitoring changes on server endpoints, and for initiating sync sessions to Azure.
- StorageSync.sys: The Azure File Sync file system filter, which is responsible for tiering files to Azure Files (when cloud tiering is enabled).
- PowerShell management cmdlets: PowerShell cmdlets that you use to interact with the Microsoft.StorageSync Azure resource provider. You can find these at the following (default) locations:
- C:\Program Files\Azure\StorageSyncAgent\StorageSync.Management.PowerShell.Cmdlets.dll
- C:\Program Files\Azure\StorageSyncAgent\StorageSync.Management.ServerCmdlets.dll
A server endpoint represents a specific location on a registered server, such as a folder on a server volume. Multiple server endpoints can exist on the same volume if their namespaces do not overlap (for example,
F:\sync2). You can configure cloud tiering policies individually for each server endpoint.
You can create a server endpoint via a mountpoint. Note, mountpoints within the server endpoint are skipped.
You can create a server endpoint on the system volume but, there are two limitations if you do so:
- Cloud tiering cannot be enabled.
- Rapid namespace restore (where the system quickly brings down the entire namespace and then starts to recall content) is not performed.
Only non-removable volumes are supported. Drives mapped from a remote share are not supported for a server endpoint path. In addition, a server endpoint may be located on the Windows system volume though cloud tiering is not supported on the system volume.
If you add a server location that has an existing set of files as a server endpoint to a sync group, those files are merged with any other files that are already on other endpoints in the sync group.
A cloud endpoint is an Azure file share that is part of a sync group. The entire Azure file share syncs, and an Azure file share can be a member of only one cloud endpoint. Therefore, an Azure file share can be a member of only one sync group. If you add an Azure file share that has an existing set of files as a cloud endpoint to a sync group, the existing files are merged with any other files that are already on other endpoints in the sync group.
Azure File Sync supports making changes to the Azure file share directly. However, any changes made on the Azure file share first need to be discovered by an Azure File Sync change detection job. A change detection job is initiated for a cloud endpoint only once every 24 hours. In addition, changes made to an Azure file share over the REST protocol will not update the SMB last modified time and will not be seen as a change by sync. For more information, see Azure Files frequently asked questions.
Cloud tiering is an optional feature of Azure File Sync in which frequently accessed files are cached locally on the server while all other files are tiered to Azure Files based on policy settings. For more information, see Understanding Cloud Tiering.
Azure File Sync system requirements and interoperability
This section covers Azure File Sync agent system requirements and interoperability with Windows Server features and roles and third-party solutions.
Before deploying Azure File Sync, you should evaluate whether it is compatible with your system using the Azure File Sync evaluation cmdlet. This cmdlet checks for potential issues with your file system and dataset, such as unsupported characters or an unsupported operating system version. Its checks cover most but not all of the features mentioned below; we recommend you read through the rest of this section carefully to ensure your deployment goes smoothly.
The evaluation cmdlet can be installed by installing the Az PowerShell module, which can be installed by following the instructions here: Install and configure Azure PowerShell.
You can invoke the evaluation tool in a few different ways: you can perform the system checks, the dataset checks, or both. To perform both the system and dataset checks:
Invoke-AzStorageSyncCompatibilityCheck -Path <path>
To test only your dataset:
Invoke-AzStorageSyncCompatibilityCheck -Path <path> -SkipSystemChecks
To test system requirements only:
Invoke-AzStorageSyncCompatibilityCheck -ComputerName <computer name>
To display the results in CSV:
$errors = Invoke-AzStorageSyncCompatibilityCheck […] $errors | Select-Object -Property Type, Path, Level, Description | Export-Csv -Path <csv path>
A server running one of the following operating system versions:
Version Supported SKUs Supported deployment options Windows Server 2019 Datacenter and Standard Full and Core Windows Server 2016 Datacenter and Standard Full and Core Windows Server 2012 R2 Datacenter and Standard Full and Core Windows Server IoT 2019 for Storage Datacenter and Standard Full and Core Windows Storage Server 2016 Datacenter and Standard Full and Core Windows Storage Server 2012 R2 Datacenter and Standard Full and Core
Future versions of Windows Server will be added as they are released.
We recommend keeping all servers that you use with Azure File Sync up to date with the latest updates from Windows Update.
A server with a minimum of 2 GiB of memory.
If the server is running in a virtual machine with dynamic memory enabled, the VM should be configured with a minimum 2048 MiB of memory.
A locally attached volume formatted with the NTFS file system.
File system features
|Access control lists (ACLs)||Fully supported||Windows ACLs are preserved by Azure File Sync, and are enforced by Windows Server on server endpoints. Windows ACLs are not (yet) supported by Azure Files if files are accessed directly in the cloud.|
|Mount points||Partially supported||Mount points might be the root of a server endpoint, but they are skipped if they are contained in a server endpoint's namespace.|
|Junctions||Skipped||For example, Distributed File System DfrsrPrivate and DFSRoots folders.|
|NTFS compression||Fully supported|
|Sparse files||Fully supported||Sparse files sync (are not blocked), but they sync to the cloud as a full file. If the file contents change in the cloud (or on another server), the file is no longer sparse when the change is downloaded.|
|Alternate Data Streams (ADS)||Preserved, but not synced||For example, classification tags created by the File Classification Infrastructure are not synced. Existing classification tags on files on each of the server endpoints are left untouched.|
Only NTFS volumes are supported. ReFS, FAT, FAT32, and other file systems are not supported.
|pagefile.sys||File specific to system|
|Desktop.ini||File specific to system|
|thumbs.db||Temporary file for thumbnails|
|ehthumbs.db||Temporary file for media thumbnails|
|~$*.*||Office temporary file|
|*.laccdb||Access DB locking file|
|635D02A9D91C401B97884B82B3BCDAEA.*||Internal Sync file|
|\System Volume Information||Folder specific to volume|
|\SyncShareState||Folder for Sync|
Windows Server Failover Clustering is supported by Azure File Sync for the "File Server for general use" deployment option. Failover Clustering is not supported on "Scale-Out File Server for application data" (SOFS) or on Clustered Shared Volumes (CSVs).
The Azure File Sync agent must be installed on every node in a Failover Cluster for sync to work correctly.
Windows Server 2016 and Windows Server 2019
Data Deduplication is supported on volumes with cloud tiering enabled on Windows Server 2016 and Windows Server 2019. Enabling Data Deduplication on a volume with cloud tiering enabled lets you cache more files on-premises without provisioning more storage.
When Data Deduplication is enabled on a volume with cloud tiering enabled, Dedup optimized files within the server endpoint location will be tiered similar to a normal file based on the cloud tiering policy settings. Once the Dedup optimized files have been tiered, the Data Deduplication garbage collection job will run automatically to reclaim disk space by removing unnecessary chunks that are no longer referenced by other files on the volume.
Note the volume savings only apply to the server; your data in the Azure file share will not be deduped.
To support Data Deduplication on volumes with cloud tiering enabled on Windows Server 2019, Windows update KB4520062 must be installed and Azure File Sync agent version 188.8.131.52 or newer is required.
Windows Server 2012 R2
Azure File Sync does not support Data Deduplication and cloud tiering on the same volume on Windows Server 2012 R2. If Data Deduplication is enabled on a volume, cloud tiering must be disabled.
If Data Deduplication is installed prior to installing the Azure File Sync agent, a restart is required to support Data Deduplication and cloud tiering on the same volume.
If Data Deduplication is enabled on a volume after cloud tiering is enabled, the initial Deduplication optimization job will optimize files on the volume that are not already tiered and will have the following impact on cloud tiering:
- Free space policy will continue to tier files as per the free space on the volume by using the heatmap.
- Date policy will skip tiering of files that may have been otherwise eligible for tiering due to the Deduplication optimization job accessing the files.
For ongoing Deduplication optimization jobs, cloud tiering with date policy will get delayed by the Data Deduplication MinimumFileAgeDays setting, if the file is not already tiered.
- Example: If the MinimumFileAgeDays setting is seven days and cloud tiering date policy is 30 days, the date policy will tier files after 37 days.
- Note: Once a file is tiered by Azure File Sync, the Deduplication optimization job will skip the file.
If a server running Windows Server 2012 R2 with the Azure File Sync agent installed is upgraded to Windows Server 2016 or Windows Server 2019, the following steps must be performed to support Data Deduplication and cloud tiering on the same volume:
- Uninstall the Azure File Sync agent for Windows Server 2012 R2 and restart the server.
- Download the Azure File Sync agent for the new server operating system version (Windows Server 2016 or Windows Server 2019).
- Install the Azure File Sync agent and restart the server.
Note: The Azure File Sync configuration settings on the server are retained when the agent is uninstalled and reinstalled.
Distributed File System (DFS)
Azure File Sync supports interop with DFS Namespaces (DFS-N) and DFS Replication (DFS-R).
DFS Namespaces (DFS-N): Azure File Sync is fully supported on DFS-N servers. You can install the Azure File Sync agent on one or more DFS-N members to sync data between the server endpoints and the cloud endpoint. For more information, see DFS Namespaces overview.
DFS Replication (DFS-R): Since DFS-R and Azure File Sync are both replication solutions, in most cases, we recommend replacing DFS-R with Azure File Sync. There are however several scenarios where you would want to use DFS-R and Azure File Sync together:
- You are migrating from a DFS-R deployment to an Azure File Sync deployment. For more information, see Migrate a DFS Replication (DFS-R) deployment to Azure File Sync.
- Not every on-premises server that needs a copy of your file data can be connected directly to the internet.
- Branch servers consolidate data onto a single hub server, for which you would like to use Azure File Sync.
For Azure File Sync and DFS-R to work side by side:
- Azure File Sync cloud tiering must be disabled on volumes with DFS-R replicated folders.
- Server endpoints should not be configured on DFS-R read-only replication folders.
For more information, see DFS Replication overview.
Using sysprep on a server that has the Azure File Sync agent installed is not supported and can lead to unexpected results. Agent installation and server registration should occur after deploying the server image and completing sysprep mini-setup.
If cloud tiering is enabled on a server endpoint, files that are tiered are skipped and not indexed by Windows Search. Non-tiered files are indexed properly.
Because antivirus works by scanning files for known malicious code, an antivirus product might cause the recall of tiered files. In versions 4.0 and above of the Azure File Sync agent, tiered files have the secure Windows attribute FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS set. We recommend consulting with your software vendor to learn how to configure their solution to skip reading files with this attribute set (many do it automatically).
Microsoft's in-house antivirus solutions, Windows Defender and System Center Endpoint Protection (SCEP), both automatically skip reading files that have this attribute set. We have tested them and identified one minor issue: when you add a server to an existing sync group, files smaller than 800 bytes are recalled (downloaded) on the new server. These files will remain on the new server and will not be tiered since they do not meet the tiering size requirement (>64kb).
Antivirus vendors can check compatibility between their product and Azure File Sync using the Azure File Sync Antivirus Compatibility Test Suite, which is available for download on the Microsoft Download Center.
Like antivirus solutions, backup solutions might cause the recall of tiered files. We recommend using a cloud backup solution to back up the Azure file share instead of an on-premises backup product.
If you are using an on-premises backup solution, backups should be performed on a server in the sync group that has cloud tiering disabled. When performing a restore, use the volume-level or file-level restore options. Files restored using the file-level restore option will be synced to all endpoints in the sync group and existing files will be replaced with the version restored from backup. Volume-level restores will not replace newer file versions in the Azure file share or other server endpoints.
Bare-metal (BMR) restore can cause unexpected results and is not currently supported.
With Version 9 of the Azure File Sync agent, VSS snapshots (including Previous Versions tab) are now supported on volumes which have cloud tiering enabled. However, you must enable previous version compatibility through PowerShell. Learn how.
Support for encryption solutions depends on how they are implemented. Azure File Sync is known to work with:
- BitLocker encryption
- Azure Information Protection, Azure Rights Management Services (Azure RMS), and Active Directory RMS
Azure File Sync is known not to work with:
- NTFS Encrypted File System (EFS)
In general, Azure File Sync should support interoperability with encryption solutions that sit below the file system, such as BitLocker, and with solutions that are implemented in the file format, such as Azure Information Protection. No special interoperability has been made for solutions that sit above the file system (like NTFS EFS).
Other Hierarchical Storage Management (HSM) solutions
No other HSM solutions should be used with Azure File Sync.
Azure File Sync is available only in the following regions:
|Australia East||New South Wales|
|Brazil South||Sao Paulo State|
|Canada East||Quebec City|
|East Asia||Hong Kong SAR|
|Japan East||Tokyo, Saitama|
|North Central US||Illinois|
|South Africa North||Johannesburg|
|South Africa West*||Cape Town|
|South Central US||Texas|
|US Gov Arizona||Arizona|
|US Gov Texas||Texas|
|US Gov Virginia||Virginia|
|UAE Central*||Abu Dhabi|
|West Central US||Wyoming|
|West US 2||Washington|
Azure File Sync supports syncing only with an Azure file share that's in the same region as the Storage Sync Service.
For the regions marked with asterisks, you must contact Azure Support to request access to Azure Storage in those regions. The process is outlined in this document.
Azure disaster recovery
To protect against the loss of an Azure region, Azure File Sync integrates with the geo-redundant storage redundancy (GRS) option. GRS storage works by using asynchronous block replication between storage in the primary region, with which you normally interact, and storage in the paired secondary region. In the event of a disaster that causes an Azure region to go temporarily or permanently offline, Microsoft will failover storage to the paired region.
If you are using your Azure file share as a cloud endpoint in a GRS storage account, you shouldn't initiate storage account failover. Doing so will cause sync to stop working and may also cause unexpected data loss in the case of newly tiered files. In the case of loss of an Azure region, Microsoft will trigger the storage account failover in a way that is compatible with Azure File Sync.
To support the failover integration between geo-redundant storage and Azure File Sync, all Azure File Sync regions are paired with a secondary region that matches the secondary region used by storage. These pairs are as follows:
|Primary region||Paired region|
|Australia East||Australia Southeast|
|Australia Southeast||Australia East|
|Brazil South||South Central US|
|Canada Central||Canada East|
|Canada East||Canada Central|
|Central India||South India|
|Central US||East US 2|
|East Asia||Southeast Asia|
|East US||West US|
|East US 2||Central US|
|France Central||France South|
|France South||France Central|
|Japan East||Japan West|
|Japan West||Japan East|
|Korea Central||Korea South|
|Korea South||Korea Central|
|North Europe||West Europe|
|North Central US||South Central US|
|South Africa North||South Africa West|
|South Africa West||South Africa North|
|South Central US||North Central US|
|South India||Central India|
|Southeast Asia||East Asia|
|UK South||UK West|
|UK West||UK South|
|US Gov Arizona||US Gov Texas|
|US Gov Iowa||US Gov Virginia|
|US Gov Virginia||US Gov Texas|
|West Europe||North Europe|
|West Central US||West US 2|
|West US||East US|
|West US 2||West Central US|
Azure File Sync agent update policy
The Azure File Sync agent is updated on a regular basis to add new functionality and to address issues. We recommend you configure Microsoft Update to get updates for the Azure File Sync agent as they're available.
Major vs. minor agent versions
- Major agent versions often contain new features and have an increasing number as the first part of the version number. For example: *2.*.**
- Minor agent versions are also called "patches" and are released more frequently than major versions. They often contain bug fixes and smaller improvements but no new features. For example: **.3.**
There are four approved and tested ways to install the Azure File Sync agent updates.
- (Preferred) Configure Microsoft Update to automatically download and install agent updates.
We always recommend taking every Azure File Sync update to ensure you have access to the latest fixes for the server agent. Microsoft Update makes this process seamless, by automatically downloading and installing updates for you.
- Use AfsUpdater.exe to download and install agent updates.
The AfsUpdater.exe is located in the agent installation directory. Double-click the executable to download and install agent updates.
- Patch an existing Azure File Sync agent by using a Microsoft Update patch file, or a .msp executable. The latest Azure File Sync update package can be downloaded from the Microsoft Update Catalog.
Running a .msp executable will upgrade your Azure File Sync installation with the same method used automatically by Microsoft Update in the previous upgrade path. Applying a Microsoft Update patch will perform an in-place upgrade of an Azure File Sync installation.
- Download the newest Azure File Sync agent installer from the Microsoft Download Center.
To upgrade an existing Azure File Sync agent installation, uninstall the older version and then install the latest version from the downloaded installer. The server registration, sync groups, and any other settings are maintained by the Azure File Sync installer.
Automatic agent lifecycle management
With agent version 6, the file sync team has introduced an agent auto-upgrade feature. You can select either of two modes and specify a maintenance window in which the upgrade shall be attempted on the server. This feature is designed to help you with the agent lifecycle management by either providing a guardrail preventing your agent from expiration or allowing for a no-hassle, stay current setting.
- The default setting will attempt to prevent the agent from expiration. Within 21 days of the posted expiration date of an agent, the agent will attempt to self-upgrade. It will start an attempt to upgrade once a week within 21 days prior to expiration and in the selected maintenance window. This option does not eliminate the need for taking regular Microsoft Update patches.
- Optionally, you can select that the agent will automatically upgrade itself as soon as a new agent version becomes available (currently not applicable to clustered servers). This update will occur during the selected maintenance window and allow your server to benefit from new features and improvements as soon as they become generally available. This is the recommended, worry-free setting that will provide major agent versions as well as regular update patches to your server. Every agent released is at GA quality. If you select this option, Microsoft will flight the newest agent version to you. Clustered servers are excluded. Once flighting is complete, the agent will also become available on Microsoft Download Center aka.ms/AFS/agent.
Changing the auto-upgrade setting
The following instructions describe how to change the settings after you've completed the installer, if you need to make changes.
Open a PowerShell console and navigate to the directory where you installed the sync agent then import the server cmdlets. By default this would look something like this:
cd 'C:\Program Files\Azure\StorageSyncAgent' Import-Module -Name \StorageSync.Management.ServerCmdlets.dll
You can run
Get-StorageSyncAgentAutoUpdatePolicy to check the current policy setting and determine if you want to change it.
To change the current policy setting to the delayed update track, you can use:
Set-StorageSyncAgentAutoUpdatePolicy -PolicyMode UpdateBeforeExpiration
To change the current policy setting to the immediate update track, you can use:
Set-StorageSyncAgentAutoUpdatePolicy -PolicyMode InstallLatest
Agent lifecycle and change management guarantees
Azure File Sync is a cloud service, which continuously introduces new features and improvements. This means that a specific Azure File Sync agent version can only be supported for a limited time. To facilitate your deployment, the following rules guarantee you have enough time and notification to accommodate agent updates/upgrades in your change management process:
- Major agent versions are supported for at least six months from the date of initial release.
- We guarantee there is an overlap of at least three months between the support of major agent versions.
- Warnings are issued for registered servers using a soon-to-be expired agent at least three months prior to expiration. You can check if a registered server is using an older version of the agent under the registered servers section of a Storage Sync Service.
- The lifetime of a minor agent version is bound to the associated major version. For example, when agent version 3.0 is released, agent versions 2.* will all be set to expire together.
Installing an agent version with an expiration warning will display a warning but succeed. Attempting to install or connect with an expired agent version is not supported and will be blocked.
Recommended Azure File Sync machine configuration
Azure File Sync machine requirements are determined by the number of objects in the namespace and the churn on the dataset. A single server can be attached to multiple sync groups and the number of objects listed in the following table accounts for the full namespace that a server is attached to. For example, server endpoint A with 10 million objects + server endpoint B with 10 million objects = 20 million objects. For that example deployment, we would recommend 8CPU, 16GiB of memory for steady state, and (if possible) 48GiB of memory for the initial migration.
Namespace data is stored in memory for performance reasons. Because of that, bigger namespaces require more memory to maintain good performance, and more churn requires more CPU to process.
In the following table, we have provided both the size of the namespace as well as a conversion to capacity for typical general purpose file shares, where the average file size is 512KiB. If your file sizes are smaller, consider adding additional memory for the same amount of capacity. Base your memory configuration on the size of the namespace.
|Namespace size - files & directories (millions)||Typical capacity (TiB)||CPU Cores||Recommended memory (GiB)|
|3||1.4||2||8 (initial sync)/ 2 (typical churn)|
|5||2.4||2||16 (initial sync)/ 4 (typical churn)|
|10||4.8||4||32 (initial sync)/ 8 (typical churn)|
|30||14.3||8||48 (initial sync)/ 16 (typical churn)|
|50||23.8||16||64 (initial sync)/ 32 (typical churn)|
|100*||47.7||32||128 (initial sync)/ 32 (typical churn)|
*More than 100 million files & directories has not been tested. This is a soft limit.
Initial synchronization of a namespace is an intensive operation and we recommend allocating more memory until initial synchronization is complete. This isn't required but, may speed up initial sync.
Typical churn is 0.5% of the namespace changing per day. For higher levels of churn, consider adding more CPU.