StorSimple 1200 migration to Azure File Sync
StorSimple 1200 series is a virtual appliance that is run in an on-premises data center. It is possible to migrate the data from this appliance to an Azure File Sync environment. Azure File Sync is the default and strategic long-term Azure service that StorSimple appliances can be migrated to.
StorSimple 1200 series will reach its end-of-life in December 2022. It is important to begin planning your migration as soon as possible. This article provides the necessary background knowledge and migrations steps for a successful migration to Azure File Sync.
Azure File Sync
Microsoft is committed to assist customers in their migration. Email AzureFilesMigration@microsoft .com for a customized migration plan as well as assistance during the migration.
Azure File Sync is a Microsoft cloud service, based on two main components:
- File synchronization and cloud tiering.
- File shares as native storage in Azure, that can be accessed over multiple protocols like SMB and file REST. An Azure file share is comparable to a file share on a Windows Server, that you can natively mount as a network drive. It supports important file fidelity aspects like attributes, permissions, and timestamps. Unlike with StorSimple, no application/service is required to interpret the files and folders stored in the cloud. The ideal, and most flexible approach to store general purpose file server data as well as some application data, in the cloud.
This article focuses on the migration steps. If before migrating you'd like to learn more about Azure File Sync, we recommend the following articles:
The goal is to guarantee the integrity of the production data as well as guaranteeing availability. The latter requires keeping downtime to a minimum, so that it can fit into or only slightly exceed regular maintenance windows.
StorSimple 1200 migration path to Azure File Sync
A local Windows Server is required to run an Azure File Sync agent. The Windows Server can be at a minimum a 2012R2 server but ideally is a Windows Server 2019.
There are numerous, alternative migration paths and it would create too long of an article to document all of them and illustrate why they bear risk or disadvantages over the route we recommend as a best practice in this article.
The previous image depicts steps that correspond to sections in this article.
Step 1: Provision your on-premises Windows Server and storage
- Create a Windows Server 2019 - at a minimum 2012R2 - as a virtual machine or physical server. A Windows Server fail-over cluster is also supported.
- Provision or add Direct Attached Storage (DAS as compared to NAS, which is not supported). The size of the Windows Server storage must be equal to or larger than the size of the available capacity of your virtual StorSimple 1200 appliance.
Step 2: Configure your Windows Server storage
In this step, you map your StorSimple storage structure (volumes and shares) to your Windows Server storage structure. If you plan to make changes to your storage structure, meaning the number of volumes, the association of data folders to volumes, or the subfolder structure above or below your current SMB/NFS shares, then now is the time to take these changes into consideration. Changing your file and folder structure after Azure File Sync is configured, is cumbersome, and should be avoided. This article assumes you are mapping 1:1, so you must take your mapping changes into consideration when you follow the steps in this article.
- None of your production data should end up on the Windows Server system volume. Cloud tiering is not supported on system volumes. However, this feature is required for the migration as well as continuous operations as a StorSimple replacement.
- Provision the same number of volumes on your Windows Server as you have on your StorSimple 1200 virtual appliance.
- Configure any Windows Server roles, features, and settings you need. We recommend you opt into Windows Server updates to keep your OS safe and up to date. Similarly, we recommend opting into Microsoft Update to keep Microsoft applications up to date, including the Azure File Sync agent.
- Do not configure any folders or shares before reading the following steps.
Step 3: Deploy the first Azure File Sync cloud resource
In this step, you need your Azure subscription credentials.
The core resource to configure for Azure File Sync is called a Storage Sync Service. We recommend that you deploy only one for all servers that are syncing the same set of files now or in the future. Create multiple Storage Sync Services only if you have distinct sets of servers that must never exchange data (for example: sync the same Azure file share). Otherwise, a single Storage Sync Service is the best practice.
Choose an Azure region for your Storage Sync Service that's close to your location. All other cloud resources must be deployed in the same region. To simplify management, create a new resource group in your subscription that houses sync and storage resources.
For more information, see the section about deploying the Storage Sync Service in the article about deploying Azure File Sync. Follow only this part of the article. There will be links to other sections of the article in later steps.
Step 4: Match your local volume and folder structure to Azure File Sync and Azure file share resources
In this step, you're evaluating how many Azure file shares you need. A single Windows Server instance (or cluster) can sync up to 30 Azure file shares.
You may have more folders on your volumes that you currently share out locally as SMB shares to your users and apps. The easiest way is to envision an on-premises share that maps 1:1 to an Azure file share. If you have a small-enough number, below 30 for a single Windows Server instance, then we recommend a 1:1 mapping.
If you have more shares than 30, it's often unnecessary to map an on-premises share 1:1 to an Azure file share. Consider the following options.
If your HR department (for instance) has a total of 15 shares, you might consider storing all of the HR data in a single Azure file share. Storing multiple on-premises shares in one Azure file share doesn't prevent you from creating the usual 15 SMB shares on your local Windows Server instance. It only means that you organize the root folders of these 15 shares as subfolders under a common folder. You then sync this common folder to an Azure file share. That way, only a single Azure file share in the cloud is needed for this group of on-premises shares.
Azure File Sync supports syncing the root of a volume to an Azure file share. If you sync the root folder, then all subfolders and files will go to the same Azure file share.
Syncing the root of the volume isn't always the best answer. There are benefits in syncing multiple locations. For example, doing so helps keep the number of items lower per sync scope. Setting up Azure File Sync with a lower number of items is not just beneficial for file sync. A lower number of items also benefits scenarios like these:
- Cloud-side restore from an Azure file share snapshot can be taken as a backup.
- Disaster recovery of an on-premises server can speed up significantly.
- Changes made directly in an Azure file share (outside sync) can be detected and synced faster.
A structured approach to a deployment map
Before you deploy cloud storage in a later step, it's important to create a map between on-premises folders and Azure file shares. This mapping will then inform how many and which Azure File Sync sync group resources you'll provision. A sync group ties the Azure file share and the folder on your server together and establishes a sync connection.
To make the decision about how many Azure file shares you need, review the following limits and best practices. Doing so will help you optimize your map.
A server with the Azure File Sync agent installed can sync with up to 30 Azure file shares.
An Azure file share is deployed inside a storage account. That makes the storage account a scale target for performance numbers such as IOPS and throughput.
Two standard (not premium) Azure file shares can theoretically saturate the maximum performance that a storage account can deliver. If you plan to only attach Azure File Sync to these file shares, then grouping several Azure file shares into the same storage account won't create a problem. Review the Azure file share performance targets for deeper insight into the relevant metrics to consider.
If you plan on lifting an app to Azure that will use the Azure file share natively, then you might need more performance from your Azure file share. If this is a possibility, even in the future, then mapping an Azure file share to its own storage account is best.
There's a limit of 250 storage accounts per subscription in a single Azure region.
With this information in mind, it often becomes necessary to group multiple top-level folders on your volumes into a common, new root directory. You then sync this new root directory, and all the folders you grouped into it, to a single Azure file share. This technique allows you to stay within the limit of 30 Azure file share syncs per server.
This grouping under a common root has no impact on access to your data. Your ACLs stay as is. You would only need to adjust any share paths (like SMB or NFS shares) you might have on the server folders that you now changed into a common root. Nothing else changes.
Another important aspect of Azure File Sync and a balanced performance and experience is understanding the scale factors for Azure File Sync performance. Obviously, when files are synced over the internet, larger files take more time and bandwidth to sync.
The most important scale vector for Azure File Sync is the number of items (files and folders) that need to be synchronized.
Azure File Sync supports syncing up to 100 Million items to a single Azure file share. This limit can be exceeded and only shows what the Azure File Sync team tests on a regular basis.
It's a best practice to keep the number of items per sync scope low. That's an important factor to consider in your mapping of folders to Azure file shares.
In your situation, it's possible that a set of folders can logically sync to the same Azure file share (using the new, common root folder approach mentioned earlier). But it might still be better to regroup folders such that they sync to two instead of one Azure file share. You can use this approach to keep the number of files and folders per file share balanced across the server.
Create a mapping table
Use a combination of the previous concepts to help determine how many Azure file shares you need, and which parts of your existing data will end up in which Azure file share.
Create a table that records your thoughts, so you can refer to it in the next step. Staying organized is important, because it can be easy to lose details of your mapping plan when you're provisioning many Azure resources at once. To help you in creating a complete mapping, you can download a Microsoft Excel file as a template.
|Download a namespace-mapping template.|
Step 5: Provision Azure file shares
An Azure file share is stored in the cloud in an Azure storage account. There's another level of performance considerations here.
If you have highly active shares (shares used by many users and/or applications), then two Azure file shares might reach the performance limit of a storage account.
A best practice is to deploy storage accounts with one file share each. You can pool multiple Azure file shares into the same storage account, in case you have archival shares or you expect low day-to-day activity in them.
These considerations apply more to direct cloud access (through an Azure VM) than to Azure File Sync. If you plan to use Azure File Sync only on these shares, then grouping several into a single Azure storage account is fine.
If you've made a list of your shares, you should map each share to the storage account that it will reside in.
In the previous phase, you determined the appropriate number of shares. In this step, you have a created a mapping of storage accounts to file shares. Now deploy the appropriate number of Azure storage accounts with the appropriate number of Azure file shares in them.
Make sure the region of each of your storage accounts is the same and matches the region of the Storage Sync Service resource you've already deployed.
If you create an Azure file share that has a 100-TiB limit, that share can only use locally redundant storage or zone-redundant storage redundancy options. Consider your storage redundancy needs before using 100-TiB file shares.
Azure file shares are still created with a 5-TiB limit by default. Because you're creating new storage accounts, make sure to follow the guidance to create storage accounts that allow Azure file shares with 100-TiB limits.
Another consideration when you're deploying a storage account is the redundancy of Azure Storage. See Azure Storage redundancy options.
The names of your resources are also important. For example, if you group multiple shares for the HR department into an Azure storage account, you should name the storage account appropriately. Similarly, when naming your Azure file shares, you should use names similar to the ones used for their on-premises counterparts.
Step 6: Configure Windows Server target folders
In previous steps, you have considered all aspects that will determine the components of your sync topologies. It is now time, to prepare the server to receive files for upload.
Create all folders, that will sync each to its own Azure file share. It's important that you follow the folder structure you've documented earlier. If for instance, you have decided to sync multiple, local SMB shares together into a single Azure file share, then you need to place them under a common root folder on the volume. Create this target root folder on the volume now.
The number of Azure file shares you have provisioned should match the number of folders you've created in this step + the number of volumes you will sync at the root level.
Step 7: Deploy the Azure File Sync agent
In this section, you install the Azure File Sync agent on your Windows Server instance.
The deployment guide illustrates that you need to turn off Internet Explorer Enhanced Security Configuration. This security measure is not applicable with Azure File Sync. Turning it off allows you to authenticate to Azure without any issues.
Open PowerShell and install the required PowerShell modules by using the following commands. Make sure to install the full module and the NuGet provider when you're prompted.
Install-Module -Name Az -AllowClobber Install-Module -Name Az.StorageSync
If you have any issues reaching the internet from your server, now is the time to solve them. Azure File Sync uses any available network connection to the internet. Requiring a proxy server to reach the internet is also supported. You can either configure a machine-wide proxy now, or specify a proxy that just Azure File Sync will use, during agent installation.
If configuring a proxy means you need to open your firewalls for this server, that might be an acceptable approach to you. At the end of the server installation, after you've completed server registration, a network connectivity report will show you the exact endpoint URLs in Azure that Azure File Sync needs to communicate with for the region you've selected. The report also tells you the reason why communication is needed. You can use the report to lock down the firewalls around this server to specific URLs.
You can also follow a more conservative approach in which you don't open the firewalls wide, but instead limit the server to communicate with higher-level DNS namespaces. For more information, see Azure File Sync proxy and firewall settings. Follow your own networking best practices.
At the end of the server installation wizard, a server registration wizard will open. Register the server to your Storage Sync Service's Azure resource from earlier.
These steps are described in more detail in the deployment guide, including the PowerShell modules that you should install first: Azure File Sync agent installation.
Use the latest agent. You can download it from the Microsoft Download Center: Azure File Sync Agent.
After a successful installation and server registration, you can check that you have successfully completed this step. Go to the Storage Sync Service resource in the Azure portal, and then follow the left menu to Registered servers. You'll see your server listed there.
Step 8: Configure sync
This step ties together all resources and folders you've set up on your Windows Server instance during the previous steps.
- Sign in to the Azure portal.
- Locate your Storage Sync Service resource.
- Create a new sync group within the Storage Sync Service resource for each Azure file share. In Azure File Sync terminology, the Azure file share will become a cloud endpoint in the sync topology that you're describing with the creation of a sync group. As you're creating the sync group, give it a familiar name so that you recognize which set of files syncs here. Make sure you reference the Azure file share with a matching name.
- After the sync group is created, a row for it will appear in the list of sync groups. Select the name (a link) to display the contents of the sync group. You'll see your Azure file share under Cloud endpoints.
- Locate the + Add Server Endpoint button. The folder on the local server that you've provisioned will become the path for this server endpoint.
Be sure to turn on cloud tiering! This is required if your local server does not have enough space to store the total size of your data in the StorSimple cloud storage. Set your tiering policy, temporarily for the migration, to 99% volume free space.
Repeat the steps of sync group creation and addition of the matching server folder as a server endpoint for all Azure file shares / server locations, that need to be configured for sync.
Step 9: Copy your files
The basic migration approach is a RoboCopy from your StorSimple virtual appliance to your Windows Server, and Azure File Sync to Azure file shares.
Run the first local copy to your Windows Server target folder:
- Identify the first location on your virtual StorSimple appliance.
- Identify the matching folder on the Windows Server, that already has Azure File Sync configured on it.
- Start the copy using RoboCopy
The following RoboCopy command will recall files from your StorSimple Azure storage to your local StorSimple and then move them over to the Windows Server target folder. The Windows Server will sync it to the Azure file share(s). As the local Windows Server volume gets full, cloud tiering will kick in and tier files that have successfully synced already. Cloud tiering will generate enough space to continue the copy from the StorSimple virtual appliance. Cloud tiering checks once an hour to see what has synced and to free up disk space to reach the 99% volume free space.
Robocopy /MT:32 /UNILOG:<file name> /TEE /B /MIR /COPYALL /DCOPY:DAT <SourcePath> <Dest.Path>
Allows for RoboCopy to run multi-threaded. Default is 8, max is 128.
Outputs status to LOG file as UNICODE (overwrites existing log).
Outputs to console window. Used in conjunction with output to a log file.
Runs RoboCopy in the same mode a backup application would use. It allows RoboCopy to move files that the current user does not have permissions to.
Allows to run this RoboCopy command several times, sequentially on the same target / destination. It identifies what has been copied before and omits it. Only changes, additions and "deletes" will be processed, that occurred since the last run. If the command wasn't run before, nothing is omitted. This is an excellent option for source locations that are still actively used and changing.
fidelity of the file copy (default is /COPY:DAT), copy flags: D=Data, A=Attributes, T=Timestamps, S=Security=NTFS ACLs, O=Owner info, U=aUditing info
COPY ALL file info (equivalent to /COPY:DATSOU)
fidelity for the copy of directories (default is /DCOPY:DA), copy flags: D=Data, A=Attributes, T=Timestamps
When you run the RoboCopy command for the first time, your users and applications are still accessing the StorSimple files and folders and potentially change it. It is possible, that RoboCopy has processed a directory, moves on to the next and then a user on the source location (StorSimple) adds, changes, or deletes a file that will now not be processed in this current RoboCopy run. That is fine.
The first run is about moving the bulk of the data back to on-premises, over to your Windows Server and backup into the cloud via Azure File Sync. This can take a long time, depending on:
- your download bandwidth
- the recall speed of the StorSimple cloud service
- the upload bandwidth
- the number of items (files and folders), that need to be processed by either service
Once the initial run is complete, run the command again.
The second time it will finish faster, because it only needs to transport changes that happened since the last run. Those changes are likely local to the StorSimple already, because they are recent. That is further reducing the time because the need for recall from the cloud is reduced. During this second run, still, new changes can accumulate.
Repeat this process until you are satisfied that the amount of time it takes to complete is an acceptable downtime.
When you consider the downtime acceptable and you are prepared to take the StorSimple location offline, then do so now: For example, remove the SMB share so that no user can access the folder or take any other appropriate step that prevents content to change in this folder on StorSimple.
Run one last RoboCopy round. This will pick up any changes, that might have been missed. How long this final step takes, is dependent on the speed of the RoboCopy scan. You can estimate the time (which is equal to your downtime) by measuring how long the previous run took.
Create a share on the Windows Server folder and possibly adjust your DFS-N deployment to point to it. Be sure to set the same share-level permissions as on your StorSimple SMB share.
You have finished migrating a share / group of shares into a common root or volume. (Depending on what you mapped and decided that needed to go into the same Azure file share.)
You can try to run a few of these copies in parallel. We recommend processing the scope of one Azure file share at a time.
Once you have moved all the data from you StorSimple to the Windows Server, and your migration is complete: Return to all sync groups in the Azure portal and adjust the cloud tiering volume free space percent value to something better suited for cache utilization, say 20%.
The cloud tiering volume free space policy acts on a volume level with potentially multiple server endpoints syncing from it. If you forget to adjust the free space on even one server endpoint, sync will continue to apply the most restrictive rule and attempt to keep 99% free disk space, making the local cache not performing as you might expect. Unless it is your goal to only have the namespace for a volume that only contains rarely accessed, archival data.
The most likely issue you can run into, is that the RoboCopy command fails with "Volume full" on the Windows Server side. If that is the case, then your download speed is likely better than your upload speed. Cloud tiering acts once every hour to evacuate content from the local Windows Server disk, that has synced.
Let sync progress and cloud tiering free up disk space. You can observe that in File Explorer on your Windows Server.
When your Windows Server has sufficient available capacity, rerunning the command will resolve the problem. Nothing breaks when you get into this situation and you can move forward with confidence. Inconvenience of running the command again is the only consequence.
You can also run into other Azure File Sync issues. As unlikely as they may be, if that happens, take a look at the LINK Azure File Sync troubleshooting guide.
Azure File Sync content: