Offline seeding using Azure Data Box (Preview)
This feature is applicable for Data Protection Manager (DPM) 2019 UR2 and later.
This article explains about how you can use Azure Data Box to seed initial Backup data offline from DPM to an Azure Recovery Services vault.
You can use Azure Data Box to seed your large initial DPM backups offline (without using the network) to a Recovery Services vault. This process saves time and network bandwidth that would otherwise be consumed moving large amounts of backup data online over a high-latency network. This feature is currently in preview.
Offline backup based on Azure Data Box provides two distinct advantages over offline backup based on the Azure Import/Export service:
You do not need to procure your own Azure-compatible disks and connectors. Azure Data Box ships the disks associated with the selected Data Box SKU.
Azure Backup (MARS Agent) can directly write backup data onto the supported SKUs of Azure Data Box. This capability eliminates the need for you to provision a staging location for your initial backup data. You also don't need utilities to format and copy that data onto the disks.
The following platforms are supported:
- Windows Server 2019 64 bit (Standard, Datacenter, Essentials)
- Windows Server 2016 64 bit (Standard, Datacenter, Essentials)
Backup data size and supported Data Box SKUs
The following Data Box SKUs are supported:
|Backup Data Size (post compression by MARS)* per server||Supported Azure Data Box SKU|
|<= 7.2 TB||Azure Data Box Disk|
|> 7.2 TB and <= 80 TB**||Azure Data Box (100 TB)|
*Typical compression rates vary between 10-20%
**Reach out to SystemCenterFeedback@microsoft.com if you expect to have more than 80 TB of initial backup data for a single data source.
Initial backup data from a single data source must be contained within a single Azure Data Box or Azure Data Box disk, and cannot be shared between multiple devices of the same or different SKUs. However, an Azure Data Box may contain initial backups from multiple data sources.
Before you begin
MARS agent running on DPM should be upgraded to the latest version (2.0.9171.0 or later).
Ensure the following:
Azure Subscription and required permissions
- A valid Azure subscription.
- The user intended to perform the offline backup policy must be an owner of the Azure subscription.
- The Data Box job and the Recovery Services Vault to which the data needs to be seeded must be available in the same subscriptions.
We recommend that the target storage account and the Recovery Services Vault be in the same region. However, this is not necessary.
Order and receive the Data Box device
Ensure that the required Data Box devices are in Delivered state before triggering Offline backup. See Backup Data Size and supported Data Box SKUs to order the most suitable SKU for your requirement. Follow the steps in this article to order and receive your Data Box devices.
Do not select BlobStorage for the Account kind. The DPM server requires an account that supports Page Blobs which is not supported when BlobStorage is selected. Select Storage V2 (general purpose v2) as the Account kind when creating the target storage account for your Azure Data Box job.
Setup Azure Data Box device(s)
Once you receive the Azure Data Box device, depending on the Azure Data Box SKU you have ordered, perform the steps in the appropriate sections below to set up and prepare the Data Box device(s) for the DPM Server to identify and transfer the initial backup data.
Setup Azure Data Box disk
If you ordered one or more Azure Data Box disks (up to 8 TB each), follow the steps mentioned here to unpack, connect, and unlock your Data Box disk.
It is possible that the DPM does not have an USB port. In such a scenario, you can connect your Azure Data Box disk to another server/client and expose the root of the device as a network share.
Setup Azure Data Box
If you ordered an Azure Data Box (up to 100 TB), follow the steps mentioned here to set up your Data Box.
Mount your Azure Data Box as local system
The DPM server operates in the System context and therefore requires the same level of privilege to be provided to the mount path where the Azure Data Box is connected. Follow the steps below to ensure you are able to mount your Data Box device as local system using the NFS protocol.
Enable the Client for NFS feature on the DPM Server. Specify alternate source: WIM:D:\Sources\Install.wim:4
Download PSExec from https://download.sysinternals.com/files/PSTools.zip to the DPM server.
Open an elevated command prompt and execute the following command with the directory containing PSExec.exe as current directory.
psexec.exe -s -i cmd.exe
The command window that opens as a result of the command above is in Local System context. Use this command window to execute steps to mount the Azure Page Blob Share as a network drive on your Windows Server.
Follow the steps here to connect your DPM Server to the Data Box device via NFS and execute the following command on the Local System command prompt to mount the Azure Page Blobs share:
mount -o nolock \\<DeviceIPAddres>\<StorageAccountName_PageBlob X:
Once mounted, check if you can access X: from your server. If yes, continue with the next section of this article.
Transfer Initial Backup data to Azure Data Box device(s)
On your DPM Server, follow the steps to create new protection group. If you are adding an online protection to the existing protection group, right-click the existing protection group, and select Add Online Protection and start from step 8.
On the Select Group Members page, specify the computers and sources you want to back up.
On the Select data protection method page, specify how you want to handle short and long-term backup. Make sure you select I want online protection.
On Select short-term goals page, specify how you want to back up to short-term storage on disk.
On the Review disk allocation page, review the storage pool disk space allocated for the protection group.
On the Choose replica creation method page, select Automatically over the network.
On the Choose consistency check options page, select how you want to automate consistency checks.
On the Specify online protection data: page, select the member you want enable online protection.
On the Specify online backup schedule page, specify how often incremental backups to Azure should occur.
On the Specify online retention policy page, specify how the recovery points created from the daily/weekly/monthly/yearly backups are retained in Azure.
On the Choose online replication screen of the wizard, choose the option Transfer using Microsoft owned disks and click Next.
Sign into Azure when prompted, using the user credentials that have owner access on the Azure Subscription. After successful login, the following screen is displayed:
The DPM server will then fetch the Data Box jobs in the subscription that are in Delivered state.
First time login takes longer than usual. The Azure PowerShell module gets installed in the background, and also the Azure AD Application is registered.
- The following PowerShell modules are installed:
- AzureRM.Profile 5.8.3
- AzureRM.Resources 6.7.3
- AzureRM.Storage 5.2.0
- Azure.Storage 4.6.1
- The Azure AD application is registered as AzureOfflineBackup_<object GUID of the user>.
- The following PowerShell modules are installed:
Select the correct Data box order for which you have unpacked, connected, and unlocked your Data Box disk. Click Next.
On the Detect the DataBox screen, enter the path of your Data Box device, and then click Detect Device.
Provide the network path to the root directory of the Azure Data Box disk. This directory must contain a directory by the name PageBlob as shown below:
For example, if the path of the disk is
\\mydomain\myserver\disk1\and disk1 contains a directory called PageBlob, the path to be provided on the DPM Server wizard is
\\mydomain\myserver\disk1\. If you setup an Azure Data Box 100 TB device, provide the following as the network path to the device
Click Next. On the Summary page, review your settings and click Create Group.
The following screen confirms that the Protection Group is created successfully.
Click Close on the screen above.
With this, the initial replication of the data occurs to DPM disk. When it finishes the protection, group status will show protection status as OK on the Protection page.
To initiate the offline-backup copy to your Azure Data Box device, right-click the Protection Group , and then choose the Create recovery point option. You then choose the Online Protection option.
The DPM server will start backing up the data you selected to the Azure Data Box device. This might take from several hours to a few days, depending on the size of the data and connection speed between the DPM server and the Azure Data Box Disk.
You can monitor the status of the job in the Monitoring pane. Once the backup of the data is complete, you will see a screen that resembles the one below:
Follow these steps once the data backup to the Azure Data Box Disk is successful.
Monitor the Data Box job in the Azure portal. Once the Azure Data Box job is Complete, the DPM server automatically moves the data from the Storage Account to the Recovery Services Vault at the time of the next scheduled backup. It will then mark the backup job as Job Completed if a recovery point is successfully created.
The DPM server triggers the backups at the times scheduled during protection group creation. However, these jobs will flag Waiting for Azure Data Box job to be completed until the time the job is complete.
After the DPM Server successfully creates a recovery point corresponding to the initial backup, you may delete the Storage Account (or specific contents) associated with the Azure Data Box job.
The Microsoft Azure Backup (MAB) agent on the DPM server creates an Azure AD application for you, in your tenant. This application requires a certificate for authentication that is created and uploaded when configuring offline seeding policy.
We use Azure PowerShell for creating and uploading the certificate to the Azure AD Application.
At the time of configuring offline backup, due to a known code defect in the Azure PowerShell cmdlet you are unable to add multiple certificates to the same Azure AD Application created by the MAB agent. This will impact you if you have configured offline seeding policy for the same or a different server.
Verify if the issue is caused by this specific root cause
To ensure that the failure is due to the Issue above, perform one of the following steps:
Check if you see the following error message in the DPM console at the time of configuring offline backup:
- Open the Temp folder in the installation path (default temp folder path is C:\Program Files\Microsoft Azure Recovery Services Agent\Temp. Look for the CBUICurr file and open the file.
- In the CBUICurr file, scroll to the last line and check if the failure is due to "Unable to create an Azure AD application credential in customer's account. Exception: Update to existing credential with KeyId <some guid> is not allowed".
To resolve this issue, do the following steps and retry the policy configuration.
Sign into Azure login page that appears on the DPM server UI using a different account with admin access on the subscription that will have the import export job created.
If no other server has offline seeding configured and no other server is dependent on the
AzureOfflineBackup_<Azure User Id>application, then delete this application from Azure portal > Azure Active Directory > App registrations.
Check if the application
AzureOfflineBackup_<Azure User Id>does not have any other offline seeding configured and also no other server is dependent on this application. Go to Settings > Keys under the Public Keys section it should not have any other public keys added. See the following screenshot for reference:
From the DPM server you are trying to configure offline backup, do the following actions:
Open the Manage computer certificate application > Personal tab and look for the certificate with the name
Select the above certificate, right-click All Tasks and Export without private key, in the .cer format.
Go to the Azure Offline Backup application mentioned in point 2. In the Settings > Keys > Upload Public Key, upload the certificate exported in the step above.
In the server, open the registry by typing regedit in the run window.
Go to the registry Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Azure Backup\Config\CloudBackupProvider. Right-click CloudBackupProvider and add a new string value with name
AzureADAppCertThumbprint_<Azure User Id>.
To get the Azure user ID, perform one of these actions:
- From the Azure-connected PowerShell, run the
Get-AzureRmADUser -UserPrincipalName "Account Holder's email as defined in the portal"command.
- Navigate to the registry path
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Azure Backup\DbgSettings\OnlineBackupwith the name CurrentUserId.
- From the Azure-connected PowerShell, run the
Right-click the string added in the step above and select Modify. In the value, provide the thumbprint of the certificate you exported in point 2 and click OK.
To get the value of thumbprint, double-click on the certificate, then select Details and scroll-down until you see the thumbprint field. Click on Thumbprint and copy the value.