Move data to and from Azure Blob Storage using AzCopy

AzCopy is a command-line utility designed for uploading, downloading, and copying data to and from Microsoft Azure blob, file, and table storage.

For instructions on installing AzCopy and additional information on using it with the Azure platform, see Getting Started with the AzCopy Command-Line Utility.

This menu links to technologies you can use to move data to and from Azure Blob storage:

Note

If you are using VM that was set up with the scripts provided by Data Science Virtual machines in Azure, then AzCopy is already installed on the VM.

Note

For a complete introduction to Azure blob storage, refer to Azure Blob Basics and to Azure Blob Service.

Prerequisites

This document assumes that you have an Azure subscription, a storage account and the corresponding storage key for that account. Before uploading/downloading data, you must know your Azure storage account name and account key.

Run AzCopy commands

To run AzCopy commands, open a command window and navigate to the AzCopy installation directory on your computer, where the AzCopy.exe executable is located.

The basic syntax for AzCopy commands is:

AzCopy /Source:<source> /Dest:<destination> [Options]

Note

You can add the AzCopy installation location to your system path and then run the commands from any directory. By default, AzCopy is installed to %ProgramFiles(x86)%\Microsoft SDKs\Azure\AzCopy or %ProgramFiles%\Microsoft SDKs\Azure\AzCopy.

Upload files to an Azure blob

To upload a file, use the following command:

# Upload from local file system
AzCopy /Source:<your_local_directory> /Dest: https://<your_account_name>.blob.core.windows.net/<your_container_name> /DestKey:<your_account_key> /S

Download files from an Azure blob

To download a file from an Azure blob, use the following command:

# Downloading blobs to local file system
AzCopy /Source:https://<your_account_name>.blob.core.windows.net/<your_container_name>/<your_sub_directory_at_blob>  /Dest:<your_local_directory> /SourceKey:<your_account_key> /Pattern:<file_pattern> /S

Transfer blobs between Azure containers

To transfer blobs between Azure containers, use the following command:

# Transferring blobs between Azure containers
AzCopy /Source:https://<your_account_name1>.blob.core.windows.net/<your_container_name1>/<your_sub_directory_at_blob1> /Dest:https://<your_account_name2>.blob.core.windows.net/<your_container_name2>/<your_sub_directory_at_blob2> /SourceKey:<your_account_key1> /DestKey:<your_account_key2> /Pattern:<file_pattern> /S

<your_account_name>: your storage account name
<your_account_key>: your storage account key
<your_container_name>: your container name
<your_sub_directory_at_blob>: the sub directory in the container
<your_local_directory>: directory of local file system where files to be uploaded from or the directory of local file system files to be downloaded to
<file_pattern>: pattern of file names to be transferred. The standard wildcards are supported

Tips for using AzCopy

Tip

  1. When uploading files, /S uploads files recursively. Without this parameter, files in subdirectories are not uploaded.
  2. When downloading file, /S searches the container recursively until all files in the specified directory and its subdirectories, or all files that match the specified pattern in the given directory and its subdirectories, are downloaded.
  3. You cannot specify a specific blob file to download using the /Source parameter. To download a specific file, specify the blob file name to download using the /Pattern parameter. /S parameter can be used to have AzCopy look for a file name pattern recursively. Without the pattern parameter, AzCopy downloads all files in that directory.