您现在访问的是微软AZURE全球版技术文档网站,若需要访问由世纪互联运营的MICROSOFT AZURE中国区技术文档网站,请访问 https://docs.azure.cn.

教程:使用 AzCopy 将本地数据迁移到云存储空间Tutorial: Migrate on-premises data to cloud storage with AzCopy

AzCopy 是一个命令行工具,借助该工具,可使用简单命令将数据复制到 Azure Blob 存储、Azure 文件和 Azure 表存储或从其中复制出数据。AzCopy is a command-line tool for copying data to or from Azure Blob storage, Azure Files, and Azure Table storage, by using simple commands. 这些命令旨在实现最佳性能。The commands are designed for optimal performance. 使用 AzCopy,可在文件系统和存储帐户之间或在存储帐户之间复制数据。Using AzCopy, you can either copy data between a file system and a storage account, or between storage accounts. AzCopy 可以用来将数据从本地复制到存储帐户。AzCopy may be used to copy data from local (on-premises) data to a storage account.

本教程介绍如何执行下列操作:In this tutorial, you learn how to:

  • 创建存储帐户。Create a storage account.
  • 使用 AzCopy 上传所有数据。Use AzCopy to upload all your data.
  • 修改用于测试目的的数据。Modify the data for test purposes.
  • 创建一个计划任务或 cron 作业,以标识要上传的新文件。Create a scheduled task or cron job to identify new files to upload.

如果还没有 Azure 订阅,可以在开始前创建一个免费帐户If you don’t have an Azure subscription, create a free account before you begin.

先决条件Prerequisites

若要完成本教程,请下载最新版 AzCopy。To complete this tutorial, download the latest version of AzCopy. 请参阅 AzCopy 入门See Get started with AzCopy.

如果使用 Windows,则需 Schtasks,因为本教程使用它来计划任务。If you're on Windows, you will require Schtasks as this tutorial makes use of it in order to schedule a task. Linux 用户会改用 crontab 命令。Linux users will make use of the crontab command, instead.

若要在 Azure 门户中创建常规用途 v2 存储帐户,请执行以下步骤:To create a general-purpose v2 storage account in the Azure portal, follow these steps:

  1. 在 Azure 门户菜单中,选择“所有服务” 。On the Azure portal menu, select All services. 在资源列表中,键入“存储帐户” 。In the list of resources, type Storage Accounts. 开始键入时,会根据输入筛选该列表。As you begin typing, the list filters based on your input. 选择“存储帐户” 。Select Storage Accounts.

  2. 在显示的“存储帐户”窗口中,选择“添加”。 On the Storage Accounts window that appears, choose Add.

  3. 选择要在其中创建存储帐户的订阅。Select the subscription in which to create the storage account.

  4. 在“资源组” 字段下,选择“新建” 。Under the Resource group field, select Create new. 输入新资源组的名称,如下图中所示。Enter a name for your new resource group, as shown in the following image.

    显示如何在门户中创建资源组的屏幕截图

  5. 然后,输入存储帐户的名称。Next, enter a name for your storage account. 所选名称在 Azure 中必须唯一。The name you choose must be unique across Azure. 该名称还必须为 3 到 24 个字符,并且只能包含数字和小写字母。The name also must be between 3 and 24 characters in length, and can include numbers and lowercase letters only.

  6. 选择存储帐户的位置或使用默认位置。Select a location for your storage account, or use the default location.

  7. 将这些字段设置为其默认值:Leave these fields set to their default values:

    字段Field Value
    部署模型Deployment model 资源管理器Resource Manager
    性能Performance 标准Standard
    帐户类型Account kind StorageV2(常规用途 v2)StorageV2 (general-purpose v2)
    复制Replication 读取访问异地冗余存储 (RA-GRS)Read-access geo-redundant storage (RA-GRS)
    访问层Access tier Hot
  8. 选择“查看+创建” 可查看存储帐户设置并创建帐户。Select Review + Create to review your storage account settings and create the account.

  9. 选择“创建” 。Select Create.

有关存储帐户类型和其他存储帐户设置的详细信息,请参阅 Azure 存储帐户概述For more information about types of storage accounts and other storage account settings, see Azure storage account overview. 有关资源组的详细信息,请参阅 Azure 资源管理器概述For more information on resource groups, see Azure Resource Manager overview.

创建容器Create a container

第一步是创建容器,因为 Blob 始终必须上传到容器中。The first step is to create a container, because blobs must always be uploaded into a container. 容器用作组织 Blob 组的方法,就像将计算机上的文件组织到文件夹中一样。Containers are used as a method of organizing groups of blobs like you would files on your computer, in folders.

按照这些步骤创建容器:Follow these steps to create a container:

  1. 选择主页上的“存储帐户” 按钮,然后选择创建的存储帐户。Select the Storage accounts button from the main page, and select the storage account that you created.

  2. 选择“服务” 下的“Blob” ,然后选择“容器” 。Select Blobs under Services, and then select Container.

    创建容器

容器名必须以字母或数字开头。Container names must start with a letter or number. 名称中只能包含字母、数字和连字符 (-)。They can contain only letters, numbers, and the hyphen character (-). 有关命名 Blob 和容器的更多规则,请参阅命名和引用容器、Blob 和元数据For more rules about naming blobs and containers, see Naming and referencing containers, blobs, and metadata.

下载 AzCopyDownload AzCopy

下载 AzCopy V10 可执行文件。Download the AzCopy V10 executable file.

将 AzCopy 文件置于计算机上的任何位置。Place the AzCopy file anywhere on your computer. 将文件位置添加到系统路径变量,这样即可从计算机上的任何文件夹引用该可执行文件。Add the location of the file to your system path variable so that you can refer to this executable file from any folder on your computer.

使用 Azure AD 进行身份验证Authenticate with Azure AD

首先,为标识分配存储 Blob 数据参与者角色。First, assign the Storage Blob Data Contributor role to your identity. 请参阅在 Azure 门户中使用 RBAC 授予对 Azure Blob 和队列数据的访问权限See Grant access to Azure blob and queue data with RBAC in the Azure portal.

然后打开命令提示符,键入以下命令,按 ENTER 键。Then, open a command prompt, type the following command, and press the ENTER key.

azcopy login

此命令返回身份验证代码和网站的 URL。This command returns an authentication code and the URL of a website. 打开网站,提供代码,然后选择“下一步”按钮。 Open the website, provide the code, and then choose the Next button.

创建容器

此时会出现登录窗口。A sign-in window will appear. 在该窗口中,使用 Azure 帐户凭据登录到 Azure 帐户。In that window, sign into your Azure account by using your Azure account credentials. 成功登录后,可以关闭浏览器窗口,开始使用 AzCopy。After you've successfully signed in, you can close the browser window and begin using AzCopy.

将文件夹的内容上传到 Blob 存储Upload contents of a folder to Blob storage

可使用 AzCopy 将文件夹中的所有文件上传到 WindowsLinux 上的 Blob 存储中。You can use AzCopy to upload all files in a folder to Blob storage on Windows or Linux. 若要上传文件夹中的所有 Blob,请输入以下 AzCopy 命令:To upload all blobs in a folder, enter the following AzCopy command:

azcopy copy "<local-folder-path>" "https://<storage-account-name>.<blob or dfs>.core.windows.net/<container-name>" --recursive=true
  • <local-folder-path> 占位符替换为包含文件(例如 C:\myFolder/mnt/myFolder)的文件夹的路径。Replace the <local-folder-path> placeholder with the path to a folder that contains files (For example: C:\myFolder or /mnt/myFolder).

  • <storage-account-name> 占位符替换为存储帐户的名称。Replace the <storage-account-name> placeholder with the name of your storage account.

  • 请将 <container-name> 占位符替换为所创建容器的名称。Replace the <container-name> placeholder with the name of the container that you created.

若要将指定目录的内容以递归方式上传到 Blob 存储,请指定 --recursive 选项。To upload the contents of the specified directory to Blob storage recursively, specify the --recursive option. 使用此选项运行 AzCopy 时,会同时上传所有子文件夹及其文件。When you run AzCopy with this option, all subfolders and their files are uploaded as well.

将修改的文件上传到 Blob 存储Upload modified files to Blob storage

可基于文件的上次修改时间,使用 AzCopy 上传文件。You can use AzCopy to upload files based on their last-modified time.

若要尝试此操作,可在源目录中修改文件或创建新文件,用于测试目的。To try this, modify or create new files in your source directory for test purposes. 然后,使用 AzCopy sync 命令。Then, use the AzCopy sync command.

azcopy sync "<local-folder-path>" "https://<storage-account-name>.blob.core.windows.net/<container-name>" --recursive=true
  • <local-folder-path> 占位符替换为包含文件(例如 C:\myFolder/mnt/myFolder)的文件夹的路径。Replace the <local-folder-path> placeholder with the path to a folder that contains files (For example: C:\myFolder or /mnt/myFolder.

  • <storage-account-name> 占位符替换为存储帐户的名称。Replace the <storage-account-name> placeholder with the name of your storage account.

  • 请将 <container-name> 占位符替换为所创建容器的名称。Replace the <container-name> placeholder with the name of the container that you created.

若要详细了解 sync 命令,请参阅同步文件To learn more about the sync command, see Synchronize files.

创建计划的任务Create a scheduled task

可创建用于运行 AzCopy 命令脚本的计划任务或 cron 作业。You can create a scheduled task or cron job that runs an AzCopy command script. 此脚本标识新的本地数据,并按特定时间间隔将其上传到云存储。The script identifies and uploads new on-premises data to cloud storage at a specific time interval.

将 AzCopy 命令复制到文本编辑器。Copy the AzCopy command to a text editor. 将 AzCopy 命令的参数值更新为合适的值。Update the parameter values of the AzCopy command to the appropriate values. 将文件另存为适用于 AzCopy 的 script.sh (Linux) 或 script.bat (Windows)。Save the file as script.sh (Linux) or script.bat (Windows) for AzCopy.

这些示例假设你的文件夹名为 myFolder,你的存储帐户名为 mystorageaccount,你的容器名为 mycontainerThese examples assume that your folder is named myFolder, your storage account name is mystorageaccount and your container name is mycontainer.

备注

Linux 示例将追加 SAS 令牌。The Linux example appends a SAS token. 你需要在命令中提供一个。You'll need to provide one in your command. 当前版本的 AzCopy V10 不支持 cron 作业中的 Azure AD 授权。The current version of AzCopy V10 doesn't support Azure AD authorization in cron jobs.

azcopy sync "/mnt/myfiles" "https://mystorageaccount.blob.core.windows.net/mycontainer?sv=2018-03-28&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-05-30T06:57:40Z&st=2019-05-29T22:57:40Z&spr=https&sig=BXHippZxxx54hQn%2F4tBY%2BE2JHGCTRv52445rtoyqgFBUo%3D" --recursive=true

在本教程中,Schtasks 用于在 Windows 上创建计划任务。In this tutorial, Schtasks is used to create a scheduled task on Windows. Crontab 命令用于在 Linux 上创建 cron 作业。The Crontab command is used to create a cron job on Linux.

使用 Schtasks ,管理员能够在本地或远程计算机上创建、删除、查询、更改、运行和结束计划的任务。Schtasks enables an administrator to create, delete, query, change, run, and end scheduled tasks on a local or remote computer. 使用 Cron ,Linux 和 Unix 用户能够使用 cron 表达式在指定日期和时间运行命令或脚本。Cron enables Linux and Unix users to run commands or scripts at a specified date and time by using cron expressions.

若要在 Linux 上创建 cron 作业,请在终端上输入以下命令:To create a cron job on Linux, enter the following command on a terminal:

crontab -e
*/5 * * * * sh /path/to/script.sh

在命令中指定 cron 表达式 */5 * * * * 可指示 shell 脚本 script.sh 应每隔五分钟运行一次。Specifying the cron expression */5 * * * * in the command indicates that the shell script script.sh should run every five minutes. 可计划让脚本在每日、每月或每年的特定时间运行。You can schedule the script to run at a specific time daily, monthly, or yearly. 若要了解有关设置作业执行日期和时间的详细信息,请参阅 cron 表达式To learn more about setting the date and time for job execution, see cron expressions.

若要验证计划的任务或 cron 作业运行正常,在 myFolder 目录中创建新文件。To validate that the scheduled task/cron job runs correctly, create new files in your myFolder directory. 等待五分钟以确认已将新文件上传到存储帐户。Wait five minutes to confirm that the new files have been uploaded to your storage account. 转到日志目录,以查看计划任务或 cron 作业的输出日志。Go to your log directory to view output logs of the scheduled task or cron job.

后续步骤Next steps

若要详细了解如何在本地和 Azure 存储之间移动数据,请单击以下链接:To learn more about ways to move on-premises data to Azure Storage and vice versa, follow this link:

有关 AzCopy 的详细信息,请参阅以下任何文章:For more information about AzCopy, see any of these articles: