Transfer objects to/from Azure Blob storage using Python
In this quickstart, you learn how to use Python to upload, download, and list block blobs in a container in Azure Blob storage.
To complete this quickstart:
If you don't have an Azure subscription, create a free account before you begin.
Create a storage account by using the Azure portal
First, create a new general-purpose storage account to use for this quickstart.
- Go to the Azure portal and sign in by using your Azure account.
- Enter a unique name for your storage account. Keep these rules in mind for naming your storage account:
- The name must be 3 to 24 characters in length.
- The name can contain numbers and lowercase letters only.
- Make sure that the following default values are set:
- Deployment model is set to Resource Manager.
- Account kind is set to General purpose.
- Performance is set to Standard.
- Replication is set to Locally Redundant storage (LRS).
- Select your subscription.
- For Resource group, create a new one and give it a unique name.
- Select the location to use for your storage account.
- Select Pin to dashboard and select Create to create your storage account.
After your storage account is created, it's pinned to the dashboard. Select it to open it. Under Settings, select Access keys. Select the primary key and copy the associated connection string to the clipboard. Then paste the string into a text editor for later use.
Download the sample application
The sample application used in this quickstart is a basic Python application.
Use git to download a copy of the application to your development environment.
git clone https://github.com/Azure-Samples/storage-blobs-python-quickstart.git
This command clones the repository to your local git folder. To open the Python program, look for the storage-blobs-python-quickstart folder, and example.py file.
Configure your storage connection string
In the application, you must provide your storage account name and account key to create a
BlockBlobService object. Open the
example.py file from the Solution Explorer in your IDE. Replace the accountname and accountkey values with your account name and key.
block_blob_service = BlockBlobService(account_name='accountname', account_key='accountkey')
Run the sample
This sample creates a test file in the 'Documents' folder. The sample program uploads the test file to Blob storage, lists the blobs in the container, and downloads the file with a new name.
Run the sample. The following output is an example of the output returned when running the application:
Temp file = C:\Users\azureuser\Documents\QuickStart_9f4ed0f9-22d3-43e1-98d0-8b2c05c01078.txt Uploading to Blob storage as blobQuickStart_9f4ed0f9-22d3-43e1-98d0-8b2c05c01078.txt List blobs in the container Blob name: QuickStart_9f4ed0f9-22d3-43e1-98d0-8b2c05c01078.txt Downloading blob to C:\Users\azureuser\Documents\QuickStart_9f4ed0f9-22d3-43e1-98d0-8b2c05c01078_DOWNLOADED.txt
When you press any key to continue, the sample program deletes the storage container and the files. Before you continue, check your 'Documents' folder for the two files. You can open them and see they are identical.
You can also use a tool such as the Azure Storage Explorer to view the files in Blob storage. Azure Storage Explorer is a free cross-platform tool that allows you to access your storage account information.
After you've verified the files, hit any key to finish the demo and delete the test files. Now that you know what the sample does, open the example.py file to look at the code.
Understand the sample code
Next, we walk through the sample code so that you can understand how it works.
Get references to the storage objects
The first thing to do is create the references to the objects used to access and manage Blob storage. These objects build on each other, and each is used by the next one in the list.
Instantiate the BlockBlobService object, which points to the Blob service in your storage account.
Instantiate the CloudBlobContainer object, which represents the container you are accessing. Containers are used to organize your blobs like you use folders on your computer to organize your files.
Once you have the Cloud Blob container, you can instantiate the CloudBlockBlob object that points to the specific blob in which you are interested, and perform operations such as upload, download, and copy.
Container names must be lowercase. See Naming and Referencing Containers, Blobs, and Metadata for more information about container and blob names.
In this section, you instantiate the objects, create a new container, and then set permissions on the container so the blobs are public. The container is called quickstartblobs.
# Create the BlockBlockService that is used to call the Blob service for the storage account block_blob_service = BlockBlobService(account_name='accountname', account_key='accountkey') # Create a container called 'quickstartblobs'. container_name ='quickstartblobs' block_blob_service.create_container(container_name) # Set the permission so the blobs are public. block_blob_service.set_container_acl(container_name, public_access=PublicAccess.Container)
Upload blobs to the container
Blob storage supports block blobs, append blobs, and page blobs. Block blobs are the most commonly used, and that is what is used in this quickstart.
To upload a file to a blob, get the full path of the file by joining the directory name and the file name on your local drive. You can then upload the file to the specified path using the create_blob_from_path method.
The sample code creates a local file to be used for the upload and download, storing the file to be uploaded as file_path_to_file and the name of the blob as local_file_name. The following example uploads the file to your container called quickstartblobs.
# Create a file in Documents to test the upload and download. local_path=os.path.expanduser("~\Documents") local_file_name ="QuickStart_" + str(uuid.uuid4()) + ".txt" full_path_to_file =os.path.join(local_path, local_file_name) # Write text to the file. file = open(full_path_to_file, 'w') file.write("Hello, World!") file.close() print("Temp file = " + full_path_to_file) print("\nUploading to Blob storage as blob" + local_file_name) # Upload the created file, use local_file_name for the blob name block_blob_service.create_blob_from_path(container_name, local_file_name, full_path_to_file)
There are several upload methods that you can use with Blob storage. For example, if you have a memory stream, you can use the create_blob_from_stream method rather than the create_blob_from_path.
Block blobs can be as large as 4.7 TB, and can be anything from Excel spreadsheets to large video files. Page blobs are primarily used for the VHD files used to back IaaS VMs. Append blobs are used for logging, such as when you want to write to a file and then keep adding more information. Most objects stored in Blob storage are block blobs.
List the blobs in a container
Get a list of files in the container using the list_blobs method. This method returns a generator. The following code retrieves the list of blobs, then loops through them, showing the names of the blobs found in a container.
# List the blobs in the container print("\nList blobs in the container") generator = block_blob_service.list_blobs(container_name) for blob in generator: print("\t Blob name: " + blob.name)
Download the blobs
Download blobs to your local disk using the get_blob_to_path method. The following code downloads the blob uploaded in a previous section. "_DOWNLOADED" is added as a suffix to the blob name so you can see both files on local disk.
# Download the blob(s). # Add '_DOWNLOADED' as prefix to '.txt' so you can see both files in Documents. full_path_to_file2 = os.path.join(local_path, string.replace(local_file_name ,'.txt', '_DOWNLOADED.txt')) print("\nDownloading blob to " + full_path_to_file2) block_blob_service.get_blob_to_path(container_name, local_file_name, full_path_to_file2)
Clean up resources
If you no longer need the blobs uploaded in this quickstart, you can delete the entire container using the delete_container. If the files created are no longer needed, you use the delete_blob method to delete the files.
# Clean up resources. This includes the container and the temp files block_blob_service.delete_container(container_name) os.remove(full_path_to_file) os.remove(full_path_to_file2)
In this quickstart, you learned how to transfer files between a local disk and Azure blob storage using Python. To learn more about working with blob storage, continue to the Blob storage How-to.
For more information about the Storage Explorer and Blobs, see Manage Azure Blob storage resources with Storage Explorer.