Get started with Azure Blob Storage using F#
Azure Blob Storage is a service that stores unstructured data in the cloud as objects/blobs. Blob storage can store any type of text or binary data, such as a document, media file, or application installer. Blob storage is also referred to as object storage.
This article shows you how to perform common tasks using Blob storage. The samples are written using F# using the Azure Storage Client Library for .NET. The tasks covered include how to upload, list, download, and delete blobs.
For a conceptual overview of blob storage, see the .NET guide for blob storage.
Prerequisites
To use this guide, you must first create an Azure storage account. You also need your storage access key for this account.
Create an F# script and start F# interactive
The samples in this article can be used in either an F# application or an F# script. To create an F# script, create a file with the .fsx
extension, for example blobs.fsx
, in your F# development environment.
How to execute scripts
F# Interactive, dotnet fsi
, can be launched interactively, or it can be launched from the command line to run a script. The command-line syntax is
> dotnet fsi [options] [ script-file [arguments] ]
Add packages in a script
Next, use #r
nuget:package name
to install the Azure.Storage.Blobs
package and open
namespaces.Such as
> #r "nuget: Azure.Storage.Blobs"
open Azure.Storage.Blobs
open Azure.Storage.Blobs.Models
open Azure.Storage.Blobs.Specialized
Add namespace declarations
Add the following open
statements to the top of the blobs.fsx
file:
open System
open System.IO
open Azure.Storage.Blobs // Namespace for Blob storage types
open Azure.Storage.Blobs.Models
open Azure.Storage.Blobs.Specialized
open System.Text
Get your connection string
You need an Azure Storage connection string for this tutorial. For more information about connection strings, see Configure Storage Connection Strings.
For the tutorial, you enter your connection string in your script, like this:
let storageConnString = "..." // fill this in from your storage account
Create some local dummy data
Before you begin, create some dummy local data in the directory of our script. Later you upload this data.
// Create a dummy file to upload
let localFile = "./myfile.txt"
File.WriteAllText(localFile, "some data")
Create the blob service client
The BlobContainerClient
type enables you to create containers and retrieve blobs stored in Blob storage. Here's one way to create the container client:
let container = BlobContainerClient(storageConnString, "myContainer")
Now you are ready to write code that reads data from and writes data to Blob storage.
Create a container
This example shows how to create a container if it does not already exist:
container.CreateIfNotExists()
By default, the new container is private, meaning that you must specify your storage access key to download blobs from this container. If you want to make the files within the container available to everyone, you can set the container to be public using the following code:
let permissions = PublicAccessType.Blob
container.SetAccessPolicy(permissions)
Anyone on the Internet can see blobs in a public container, but you can modify or delete them only if you have the appropriate account access key or a shared access signature.
Upload a blob into a container
Azure Blob Storage supports block blobs and page blobs. In most cases, a block blob is the recommended type to use.
To upload a file to a block blob, get a container client and use it to get a block blob reference. Once you have a blob reference, you can upload any stream of data to it by calling the Upload
method. This operation overwrites the contents of the blob, creating a new block blob if none exists.
// Retrieve reference to a blob named "myblob.txt".
let blockBlob = container.GetBlobClient("myblob.txt")
// Create or overwrite the "myblob.txt" blob with contents from the local file.
use fileStream = new FileStream(localFile, FileMode.Open, FileAccess.Read, FileShare.Read)
do blockBlob.Upload(fileStream)
List the blobs in a container
To list the blobs in a container, first get a container reference. You can then use the container's GetBlobs
method to retrieve the blobs and/or directories within it. To access the rich set of properties and methods for a returned BlobItem
.
for item in container.GetBlobsByHierarchy() do
printfn $"Blob name: {item.Blob.Name}"
For example, consider the following set of block blobs in a container named photos
:
photo1.jpg
2015/architecture/description.txt
2015/architecture/photo3.jpg
2015/architecture/photo4.jpg
2016/architecture/photo5.jpg
2016/architecture/photo6.jpg
2016/architecture/description.txt
2016/photo7.jpg\
When you call GetBlobsByHierarchy
on a container (as in the above sample), a hierarchical listing is returned.
Directory: https://<accountname>.blob.core.windows.net/photos/2015/
Directory: https://<accountname>.blob.core.windows.net/photos/2016/
Block blob of length 505623: https://<accountname>.blob.core.windows.net/photos/photo1.jpg
Download blobs
To download blobs, first retrieve a blob reference and then call the DownloadTo
method. The following example uses the DownloadTo
method to transfer the blob contents to a stream object that you can then persist to a local file.
// Retrieve reference to a blob named "myblob.txt".
let blobToDownload = container.GetBlobClient("myblob.txt")
// Save blob contents to a file.
do
use fileStream = File.OpenWrite("path/download.txt")
blobToDownload.DownloadTo(fileStream)
You can also use the DownloadContent
method to download the contents of a blob as a text string.
let text = blobToDownload.DownloadContent().Value.Content.ToString()
Delete blobs
To delete a blob, first get a blob reference and then call the
Delete
method on it.
// Retrieve reference to a blob named "myblob.txt".
let blobToDelete = container.GetBlobClient("myblob.txt")
// Delete the blob.
blobToDelete.Delete()
List blobs in pages asynchronously
If you are listing a large number of blobs, or you want to control the number of results you return in one listing operation, you can list blobs in pages of results. This example shows how to return results in pages.
This example shows a hierarchical listing, by using the GetBlobsByHierarchy
method of the BlobClient
.
let ListBlobsSegmentedInHierarchicalListing(container:BlobContainerClient) =
// List blobs to the console window, with paging.
printfn "List blobs in pages:"
// Call GetBlobsByHierarchy to return an async collection
// of blobs in this container. AsPages() method enumerate the values
//a Page<T> at a time. This may make multiple service requests.
for page in container.GetBlobsByHierarchy().AsPages() do
for blobHierarchyItem in page.Values do
printf $"The BlobItem is : {blobHierarchyItem.Blob.Name} "
printfn ""
We can now use this hierarchical listing routine as follows. First, upload some dummy data (using the local file created earlier in this tutorial).
for i in 1 .. 100 do
let blob = container.GetBlobClient($"myblob{i}.txt")
use fileStream = System.IO.File.OpenRead(localFile)
blob.Upload(localFile)
Now, call the routine.
ListBlobsSegmentedInHierarchicalListing container
Writing to an append blob
An append blob is optimized for append operations, such as logging. Like a block blob, an append blob is composed of blocks, but when you add a new block to an append blob, it is always appended to the end of the blob. You cannot update or delete an existing block in an append blob. The block IDs for an append blob are not exposed as they are for a block blob.
Each block in an append blob can be a different size, up to a maximum of 4 MB, and an append blob can include a maximum of 50,000 blocks. The maximum size of an append blob is therefore slightly more than 195 GB (4 MB X 50,000 blocks).
The following example creates a new append blob and appends some data to it, simulating a simple logging operation.
// Get a reference to a container.
let appendContainer = BlobContainerClient(storageConnString, "my-append-blobs")
// Create the container if it does not already exist.
appendContainer.CreateIfNotExists() |> ignore
// Get a reference to an append blob.
let appendBlob = appendContainer.GetAppendBlobClient("append-blob.log")
// Create the append blob. Note that if the blob already exists, the
// CreateOrReplace() method will overwrite it. You can check whether the
// blob exists to avoid overwriting it by using CloudAppendBlob.Exists().
appendBlob.CreateIfNotExists()
let numBlocks = 10
// Generate an array of random bytes.
let rnd = Random()
let bytesArray = Array.zeroCreate<byte>(numBlocks)
rnd.NextBytes(bytesArray)
// Simulate a logging operation by writing text data and byte data to the
// end of the append blob.
for i in 0 .. numBlocks - 1 do
let msg = sprintf $"Timestamp: {DateTime.UtcNow} \tLog Entry: {bytesArray.[i]}\n"
let array = Encoding.ASCII.GetBytes(msg);
use stream = new MemoryStream(array)
appendBlob.AppendBlock(stream)
// Read the append blob to the console window.
let downloadedText = appendBlob.DownloadContent().ToString()
printfn $"{downloadedText}"
See Understanding Block Blobs, Page Blobs, and Append Blobs for more information about the differences between the three types of blobs.
Concurrent access
To support concurrent access to a blob from multiple clients or multiple process instances, you can use ETags or leases.
Etag - provides a way to detect that the blob or container has been modified by another process
Lease - provides a way to obtain exclusive, renewable, write, or delete access to a blob for a period of time
For more information, see Managing Concurrency in Microsoft Azure Storage.
Naming containers
Every blob in Azure storage must reside in a container. The container forms part of the blob name. For example, mydata
is the name of the container in these sample blob URIs:
https://storagesample.blob.core.windows.net/mydata/blob1.txt
https://storagesample.blob.core.windows.net/mydata/photos/myphoto.jpg
A container name must be a valid DNS name, conforming to the following naming rules:
- Container names must start with a letter or number, and can contain only letters, numbers, and the dash (-) character.
- Every dash (-) character must be immediately preceded and followed by a letter or number; consecutive dashes are not permitted in container names.
- All letters in a container name must be lowercase.
- Container names must be from 3 through 63 characters long.
The name of a container must always be lowercase. If you include an upper-case letter in a container name, or otherwise violate the container naming rules, you may receive a 400 error (Bad Request).
Managing security for blobs
By default, Azure Storage keeps your data secure by limiting access to the account owner, who is in possession of the account access keys. When you need to share blob data in your storage account, it is important to do so without compromising the security of your account access keys. Additionally, you can encrypt blob data to ensure that it is secure going over the wire and in Azure Storage.
Controlling access to blob data
By default, the blob data in your storage account is accessible only to storage account owner. Authenticating requests against Blob storage requires the account access key by default. However, you might want to make certain blob data available to other users.
Encrypting blob data
Azure Storage supports encrypting blob data both at the client and on the server.
See also
Feedback
https://aka.ms/ContentUserFeedback.
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see:Submit and view feedback for