Upload file to Azure Blob Storage with an Azure Function
This article shows you how to create an Azure Function API, which uploads a file to Azure Storage using an out binding to move the file contents from the API to Storage.
Solution architecture considerations
The Azure Function file upload limit is 100 MB. If you need to upload larger files, consider either a browser-based approach or a server app.
This sample uses an Azure Function out binding instead of the Azure Storage npm package. By using the binding, you have to configure your function to correctly use the outbound binding to move the file from this function to the storage resource without writing code to interact with Azure Storage.
The out binding usage, used in this article, has some pros and cons:
| Pros | Cons |
|---|---|
| * No code to write to move a file from the function to storage * No npm dependency for storage |
* function.json must be configured correctly * Connection string to storage must be configured correctly in environment |
The code required to read the uploaded file and convert it into a format that can be sent to storage is required, regardless if you use an out binding or an npm package to integrate with Azure Storage directly.
Prepare your development environment
Make sure the following are installed on your local developer workstation:
- An Azure account with an active subscription which you own. Create an account for free. Ownership is required to provide the correct Azure Active folder permissions to complete these steps.
- Node.js LTS and npm - for local development.
- Visual Studio Code - to develop locally and to deploy to Azure.
- Visual Studio Code extensions:
1. Create a resource group
A resource group holds both the Azure Function resource and the Azure Storage resource. Because both resources are in a single resource group, when you want to remove these resources, you remove the resource group. That action removes all resources in the resource group.
In Visual Studio Code, select the Azure explorer, then select the + (Plus/Addition) icon under Resource Groups.
Use the following table to finish creating the resource group:
Prompt Value Notes Enter the name of the new resource group. blob-storage-upload-function-groupIf you choose a different name, remember to use it as a replacement for this name when you see it in the rest of this article. Select a location for new resources. Select a region close to you.
2. Create the local Functions app
Create a new folder on your local workstation, then open Visual Studio Code in this folder.
In Visual Studio Code, select the Azure explorer, then expand the Azure Functions explorer, then select the Create New Project command:
Use the following table to finish creating the local Azure Function project:
Prompt Value Notes Select the folder that will contain your function project. Select the current folder, which is the default value. Select a language TypeScript Select a template for your project's first function HTTP Trigger API is invoked with an HTTP request. Provide a function name uploadAPI route is /api/uploadAuthorization Level Function This locks the remote API to requests that pass the function key with the request. While developing locally, you won't need the function key. This process doesn't create cloud-based Azure Function resource yet. That step will come later.
Return to the Visual Studio Code File Explorer.
After a few moments, Visual Studio Code completes creation of the local project, including a folder named for the function, upload, within which are three files:
Filename Description index.ts The source code that responds to the HTTP request. function.json The binding configuration for the HTTP trigger. sample.dat A placeholder data file to demonstrate that you can have other files in the folder. You can delete this file, if desired, as it's not used in this tutorial.
3. Install dependencies
In Visual Studio Code, open an integrated bash terminal, Ctrl + `.
Install npm dependencies:
npm install
4. Install and start Azurite storage emulator
Now that the basic project folder structure and files are in place, add local storage emulation.
To emulate the Azure Storage service locally, install Azurite.
npm install azuriteCreate a folder to hold the storage files inside your local project folder:
mkdir azureStorageTo start the Azurite emulator, add an npm script to the end of the
scriptsproperty items in the package.json file:"start-azurite": "azurite --silent --location azureStorage --debug azureStorage/debug.log"This action uses the local folder
azureStorageto hold the storage files and logs.In a new Visual Studio Code bash terminal, start the emulator:
npm run start-azuriteDon't close this terminal during the article until the cleanup step.
5. Add code to manage file upload
In a new Visual Studio Code integrated bash terminal, add npm packages to handle file tasks:
npm install http-status-enum parse-multipart @types/parse-multipartLeave this terminal open to use other script commands. You should have two terminal windows open: one window running Azurite storage emulator, and this terminal for commands.
Open the
./upload/index.tsfile and replace the contents with the following code:import { AzureFunction, Context, HttpRequest } from "@azure/functions" import HTTP_CODES from "http-status-enum"; import * as multipart from "parse-multipart"; const httpTrigger: AzureFunction = async function (context: Context, req: HttpRequest): Promise<any> { context.log('upload HTTP trigger function processed a request.'); if (!req.query?.username) { context.res.body = `username is not defined`; context.res.status = HTTP_CODES.BAD_REQUEST } // `filename` is required property to use multi-part npm package if (!req.query?.filename) { context.res.body = `filename is not defined`; context.res.status = HTTP_CODES.BAD_REQUEST } if (!req.body || !req.body.length){ context.res.body = `Request body is not defined`; context.res.status = HTTP_CODES.BAD_REQUEST } // Content type is required to know how to parse multi-part form if (!req.headers || !req.headers["content-type"]){ context.res.body = `Content type is not sent in header 'content-type'`; context.res.status = HTTP_CODES.BAD_REQUEST } context.log(`*** Username:${req.query?.username}, Filename:${req.query?.filename}, Content type:${req.headers["content-type"]}, Length:${req.body.length}`); if(process?.env?.Environment==='Production' && (!process?.env?.AzureWebJobsStorage || process?.env?.AzureWebJobsStorage.length<10)){ throw Error("Storage isn't configured correctly - get Storage Connection string from Azure portal"); } try { // Each chunk of the file is delimited by a special string const bodyBuffer = Buffer.from(req.body); const boundary = multipart.getBoundary(req.headers["content-type"]); const parts = multipart.Parse(bodyBuffer, boundary); // The file buffer is corrupted or incomplete ? if (!parts?.length){ context.res.body = `File buffer is incorrect`; context.res.status = HTTP_CODES.BAD_REQUEST } // filename is a required property of the parse-multipart package if(parts[0]?.filename)console.log(`Original filename = ${parts[0]?.filename}`); if(parts[0]?.type)console.log(`Content type = ${parts[0]?.type}`); if(parts[0]?.data?.length)console.log(`Size = ${parts[0]?.data?.length}`); // Passed to Storage context.bindings.storage = parts[0]?.data; // returned to requestor context.res.body = `${req.query?.username}/${req.query?.filename}`; } catch (err) { context.log.error(err.message); { context.res.body = `${err.message}`; context.res.status = HTTP_CODES.INTERNAL_SERVER_ERROR } } return context.res; }; export default httpTrigger;The
filenamequery string parameter is required because the out binding needs to know the name of the file to create. Theusernamequery string parameter is required because it becomes the Storage container (folder) name. For example, if the user name isjsmithand the file name istest-file.txt, the Storage location isjsmith/test-file.txt.The code to read the file and send it to the out binding is highlighted.
6. Connect Azure Function to Azure Storage
Open the
./upload/function.jsonfile and replace the contents with the following code:{ "bindings": [ { "authLevel": "Function", "type": "httpTrigger", "direction": "in", "dataType": "binary", "name": "req", "methods": [ "post" ] }, { "type": "http", "direction": "out", "name": "$return" }, { "name": "storage", "type": "blob", "path": "{username}/{filename}", "direction": "out", "connection": "AzureWebJobsStorage" } ], "scriptFile": "../dist/upload/index.js" }The first highlighted object defines the out binding to read the returned object from the function. The second highlighted object defines how to use the read information. The connection string for the Storage resource is defined in the connection property with the
AzureWebJobsStoragevalue.Open the
./local.settings.jsonfile and replace the AzureWebJobsStorage property's value withUseDevelopmentStorage=trueto ensure that when you develop locally, the function uses the local Azurite storage emulator:{ "IsEncrypted": false, "Values": { "Environment": "Development", "AzureWebJobsStorage": "UseDevelopmentStorage=true", "FUNCTIONS_WORKER_RUNTIME": "node" } }
7. Run the local function
In the integrated terminal window for commands (not the terminal window running Azurite), start the function:
npm startWait until you see the URL for the function. This indicates your function started correctly.
upload: [POST] http://localhost:7071/api/uploadCreate a new file in the root of the project named
test-file.txtand copy in the text:https://azure.microsoft.com/en-us/overview/what-is-azure/ The Azure cloud platform is more than 200 products and cloud services designed to help you bring new solutions to life—to solve today’s challenges and create the future. Build, run, and manage applications across multiple clouds, on-premises, and at the edge, with the tools and frameworks of your choice.In Visual Studio Code, open a new bash terminal at the root of the project to use the function API to upload the
test-file.txt:curl -X POST \ -F 'filename=@test-file.txt' \ -H 'Content-Type: text/plain' \ 'http://localhost:7071/api/upload?filename=test-file.txt&username=jsmith' --verboseCheck the response for a status code of 200:
Note: Unnecessary use of -X or --request, POST is already inferred. * Trying ::1:7071... * Trying 127.0.0.1:7071... * Connected to localhost (127.0.0.1) port 7071 (#0) > POST /api/upload?filename=README.md&username=jsmith HTTP/1.1 > Host: localhost:7071 > User-Agent: curl/7.77.0 > Accept: */* > Content-Length: 964 > Content-Type: multipart/form-data; boundary=------------------------549ebfc06c8f40ab > * We are completely uploaded and fine * Mark bundle as not supporting multiuse < HTTP/1.1 200 OK < Date: Mon, 27 Sep 2021 16:53:56 GMT < Content-Type: text/plain; charset=utf-8 < Server: Kestrel < Transfer-Encoding: chunked < { "string": "jsmith/README.md" }* Connection #0 to host localhost left intactIn Visual Studio Code, in the file explorer, expand the azureStorage/blobstorage folder and view the contents of the file.
Locally, you've called the function and uploaded the file to the storage emulator successfully.
8. Deploy to Azure with Visual Studio Code
In Visual Studio Code, open the Azure Explorer, then right-click the deployment icon under Functions to deploy your app:
Alternately, you can deploy by opening the Command Palette (F1), entering
deploy to function app, and running the Azure Functions: Deploy to Function App command.Use the following table to complete the prompts to create a new Azure Function resource.
Prompt Value Notes Select Function App in Azure Create new Function app in Azure (Advanced) Create a cloud-based resource for your function. Enter a globally unique name for the new Function App The name becomes part of the API's URL. API is invoked with an HTTP request. Valid characters for a function app name are 'a-z', '0-9', and '-'. An example is blob-storage-upload-function-app-jsmith. You can replacejsmithwith your own name, if you would prefer.Select a runtime stack Select a Node.js stack with the LTSdescriptor.LTS means long-term support. Select an OS. Windows Windows is selected specifically for the stream logs integration in Visual Studio Code. Linux log streaming is available from the Azure portal. Select a resource group for new resources. blob-storage-upload-function-groupSelect the resource group you created. Select a location for new resources. Select the recommended region. Select a hosting plan. Consumption Select a storage account. + Create new storage account Enter the name of the new storage account. blobstoragefunctionSelect an Application Insights resource for your app. + Create new Application Insights resource. Enter an Application Insights resource for your app. blob-storage-upload-function-app-insightsThe Visual Studio Code Output panel for Azure Functions shows progress:
When deploying, the entire Functions application is deployed, any changes to individual APIs are deployed at once.
9. Create an Azure Storage Resource
In Visual Studio Code, select the Azure explorer, then right-click on your subscription under Storage to select Create Storage Account (Advanced).
Use the following table to finish creating the local Azure Function project:
Prompt Value Notes Enter a globally unique name for the new Storage resource blobstoragefunctionThe name must be 3 to 24 lowercase letters and numbers only. Select a resource group for new resources. blob-storage-upload-function-groupSelect the resource group you created. Would you like to enable static website hosting? No. Select a location for new resources. Select one of the recommended locations close to use.
10. Set Storage connection string in Function app setting
- In Visual Studio Code, select the Azure explorer, then right-click on your new storage resource, and select Copy Connection String.
- Still in the Azure explorer, expand your Azure Function app, then expand the Application Settings node and right-click AzureWebJobsStorage to select Edit Setting.
- Paste in the Azure Storage connection string and press enter to complete the change.
11. Use cloud-based function
Once deployment is completed and the AzureWebJobsStorage app setting have been updated, test your Azure Function.
Open a text file and copy in the following:
curl -X POST \ -F 'filename=@test-file.txt' \ -H 'Content-Type: text/plain' \ 'REPLACE-WITH-YOUR-FUNCTION-URL' --verboseIn Visual Studio Code, select the Azure explorer, then expand the node for your Functions app, then expand Functions. Right-click the function name,
uploadand select Copy Function Url:
Paste the URL into a text file overwriting
REPLACE-WITH-YOUR-FUNCTION-URL.Append the filename and username query string name/value pairs:
Name Value username jsmithfilename test-file.txtThe final cURL command format should be similar to the following, except for your own substitutions for username and function resource name:
curl -X POST \ -F 'filename=@test-file.txt' \ -H 'Content-Type: text/plain' \ 'https://blob-storage-upload-function-app-jsmith.azurewebsites.net/api/randomnumber?code=12345&filename=test-file.txt&username=jsmith' --verboseThe value for
codein your own URL will be a much longer value.Copy the complete cURL command and run it in a Visual Studio Code bash terminal at the root of your function app to upload the root file,
test-file.txt.* Trying 000.49.104.16:443... * Connected to blob-storage-upload-function-app-jsmith.azurewebsites.net (20.49.104.16) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt * CApath: none * TLSv1.3 (OUT), TLS handshake, Client hello (1): * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 * ALPN, server did not agree to a protocol * Server certificate: * subject: CN=*.azurewebsites.net * start date: Jul 7 18:20:52 2021 GMT * expire date: Jul 7 18:20:52 2022 GMT * subjectAltName: host "blob-storage-upload-function-app-jsmith.azurewebsites.net" matched cert's "*.azurewebsites.net"* issuer: C=US; O=Microsoft Corporation; CN=Microsoft RSA TLS CA 02 * SSL certificate verify ok. > POST /api/upload?code=123456&filename=test-file.txt&username=jsmith HTTP/1.1 > Host: blob-storage-upload-function-app-jsmith.azurewebsites.net > User-Agent: curl/7.75.0 > Accept: */* > Content-Length: 566 > Content-Type: multipart/form-data; boundary=------------------------57d6fc242c9faa80 > * We are completely uploaded and fine * Mark bundle as not supporting multiuse < HTTP/1.1 200 OK < Transfer-Encoding: chunked < Content-Type: text/plain; charset=utf-8 < Request-Context: appId=cid-v1:234a5745-1c92-46c6-84a3-6b4d6bb87e40 < Date: Tue, 28 Sep 2021 16:45:52 GMT < { "string": "jsmith/test-file.txt" }* Connection #0 to host blob-storage-upload-function-app-jsmith.azurewebsites.net left intactIn Visual Studio Code, open the Azure explorer, expand your Storage blob resource, under containers, and find the container name that matches your username value in the query string.
11. Query your Azure Function logs
In Visual Studio Code, select the Azure explorer, then under Functions, right-click on your function app, then select Open in Portal.
This opens the Azure portal to your Azure Function.
Select Application Insights from the Settings, then select View Application Insights data.
This link takes you to your separate metrics resource created for you when you created your Azure Function with Visual Studio Code.
Select Logs in the Monitoring section. If a Queries pop-up window appears, select the X in the top-right corner of the pop-up to close it.
In the New Query 1 pane, on the Tables tab, double-click the traces table.
This enters the Kusto query,
tracesinto the query window.Edit the query to search for the custom logs:
traces | where message startswith "***"Select Run.
If the log doesn't display any results, it may be because there is a few minutes delay between the HTTP request to the Azure Function and the log availability in Kusto. Wait a few minutes and run the query again.
12. Clean up Azure resources
In Visual Studio Code, in the Azure explorer, find the resource group name,
blob-storage-upload-function-group, in the list.Right-click the resource group name and select Delete.
Troubleshooting
If you try to use this sample and run into an error regarding split from the parse-multipart library, verify that you are sending the filename property in your multiform data and that you are sending the content-type header into the function.NET