Get started with the Azure WebJobs SDK for event-driven background processing

This article shows how to use Visual Studio 2019 to create an Azure WebJobs SDK project, run it locally, and then deploy it to Azure App Service. Version 3.x of the WebJobs SDK supports both .NET Core and .NET Framework console apps. To learn more about working with the WebJobs SDK, see How to use the Azure WebJobs SDK for event-driven background processing.

This article shows you how to deploy WebJobs as a .NET Core console app. To deploy WebJobs as a .NET Framework console app, see WebJobs as .NET Framework console apps. If you are interested in WebJobs SDK version 2.x, which only supports .NET Framework, see Develop and deploy WebJobs using Visual Studio - Azure App Service.

Prerequisites

  • Install Visual Studio 2019 with the Azure development workload. If you already have Visual Studio but don't have that workload, add the workload by selecting Tools > Get Tools and Features.

  • You must have an Azure account to publish your WebJobs SDK project to Azure.

Create a project

  1. In Visual Studio, select Create a New Project.

  2. Select Console App (.NET Core).

  3. Name the project WebJobsSDKSample, and then select Create.

    New Project dialog

WebJobs NuGet packages

  1. Install the latest stable 3.x version of the Microsoft.Azure.WebJobs.Extensions NuGet package, which includes Microsoft.Azure.WebJobs.

    Here's the Package Manager Console command for version 3.0.2:

    Install-Package Microsoft.Azure.WebJobs.Extensions -version 3.0.2
    

Create the Host

The host is the runtime container for functions that listens for triggers and calls functions. The following steps create a host that implements IHost, which is the Generic Host in ASP.NET Core.

  1. In Program.cs, add a using statement:

    using Microsoft.Extensions.Hosting;
    
  2. Replace the Main method with the following code:

    static void Main(string[] args)
    {
        var builder = new HostBuilder();
        builder.ConfigureWebJobs(b =>
                {
                    b.AddAzureStorageCoreServices();
                });
        var host = builder.Build();
        using (host)
        {
            host.Run();
        }
    }
    

In ASP.NET Core, host configurations are set by calling methods on the HostBuilder instance. For more information, see .NET Generic Host. The ConfigureWebJobs extension method initializes the WebJobs host. In ConfigureWebJobs, you initialize specific WebJobs extensions and set properties of those extensions.

Enable console logging

In this section, you set up console logging that uses the ASP.NET Core logging framework.

  1. Install the latest stable version of the Microsoft.Extensions.Logging.Console NuGet package, which includes Microsoft.Extensions.Logging.

    Here's the Package Manager Console command for version 2.2.0:

    Install-Package Microsoft.Extensions.Logging.Console -version 2.2.0
    
  2. In Program.cs, add a using statement:

    using Microsoft.Extensions.Logging;
    
  3. Call the ConfigureLogging method on HostBuilder. The AddConsole method adds console logging to the configuration.

    builder.ConfigureLogging((context, b) =>
    {
        b.AddConsole();
    });
    

    The Main method now looks like this:

    static void Main(string[] args)
    {
        var builder = new HostBuilder();
        builder.ConfigureWebJobs(b =>
                {
                    b.AddAzureStorageCoreServices();
                });
        builder.ConfigureLogging((context, b) =>
                {
                    b.AddConsole();
                });
        var host = builder.Build();
        using (host)
        {
            host.Run();
        }
    }
    

    This update does the following:

    • Disables dashboard logging. The dashboard is a legacy monitoring tool, and dashboard logging is not recommended for high-throughput production scenarios.
    • Adds the console provider with default filtering.

Now, you can add a function that is triggered by messages arriving in an Azure Storage queue.

Install the Storage binding extension

Starting with version 3.x, you must explicitly install the Storage binding extension required by the WebJobs SDK. In prior versions, the Storage bindings were included in the SDK.

  1. Install the latest stable version of the Microsoft.Azure.WebJobs.Extensions.Storage NuGet package, version 3.x.

    Here's the Package Manager Console command for version 3.0.4:

    Install-Package Microsoft.Azure.WebJobs.Extensions.Storage -Version 3.0.4
    
  2. In the ConfigureWebJobs extension method, call the AddAzureStorage method on the HostBuilder instance to initialize the Storage extension. At this point, the ConfigureWebJobs method looks like the following example:

    builder.ConfigureWebJobs(b =>
                    {
                        b.AddAzureStorageCoreServices();
                        b.AddAzureStorage();
                    });
    

Create a function

  1. Right-click the project, select Add > New Item..., choose Class, name the new C# class file Functions.cs, and select Add.

  2. In Functions.cs, replace the generated template with the following code:

    using Microsoft.Azure.WebJobs;
    using Microsoft.Extensions.Logging;
    
    namespace WebJobsSDKSample
    {
        public class Functions
        {
            public static void ProcessQueueMessage([QueueTrigger("queue")] string message, ILogger logger)
            {
                logger.LogInformation(message);
            }
        }
    }
    

    The QueueTrigger attribute tells the runtime to call this function when a new message is written on an Azure Storage queue called queue. The contents of the queue message are provided to the method code in the message parameter. The body of the method is where you process the trigger data. In this example, the code just logs the message.

    The message parameter doesn't have to be a string. You can also bind to a JSON object, a byte array, or a CloudQueueMessage object. See Queue trigger usage. Each binding type (such as queues, blobs, or tables) has a different set of parameter types that you can bind to.

Create a storage account

The Azure Storage emulator that runs locally doesn't have all of the features that the WebJobs SDK needs. So in this section you create a storage account in Azure and configure the project to use it. If you already have a storage account, skip down to step 6.

  1. Open Server Explorer in Visual studio and sign in to Azure. Right-click the Azure node, and then select Connect to Microsoft Azure Subscription.

    Sign in to Azure

  2. Under the Azure node in Server Explorer, right-click Storage, and then select Create Storage account.

    Create Storage account menu

  3. In the Create Storage Account dialog box, enter a unique name for the storage account.

  4. Choose the same Region that you created your App Service app in, or a region close to you.

  5. Select Create.

    Create Storage account

  6. Under the Storage node in Server Explorer, select the new Storage account. In the Properties window, select the ellipsis (...) at the right of the Connection String value field.

    Connection String ellipsis

  7. Copy the connection string, and save this value somewhere that you can copy it again readily.

    Copy connection string

Configure storage to run locally

The WebJobs SDK looks for the storage connection string in the Application Settings in Azure. When you run locally, it looks for this value in the local configuration file or in environment variables.

  1. Right-click the project, select Add > New Item..., choose JavaScript JSON configuration file, name the new file appsettings.json file, and select Add.

  2. In the new file, add a AzureWebJobsStorage field, as in the following example:

    {
        "AzureWebJobsStorage": "{storage connection string}"
    }
    
  3. Replace {storage connection string} with the connection string that you copied earlier.

  4. Select the appsettings.json file in Solution Explorer and in the Properties window, set Copy to Output Directory to Copy if newer.

Later, you'll add the same connection string app setting in your app in Azure App Service.

Test locally

In this section, you build and run the project locally and trigger the function by creating a queue message.

  1. Press Ctrl+F5 to run the project.

    The console shows that the runtime found your function and is waiting for queue messages to trigger it. The following output is generated by the v3.x host:

     info: Microsoft.Azure.WebJobs.Hosting.JobHostService[0]
           Starting JobHost
     info: Host.Startup[0]
           Found the following functions:
           WebJobsSDKSample.Functions.ProcessQueueMessage
    
     info: Host.Startup[0]
           Job host started
     Application started. Press Ctrl+C to shut down.
     Hosting environment: Development
     Content root path: C:\WebJobsSDKSample\WebJobsSDKSample\bin\Debug\netcoreapp2.1\
    
  2. Close the console window.

  3. In Server Explorer in Visual Studio, expand the node for your new storage account, and then right-click Queues.

  4. Select Create Queue.

  5. Enter queue as the name for the queue, and then select OK.

    Create queue

  6. Right-click the node for the new queue, and then select View Queue.

  7. Select the Add Message icon.

    Create queue

  8. In the Add Message dialog, enter Hello World! as the Message text, and then select OK. There is now a message in the queue.

    Create queue

  9. Run the project again.

    Because you used the QueueTrigger attribute in the ProcessQueueMessage function, the WeJobs SDK runtime listens for queue messages when it starts up. It finds a new queue message in the queue named queue and calls the function.

    Due to queue polling exponential backoff, it might take as long as 2 minutes for the runtime to find the message and invoke the function. This wait time can be reduced by running in development mode.

    The console output looks like this:

     info: Function.ProcessQueueMessage[0]
           Executing 'Functions.ProcessQueueMessage' (Reason='New queue message detected on 'queue'.', Id=2c319369-d381-43f3-aedf-ff538a4209b8)
     info: Function.ProcessQueueMessage[0]
           Trigger Details: MessageId: b00a86dc-298d-4cd2-811f-98ec39545539, DequeueCount: 1, InsertionTime: 1/18/2019 3:28:51 AM +00:00
     info: Function.ProcessQueueMessage.User[0]
           Hello World!
     info: Function.ProcessQueueMessage[0]
           Executed 'Functions.ProcessQueueMessage' (Succeeded, Id=2c319369-d381-43f3-aedf-ff538a4209b8)
    
  10. Close the console window.

  11. Go back to the Queue window and refresh it. The message is gone, since it has been processed by your function running locally.

Add Application Insights logging

When the project runs in Azure, you can't monitor function execution by viewing console output. The monitoring solution we recommend is Application Insights. For more information, see Monitor Azure Functions.

In this section, you do the following tasks to set up Application Insights logging before you deploy to Azure:

  • Make sure you have an App Service app and an Application Insights instance to work with.
  • Configure the App Service app to use the Application Insights instance and the storage account that you created earlier.
  • Set up the project for logging to Application Insights.

Create App Service app and Application Insights instance

  1. If you don't already have an App Service app that you can use, create one. When you create your app, you can also create a connected Application Insights resource. When you do this, the APPINSIGHTS_INSTRUMENTATIONKEY is set for you in your app.

  2. If you don't already have an Application Insights resource that you can use, create one. Set Application type to General, and skip the sections that follow Copy the instrumentation key.

  3. If you already have an Application Insights resource that you want to use, copy the instrumentation key.

Configure app settings

  1. In Server Explorer in Visual Studio, expand the App Service node under Azure.

  2. Expand the resource group that your App Service app is in, and then right-click your App Service app.

  3. Select View Settings.

  4. In the Connection Strings box, add the following entry.

    Name connection String Database Type
    AzureWebJobsStorage {the Storage connection string that you copied earlier} Custom
  5. If the Application Settings box doesn't have an Application Insights instrumentation key, add the one that you copied earlier. (The instrumentation key may already be there, depending on how you created the App Service app.)

    Name Value
    APPINSIGHTS_INSTRUMENTATIONKEY {instrumentation key}
  6. Replace {instrumentation key} with the instrumentation key from the Application Insights resource that you're using.

  7. Select Save.

  8. Add the Application Insights connection to the project so that you can run it locally. In the appsettings.json file, add an APPINSIGHTS_INSTRUMENTATIONKEY field, as in the following example:

    {
        "AzureWebJobsStorage": "{storage connection string}",
        "APPINSIGHTS_INSTRUMENTATIONKEY": "{instrumentation key}"
    }
    

    Replace {instrumentation key} with the instrumentation key from the Application Insights resource that you're using.

  9. Save your changes.

Add Application Insights logging provider

To take advantage of Application Insights logging, update your logging code to do the following:

  • Add an Application Insights logging provider with default filtering; all Information and higher-level logs goes to both the console and Application Insights when you're running locally.
  • Put the LoggerFactory object in a using block to ensure that log output is flushed when the host exits.
  1. Install the latest stable 3.x version of the NuGet package for the Application Insights logging provider: Microsoft.Azure.WebJobs.Logging.ApplicationInsights.

    Here's the Package Manager Console command for version 3.0.2:

    Install-Package Microsoft.Azure.WebJobs.Logging.ApplicationInsights -Version 3.0.2
    
  2. Open Program.cs and replace the code in the Main method with the following code:

    static void Main(string[] args)
    {
        var builder = new HostBuilder();
        builder.UseEnvironment(EnvironmentName.Development);
        builder.ConfigureWebJobs(b =>
                {
                    b.AddAzureStorageCoreServices();
                    b.AddAzureStorage();
                });
        builder.ConfigureLogging((context, b) =>
                {
                    b.AddConsole();
    
                    // If the key exists in settings, use it to enable Application Insights.
                    string instrumentationKey = context.Configuration["APPINSIGHTS_INSTRUMENTATIONKEY"];
                    if (!string.IsNullOrEmpty(instrumentationKey))
                    {
                        b.AddApplicationInsights(o => o.InstrumentationKey = instrumentationKey);
                    }
                });
        var host = builder.Build();
        using (host)
        {
            host.Run();
        }
    }
    

    This adds the Application Insights provider to the logging, using the key you added earlier to your app settings.

Test Application Insights logging

In this section, you run locally again to verify that logging data is now going to Application Insights as well as to the console.

  1. Use Server Explorer in Visual Studio to create a queue message, the same way you did earlier, except enter Hello App Insights! as the message text.

  2. Run the project.

    The WebJobs SDK processes the queue message and you see the logs in the console window.

  3. Close the console window.

  4. Open the Azure portal, and go to your Application Insights resource.

  5. Select Search.

    Select Search

  6. If you don't see the Hello App Insights! message, select Refresh periodically for several minutes. (Logs don't appear immediately because it takes a while for the Application Insights client to flush the logs it processes.)

    Logs in Application Insights

  7. Close the console window.

Deploy to Azure

During deployment, you create an app service instance in which to run your functions. When you publish a .NET Core console app to App Service in Azure, it automatically gets run as a WebJob. To learn more about publishing, see Develop and deploy WebJobs using Visual Studio.

  1. In Solution Explorer, right-click the project and select Publish.

  2. In the Publish dialog, select Microsoft Azure App Service, choose Create New, and then select Publish.

    Pick publish target

  3. In the Create App Service dialog, use the hosting settings as specified in the table below the image:

    Create App Service dialog

    Setting Suggested value Description
    App Name Globally unique name Name that uniquely identifies your new function app.
    Subscription Choose your subscription The Azure subscription to use.
    Resource Group myResourceGroup Name of the resource group in which to create your function app. Choose New to create a new resource group.
    Hosting Plan App Service plan An App Service plan specifies the location, size, and features of the web server farm that hosts your app. You can save money when hosting multiple apps by configuring the web apps to share a single App Service plan. App Service plans define the region, instance size, scale count, and SKU (Free, Shared, Basic, Standard, or Premium). Choose New to create a new App Service plan.
  4. Click Create to create a WebJob and related resources in Azure with these settings and deploy your project code.

Trigger the function in Azure

  1. Make sure you're not running locally (close the console window if it's still open). Otherwise the local instance might be the first to process any queue messages you create.

  2. In the Queue page in Visual Studio, add a message to the queue as before.

  3. Refresh the Queue page, and the new message disappears because it has been processed by the function running in Azure.

    Tip

    When you're testing in Azure, use development mode to ensure that a queue trigger function is invoked right away and avoid delays due to queue polling exponential backoff.

View logs in Application Insights

  1. Open the Azure portal, and go to your Application Insights resource.

  2. Select Search.

  3. If you don't see the Hello Azure! message, select Refresh periodically for several minutes.

    You see the logs from the function running in a WebJob, including the Hello Azure! text that you entered in the preceding section.

Add an input binding

Input bindings simplify code that reads data. For this example, the queue message will be a blob name and you'll use the blob name to find and read a blob in Azure Storage.

  1. In Functions.cs, replace the ProcessQueueMessage method with the following code:

    public static void ProcessQueueMessage(
        [QueueTrigger("queue")] string message,
        [Blob("container/{queueTrigger}", FileAccess.Read)] Stream myBlob,
        ILogger logger)
    {
        logger.LogInformation($"Blob name:{message} \n Size: {myBlob.Length} bytes");
    }
    

    In this code, queueTrigger is a binding expression, which means it resolves to a different value at runtime. At runtime, it has the contents of the queue message.

  2. Add a using:

    using System.IO;
    
  3. Create a blob container in your storage account.

    a. In Server Explorer in Visual Studio, expand the node for your storage account, right-click Blobs, and then select Create Blob Container.

    b. In the Create Blob Container dialog, enter container as the container name, and then click OK.

  4. Upload the Program.cs file to the blob container. (This file is used here as an example; you could upload any text file and create a queue message with the file's name.)

    a. In Server Explorer, double-click the node for the container you created.

    b. In the Container window, select the Upload button.

    Blob upload button

    c. Find and select Program.cs, and then select OK.

  5. Create a queue message in the queue you created earlier, with Program.cs as the text of the message.

    Queue message Program.cs

  6. Run the project locally.

    The queue message triggers the function, which then reads the blob and logs its length. The console output looks like this:

    Found the following functions:
    ConsoleApp1.Functions.ProcessQueueMessage
    Job host started
    Executing 'Functions.ProcessQueueMessage' (Reason='New queue message detected on 'queue'.', Id=5a2ac479-de13-4f41-aae9-1361f291ff88)
    Blob name:Program.cs
    Size: 532 bytes
    Executed 'Functions.ProcessQueueMessage' (Succeeded, Id=5a2ac479-de13-4f41-aae9-1361f291ff88)
    

Add an output binding

Output bindings simplify code that writes data. This example modifies the previous one by writing a copy of the blob instead of logging its size. Blob storage bindings are included in the Azure Storage extension package that we installed previously.

  1. Replace the ProcessQueueMessage method with the following code:

    public static void ProcessQueueMessage(
        [QueueTrigger("queue")] string message,
        [Blob("container/{queueTrigger}", FileAccess.Read)] Stream myBlob,
        [Blob("container/copy-{queueTrigger}", FileAccess.Write)] Stream outputBlob,
        ILogger logger)
    {
        logger.LogInformation($"Blob name:{message} \n Size: {myBlob.Length} bytes");
        myBlob.CopyTo(outputBlob);
    }
    
  2. Create another queue message with Program.cs as the text of the message.

  3. Run the project locally.

    The queue message triggers the function, which then reads the blob, logs its length, and creates a new blob. The console output is the same, but when you go to the blob container window and select Refresh, you see a new blob named copy-Program.cs.

Republish the updates to Azure

  1. In Solution Explorer, right-click the project and select Publish.

  2. In the Publish dialog, make sure that the current profile is selected and then choose Publish. Results of the publish are detailed in the Output window.

  3. Verify the function in Azure by again uploading a file to the blob container and adding a message to the queue that is the name of the uploaded file. You see the message get removed from the queue and a copy of the file created in the blob container.

Next steps

This article showed you how to create, run, and deploy a WebJobs SDK 3.x project.