Get started with the Azure WebJobs SDK for event-driven background processing

This article shows how to create an Azure WebJobs SDK project, run it locally, and deploy it to Azure App Service.

The instructions are for Visual Studio 2017, but the same tasks can be accomplished with other tools, such as Visual Studio Code.

What is the Azure WebJobs SDK

The Azure WebJobs SDK is a framework that simplifies the task of writing background processing code that accesses data in Azure services. The SDK features a declarative syntax for specifying events that should trigger a function, such as a new message added to a queue. Similar declarative syntax controls reading and writing data once a function has been triggered. This system of triggers and bindings takes care of most of the low-level coding tasks associated with accessing Azure and third-party services.

Functions, triggers and bindings

A WebJobs SDK project defines one or more functions. A function is a method that has a trigger attribute in its method signature. Triggers specify conditions for calling a function, and bindings specify what to read and write. For example, the trigger attribute in the following function tells the runtime to call the function whenever a queue message appears in the items queue. The Blob attribute tells the runtime to use the queue message to read a blob in the workitems container. The content of the queue message — provided in the queueTrigger parameter — is the name of the blob.

public static void Run(
    [QueueTrigger("items")] string myQueueItem,
    [Blob("workitems/{queueTrigger}", FileAccess.Read)] Stream myBlob,
    TraceWriter log)
{
    log.Info($"BlobInput processed blob\n Name:{myQueueItem} \n Size: {myBlob.Length} bytes");
}

Versions 2.x and 3.x

The instructions tell how to create a WebJobs SDK version 2.x project. The latest version of the WebJobs SDK is 3.x, but it is currently in preview and this article doesn't have instructions for that version yet. The main change introduced by version 3.x is the use of .NET Core instead of .NET Framework.

Azure Functions

Azure Functions is based on the WebJobs SDK and is an option when you don't need to use the WebJobs SDK directly. Azure Functions 1.x uses the WebJobs SDK 2.x. For more information, see comparison between Azure Functions and the WebJobs SDK.

Prerequisites

This article assumes you have an Azure account and experience with apps in Azure App Service. To complete the steps in this article:

  • Install Visual Studio 2017 with the Azure development workload. If you already have Visual Studio but don't have that workload, add the workload by selecting Tools > Get Tools and Features.
  • Create an App Service app. If you already have one that you can deploy a WebJob to, you can use that instead of creating a new one.

Create a project

  1. In Visual Studio, select File > New project.

  2. Select Windows Classic Desktop > Console App (.NET Framework).

  3. Name the project WebJobsSDKSample, and then select OK.

    New Project dialog

Add WebJobs NuGet package

  1. Install the latest stable 2.x version of the NuGet package Microsoft.Azure.WebJobs.

    Here's the Package Manager Console command for version 2.2.0:

    Install-Package Microsoft.Azure.WebJobs -version 2.2.0
    

Create the JobHost

The JobHost object is the runtime container for functions: it listens for triggers and calls functions.

  1. In Program.cs, add a using statement:

    using Microsoft.Azure.WebJobs;
    
  2. Replace the Main method with the following code:

    static void Main()
    {
        var config = new JobHostConfiguration();
        var host = new JobHost(config);
        host.RunAndBlock();
    }
    

Enable console logging

There are several options for logging in WebJobs SDK project. The one we recommend is the logging framework that was developed for ASP.NET Core. This framework offers better performance and more flexibility in storage media and filtering.

In this section, you set up console logging that uses the new framework.

  1. Install the latest stable version of the following NuGet packages:

    • Microsoft.Extensions.Logging - The logging framework.
    • Microsoft.Extensions.Logging.Console - The console provider. A provider sends logs to a particular destination, in this case to the console.

    Here are the Package Manager Console commands for version 2.0.1:

    Install-Package Microsoft.Extensions.Logging -version 2.0.1
    
    Install-Package Microsoft.Extensions.Logging.Console -version 2.0.1
    
  2. In Program.cs, add a using statement:

    using Microsoft.Extensions.Logging;
    
  3. In the Main method, add code to update the JobHostConfiguration before creating the JobHost:

    config.DashboardConnectionString = "";
    var loggerFactory = new LoggerFactory();
    config.LoggerFactory = loggerFactory
        .AddConsole();
    

    This code makes the following changes:

    • Disables dashboard logging. The dashboard is a legacy monitoring tool, and dashboard logging is not recommended for high-throughput production scenarios.
    • Adds the console provider with default filtering.

    The Main method now looks like this:

    var config = new JobHostConfiguration();
    config.DashboardConnectionString = "";
    var loggerFactory = new LoggerFactory();
    config.LoggerFactory = loggerFactory
        .AddConsole();
    var host = new JobHost(config);
    host.RunAndBlock();
    

Create a function

  1. Create Functions.cs in the project folder, and replace the template code with this code:

    using Microsoft.Azure.WebJobs;
    using Microsoft.Azure.WebJobs.Host;
    using Microsoft.Extensions.Logging;
    
    namespace WebJobsSDKSample
    {
        public class Functions
        {
            public static void ProcessQueueMessage([QueueTrigger("queue")] string message, ILogger logger)
            {
                logger.LogInformation(message);
            }
        }
    }
    

    The QueueTrigger attribute tells the runtime to call this function when a new message is written on an Azure Storage queue called queue. The contents of the queue message are provided to the method code in the message parameter. The body of the method is where you process the trigger data. In this example, the code just logs the message.

    The message parameter doesn't have to be a string. You can also bind to a JSON object, a byte array, or a CloudQueueMessage object. See Queue trigger usage. Each binding type (such as queues, blobs, or tables) has a different set of parameter types that you can bind to.

Create a storage account

The Azure Storage emulator that runs locally doesn't have all of the features that the WebJobs SDK needs. So in this section you create a Storage account in Azure and configure the project to use it.

  1. Open Server Explorer in Visual studio and sign in to Azure. Right-click the Azure node, and then select Connect to Microsoft Azure Subscription.

    Sign in to Azure

  2. Under the Azure node in Server Explorer, right-click Storage, and then select Create Storage account.

    Create Storage account menu

  3. In the Create Storage Account dialog box, enter a unique name for the storage account.

  4. Choose the same Region that you created your App Service app in, or a region close to you.

  5. Select Create.

    Create Storage account

  6. Under the Storage node in Server Explorer, select the new Storage account. In the Properties window, select the ellipsis (...) at the right of the Connection String value field.

    Connection String ellipsis

  7. Copy the connection string, and save this value somewhere that you can copy it again readily.

    Copy connection string

Configure storage for running locally

The WebJobs SDK looks for the Storage connection string in the App Settings collection. When you run locally, it looks for this value in the App.config file or environment variables.

  1. Add the following XML to the App.config file, immediately after the opening <configuration> tag.

    <connectionStrings>
      <add name="AzureWebJobsStorage" connectionString="{storage connection string}" />
    </connectionStrings>
    
  2. Replace {storage connection string} with the connection string that you copied earlier.

    Later you'll use the connection string again, when you configure the App Service app in Azure.

Test locally

In this section, you build and run the project locally and trigger the function by creating a queue message.

  1. Press Ctrl+F5 to run the project.

    The console shows that the runtime found your function and is waiting for queue messages to trigger it.

    Found the following functions:
    WebJobsSDKSample.Functions.ProcessQueueMessage
    info: Host.Startup[0]
          Found the following functions:
          WebJobsSDKSample.Functions.ProcessQueueMessage
    Job host started
    info: Host.Startup[0]
          Job host started
    

    You may see a warning message about a ServicePointManager setting. For the testing you'll be doing with this project, you can ignore the warning. For more information about the warning, see How to use the WebJobs SDK.

  2. Close the console window.

  3. In Server Explorer in Visual Studio, expand the node for your new storage account, and then right-click Queues.

  4. Select Create Queue.

  5. Enter queue as the name for the queue, and then select OK.

    Create queue

  6. Right-click the node for the new queue, and then select View Queue.

  7. Select the Add Message icon.

    Create queue

  8. In the Add Message dialog, enter Hello World! as the Message text, and then select OK.

    Create queue

  9. Run the project again.

    Because you used the QueueTrigger attribute in the ProcessQueueMessage function, the WeJobs SDK runtime listens for queue messages when it starts up. It finds a new queue message in the queue named queue and calls the function.

    Due to queue polling exponential backoff, it might take as long as 2 minutes for the runtime to find the message and invoke the function. This wait time can be reduced by running in development mode.

    The console output looks like this:

    Found the following functions:
    WebJobsSDKSample.Functions.ProcessQueueMessage
    info: Host.Startup[0]
          Found the following functions:
          WebJobsSDKSample.Functions.ProcessQueueMessage
    Job host started
    info: Host.Startup[0]
          Job host started
    Executing 'Functions.ProcessQueueMessage' (Reason='New queue message detected on 'queue'.', Id=ebcb275d-0d7c-4293-a1af-93e0804b9e49)
    info: Function[0]
          Hello World!
    info: Host.Results[0]
          Executed 'Functions.ProcessQueueMessage' (Succeeded, Id=ebcb275d-0d7c-4293-a1af-93e0804b9e49)
    Executed 'Functions.ProcessQueueMessage' (Succeeded, Id=ebcb275d-0d7c-4293-a1af-93e0804b9e49)
    
  10. Close the console window.

Add Application Insights logging

When the project runs in Azure, you can't monitor function execution by viewing console output. The monitoring solution we recommend is Application Insights. For more information, see Monitor Azure Functions.

In this section, you do the following tasks to set up Application Insights logging before you deploy to Azure:

  • Make sure you have an App Service app and an Application Insights instance to work with.
  • Configure the App Service app to use the Application Insights instance and the storage account that you created earlier.
  • Set up the project for logging to Application Insights.

Create App Service app and Application Insights instance

  1. If you don't already have an App Service app that you can use, create one.

  2. If you don't already have an Application Insights resource that you can use, create one. Set Application type to General, and skip the sections that follow Copy the instrumentation key.

  3. If you already have an Application Insights resource that you want to use, copy the instrumentation key.

Configure app settings

  1. In Server Explorer in Visual Studio, expand the App Service node under Azure.

  2. Expand the resource group that your App Service app is in, and then right-click your App Service app.

  3. Select View Settings.

  4. In the Connection Strings box, add the following entry.

    Name connection String Database Type
    AzureWebJobsStorage {the Storage connection string that you copied earlier} Custom
  5. If the Application Settings box doesn't have an Application Insights instrumentation key, add the one that you copied earlier. (The instrumentation key may already be there, depending on how you created the App Service app.)

    Name Value
    APPINSIGHTS_INSTRUMENTATIONKEY {instrumentation key}
  6. Replace {instrumentation key} with the instrumentation key from the Application Insights resource that you're using.

  7. Select Save.

  8. Add the following XML to the App.config file, immediately after the connection strings collection.

    <appSettings>
      <add key="APPINSIGHTS_INSTRUMENTATIONKEY" value="{instrumentation key}" />
    </appSettings>
    
  9. Replace {instrumentation key} with the instrumentation key from the Application Insights resource that you're using.

    Adding this data to the App.config file enables you to test the Application Insights connection when you run the project locally.

  10. Save your changes.

Add Application Insights logging provider

  1. Install the latest stable 2.x version of the NuGet package for the Application Insights logging provider: Microsoft.Azure.WebJobs.Logging.ApplicationInsights.

    Here's the Package Manager Console command for version 2.2.0:

    Install-Package Microsoft.Azure.WebJobs.Logging.ApplicationInsights -version 2.2.0
    
  2. Install the latest stable 4.x version of the NuGet package for the .NET configuration manager: System.Configuration.ConfigurationManager.

    Here's the Package Manager Console command for version 4.4.1:

    Install-Package System.Configuration.ConfigurationManager -version 4.4.1
    
  3. Open Program.cs and add a using statement for the configuration manager:

    using System.Configuration;
    
  4. Replace the code in the Main method with the following code:

    using (var loggerFactory = new LoggerFactory())
    {
        var config = new JobHostConfiguration();
        var instrumentationKey =
            ConfigurationManager.AppSettings["APPINSIGHTS_INSTRUMENTATIONKEY"];
        config.DashboardConnectionString = "";
        config.LoggerFactory = loggerFactory
            .AddApplicationInsights(instrumentationKey, null)
            .AddConsole();
        var host = new JobHost(config);
        host.RunAndBlock();
    }
    

    This code makes the following changes:

    • Adds an Application Insights logging provider with default filtering; all Information and higher level logs will now go to both the console and Application Insights when you're running locally.
    • Puts the LoggerFactory object in a using block to ensure that log output is flushed when the host exits.

Test Application Insights logging

In this section you run locally again to verify that logging data is now going to Application Insights as well as to the console.

  1. Use Server Explorer in Visual Studio to create a queue message, the same way you did earlier, except enter Hello App Insights! as the message text.

  2. Run the project.

    The WebJobs SDK processes the queue message and you see the logs in the console window.

  3. Close the console window.

  4. Open the Azure portal, and go to your Application Insights resource.

  5. Select Search.

    Select Search

  6. If you don't see the Hello App Insights! message, select Refresh periodically for several minutes. (Logs don't appear immediately because it takes a while for the Application Insights client to flush the logs it processes.)

    Logs in Application Insights

  7. Close the console window.

Deploy as a WebJob

In this section you deploy the project as a WebJob. You deploy it to an App Service app that you created earlier. To test your code while it runs in Azure, you'll trigger a function invocation by creating a queue message.

  1. In Solution Explorer, right-click the project, and then select Publish as Azure WebJob.

  2. In the Add Azure WebJob dialog, select OK.

    Add Azure WebJob

    Visual Studio automatically installs a NuGet package for WebJob publishing.

  3. In the Profile step of the Publish wizard, select Microsoft Azure App Service.

    Publish dialog

  4. In the App Service dialog, select your resource group > your App Service app, and then select OK.

    App Service dialog

  5. In the Connection step of the wizard, select Publish.

Trigger the function in Azure

  1. Make sure you're not running locally (close the console window if it's still open). Otherwise the local instance might be the first to process any queue messages you create.

  2. Refresh the Queue page in Visual Studio, and the new message has disappeared because the function running in Azure App Service processed it.

    Tip

    When you're testing in Azure, use development mode to ensure that a queue trigger function is invoked right away and avoid delays due to queue polling exponential backoff.

View logs in Application Insights

  1. Open the Azure portal, and go to your Application Insights resource.

  2. Select Search.

  3. If you don't see the Hello Azure! message, select Refresh periodically for several minutes.

    You see the logs from the function running in a WebJob, including the Hello Azure! text that you entered in the preceding section.

Add an input binding

Input bindings simplify code that reads data. For this example, the queue message will be a blob name and you'll use the blob name to find and read a blob in Azure Storage.

  1. In Functions.cs, replace the ProcessQueueMessage method with the following code:

    public static void ProcessQueueMessage(
        [QueueTrigger("queue")] string message,
        [Blob("container/{queueTrigger}", FileAccess.Read)] Stream myBlob,
        ILogger logger)
    {
        logger.LogInformation($"Blob name:{message} \n Size: {myBlob.Length} bytes");
    }
    

    In this code, queueTrigger is a binding expression, which means it resolves to a different value at runtime. At runtime it has the contents of the queue message.

  2. Add a using:

    using System.IO;
    
  3. Create a blob container in your storage account.

    a. In Server Explorer in Visual Studio, expand the node for your storage account, right-click Blobs, and then select Create Blob Container.

    b. In the Create Blob Container dialog, enter container as the container name, and then click OK.

  4. Upload the Program.cs file to the blob container. (This file is used here as an example; you could upload any text file and create a queue message with the file's name.)

    a. In Server Explorer, double-click the node for the container you just created.

    b. In the Container window, select the Upload button.

    Blob upload button

    c. Find and select Program.cs, and then select OK.

  5. Create a queue message in the queue you created earlier, with Program.cs as the text of the message.

    Queue message Program.cs

  6. Run the project.

    The queue message triggers the function, which then reads the blob and logs its length. The console output looks like this:

    Found the following functions:
    ConsoleApp1.Functions.ProcessQueueMessage
    Job host started
    Executing 'Functions.ProcessQueueMessage' (Reason='New queue message detected on 'queue'.', Id=5a2ac479-de13-4f41-aae9-1361f291ff88)
    Blob name:Program.cs
    Size: 532 bytes
    Executed 'Functions.ProcessQueueMessage' (Succeeded, Id=5a2ac479-de13-4f41-aae9-1361f291ff88)
    

Add an output binding

Output bindings simplify code that writes data. This example modifies the previous one by writing a copy of the blob instead of logging its size.

  1. Replace the ProcessQueueMessage method with the following code:

    public static void ProcessQueueMessage(
        [QueueTrigger("queue")] string message,
        [Blob("container/{queueTrigger}", FileAccess.Read)] Stream myBlob,
        [Blob("container/copy-{queueTrigger}", FileAccess.Write)] Stream outputBlob,
        ILogger logger)
    {
        logger.LogInformation($"Blob name:{message} \n Size: {myBlob.Length} bytes");
        myBlob.CopyTo(outputBlob);
    }
    
  2. Create another queue message with Program.cs as the text of the message.

  3. Run the project.

    The queue message triggers the function, which then reads the blob, logs its length, and creates a new blob. The console output is the same, but when you go to the blob container window and select Refresh, you see a new blob named copy-Program.cs.

Next steps

This guide has shown how to create, run, and deploy a WebJobs SDK project.

To show everything that goes into a WebJobs SDK project, the instructions had you create a project from scratch. However, when you create your next project, consider using the Azure WebJob template in the Cloud category. This template creates a project with NuGet packages and sample code already set up. Note that the sample code may need to be changed to use the new logging framework.

For more information, see How to use the WebJobs SDK.