Monitor scenario in Durable Functions - Weather watcher sample

The monitor pattern refers to a flexible recurring process in a workflow - for example, polling until certain conditions are met. This article explains a sample that uses Durable Functions to implement monitoring.

Prerequisites

Complete the quickstart article:

Scenario overview

This sample monitors a location's current weather conditions and alerts a user by SMS when the skies are clear. You could use a regular timer-triggered function to check the weather and send alerts. However, one problem with this approach is lifetime management. If only one alert should be sent, the monitor needs to disable itself after clear weather is detected. The monitoring pattern can end its own execution, among other benefits:

  • Monitors run on intervals, not schedules: a timer trigger runs every hour; a monitor waits one hour between actions. A monitor's actions will not overlap unless specified, which can be important for long-running tasks.
  • Monitors can have dynamic intervals: the wait time can change based on some condition.
  • Monitors can terminate when some condition is met or be terminated by another process.
  • Monitors can take parameters. The sample shows how the same weather-monitoring process can be applied to any requested location and phone number.
  • Monitors are scalable. Because each monitor is an orchestration instance, multiple monitors can be created without having to create new functions or define more code.
  • Monitors integrate easily into larger workflows. A monitor can be one section of a more complex orchestration function, or a sub-orchestration.

Configuring Twilio integration

This sample involves using the Twilio service to send SMS messages to a mobile phone. Azure Functions already has support for Twilio via the Twilio binding, and the sample uses that feature.

The first thing you need is a Twilio account. You can create one free at https://www.twilio.com/try-twilio. Once you have an account, add the following three app settings to your function app.

App setting name Value description
TwilioAccountSid The SID for your Twilio account
TwilioAuthToken The Auth token for your Twilio account
TwilioPhoneNumber The phone number associated with your Twilio account. This is used to send SMS messages.

Configuring Weather Underground integration

This sample involves using the Weather Underground API to check current weather conditions for a location.

The first thing you need is a Weather Underground account. You can create one for free at https://www.wunderground.com/signup. Once you have an account, you will need to acquire an API key. You can do so by visiting https://www.wunderground.com/weather/api, then selecting Key Settings. The Stratus Developer plan is free and sufficient to run this sample.

Once you have an API key, add the following app setting to your function app.

App setting name Value description
WeatherUndergroundApiKey Your Weather Underground API key.

The functions

This article explains the following functions in the sample app:

  • E3_Monitor: An orchestrator function that calls E3_GetIsClear periodically. It calls E3_SendGoodWeatherAlert if E3_GetIsClear returns true.
  • E3_GetIsClear: An activity function that checks the current weather conditions for a location.
  • E3_SendGoodWeatherAlert: An activity function that sends an SMS message via Twilio.

The following sections explain the configuration and code that are used for C# scripting and JavaScript. The code for Visual Studio development is shown at the end of the article.

The weather monitoring orchestration (Visual Studio Code and Azure portal sample code)

The E3_Monitor function uses the standard function.json for orchestrator functions.

{
    "bindings": [
      {
        "name": "monitorContext",
        "type": "orchestrationTrigger",
        "direction": "in"
      }
    ]
  }

Here is the code that implements the function:

C#

#r "Microsoft.Azure.WebJobs.Extensions.DurableTask"
#r "Microsoft.Extensions.Logging"

#load "..\shared\MonitorRequest.csx"

using System.Threading;

public static async Task Run(DurableOrchestrationContext monitorContext, ILogger log)
{
    MonitorRequest input = monitorContext.GetInput<MonitorRequest>();
    if (!monitorContext.IsReplaying) { log.LogInformation($"Received monitor request. Location: {input?.Location}. Phone: {input?.Phone}."); }

    VerifyRequest(input);

    DateTime endTime = monitorContext.CurrentUtcDateTime.AddHours(6);
    if (!monitorContext.IsReplaying) { log.LogInformation($"Instantiating monitor for {input.Location}. Expires: {endTime}."); }

    while (monitorContext.CurrentUtcDateTime < endTime)
    {
        // Check the weather
        if (!monitorContext.IsReplaying) { log.LogInformation($"Checking current weather conditions for {input.Location} at {monitorContext.CurrentUtcDateTime}."); }

        bool isClear = await monitorContext.CallActivityAsync<bool>("E3_GetIsClear", input.Location);

        if (isClear)
        {
            // It's not raining! Or snowing. Or misting. Tell our user to take advantage of it.
            if (!monitorContext.IsReplaying) { log.LogInformation($"Detected clear weather for {input.Location}. Notifying {input.Phone}."); }

            await monitorContext.CallActivityAsync("E3_SendGoodWeatherAlert", input.Phone);
            break;
        }
        else
        {
            // Wait for the next checkpoint
            var nextCheckpoint = monitorContext.CurrentUtcDateTime.AddMinutes(30);
            if (!monitorContext.IsReplaying) { log.LogInformation($"Next check for {input.Location} at {nextCheckpoint}."); }

            await monitorContext.CreateTimer(nextCheckpoint, CancellationToken.None);
        }
    }

    log.LogInformation("Monitor expiring.");
}

private static void VerifyRequest(MonitorRequest request)
{
    if (request == null)
    {
        throw new ArgumentNullException(nameof(request), "An input object is required.");
    }

    if (request.Location == null)
    {
        throw new ArgumentNullException(nameof(request.Location), "A location input is required.");
    }

    if (string.IsNullOrEmpty(request.Phone))
    {
        throw new ArgumentNullException(nameof(request.Phone), "A phone number input is required.");
    }
}

JavaScript (Functions 2.x only)

const df = require("durable-functions");
const moment = require('moment');

module.exports = df.orchestrator(function*(context) {
    const input = context.df.getInput();
    context.log("Received monitor request. location: " + (input ? input.location : undefined)
        + ". phone: " + (input ? input.phone : undefined) + ".");

    verifyRequest(input);

    const endTime = moment.utc(context.df.currentUtcDateTime).add(6, 'h');
    context.log("Instantiating monitor for " + input.location.city + ", " + input.location.state
        + ". Expires: " + (endTime) + ".");

    while (moment.utc(context.df.currentUtcDateTime).isBefore(endTime)) {
        // Check the weather
        context.log("Checking current weather conditions for " + input.location.city + ", "
            + input.location.state + " at " + context.df.currentUtcDateTime + ".");
        const isClear = yield context.df.callActivity("E3_GetIsClear", input.location);

        if (isClear) {
            // It's not raining! Or snowing. Or misting. Tell our user to take advantage of it.
            context.log("Detected clear weather for " + input.location.city + ", "
                + input.location.state + ". Notifying " + input.phone + ".");

            yield context.df.callActivity("E3_SendGoodWeatherAlert", input.phone);
            break;
        } else {
            // Wait for the next checkpoint
            var nextCheckpoint = moment.utc(context.df.currentUtcDateTime).add(30, 's');
            context.log("Next check for " + input.location.city + ", " + input.location.state
                + " at " + nextCheckpoint.toString());

            yield context.df.createTimer(nextCheckpoint.toDate());   // accomodate cancellation tokens
        }
    }

    context.log("Monitor expiring.");
});

function verifyRequest(request) {
    if (!request) {
        throw new Error("An input object is required.");
    }
    if (!request.location) {
        throw new Error("A location input is required.");
    }
    if (!request.phone) {
        throw new Error("A phone number input is required.");
    }
}

This orchestrator function performs the following actions:

  1. Gets the MonitorRequest consisting of the location to monitor and the phone number to which it will send an SMS notification.
  2. Determines the expiration time of the monitor. The sample uses a hard-coded value for brevity.
  3. Calls E3_GetIsClear to determine whether there are clear skies at the requested location.
  4. If the weather is clear, calls E3_SendGoodWeatherAlert to send an SMS notification to the requested phone number.
  5. Creates a durable timer to resume the orchestration at the next polling interval. The sample uses a hard-coded value for brevity.
  6. Continues running until the CurrentUtcDateTime (C#) or currentUtcDateTime (JavaScript) passes the monitor's expiration time, or an SMS alert is sent.

Multiple orchestrator instances can run simultaneously by sending multiple MonitorRequests. The location to monitor and the phone number to send an SMS alert to can be specified.

Strongly-typed data transfer (.NET only)

The orchestrator requires multiple pieces of data, so shared POCO objects are used for strongly-typed data transfer in C# and C# script:

#load "Location.csx"

public class MonitorRequest
{
    public Location Location { get; set; }
 
    public string Phone { get; set; }
}
public class Location
{
    public string State { get; set; }

    public string City { get; set; }

    public override string ToString() => $"{City}, {State}";
}

The JavaScript sample uses regular JSON objects as parameters.

Helper activity functions

As with other samples, the helper activity functions are regular functions that use the activityTrigger trigger binding. The E3_GetIsClear function gets the current weather conditions using the Weather Underground API and determines whether the sky is clear. The function.json is defined as follows:

{
    "bindings": [
      {
        "name": "location",
        "type": "activityTrigger",
        "direction": "in"
      }
    ]
  }

And here is the implementation. Like the POCOs used for data transfer, logic to handle the API call and parse the response JSON is abstracted into a shared class in C#. You can find it as part of the Visual Studio sample code.

C#

#r "Microsoft.Azure.WebJobs.Extensions.DurableTask"

#load "..\shared\Location.csx"
#load "..\shared\WeatherUnderground.csx"

public static async Task<bool> Run(Location location)
{
    var currentConditions = await WeatherUnderground.GetCurrentConditionsAsync(location);
    return currentConditions.Equals(WeatherCondition.Clear);
}

JavaScript (Functions 2.x only)

const request = require("request-promise-native");

const clearWeatherConditions = ['Overcast', 'Clear', 'Partly Cloudy', 'Mostly Cloudy', 'Scattered Clouds'];

module.exports = async function (context, location) {
    try {
        const data = await getCurrentConditions(location);
        return clearWeatherConditions.includes(data.weather);
    } catch (err) {
        context.log(`E3_GetIsClear encountered an error: ${err}`);
        throw new Error(err);
    }
}

async function getCurrentConditions(location) {
    const options = {
        url: `https://api.wunderground.com/api/${process.env["WeatherUndergroundApiKey"]}/conditions/q/${location.state}/${location.city}.json`,
        method: 'GET',
        json: true
    };

    const body = await request(options);
    if (body.error) {
        throw body.error;
    } else if (body.response && body.response.error) {
        throw body.response.error;
    } else {
        return body.current_observation;
    }
}

The E3_SendGoodWeatherAlert function uses the Twilio binding to send an SMS message notifying the end user that it's a good time for a walk. Its function.json is simple:

{
    "bindings": [
      {
        "name": "phoneNumber",
        "type": "activityTrigger",
        "direction": "in"
      },
      {
        "type": "twilioSms",
        "name": "message",
        "from": "%TwilioPhoneNumber%",
        "accountSidSetting": "TwilioAccountSid",
        "authTokenSetting": "TwilioAuthToken",
        "direction": "out"
      }
    ]
  }

And here is the code that sends the SMS message:

C#

#r "Microsoft.Azure.WebJobs.Extensions.DurableTask"
#r "Microsoft.Azure.WebJobs.Extensions.Twilio"
#r "Twilio"

using Twilio.Rest.Api.V2010.Account;
using Twilio.Types;

public static void Run(string phoneNumber, out CreateMessageOptions message)
{
    message = new CreateMessageOptions(new PhoneNumber(phoneNumber));
    message.Body = $"The weather's clear outside! Go take a walk!";
}

JavaScript (Functions 2.x only)

module.exports = async function(context, phoneNumber) {
    context.bindings.message = {
        body: `The weather's clear outside! Go take a walk!`,
        to: phoneNumber
    };
};

Run the sample

Using the HTTP-triggered functions included in the sample, you can start the orchestration by sending the following HTTP POST request:

POST https://{host}/orchestrators/E3_Monitor
Content-Length: 77
Content-Type: application/json

{ "location": { "city": "Redmond", "state": "WA" }, "phone": "+1425XXXXXXX" }
HTTP/1.1 202 Accepted
Content-Type: application/json; charset=utf-8
Location: https://{host}/admin/extensions/DurableTaskExtension/instances/f6893f25acf64df2ab53a35c09d52635?taskHub=SampleHubVS&connection=Storage&code={SystemKey}
RetryAfter: 10

{"id": "f6893f25acf64df2ab53a35c09d52635", "statusQueryGetUri": "https://{host}/admin/extensions/DurableTaskExtension/instances/f6893f25acf64df2ab53a35c09d52635?taskHub=SampleHubVS&connection=Storage&code={systemKey}", "sendEventPostUri": "https://{host}/admin/extensions/DurableTaskExtension/instances/f6893f25acf64df2ab53a35c09d52635/raiseEvent/{eventName}?taskHub=SampleHubVS&connection=Storage&code={systemKey}", "terminatePostUri": "https://{host}/admin/extensions/DurableTaskExtension/instances/f6893f25acf64df2ab53a35c09d52635/terminate?reason={text}&taskHub=SampleHubVS&connection=Storage&code={systemKey}"}

The E3_Monitor instance starts and queries the current weather conditions for the requested location. If the weather is clear, it calls an activity function to send an alert; otherwise, it sets a timer. When the timer expires, the orchestration will resume.

You can see the orchestration's activity by looking at the function logs in the Azure Functions portal.

2018-03-01T01:14:41.649 Function started (Id=2d5fcadf-275b-4226-a174-f9f943c90cd1)
2018-03-01T01:14:42.741 Started orchestration with ID = '1608200bb2ce4b7face5fc3b8e674f2e'.
2018-03-01T01:14:42.780 Function completed (Success, Id=2d5fcadf-275b-4226-a174-f9f943c90cd1, Duration=1111ms)
2018-03-01T01:14:52.765 Function started (Id=b1b7eb4a-96d3-4f11-a0ff-893e08dd4cfb)
2018-03-01T01:14:52.890 Received monitor request. Location: Redmond, WA. Phone: +1425XXXXXXX.
2018-03-01T01:14:52.895 Instantiating monitor for Redmond, WA. Expires: 3/1/2018 7:14:52 AM.
2018-03-01T01:14:52.909 Checking current weather conditions for Redmond, WA at 3/1/2018 1:14:52 AM.
2018-03-01T01:14:52.954 Function completed (Success, Id=b1b7eb4a-96d3-4f11-a0ff-893e08dd4cfb, Duration=189ms)
2018-03-01T01:14:53.226 Function started (Id=80a4cb26-c4be-46ba-85c8-ea0c6d07d859)
2018-03-01T01:14:53.808 Function completed (Success, Id=80a4cb26-c4be-46ba-85c8-ea0c6d07d859, Duration=582ms)
2018-03-01T01:14:53.967 Function started (Id=561d0c78-ee6e-46cb-b6db-39ef639c9a2c)
2018-03-01T01:14:53.996 Next check for Redmond, WA at 3/1/2018 1:44:53 AM.
2018-03-01T01:14:54.030 Function completed (Success, Id=561d0c78-ee6e-46cb-b6db-39ef639c9a2c, Duration=62ms)

The orchestration will terminate once its timeout is reached or clear skies are detected. You can also use TerminateAsync (.NET) or terminate (JavaScript) inside another function or invoke the terminatePostUri HTTP POST webhook referenced in the 202 response above, replacing {text} with the reason for termination:

POST https://{host}/admin/extensions/DurableTaskExtension/instances/f6893f25acf64df2ab53a35c09d52635/terminate?reason=Because&taskHub=SampleHubVS&connection=Storage&code={systemKey}

Visual Studio sample code

Here is the orchestration as a single C# file in a Visual Studio project:

Note

You will need to install the Microsoft.Azure.WebJobs.Extensions.Twilio Nuget package to run the sample code below.

// Copyright (c) .NET Foundation. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.

using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json.Linq;
#if NETSTANDARD2_0
using Twilio.Rest.Api.V2010.Account;
using Twilio.Types;
#else
using Twilio;
#endif

/* This sample demonstrates the Monitor workflow. In this pattern, the orchestrator function is
 * used to periodically check something's status and take action as appropriate. While a
 * Timer-triggered function can perform similar polling action, the Monitor has additional
 * capabilities:
 *
 *   - manual termination (via request to the orchestrator termination endpoint)
 *   - termination when some condition is met
 *   - monitoring of multiple arbitrary subjects
 *
 * To run this sample, you'll need to define the following app settings:
 *
 *   - TwilioAccountSid: your Twilio account's SID
 *   - TwilioAuthToken: your Twilio account's auth token
 *   - TwilioPhoneNumber: an SMS-capable Twilio number
 *   - WeatherUndergroundApiKey: a WeatherUnderground API key
 *
 * For Twilio trial accounts, you also need to verify the phone number in your MonitorRequest.
 *
 * Twilio: https://www.twilio.com
 * WeatherUnderground API: https://www.wunderground.com/weather/api/d/docs
 */
namespace VSSample
{
    public static class Monitor
    {
        [FunctionName("E3_Monitor")]
        public static async Task Run([OrchestrationTrigger] DurableOrchestrationContext monitorContext, ILogger log)
        {
            MonitorRequest input = monitorContext.GetInput<MonitorRequest>();
            if (!monitorContext.IsReplaying) { log.LogInformation($"Received monitor request. Location: {input?.Location}. Phone: {input?.Phone}."); }

            VerifyRequest(input);

            DateTime endTime = monitorContext.CurrentUtcDateTime.AddHours(6);
            if (!monitorContext.IsReplaying) { log.LogInformation($"Instantiating monitor for {input.Location}. Expires: {endTime}."); }

            while (monitorContext.CurrentUtcDateTime < endTime)
            {
                // Check the weather
                if (!monitorContext.IsReplaying) { log.LogInformation($"Checking current weather conditions for {input.Location} at {monitorContext.CurrentUtcDateTime}."); }

                bool isClear = await monitorContext.CallActivityAsync<bool>("E3_GetIsClear", input.Location);

                if (isClear)
                {
                    // It's not raining! Or snowing. Or misting. Tell our user to take advantage of it.
                    if (!monitorContext.IsReplaying) { log.LogInformation($"Detected clear weather for {input.Location}. Notifying {input.Phone}."); }

                    await monitorContext.CallActivityAsync("E3_SendGoodWeatherAlert", input.Phone);
                    break;
                }
                else
                {
                    // Wait for the next checkpoint
                    var nextCheckpoint = monitorContext.CurrentUtcDateTime.AddMinutes(30);
                    if (!monitorContext.IsReplaying) { log.LogInformation($"Next check for {input.Location} at {nextCheckpoint}."); }

                    await monitorContext.CreateTimer(nextCheckpoint, CancellationToken.None);
                }
            }

            log.LogInformation($"Monitor expiring.");
        }

        [FunctionName("E3_GetIsClear")]
        public static async Task<bool> GetIsClear([ActivityTrigger] Location location)
        {
            var currentConditions = await WeatherUnderground.GetCurrentConditionsAsync(location);
            return currentConditions.Equals(WeatherCondition.Clear);
        }

        [FunctionName("E3_SendGoodWeatherAlert")]
        public static void SendGoodWeatherAlert(
            [ActivityTrigger] string phoneNumber,
            ILogger log,
            [TwilioSms(AccountSidSetting = "TwilioAccountSid", AuthTokenSetting = "TwilioAuthToken", From = "%TwilioPhoneNumber%")]
#if NETSTANDARD2_0
                out CreateMessageOptions message)
#else
                out SMSMessage message)
#endif
        {
#if NETSTANDARD2_0
            message = new CreateMessageOptions(new PhoneNumber(phoneNumber));
#else
            message = new SMSMessage { To = phoneNumber };
#endif
            message.Body = $"The weather's clear outside! Go take a walk!";
        }

        private static void VerifyRequest(MonitorRequest request)
        {
            if (request == null)
            {
                throw new ArgumentNullException(nameof(request), "An input object is required.");
            }

            if (request.Location == null)
            {
                throw new ArgumentNullException(nameof(request.Location), "A location input is required.");
            }

            if (string.IsNullOrEmpty(request.Phone))
            {
                throw new ArgumentNullException(nameof(request.Phone), "A phone number input is required.");
            }
        }
    }

    public class MonitorRequest
    {
        public Location Location { get; set; }

        public string Phone { get; set; }
    }

    public class Location
    {
        public string State { get; set; }

        public string City { get; set; }

        public override string ToString() => $"{City}, {State}";
    }

    public enum WeatherCondition
    {
        Other,
        Clear,
        Precipitation,
    }

    internal class WeatherUnderground
    {
        private static readonly HttpClient httpClient = new HttpClient();
        private static IReadOnlyDictionary<string, WeatherCondition> weatherMapping = new Dictionary<string, WeatherCondition>()
        {
            { "Clear", WeatherCondition.Clear },
            { "Overcast", WeatherCondition.Clear },
            { "Cloudy", WeatherCondition.Clear },
            { "Clouds", WeatherCondition.Clear },
            { "Drizzle", WeatherCondition.Precipitation },
            { "Hail", WeatherCondition.Precipitation },
            { "Ice", WeatherCondition.Precipitation },
            { "Mist", WeatherCondition.Precipitation },
            { "Precipitation", WeatherCondition.Precipitation },
            { "Rain", WeatherCondition.Precipitation },
            { "Showers", WeatherCondition.Precipitation },
            { "Snow", WeatherCondition.Precipitation },
            { "Spray", WeatherCondition.Precipitation },
            { "Squall", WeatherCondition.Precipitation },
            { "Thunderstorm", WeatherCondition.Precipitation },
        };

        internal static async Task<WeatherCondition> GetCurrentConditionsAsync(Location location)
        {
            var apiKey = Environment.GetEnvironmentVariable("WeatherUndergroundApiKey");
            if (string.IsNullOrEmpty(apiKey))
            {
                throw new InvalidOperationException("The WeatherUndergroundApiKey environment variable was not set.");
            }

            var callString = string.Format("http://api.wunderground.com/api/{0}/conditions/q/{1}/{2}.json", apiKey, location.State, location.City);
            var response = await httpClient.GetAsync(callString);
            var conditions = await response.Content.ReadAsAsync<JObject>();

            JToken currentObservation;
            if (!conditions.TryGetValue("current_observation", out currentObservation))
            {
                JToken error = conditions.SelectToken("response.error");

                if (error != null)
                {
                    throw new InvalidOperationException($"API returned an error: {error}.");
                }
                else
                {
                    throw new ArgumentException("Could not find weather for this location. Try being more specific.");
                }
            }

            return MapToWeatherCondition((string)(currentObservation as JObject).GetValue("weather"));
        }

        private static WeatherCondition MapToWeatherCondition(string weather)
        {
            foreach (var pair in weatherMapping)
            {
                if (weather.Contains(pair.Key))
                {
                    return pair.Value;
                }
            }

            return WeatherCondition.Other;
        }
    }
}

Next steps

This sample has demonstrated how to use Durable Functions to monitor an external source's status using durable timers and conditional logic. The next sample shows how to use external events and durable timers to handle human interaction.