Creating a Great Mashup Part 2

Article
06/17/2013

This post details features used by Earthquake Explorer , a Windows 8 app that displays earthquake information on Bing maps. Earthquake Explorer was based off of the Earthquakes mashup starter kit on github . Read part 1 here.

In the first part, we created a Windows Azure Mobile Service to store data as a back end for our mashup. A big benefit of this approach is that it lets us filter/sort/query the data any way we’d like, and our application is now resilient against outages and changes in the schema. In this part, we are going to modify our existing scheduled task to consume the actual USGS data feed and store it in a table.

Getting the data is simple, thanks to the nice GeoJSON feed provided by the USGS. Read more about that here:

https://earthquake.usgs.gov/earthquakes/feed/v1.0/

From within our scheduled job, we can query it like so:

 function fetchUSGS() {
    console.log('Starting job');

    var httpRequest = require('request');
    var uri = 'https://earthquake.usgs.gov/earthquakes/feed/v0.1/summary/all_hour.geojson';

    httpRequest(uri, function (err, response, body) {
        if (err) {
            console.warn('Error connecting to service.');
        } else if (response.statusCode !== 200) {
            console.warn('Unexpected reply.');
        } else {
            //got reply, process the results
            var theData = JSON.parse(response.body);
            processData(theData);
        }
    });
}

We’re making a request to the feed (uri) and specifying the callback as the inline function. There really isn’t any robust error handling, but then, there really isn’t much you can do in the event of an error except just try again the next time the task runs.

Calling JSON.parse() will, naturally, parse the response text into a nice object collection we can parse. If we look at the JSON in fiddler, you can see the structure of the document:

The JSON contains a collection of features, where each feature contains a geometry object, an id string, and a properties object. Either through the documentation on the USGS site, or through examining the results, we’ll need to know the structure of the data.

To make working with the data a bit easier we can create a simple javascript class to hold the data:

 function earthquake(item) {
    this.latitude = item.geometry.coordinates[1];
    this.longitude = item.geometry.coordinates[0];
    this.depth = item.geometry.coordinates[2];
    this.usgsId = item.id;
    this.usgsCode = item.properties.code;
    this.tz = item.properties.tz;
    this.mag = item.properties.mag;
    this.time = item.properties.time;
    this.updated = item.properties.updated;
    this.place = item.properties.place;
    this.timestamp = new Date();
}

The one tricky part here is that the identifier of the earthquake is the USGS ID, but all data in Azure Mobile Services uses a bigint as a primary key. When we’re updating a row in Mobile Services, we’d pass the object into the update method with the appropriate ID (bigint). (Deletes require the bigint ID only.) Yes, this is a slight inefficiency because WAMS uses the surrogate bigint, and the USGS prefers the natural key of the USGS ID, and we will, for simplicity sake, use both depending on the operation. Because we’re requesting the data frequently, each item in the feed will have one of a few states: it may be new (not in the database), it may be in the database already, or it may be in the database but the feed contains updated/revised values.

 function processData(json) {

    var quakesTable = tables.getTable('earthquakes');
    var featureList = json.features;

    featureList.forEach(
        function (item) {
            var eq = new earthquake(item);

            quakesTable.where({
                usgsId: eq.usgsId
            }).read({
                success: function (results) {
                    if (results.length > 0) {
                        //record exists                    
                        eq.id = results[0].id;
                        eq.timestamp = new Date();
                        quakesTable.update(eq);
                      
                    } else {
                        //record doesn't exist                          
                        quakesTable.insert(eq);

                        //send notification?
                        //see next blog post                         
                    }
                }
            });
        }
    );
}

As we iterate through the features, we create an earthquake object from each item. We then check the table for a record with the same USGS ID property. If it already exists, we’ll update the row with the information because details may have changed. Otherwise, we’ll insert it.

There’s a general inefficiency here that we can deal with if we’re so motivated. Each item results in a query to see if the item exists, and then a query to either update or insert. We could simply build a stored procedure that handles either scenario so it’s one call, but because each feed contains about 10 items and this scheduled task is a server-side job, it’s not high on my priority list. With higher volume apps and/or with client apps, this is a modification you should make.

The other nice thing we can do is send notifications when a new earthquake is found. For example, if a new earthquake is found in the feed, we can send a live tile notification which would update the tile to look like so:

For Windows 8 apps, this is really a must-do feature. Do _something_ (anything!) with the live tile – this will differentiate your app. Mobile Services can send notifications to Windows Phone, Windows 8, iOS, and Android, so this is a great way to take the app cross platform. In the next post, we’ll look at how to do a live tile update like the one above.

Creating a Great Mashup Part 2

Additional resources